Look out bullshit peddlers, the robot fact-checkers are coming to get you. Researchers at Indiana University have developed an automated system for checking claims made online. And while it’s not the first such system, it’s definitely a sign that automated fact-checking might soon become mainstream.
The new study published today, Computational Fact Checking From Knowledge Networks, used an algorithm that was able to determine the accuracy of simple facts. And it performed just as well as humans.
From Indiana University:
In what the IU scientists describe as an “automatic game of trivia,” the team applied their algorithm to answer simple questions related to geography, history and entertainment, including statements that matched states or nations with their capitals, presidents with their spouses and Oscar-winning film directors with the movie for which they won the Best Picture awards, with the majority of tests returning highly accurate truth scores.
But the impressive part came next. Their system was able to dig deeper and find correlations outside the “infoboxes” initially established by the researchers.
“The measurement of the truthfulness of statements appears to rely strongly on indirect connections, or ‘paths,’ between concepts,” researcher Giovanni Luca Ciampaglia said in a statement. “If we prevented our fact-checker from traversing multiple nodes on the graph, it performed poorly since it could not discover relevant indirect connections. But because it’s free to explore beyond the information provided in one infobox, our method leverages the power of the full knowledge graph.”
The only potential flaw in the system? Much like a human fact-checker, the robot version is only as good as its sources. By design, Wikipedia only allows secondary sources like newspaper articles and blog posts. As we all know, these sources can get things wrong.
The algorithms developed at Indiana University aren’t quite ready for application in the real world. But they’re close! And notoriously inaccurate sources of information like Uberfacts and HistoryInPics better take notice. The robots are coming to get you.