Will robots take over fact-checking the information overload?


Aw nuts. It’s sorry enough that we lean on computers to research and write stories.

It’s a given that much information is dredged from websites.

Now, we’re on to robots for fact-checking?

Sure, says Bill Adair, the Duke University professor who earlier helped birth the PolitiFact fact-checking operation at the Tampa Bay Times and in states around the country. The Austin American-Statesman’s PolitiFact Texas is among seven state-level PolitiFact franchises established nationally since 2010. All this happened after FactCheck.org debuted at the University of Pennsylvania and while The Washington Post launched its Pinnochio-popping FactChecker.

Globally, from 2014 to 2015, formal fact-checking efforts doubled to more than 60 projects, Adair reports. An international fact-checking group held its second gathering in London this summer.

In June, meantime, researchers at Indiana University revealed they’d devised a computer program to assign “truth scores” to excerpts from Wikipedia, the crowd-sourced online encyclopedia. Upshot: The computational fact-checker did well, consistently matching the assessment of human fact-checkers in certitude about the accuracy of statements.

A news release quoted Giovanni Luca Ciampaglia, a postdoctoral fellow at the Center for Complex Networks and Systems Research in the IU Bloomington School of Informatics and Computing, summing up: “We live in an age of information overload, including abundant misinformation, unsubstantiated rumors and conspiracy theories whose volume threatens to overwhelm journalists and the public. Our experiments point to methods to abstract the vital and complex human task of fact-checking into a network analysis problem, which is easy to solve computationally.”

Filippo Menczer, director of the Center for Complex Networks and Systems Research and a co-author on the study, said: “With increasing reliance on the Internet as a source of information, we need tools to deal with the misinformation that reaches us every day. Computational fact-checkers could become part of the solution to this problem.”

And now a University of Texas-Arlington team is getting closer to helping journalists determine which presidential debate statements warrant fact-checking. Still, I was kind of happy to hear, we’re not yet to the point of a computer churning out full-fledged fact checks.

Chengkai Li, a UT-Arlington associate professor, has enlisted university students and members of the public to hone his team’s compilation of potentially checkable claims made in the U.S. presidential debates from 1960 through 2012 as part of the ClaimBuster project, which Li describes as using “computational power to do tasks that are tedious and time-consuming for fact-checkers, such as finding claims by politicians that should be checked.”

By email, Li recently talked with the Statesman about the project.

Austin American-Statesman: How does the project work?

Participants … review transcripts from past presidential debates and identify sentences that should be fact-checked. These “labeled” sentences become training examples. Then, we use natural language processing (NLP) to extract features from the training examples. The features can include the words used in the sentences, the lengths of the sentences, the speaker’s attitude, and so on. We then use machine-learning algorithms to identify the characteristics of features in factual claims that are worth checking. The outcome of the machine-learning algorithm is called a model. With the help of this model, ClaimBuster examines any transcript to find sentences that should be fact-checked.

You’ve said your goal is to play a role in the 2016 presidential election. In what way?

During a live presidential debate, for each sentence coming from a candidate, ClaimBuster will give it a score which tells journalists and professional fact-checkers if the sentence should be fact-checked and how important it is. It will rank all the sentences by their scores. In this way, ClaimBuster will speed the process of finding important factual claims and leave the journalists with more time for assessing the validity of claims.

Can you envision a computer program that consistently evaluates factual claims?

The “Holy Grail” perhaps can never be completely accomplished.

There are mainly two fundamental challenges. One is to understand what one says. Computer scientists have made leaps and bounds in speech recognition and NLP and we now have wonderful tools such as [Apple’s] Siri in our hands. But these technologies are far from perfect.

The other lies in our capability of collecting sufficient evidence for checking facts. We are in the big-data era. A huge amount of useful data is accessible to us, and more is being made available at every second. But, what is being recorded is still tiny compared to the vast amount of information the universe holds. We will never be able to capture and record all such information.

A program can check with ease simple statements on related facts with no subtle implications, such as “the daily highest temperature in Arlington, Texas has been over 100 for 7 consecutive days.”

Of course, significant claims checked by professionals are usually complex and subtle. Even if a claim checks out at the surface level, it can be [rated] “Pants on Fire.” For instance, Mitt Romney said during a 2012 Republican presidential debate that “our Navy is smaller than it’s been since 1917.” To check this claim, first we need to figure out he was using number of military ships as the measure of U.S. Navy’s size. Furthermore, we need to understand he was suggesting that smaller Navy implies weaker Navy. Even though the U.S. Navy’s size in 2012 was indeed almost the smallest since 1917, it was hardly the weakest, since it makes little sense to compare modern battleships with those 100 years ago. For this and other reasons, PolitiFact gave his claim “Pants on Fire.”

Could a computer program eventually overtake human beings as fact-checkers?

Maybe, but not within many, many, many years.

We all have heard about the dream of artificial intelligence. We are much further than where we started merely 60 years ago. But it is still a dream.



Reader Comments ...


Next Up in Local

Testimony from prosecution witnesses to continue in Criner trial
Testimony from prosecution witnesses to continue in Criner trial

More testimony from the prosecution’s witnesses is scheduled for Tuesday in the capital murder trial of Meechaiel Criner, who is accused of killing University of Texas dance student Haruka Weiser in April 2016. The seventh day of the trial gets underway at 8:45 a.m. with state District Judge David Wahlberg presiding. On Monday, Austin police...
Valdez on her humble fundraising: ‘This election will not be bought’
Valdez on her humble fundraising: ‘This election will not be bought’

Democrat Lupe Valdez’s gubernatorial campaign reported raising $335,765 between May 13 and June 20 and had $222,050 in cash on hand as of June 30. Gov. Greg Abbott’s re-election campaign reported raising $10.9 million between Feb. 25 and June 30, and had $28.9 million in cash on hand, according to its filing Monday with the Texas Ethics...
A&M chancellor to discuss new contract to run Los Alamos nuclear lab
A&M chancellor to discuss new contract to run Los Alamos nuclear lab

Texas A&M University System Chancellor John Sharp on Tuesday will hold forth in Austin on the system’s partnership with the University of California System and Battelle Memorial Institute to operate Los Alamos National Laboratory, heart of the nation’s nuclear weapons complex. The A&M-UC-Battelle team, known as Triad National Security LLC...
Horse fatally struck by vehicle outside Circuit of the Americas, authorities say
Horse fatally struck by vehicle outside Circuit of the Americas, authorities say

8 a.m. update: A horse hit by a vehicle early Tuesday outside the Circuit of the Americas has died, the Travis County sheriff’s office said. A vehicle struck a horse that was loose in the area around 5 a.m., said sheriff’s spokeswoman Kristen Dark. The incident shut down the westbound lanes of FM 812 at Texas 130 and the eastbound...
FORECAST: Saharan dust causing hazy skies; Austin temps to reach 102
FORECAST: Saharan dust causing hazy skies; Austin temps to reach 102

Tuesday forecast for Austin: Saharan dust will continue to hover over the Austin area, so expect hazy skies at sunrise or during the morning commute, the National Weather Service said. But the dust in the air won’t be as noticeable as it was Monday, weather service meteorologist Jason Runyen said. Still, residents with respiratory issues or those...
More Stories