Researchers question claim about Facebook news algorithm curbing election misinformation

(CN) — A study claiming Meta’s news feed algorithm successfully filtered out fake news has been called into question by a team of researchers.

In 2023, the journal Science published an article investigating the effects of Instagram and Facebook’s feed algorithms during the 2020 U.S. election. The research was funded by Meta, the parent company of both of the social media companies analyzed in the work, and concluded that both social platforms were not spreading misinformation and had filtered out untrustworthy election news.

Now, a team of researchers led by the University of Massachusetts Amherst are questioning the results of that study.

The researchers discovered that the previous study was conducted during a narrow window of time when Meta temporarily switched from its standard news algorithm to a more rigorous one.

“The first thing that rang alarm bells for us was when we realized that the previous researchers conducted a randomized control experiment during the same time that Facebook had made a systemic, short-term change to their news algorithm,”  lead author Chhandak Bagchi, a graduate student in the Manning College of Information and Computer Science at the University of Massachusetts Amherst, said in a statement.

Meta introduced 63 “break glass” changes to the news feed on Facebook at the start of November 2020 intended to push down untrustworthy new sources spreading information about the election, which were successful, the researchers said.

“We applaud Facebook for implementing the more stringent news feed algorithm,” senior author Przemek Grabowicz said in a statement.

The implementation of the new algorithm cut user views of misinformation by at least 24%, according to the researchers. The problem is the algorithm was only temporary, and the platform reverted back to its previous algorithm several months later in March 2021.

The study published in Science, led by a researcher from Princeton University, ran from Sept. 24, 2020, to Dec. 23, 2020, thereby overlapping significantly with the window during which Facebook’s news algorithm was more strict.

“Their paper gives the impression that the standard Facebook algorithm is good at stopping misinformation, which is questionable,” Grabowicz said.

Experiments such as the one conducted by the previous research team have to be preregistered, meaning Meta could have been informed ahead of the study exactly what the researchers were going to be looking for. Social media platforms can change their algorithms without having to publicly disclose the changes.

“This can lead to situations where social media companies could conceivably change their algorithms to improve their public image if they know they are being studied,” the authors of the new research wrote.

While Meta funded the previous study and provided 12 co-authors, the authors of the new research acknowledged that the social media company didn’t have the right to prepublication approval.

“Our results show that social media companies can mitigate the spread of misinformation by modifying their algorithms but may not have financial incentives to do so,” Anthony Paik, coauthor and professor of sociology at the University of Massachusetts Amherst said. “A key question is whether the harms of misinformation — to individuals, the public and democracy — should be more central in their business decisions.”

The new research, titled “Social media algorithms can curb misinformation, but do they?,” was published in Science on Thursday, accompanied by an editorial from the editor-in-chief of the publication, as well as an eLetter from the original authors.

Michael Wagner, an “independent rapporteur” for the original study told Science editor-in-chief H. Holden Thorp that while Facebook had publicly stated it changed its algorithm during that time period, the topic was not widely discussed among the researchers.

In response to Thorp asking Wagner whether Facebook should have been more forthcoming with the researchers, Wagner said “it doesn’t do Facebook any favors in an environment where so many researchers, journalists, and social media platform skeptics are looking for things that Facebook has hidden from them.”

Meta told Thorp that it had been forthcoming about the emergency algorithm measures.

The authors of the original study acknowledged that readers shouldn’t conclude their study applies to algorithms used outside of their study period.

“We appreciate the opportunity to underscore the message to future readers that our results were in comparison to the feed-ranking algorithm used by Facebook at the time of the study," the original authors wrote.

The authors also insisted that the conclusions of their study at large still stand, and included a portion in the original paper acknowledging that changes to the algorithm may have affected the interpretation.