(CN) — While many have lauded the potential value of antibody tests as the world continues to grapple with the Covid-19 pandemic, a study released Wednesday indicates some of the tests are so flawed they shouldn’t be used.
Since the start of the coronavirus pandemic in the early part of this year, few subjects have received more airtime and discussion as accessible and accurate Covid-19 testing resources. While a great deal of this focus has been on molecular tests — which determine whether someone has the illness in that moment — much has also been made about serological tests which detect coronavirus antibodies in people who were previously infected.
Serological tests have been widely viewed as critical to help us better understand how and when populations become most exposed to the virus and to help health professionals more accurately gauge infection trends. British Prime Minister Boris Johnson — who suffered a bout of Covid-19 and was even hospitalized in intensive care for a time — personally hailed these serological tests as “game-changing.”
But this optimism may not be backed by the data.
A study published Wednesday in The BMJ by an international team of researchers reveals that significant weaknesses have been identified in the evidence base for serological tests. The evidence is particularly weak for point-of-care tests, conducted directly with a patient outside of a traditional laboratory environment, with the researchers saying the evidence doesn’t even support their continued use.
Researchers made these troubling observations by combing through 40 different studies for measuring sensitivity or specificity of a Covid-19 antibody test compared with a control test. They found the studies in medical databases and preprint servers from January to April.
They discovered at least half of the studies, the majority of which came from China, were not peer reviewed. Worse, most showed a either high or uncertain possibility of bias that would have wrongly skewed the final results.
Of the 40 studies, researchers noted that only four included data related to outpatients and just two examined tests at the point of care.
When they collected all of the sensitivity data from the studies — measurements of those who were correctly identified as having a disease — the researchers found anywhere between 2.2% and 34% of the patients with Covid-19 would have been missed.
“Overall, the poor performance of existing serological tests for Covid-19 raises questions about the utility of using such methods for medical decision making, particularly given time and effort required to do these tests and the challenging workloads many clinics are facing,” the study states.
Researchers also looked closely at lateral flow immunoassay tests, a point of care test that has been recently considered as a way to provide people with “immunity passports.” They weren’t impressed with the results.
According to their data, if lateral flow immunoassay tests used on a population with a Covid-19 prevalence of 10%, of 1,000 people tested there would be at least 31 who would be declared immune without ever having contracted the virus. Another 34 people who actually did have Covid-19 would be told they had never been infected, the researchers found.
Governments around the world should therefore not put too much stock in the accuracy of these tests, the researchers said.
“Our findings should also give pause to governments that are contemplating the use of serological test s— in particular, point-of-care tests — to issue immunity ‘certificates’ or ‘passports.’”
Faiz Ahmad Khan, one of the authors of the study and scientist at the Research Institute of the McGill University Health Centre, said that if we want to better understand our testing effectiveness, better clinical studies is a solid way to get it.
“We need more rigorously designed clinical studies, that actually see how these serological tests perform at the point of care and when used in the same populations where they will eventually be applied,” Khan said in an email. “Such studies could be rapidly conducted with international collaboration — that will be key to get answers that are applicable in a diversity of settings.”