Sub-Total: €0.00
Diagnostic Accuracy of Rapid Antigen Test Kits for Detecting SARS-CoV-2: A Systematic Review and Meta-Analysis of 17,171 Suspected COVID-19 Patients
Abstract
Early diagnosis is still as crucial as the initial stage of the COVID-19 pandemic. As RT-PCR sometimes is not feasible in developing nations or rural areas, health professionals may use a rapid antigen test (RAT) to lessen the load of diagnosis. However, the efficacy of RAT is yet to be investigated thoroughly. Hence, we tried to evaluate the overall performance of RAT in SARS-CoV-2 diagnosis. Based on our PROSPERO registered protocol (CRD42021231432), we searched online databases (i.e., PubMed, Google Scholar, Scopus, and Web of Science) and analysed overall pooled specificity and sensitivity of RAT along with study quality, publication bias, heterogeneity and more. The overall pooled specificity and sensitivity of RAT were detected as 99.4% (95% CI: 99.1–99.8; I2 = 90%) and 68.4% (95% CI: 60.8–75.9; I2 = 98%), respectively. In subgroup analyses, nasopharyngeal specimens and symptomatic patient’s samples were more sensitive in RAT, while cycle threshold (Ct) values were found to have an inverse relationship with sensitivity. In the European and American populations, RAT showed better performance. Although the sensitivity of RAT is yet to be improved, it could still be an alternative in places with poor laboratory set up. Nevertheless, the negative samples of RAT can be re-tested using RT-PCR to reduce false negative results.
1. Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is responsible for the coronavirus disease 2019 (COVID-19), characterised mainly by fever, cough, sore throat, fatigue, joint and muscle pain and loss of smell and taste [1,2,3,4,5]. It was first identified in Hubei province in Wuhan, China, in December 2019 and until now, it continues to be a significant health issue worldwide. Because there is no specific therapy or medication, early and accurate diagnosis is critical in preventing the spread of the disease [6,7]. In vitro diagnostics have long been recognised as a valuable tool for outbreak control and patient care [8]. The current gold standard diagnostic test for laboratory diagnosis of SARS-CoV-2 is nucleic acid amplification tests (NAAT); reverse transcription polymerase chain reaction (RT-PCR) to be specific [9]. The advantage this has over other tests is its high sensitivity due to the nucleic acid amplification step while also having a high specificity due to the specific primers used. This is consistent with its purpose at referral centres, which is to advise clinical care of patients. However, since it can take several hours to obtain results from RT-PCR, it may be inappropriate to use during an emergency. It also requires costly technology and competent workers, both of which may not be accessible in remote health clinics, particularly in underdeveloped countries.
Rapid antigen test (RAT) kits have emerged as an important alternative tool to aid in the clinical diagnosis of COVID-19. RAT is based on immunochromatography, which employs antibodies spotted onto nitrocellulose membranes that interact with specific antigens from the patient sample. The antigen–antibody interaction can be visualised manually or by using an immunofluorescence machine reader. For the diagnosis of COVID-19, the target analyte is often the virus’ nucleocapsid protein. A similar strategy has been used for the rapid diagnosis of HIV, malaria, influenza and other diseases [9]. It is a reasonably inexpensive and simple test with quick results that may be used for point-of-care testing [10].
The advantage of RAT is that the test is more accessible to patients. It also enables appropriate infection control measures to be instituted earlier which is important in a pandemic [10,11]. However, a negative RAT result cannot rule out COVID-19 infection [12]. Patients with typical clinical presentation or close contacts of COVID-19 cases should undergo repeat testing [12,13].
For diagnosis of SARS-CoV-2, the World Health Organisation has recommended that a RAT kit needs to meet a minimum performance requirement of at least 80% sensitivity and 97% specificity compared with a NAAT reference assay to be used [14]. Studies regarding the diagnostic accuracy of RAT have produced a wide range of sensitivity. The varied results may be due to the various study designs, manufacturer of the RAT kits, patient selection, types of specimens and the phase of illness at the point of sample collection. While research and development of RAT to detect SARS-CoV-2 continue, this systematic review and meta-analysis aims to provide an update on the diagnostic accuracy in terms of estimating the specificity and sensitivity of RAT kits in patients with suspected COVID-19.
2. Methods
2.1. Study Protocol and Guideline
Based on the current literature, this systematic review and meta-analysis on the diagnostic accuracy of the available RAT kits for COVID-19 diagnosis was undertaken according to the PRISMA guideline [15]. The protocol of this study was registered in the PROSPERO database (CRD42021231432).
2.2. Eligibility Criteria
As the objective was to investigate the pooled sensitivity and specificity of the available RAT kits detecting SARS-CoV-2, we included studies in which RAT kits were used to identify SARS-CoV-2 to confirm COVID-19. Original studies without restricting study design or language were included. Review articles, opinions, case reports, news, press releases, blogs and data from websites were not considered eligible.
2.3. Search Strategies
Based on the eligibility criteria, we searched online databases such as PubMed, Google Scholar, Scopus, and Web of Science to identify studies of our interest published between 1 January 2020 and 13 January 2021. The following keywords were used to search different databases: COVID-19, SARS-CoV-2, coronavirus, nCoV, antigen, detection, diagnostic, diagnosis, test, testing, assay, assays and combined with appropriate Boolean operators (Table S1). Additionally, the reference lists of the included studies were also reviewed to identify any potentially eligible studies. EndNote X8 software (Clarivate Analytics, Philadelphia, PA, USA) was used to identify and exclude duplicate studies.
2.4. Study Selection
Three authors (S.S.K., N.H.H.N.H. and R.H.S.) independently screened the original pool of papers, then assessed their eligibility to be included in this meta-analysis using title, abstract, and full-text evaluation. Disagreements about whether a study should be included or excluded were discussed with the other authors (M.A.I. and Z.Z.D.) and resolved with acceptable consensus.
2.5. Data Extraction
Three authors (S.S.K., N.H.H.N.H. and R.H.S.) independently undertook the data extraction, and two authors (M.A.I. and Z.Z.D.) validated it. Initially, the data of the total number of positive and negative specimens confirmed by a reference standard followed by the number of positive and negative results of those same specimens were evaluated through RAT kits and they were extracted from each of the eligible studies. The major characteristics of the studies including the Study ID (last name of the first author and year of publication), location, total number of subjects, percentage of female subjects, the mean or median age of the participants, type of participants, specimen types, the test method of RAT kit used, percentages of positive samples detected by the reference standard (RT-PCR), ranges of Ct values of the reference standard and the manufacturer of the RAT kits were documented.
2.6. Quality Assessment
Two authors (S.S.K. and M.A.I.) independently assessed the quality of the included studies following the diagnostic test accuracy quality assessment tool of the Joanna Briggs Institute (JBI). To resolve the discrepancies, all authors took part in the discussion and resolved with consensus. If the total score was ≤49, 50–69, or ≥70%, the studies were classed as low-quality (high-risk of bias), moderate-quality (moderate-risk of bias), or high-quality (low risk of bias) [16]. To detect publication bias, funnel plots were constructed, and the Egger’s test was performed.
2.7. Data Analyses
A random-effects model with 95% confidence intervals (CIs) was used to analyse the overall pooled sensitivity and specificity of the RAT kit to detect COVID-19. I2 statistics were used to analyse heterogeneity among the included studies (I2 > 75% indicating substantial heterogeneity), followed by Cochran’s Q test to determine the significance of the heterogeneity (p < 0.05 was considered statistically significant). Furthermore, a Galbraith plot was generated to determine the outlier studies.
2.8. Subgroup and Sensitivity Analyses
Subgroup analyses were undertaken to investigate sensitivity and specificity based on symptomatic and asymptomatic patients, days of symptom onset, types of specimen, Ct values, countries, continents and manufacturers of RAT. To investigate the robustness of results and the possible source of heterogeneity, sensitivity analyses were performed through strategies such as excluding small studies (<100), excluding low- or moderate-quality studies, using a fixed-effects model, and excluding outlier studies. The analyses and plots were constructed by using the metaprop codes in the meta (version 4.15-1) and metafor (version 2.4-0) packages of R (version 3.6.3) in RStudio (RStudio, Inc., Boston, MA, USA) (version 1.3.1093).
3. Results
3.1. Study Selection
Primarily, based on the search strategies, a total of 1201 published articles were identified from the online databases. During the initial screening process, 872 articles including review articles (n = 31), case reports (n = 7), articles that included non-human subjects (n = 18), editorials, letters and comment (n = 65) and duplicate studies (n = 751) were excluded. From the remaining 329 studies, 300 studies did not comply with the objective of this meta-analysis (irrelevant to the objective of the meta-analysis, repetitive studies, protocols only or missing data of interest); hence regarded as ineligible. The remaining 29 studies fulfilled the eligibility criteria and were finally included in this meta-analysis (Figure 1).
PRISMA flow diagram of study selection.
3.2. Characteristics of the Included Studies
A total of 17,171 COVID-19 suspects, including RT-PCR-positive and negative participants who were further tested in RAT, were reported in this meta-analysis. Different types of specimen (i.e., nasopharyngeal swab, saliva, nasal swab, sputum, throat swab or endotracheal aspirates) were taken from suspected symptomatic or asymptomatic participants. The RT-PCR-positive participants had a wide range of Ct values. Although a total of 15 different RAT kits were assessed from 13 different manufacturers, the RAT test methods were based on either immunochromatographic (ICG) assay or fluorescence immunoassay (FIA). The studies were carried out in several countries encompassing five continents. Table 1 shows the detailed features of the studies we included.
Table 1
Major characteristics of the included studies.
| Study ID [References] | Location | Total Subjects (% Female) Mean/Median Age | Type of Participants | Specimen Types | Positive Sample by RT-PCR (%) | Range of Ct Values | Testing Method | Rapid Antigen Test Kit (Manufacturer, Country) |
|---|---|---|---|---|---|---|---|---|
| Abdelrazik 2020 [17] | Egypt | 310 (40.6, 42) | C-19 (n = 160) and HCW + CC (n = 150) | NPS | 60.6 | 15.8–32.3 | ICG | Biocredit COVID-19 Ag Detection Kit (RapiGEN, Korea) |
| Agulló 2020 [18] | Spain | 659 (56.4, 38) | SS (n = 394) and CC (n = 265) | NS, S | 20.0 | 14.0–33.0 (IQR) | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) |
| Albert 2020 [12] | Spain | 412 (58.0, 31) | SS (n = 412) | NPS | 13.1 | ≤25–≤34 | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) |
| Alemany 2020 [19] | Spain | 1406 (NR, 40) | SS (n = 446), CC (n = 473) and GS (n = 487) | NPS, NMT | 67.6 | 19.7–27.3 (IQR) | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) |
| Azzi 2020 † [20] | Italy | 122 (67.2, 54) | C-19, SS and HCW | NPS, S | 23.8 | NR | ICG | In house |
| Cerutti 2020 [21] | Italy | 185 (NR, 45) | SS (n = 185) | NPS | 56.2 | 12.3–38.1 | ICG | Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) |
| 145 (NR, 36) | T (n = 145) | 3.4 | ||||||
| Chaimayo 2020 [22] | Thailand | 454 (56.2, 58) | CC, SS, T and POS. | NPS, TS, EA | 13.2 | 10.4–35.0 | ICG | Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) |
| Diao 2020 † [23] | China | 251 (51.4, 40) | SS (n = 251) | NPS | 80.1 | ≤37.0–≤40.0 | FIA | In house |
| Fenollar 2020 [24] | France | 341 (NR, NR) | SS (n = 182) and CC (n = 159) | NPS | 59.8 | 9.0–34.0 | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) |
| Gremmels 2020 † [25] | The Netherlands | 1367 (61.7, 36) | GS (n = 1367) | NPS, TS | 10.2 | <32.0–≥32.0 | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, USA) |
| Aruba | 208 (NR, NR) | GS (n = 208) | 30.3 | |||||
| Gupta 2020 † [26] | India | 330 (30.0, 34) | SS (n = 204) and CC (n = 126) | NS, TS | 23.3 | 10.0–35.4 | ICG | Standard™ Q COVID-19 Ag kit (SD Biosensor, India) |
| Krüttgen 2020 [27] | Germany | 150 (NR, NR) # | C-19 (n = 75) and non-C-19 (n = 75) | NPS | 50.0 | <25.0–≥35.0 | ICG | SARS-CoV-2 Rapid Antigen Test (Roche, Switzerland) |
| Linares 2020 [28] | Spain | 255 (51.4, 46 *) | SS, CC and AS | NPS | 23.5 | <25.0–<40.0 | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) |
| Lindner 2020 † [29] | Germany | 287 (42.9, 35) | SS | NS ** | 13.5 | 17.3–≥35.5 | ICG | Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) |
| Liotti 2020 [30] | Italy | 359 (NR, NR) # | C-19 (n = 104) and non-C-19 (n = 255) | NPS | 29.0 | 15.3–39.7 | FIA | Standard™ F COVID-19 Ag (SD Biosensor, Korea) |
| Mak 2020a [31] | Hong Kong | 280 (NR, NR) # | C-19 (n = 280) | S, NPS, NPA, TS | 100.0 | <18.6–>28.7 | ICG | COVID-19 Ag Respi-Strip (Coris Bioconcept, Belgium) |
| NADAL COVID-19 Ag Test (Nal Von Minden, Germany) | ||||||||
| Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) | ||||||||
| Mak 2020b [32] | Hong Kong | 105 (NR, NR) # | C-19 (n = 105) | NPS, TS, S | 100.0 | <18.6–>28.7 | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) |
| Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) | ||||||||
| Mak 2020c [33] | Hong Kong | 160 (NR, NR) # | C-19 (n = 160) | S, NPS, TS, NPA, SP | 100.0 | <18.6–>28.7 | ICG | Biocredit COVID-19 Ag Detection Kit, (RapiGEN, Korea) |
| Nalumansi 2020 [34] | Uganda | 262 (10.7, 34) | SS (n = 136) and AS (n = 124) | NPS | 34.4 | <29–39 | ICG | Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) |
| Pilarowski 2020a [35] | USA | 878 (46.0, NR) | GS (n = 878) | NS | 3.0 | 28.8–30.3 | ICG | Abbott BinaxNOW™ COVID-19 Ag (Abbott Diagnostics, USA) |
| Pilarowski 2020b [36] | USA | 3302 (45.4, NR) | GS (n = 3302) | NS | 7.2 | <30.0–<35.0 | ICG | Abbott BinaxNOW™ COVID-19 Ag (Abbott Diagnostics, USA) |
| Porte 2020 † [37] | Chile | 127 (46.5, 38) | SS + T + CC (n = 127) | NPS, OP | 64.6 | 14.2–25.1 (IQR) | FIA | Bioeasy™ 2019-nCoV Ag RTK (Bioeasy Biotechnology, China) |
| Scohy 2020 [38] | Belgium | 148 (56.8, 58) | SS (n = 148) | NPS | 71.6 | 16.0–36.0 | ICG | COVID-19 Ag Respi-Strip (Coris Bioconcept, Belgium) |
| Strömer 2021 ‡ [39] | Germany | 134 (NR, NR) # | C-19 (n = 124) and non-C-19 (n = 10) | NPS | 92.5 | 17.0–37.0 | ICG | NADAL COVID-19 Ag Test (Nal Von Minden, Germany) |
| Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) | ||||||||
| Toptan 2021 [40] | Germany | 67 (NR, NR) # | C-19 (n = 58) and Non-C-19 (n = 9) | OP, NS | 86.6 | 18.7–40.0 | ICG | RIDA®QUICK SARS-CoV-2 Ag test (R-Biopharm, Germany) |
| 70 (NR, NR) | GS (n = 70) | NS | 45.7 | 18.0–35.9 | ICG | RIDA®QUICK SARS-CoV-2 Ag test (R-Biopharm, Germany) | ||
| Torress 2021 [41] | Spain | 634 (56.0, 37) | CC (n = 634) | NPS | 12.4 | ≤20.0–>35.0 | ICG | Panbio™ COVID-19 Ag-RTD (Abbott Diagnostics, Germany) |
| Turcato 2020 [42] | Italy | 3410 (NR, NR) | SS (n = 991) and AS (n = 2419) | NR | 6.5 | NR | ICG | Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) |
| Weitzel 2020 † [43] | Chile | 111 (55.0, 40) | SS (n = 111) | NPS | 72.1 | 10.7–37.7 | ICG | Biocredit COVID-19 Ag Detection Kit (RapiGEN, Korea) |
| ICG | StrongStep COVID-19 Antigen test (Liming Bioproducts, China) | |||||||
| FIA | Huaketai New Coronavirus (Savant Biotechnology, China) | |||||||
| FIA | Bioeasy™ 2019-nCoV Ag RTK (Bioeasy Biotechnology, China) | |||||||
| Yamayoshi 2020 [44] | Japan | 76 (NR, NR) # | C-19 (n = 76) | S, TS, NS, NPS, SP, EA | 100.0 | 18.8–36.0 | ICG | Standard™ Q COVID-19 Ag kit (SD Biosensor, Korea) |
| Espline® SARS-CoV-2 (Fujirebio, Japan) | ||||||||
| QuickNavi™-COVID19 Ag (Denka Seiken, Japan) | ||||||||
| ImmunoAce SARS-CoV-2 (Tauns Laboratories, Japan) |
RT-PCR: Reverse transcription polymerase chain reaction, NR: Not reported, ICG: Immunochromatographic assay, FIA: Fluorescence immunoassay, IQR: Interquartile range, C-19: Confirmed COVID-19 patient, SS: Symptomatic patient (suggestive of COVID-19), AS: Asymptomatic patient, CC: Asymptomatic contact with known COVID-19 or symptomatic patient, HCW: Healthcare workers, GS: General screening, T: Travelers, POS: Pre-operative screening, NPS: Nasopharyngeal swab, S: Saliva, NS: Nasal swab, TS: Throat swab, EA: Endotracheal aspirates, OP: Oro-pharyngeal swab, NPA: Nasopharyngeal aspirate, SP: Sputum, RTD: Rapid test device, NMT: Nasal mid-turbinate. # Archived samples of known RT-PCR result, † Reader-blinded study, ‡ With photo and intensity reader to support the reading, * Based on presumption mean calculation, ** Self-collected.
3.3. Quality Assessment and Publication Bias
The quality of each of the included studies was extensively examined using the JBI diagnostic accuracy checklist where the highest and lowest quality scores of the included studies were recorded as 88.8% (five studies) and 55.5% (two studies), respectively. Overall, there were no low-quality studies, 26.6% of high-quality, and 72.4% of moderate-quality studies (Table S2). There was no indication of substantial publication bias in the funnel plots evaluating the specificity and sensitivity of RAT kits to confirm SARS-CoV-2 (Figure 2).
Funnel plots representing no evidence of significant publication bias estimating (A) specificity and (B) sensitivity of rapid antigen tests in confirming COVID-19.
3.4. Meta-Analysis
The overall pooled specificity and sensitivity of RAT were 99.4% (95% CI: 99.1–99.8; I2 = 90%) and 68.4% (95% CI: 60.8–75.9; I2 = 98%), respectively (Figure 3). Four outlier studies [20,33,38,44] were identified using the Galbraith plot (Figure 4). Except for the pooled specificity of subgroups based on specimen types (i.e., nasopharyngeal swab: 71.0% (95% CI: 14.1–100.1) and saliva: 80.7% (95% CI: 41.8–100.0)), the specificity did not vary in most of the subgroups and ranged between 99 and 100% (Table 2 and Figure S1). On the other hand, in subgroup analyses, the sensitivity for each subgroup was lower than the specificity. Each of the subgroups estimating the sensitivity had high levels of heterogeneity except for the subgroups defined by the onset of symptoms (<5 and >5 days; I2 = 0%) and Ct values (Ct values ≤20 and 36–40; I2 = 0%). When compared with asymptomatic patients (54.5%), the sensitivity of RAT kits was higher in symptomatic patients (78.5%). The sensitivity of the nasopharyngeal swab was higher (70.1%) than that of saliva (50.4%) and throat swab or saliva (38.4%). The CT indicates the number of cycles needed for the fluorescence signals to reach the threshold. Ct value for a particular gene (i.e., N, ORF, E, S or M) is inversely correlated with viral load in a targeted specimen [45,46]. Surprisingly, the sensitivity of RAT kits was shown to have an inverse relationship with the Ct values, which also corresponded with the sensitivity of subgroups based on the symptom onset days (Table 2 and Figure S1). The sensitivity of RAT kits dropped to 15.1% and 16.5% when Ct values were 31–35 and 36–40, respectively. On the other hand, the RAT kits had a lower sensitivity when used in the African (56.4%) and Asian (65.0%) populations compared with the European (70.0%) and American populations (74.1%). Kits from different manufacturers exhibited various sensitivity. PanbioTM showed the highest sensitivity (75.1%) followed by Abbott BinaxNOW™ (74.8%), Standard™ (66.4%) and Biocredit (42.7%).
Forest plots representing estimating of (A) specificity and (B) sensitivity of rapid antigen tests in confirming COVID-19.
