Background
Health strategies for the management of COVID-19 pandemic rely on diagnostic tests. External Quality Assessment Programs provide an independent assessment of the effectiveness of analytical systems, improving the health strategy’s quality.
Aim
Here, we aimed to assess the diagnostic accuracy of molecular methods for SARS-CoV-2 by analyzing the results of an External Quality Assessment Program conducted by Controllab, accredited by ABNT NBR ISO/IEC 17043:2011, in partnership with the Brazilian Society of Clinical Pathology/Laboratory Medicine (SBPC/ML).
Methods
The quality control materials were inactivated lyophilized suspension of Vero cells (BCRJ 0245/ATCC CCL-81) infected with viable SARS-CoV2 particles from different strains, including the variants of concern, Gamma (Variant B.1.1.28.1 or P.1- Brazilian Variant -Manaus), BETA (Variant B. 1.351 – South African variant), Omicron (B.1.529 variant), and other strains. The strains were propagated under BSL-3 conditions and maintained in an atmosphere of 5% CO2 at 37ºC in Dulbecco’s Modified Eagle’s Medium, supplemented with 5% fetal bovine serum. The External Quality Assessment Program surveys were conducted from May 29, 2020, to November 1, 2022, and the accuracy of several molecular tests was assessed. The percentage of correct results, sensitivity, specificity, false positive, and false negative were calculated and analyzed according to the applied method (RT-PCR, RT-LAMP, and Nicking Enzyme Amplification Reaction – NEAR). The methods were also classified as laboratory-developed or in vitro diagnosis. Standard descriptive analyses were carried out. For statistical analysis, it was applied the independence chi-square test, and was used RStudio software to perform test.
Results
A total of 351 laboratories from 10 countries participated, and 5121 datasets were analyzed. RT-LAMP presented lower percentage of correct results, sensitivity, specificity, and higher percentage of false positive and false negative results than the other methods (P<0.05, for all comparisons) (Table 1). Laboratory-developed tests had lower percentage of correct results and specificity and higher percentage of false positive results compared to In vitro diagnosis test (P<0.05, for all comparisons) (Table 2). The External Quality Assessment Program also revealed that 90% of laboratories using RT-PCR, 100% using NEAR, and 50% using RT-LAMP presented ≥ 80% of correct results.
Conclusion
NEAR and RT-PCR showed similar diagnostic accuracy and both higher compared to RT-LAMP. In vitro diagnosis tests had higher diagnostic performance compared to laboratory-developed tests. The External Quality Assessment program revealed overall good performance for laboratories. We cannot exclude the impact of a small sample size on RT-LAMP results. Attention points and improvement opportunities go for those using RT-LAMP and laboratory-developed tests.
Disclosure
The authors confirm that they don’t have any conflict of interest to declare.
Bibliographic References
Oliveira CA, Mendes ME (orgs.). Gestão da fase analítica do laboratório: como assegurar a qualidade na prática. 1 ed., v. 2. Rio de Janeiro: Controllab: 2011, 184p.
ISO/IEC 17043:2010 – Conformity assessment – General requirements for proficiency testing
-Matheeussen V, Corman VM, Donoso Mantke O, McCulloch E, Lammens C, Goossens H, Niemeyer D, Wallace PS, Klapper P, Niesters HG, Drosten C, Ieven M; RECOVER project and collaborating networks. International external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020. Euro Surveill. 2020 Jul;25(27):2001223. doi: 10.2807/1560-7917.ES.2020.25.27.2001223. PMID: 32672149; PMCID: PMC7364759.
-Kaur H, Mukhopadhyay L, Gupta N, Aggarwal N, Sangal L, Potdar V, Inbanathan FY, Narayan J, Gupta S, Rana S, Vijay N, Singh H, Kaur J, Kumar V, Kaundal N, Abraham P, Ravi V. External quality assessment of COVID-19 real time reverse transcription PCR laboratories in India. PLoS One. 2022 Feb 8;17(2):e0263736. doi: 10.1371/journal.pone.0263736. PMID: 35134089; PMCID: PMC8824319.