Evaluating the performance of two different chemiluminescence analyzers systems in screening donors’ samples for HCV infection

Abstract

Background

The approved screening test for HCV infection among blood donors is the HCV antibody test. The diagnostic performances of the available immunoassays in the market that target different HCV antigens have not yet been thoroughly analyzed. Our study aimed to assess the diagnostic performances of two different chemiluminescence immunoassay (CLIA) assays.

Methods

We analyzed 1909 samples using two assay systems (COBAS e601 ROCHE® & Vitros 3600 ORTHO®) and evaluated the agreement of each with the gold standard method ELISA, as well as studying their diagnostic performances.

Results

The Cohen’s Kappa statistics revealed excellent agreement between ELISA and both CLIA methods performed on Cobas e 601 ROCHE and Vitrous 3600 ORTHO (0.81 & 0.994 respectively). The sensitivities and positive predictive values were 95.05% and 73.85% for Cobas e 601 ROCHE, 100.00%, and 98.97% for Vitrous 3600 ORTHO, respectively.

Conclusions

Excellent diagnostic performance was detected by both assays; however, Vitros 3600 ORTHO outperformed Cobas e 601 ROCHE in terms of sensitivity and specificity.

Introduction

In 1989, an enveloped positive-strand RNA virus Hepatitis C virus (HCV) was identified. It was classified in the Hepacivirus genus in the family Flaviviridae [1]. The HCV genome is divided into non-coding regions (NCR) and an open reading frame that encodes structural and non-structural proteins. The core protein and two envelope proteins (E1 and E2) are considered the structural proteins that form the viral particles. On the other hand, the non-structural proteins (NS) are required for viral genome replication (NS3, and NS5), as well as for the assembly of viral particles and release of infectious virions (NS2 and p7) [2].

HCV infection is a significant public health concern causing liver-related morbidity and mortality. Globally, the prevalence of infection is 0.8% in the general population with the highest one in the Eastern Mediterranean Region (1.6%) [3]. Egypt was one of the world countries with the highest prevalence of HCV infection. To achieve the World Health Organization (WHO) target aiming to eliminate viral hepatitis by 2030, Egypt made many efforts, and a large Egyptian study showed a marked decrease in mortality in Egypt [4].

Before 1992, HCV-infected blood supply was a major issue because of the lack of laboratory tests for its detection. The diagnostic screening blood tests include antibody detection tests (ELISA and chemiluminescence immunoassay), a confirmatory recombinant immunoblot assay (RIBA) and the most sensitive test for Ribonucleic acid (RNA) detection is polymerase chain reaction [5].

The gold standard serological test for HCV detection is ELISA which depends on the optical density principle. ELISA HCV antibody detection has gone through numerous stages, the first of which simply uses a recombinant peptide (c100-3) to detect antibodies against the NS4 region of the genome. This generation's sensitivity and specificity were relatively poor. Antigens from the HCV core region (c22-3) and NS3 (c33c) areas, as well as an antigen from the NS4 region, were integrated into the second generation. Including synthetic antigens as NS5 reinforced the third-generation sensitivity and specificity to be around 99.4 – 100%, as well as contributed to shortening the window period to 7–8 weeks [6]. ORTHO HCV Version 3.0 ELISA Test System (Ortho Diagnostic Systems, USA) and Murex anti-HCV 4.0 (Murex Diagnostics, UK) are examples of the third generation of enzyme immunoassays (EIA) commercial systems that detect anti-HCV antibodies against core, NS3, NS4, and NS5 recombinant antigens [7]. Finally, the fourth generation, known as the antigen–antibody combo assay, identified circulating antibodies to viral antigens and uses also synthetic antigens [8]. Despite the progress in ELISA, its inherent instability, lack of automation, and the Hook effect limited its use [9].

Consequently, RIBA was employed as a secondary confirmatory test if a first-line HCV screening test (ELISA) was positive or grey zone. This test was abandoned due to the development of CLIA technologies [10].

As regards chemiluminescence immunoassay (CLIA), a luminous molecule emits visible or near-visible (300–800 nm) radiation, acting as a signal of the analytical process. The CLIA enzymes are used to change a substrate into a reaction product which in turn emits photons. It is an epitope-specific antibody detection test [11]. CLIA analyzers are characterized by excellent precision, greater positive predictive value, high-speed throughput, random access, and technical simplicity. Furthermore, they have similar sensitivity and specificity to the third-generation EIA test [12].

Today, high-volume clinical laboratories benefit from automated CLIA analyzers and antibody screening has been implemented in the blood bank setting to further prevent HCV transmission by transfusion. This study was designed to compare the performance of two different anti-HCV automated CLIA analyzer systems (COBAS e601 ROCHE® and Vitros 3600 ORTHO®) in screening donors' samples for HCV infection.

Subjects and methods

Subjects

This cross-sectional study was carried out on 1909 blood donors, attending Ain Shams University Hospitals (ASUH) Central Blood Bank, during the period from February to August 2023 to evaluate the performance of two different CLIA analyzer systems in screening donor samples for hepatitis C virus infection (1012 samples by COBAS e601 ROCHE® and 897 samples by Vitros 3600 ORTHO®). All results were validated by retesting the samples with the ELISA technique (HCV Version 3.0 ORTHO®), which is considered the golden standard method [13].

All enrolled donors signed an informed consent. The study follows the declaration of Helsinki and the Scientific and Ethical Committee, Ain-Shams University approved it (FMASU R254/2022).

Inclusion criteria

Normal blood donors typically fit the donor selection criteria defined by the Egyptian national guidelines.

Exclusion criteria

Any samples showing abnormal sample coloration (e.g., icterus, lipemic, and hemolyzed samples).

Sampling

Donors' blood samples were collected in Ethylene Diamine Tetra Acetic Acid (EDTA) and plain tubes for serological testing according to the standard protocols of the central blood bank of ASUH. Samples tested by CLIA were performed on the same day of collection. On the other hand, ELISA samples were centrifuged, and the separated plasma was frozen at -80°c till use.

The minimum required sample volume for performing the test on COBAS e601 ROCHE® and Vitros 3600 ORTHO® are 50µl and 20µl respectively.

Serological tests

1012 samples were examined by Elecsys Anti-HCV II assay, Cobas e 601, Roche® Diagnostics, Mannheim, Germany. The Elecsys Anti‑HCV II assay uses peptides and recombinant antigens representing core, NS3, and NS4 proteins for the determination of anti‑HCV antibodies.

Eight hundred ninety-seven samples were tested by Vitros 3600 ORTHO® which detects antibodies against HCV structural and non-structural antigens (Core, E1, E2, NS3, NS4, and NS5).

All the samples were re-tested by the ELISA technique (HCV Version 3.0 ORTHO®).

The analysis was performed following the recommendations of the manufacturer. Moreover, positive and negative controls were applied before each run of CLIA and checked according to the Levey–Jennings rules. Also, in each ELISA run, controls were included and validated according to the manufacturer’s recommendations.

The result of a sample by CLIA was given in the form of a cutoff index (signal sample/cutoff). The test results were calculated as the cut-off of signal (S/Co) value obtained from the sample, and according to the manufacturer’s recommendation, results < 1 S/Co value were considered nonreactive, while samples with ≥ 1 S/Co value were considered reactive.

Table 1 summarises the technical differences between both CLIA machines.

Table 1 Technical parameters for Cobas e601 ROCHE and Vitros 3600 ORTHO

Statistical analysis

The collected data was revised, coded, tabulated, and introduced to a personal computer (PC) using the Statistical Package for Social Science (SPSS 25). Data was presented and suitable analysis was done according to the type of data obtained for each parameter.

Descriptive statistics

Frequency and percentage of categorical data.

Analytical statistics

Kappa statistics to compute the measure of agreement between two investigational methods. Kappa over 0.75 is excellent, 0.40 to 0.75 is fair to good, and below 0.40 is poor.

Sensitivity of Diagnostic measures = True positive by the test / (True positive by the test + False Negative by the test).

Specificity = True Negative by the test / (True Negative by the test + False positive by the test).

Positive predictive value = True positive by test / All positive by the test (True positive by the test + False Positive by the test).

Negative predictive value = True negative by test / All negative by the test (True negative by the test + False negative by the test).

Accuracy = (True negative + True positive by test) / Grand total.

The predictive value of continuous variables was evaluated using receiver-operating characteristic (ROC) curve analysis.

Results

The present study was conducted on 1909 donors. 1012 samples were examined by Elecsys Anti-HCV II assay, Cobas e 601 ROCHE, and 897 samples were tested by Vitros 3600 ORTHO®. Table 2 summarises the results of all samples.

Table 2 Results of analysis of anti-HCV samples performed by CLIA (Cobas e 601 ROCHE & Vitros 3600 ORTHO) and ELISA

Cohen’s Kappa statistics revealed excellent agreement between ELISA (the gold standard method) and both CLIA methods performed on Cobas e 601 ROCHE and Vitrous 3600 ORTHO (0.81 & 0.994 respectively) which means that both instruments’ results are reliable and consistent. Moreover, The Cobas e 601 ROCHE test had a low false positive rate (3.4%) and a low false negative rate (0.5%). On the other hand, the Vitros 3600 ORTHO test had a 0.1% false positive rate and 0% false negative rate which denoted its higher sensitivity and specificity (Tables 3 and 4).

Table 3 Contingency table of frequencies of anti-HCV determination using COBAS e601 ROCHE® and ELISA (Gold Standard Method)

Table 4 Contingency table of frequencies of anti-HCV determination using Vitrous 3600 ORTHO and ELISA (Gold Standard Method)

The five samples showing discrepant negative results by Cobas e 601 ROCHE were retested by a third CLIA (Alignity i Abott diagnostics). The results of the 5 samples revealed and confirmed the anti-HCV's positivity. These instances were also taken into consideration for an RNA-PCR analysis. Unfortunately, we were only able to contact 2 donors, and the HCV findings were negative.

To assess the performance of each instrument in detecting the disease, the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of each instrument were calculated via the application of the ROC curve. Vitros 3600 ORTHO had a higher sensitivity, specificity, and accuracy than Cobas e 601 ROCHE (100.00% versus 95.05%; 99.88% vs 96.27% and 99.89% vs 96.15% respectively). Furthermore, the log-rank test revealed a statistically significant difference between the two instruments’ performance (p < 0.001), concluding that Vitros 3600 ORTHO is a more reliable and accurate method for screening HCV antibodies than Cobas e 601 ROCHE (Table 5) (Fig. 1).

Table 5 Performance parameters for Cobas e601 ROCHE and Vitros 3600 ORTHO

Fig. 1

POC curve of HCV antibody expression level as a screening test detected by A Vitros 3600 ORTHO and B Cobas e601 ROCHe

Discussion

Hepatitis C virus (HCV) is a global health issue that is most typically found in developing countries. Accurate and early detection of HCV antibodies is the first step in the management of the infection as well as the identification of patients who need treatment. Access to affordable and simple diagnostic tools, especially in low and middle-income countries, is a major obstacle resulting in underdiagnosis and the inability to eradicate HCV worldwide [14].

Moreover, it is worth noting that the avoidance of false-negative results in screening a population of blood donors for HCV is mandatory, thus we should use the most sensitive test. On the other hand, false-positive results should be avoided when screening patients to be treated [12]. Thus, evaluating the diagnostic performance of any assays introduced in the market is highly recommended.

The current study aims to evaluate the detection of HCV antibodies by two chemiluminescence machines (Cobas e 601 and Vitros 3600 ORTHO®) in relation to the gold standard approach, the 3rd generation ELISA (HCV Version 3.0 ORTHO®).

The study detected that the seropositivity among studied subjects ranged between 10.81% and 12.85% upon testing by CLIA. These values decreased to 5.03% when ELISA was performed. The results of CLIA were near El-Ghitany and Farghaly's results which showed that anti-HCV seroprevalence was 14.8% [15].

As regards the agreement between the studied assays and ELISA, our study supports Majumder et al.'s conclusion that CLIA is equivalent to ELISA [13]. Also, it follows Kim and his colleagues, who showed good agreement for the identification of anti-HCV antibodies upon analysis by the Elecsys Anti-HCV Assay and the Vitros Anti-HCV Assay (with a range of 94.5% to 98.1) [16].

To assess the performance of each instrument to detect the disease, the ROC curve was applied to measure the sensitivity, specificity, positive predictive value (PPV) as well as negative predictive value (NPV). Vitros 3600 ORTHO had 100% sensitivity, 99.88% specificity, and 99.89% accuracy. Our results go with Park et al., who evaluated the performance of Vitros anti-HCV assay and found that its sensitivity and specificity were 99.5% and 99.5% respectively [17]. Also, Majumder et al., who evaluated the performance of Vitros 3600 ORTHO compared with the ELISA, stated the sensitivity and specificity as 96.77% and 96.07% respectively [13].

Upon studying the performance of Cobas, Gaballah, and Esawy demonstrated that sensitivity and specificity using Cobas e 411® Elecsys Anti-HCVII were 97% and 96% respectively, which was in concordance with our results (95.05% sensitivity, 96.27% specificity) [18]. On the contrary, Gupta et al. revealed a substantial agreement (kappa 0.66) between ELISA and COBAS e601 for HCV [19].

Finally, our study found that Vitros 3600 ORTHO outperformed Cobas e 601 ROCHE in terms of sensitivity and specificity (100.00% vs 95.05%; 99.88% vs 96.27%, respectively). This finding was in contradiction with Kim et al., who evaluated the performance of four CLIA machines, and found that the clinical specificity for Elecsys was higher than that of the Vitros assay (98.2% vs 96.5%). They stated that the addition of NS5 in the Vitros Anti-HCV assay could be the cause of false positive results and thus lower its specificity [16].

As regards the five false negative results obtained by Roche; these samples were re-evaluated by a third completely automated system (Alinity s), exposing and confirming the anti-HCV's positivity. But this can be explained by finding that HCr43 and c100-3 HCV antigens in the Architect assay (the same manufacturer for the Alinity s), are known to be prepared under contract agreement by Ortho Diagnostic Systems and the Chiron Corporation [20]. These instances were also taken into consideration for an RNA-PCR analysis and, unfortunately, we were only able to contact 2 donors, and the HCV findings were negative.

Conclusion

The cornerstone of any Blood Transfusion Service requires meticulous testing for transfusion-transmitted disease (TTD) markers in donated blood to provide safe blood. Nowadays, high-volume clinical laboratories use automated CLIA analyzers. Those types of equipment provide superb accuracy and dependability, rapid data throughput, random access, and the technical ease of complete automation. In this study, we compared two CLIA machines to the gold standard method, and we came to the conclusion that both machines statistically agreed with the results of the ELISA, however, the Vitros machine displayed better values in terms of sensitivity and specificity.