COMMUNICATION FROM THE COMMISSION

Guidelines on COVID-19 in vitro diagnostic tests and their performance

(2020/C 122 I/01)

1.   Objective and Scope

Testing for the presence of or past exposure to the SARS-CoV-2 virus is an essential aspect of combatting the COVID-19 outbreak and the associated public health crisis. Wider testing is urgently needed for successful pandemic control. For effective management of the different stages of the pandemic it is vital to understand first what information different tests can deliver, i.e. what is the intended purpose of a given test, and second the level of a test’s performance, i.e. how well it is able to achieve that purpose.

This document provides guidance on these aspects. It outlines the regulatory context of COVID-19-related in vitro diagnostic testing devices in the EU and gives an overview of different types of tests and their purposes. It includes considerations on device performance and validating that performance. It provides elements to be considered by Member States in defining national strategies, and by economic operators in placing devices on the market, with the objective of ensuring that safe and effective devices for COVID-19-related testing are available in the EU. This is particularly important in the context of assessing the effect of public health countermeasures and designing safe de-escalation strategies.

2.   The EU regulatory context of COVID-19 test devices

Directive 98/79/EC on in vitro diagnostic medical devices (IVD) (1) currently applies to COVID-19 tests. In order to place these tests on the EU market, the manufacturer has to comply with the relevant provisions of the Directive. In particular, the manufacturer must draw up a technical file which explicitly shows that the test is safe and performs as intended, by demonstrating compliance with the requirements laid down in Annex I of the Directive. As explained in point 3 of these guidelines, tests can be intended by the manufacturer for use by health professionals or by lay users (self-tests). For COVID-19 tests intended for use by health professionals, the CE-mark may be affixed following a declaration by the manufacturer that the requirements of the Directive are satisfied (declaration of conformity). Devices intended for self-testing require the involvement of a notified body which must carry out additional verification of the technical documentation.

Exceptionally, in the interest of protection of health, the Directive states that a Member State may, in response to a duly justified request, authorise the placing on the market within its territory of individual devices for which the applicable conformity assessment procedures have not been carried out yet (e.g. pending the completion of the device’s evaluation). In adopting such national derogations, the national competent authority of the Member State must carefully consider any risk against the benefit of having the device available for immediate use. The national processes for adopting these derogations vary across Member States.

The Directive exempts devices that are manufactured and used within the same health institution (so-called in-house devices) from the abovementioned requirements. Such practices are generally covered by national legislation, which can vary significantly between Member States.

From 26 May 2022, the Directive will be replaced by Regulation (EU) 2017/746 on in vitro diagnostic medical devices (2). However, the Regulation already provides for a transitional period starting on the date of its entry into force (May 2017) during which the conformity of in vitro diagnostic medical devices can be assessed either under the Regulation or under the Directive.

3.   Overview of tests and their purposes

Tests can be grouped by scientific rationale, type of technology, intended user and location of testing. Correct understanding of the interplay of the related definitions is essential for appropriate use of the devices.

In terms of the scientific rationale, the COVID-19 tests available today fall broadly into two categories: those detecting the SARS-CoV-2 virus and those detecting past exposure to the virus (more specifically the immune response of the human body to the infection).

For the first category, there are two subtypes: on the one hand those detecting the virus genetic material (by reverse transcription polymerase chain reaction, or RT-PCR), and on the other hand those detecting components of the virus such as proteins on its surface (antigen tests). These tests are typically performed on nasal or throat secretions (i.e. swabs or washes). RT-PCR tests are those currently recommended by the World Health Organisation (WHO) (3) and the European Centre for Disease Prevention and Control (ECDC) (4) for diagnosis of COVID-19. Antigen tests could also in principle be used for diagnosis but not many of those have so far been developed.

Tests detecting exposure are typically those that detect antibodies in the blood which are produced by the patient’s body in response to the infection with the virus. They are also termed serological tests as they are typically performed on blood serum. Anti-SARS-CoV-2 virus antibody tests are abundant on the market. The effectiveness of antibody tests in early COVID-19 diagnosis is very limited because antibodies become detectable in the patient’s blood only several days after infection. This depends on the one hand on the individual’s immune system and on the other hand on the sensitivity of the technique employed (5). In addition, antibodies persist for some time after the infection has cleared. They do not give a definite answer on the presence or absence of the SARS-CoV-2 virus and thus they are not suitable to assess if the tested individual may be contagious for others. Nevertheless, antibody tests could prove essential for performing large-scale sero-epidemiological population surveys for assessing e.g. the immune status of workers and as one of the elements for guiding de-escalation strategies when the pandemic is under control. (4)

In terms of type of technology, two categories of commercial CE-marked tests can be distinguished: tests automated for use on analyser machines, and rapid tests, defined as qualitative or semi-quantitative devices, used singly or in a small series, which involve non-automated procedures and have been designed to give a fast result (6). To note, automated tests can also be fast and designed in portable equipment form, but they do not fall under the above definition of rapid tests. Commercial RT-PCR tests are generally non-rapid automated tests, with some portable devices becoming available. Antigen tests are found in rapid test form (some of them include readers to help interpret the result). Antibody tests exist both as automated and as rapid tests.

As of early April 2020, the following approximate numbers of COVID-19 devices have been CE-marked under Directive 98/79/EC (7): 78 for RT-PCR tests, 13 rapid antigen tests, 101 antibody tests, most of them rapid. It should be noted that the availability varies widely between Member States. This is because the manufacturer may destine them for non-EU markets or there may not be distributors selling these devices in all Member States. Availability also varies over time depending on, for example, logistic factors in manufacturing and distribution.

In terms of the intended user, tests can be designed by the manufacturer to be used by health professionals or by lay users (self-tests).

Finally, in terms of location of testing, devices can be either laboratory-based or near-patient, also termed point-of-care, i.e. performed near a patient and outside of laboratory testing facilities (8). In the EU, near-patient tests are intended to be used only by a health professional. The term near-patient test is not used for devices intended for lay users, which must be classified as self-tests.

It should be noted that, to inform national strategies, it is especially important to contextualise each type of test for its intended purpose, for example diagnosis or screening. This includes considering the specific target population (e.g. health professionals, general public), the phase of the disease they are meant to be used for (e.g. asymptomatic/healthy subjects, confirmed cases, discharge after hospital admissions etc.) and the clinical decision taken based on the test’s results.

4.   Considerations on test performance

Directive 98/79/EC lays down that devices must be designed and manufactured in such a way that they are suitable for the intended purpose specified by the manufacturer, taking account of the generally acknowledged state of the art. They must achieve the relevant performance, in particular in terms of analytical sensitivity, diagnostic sensitivity, analytical specificity, diagnostic specificity, accuracy, repeatability, reproducibility, including control of known relevant interference, and limits of detection, stated by the manufacturer (9).

The intended purpose must be specified in the instructions for use and/or on the label, unless it is obvious to the user (10). This should be complete and precise, including the intended user and clinical aspects such as the target population. The instructions for use must also contain the levels of performance for the parameters above (11). The technical documentation of the device must contain adequate performance evaluation data showing the performances claimed by the manufacturer and supported by a reference measurement system (when available), with information on the reference methods, the reference materials, the known reference values, the accuracy and measurement units used. The data should originate from studies in a clinical or other appropriate environment or result from relevant references (12). The information on establishment of performance should be complete to allow an assessment of its quality. For example, manufacturers should clearly identify comparator methods, how many subjects were part of the performance study, how those subjects were qualified as positive or negative.

The manufacturer must confirm that the device has been manufactured taking into account the “state of the art” in terms of the performance parameters listed in the first paragraph of this section. “State of the art” does not mean that the device has to be the best in its class. However, the device may not fall behind what can reasonably be achieved and is achieved by a majority of devices (13). It is clear that in the case of COVID-19 the state of the art is evolving rapidly. Nevertheless, manufacturers should strive to apply this approach as far as reasonably possible.

There can be trade-offs between parameters, for example between the sensitivity of the test (detecting the maximum number of positive individuals) and its specificity (ability to distinguish between true and false positives). A test that is very sensitive at detecting the target of interest is more likely to also detect related but distinct targets that are not of interest, i.e. it may be less specific. As another example a low limit of detection may result in a lower reproducibility of the test result. Various choices of parameter combinations can be justified, subject to the purpose of the device: fast-track screening, diagnosis, confirmation etc. For example, when a person is examined for the first time, this should be done with a test that is very precise and that has a low level of false positives, and certainly a very low level of false negatives. If the person tests positive and is examined a few days later, the test may have some tolerance for false positives (since the individual is most likely still positive), but not for false negatives (as this would lead to wrong conclusions). Also, if a test is easy to use and cheap but has a relatively low specificity, this can be overcome by repeating the test two or even three times.

Manufacturers are obliged to explain their choices of performance levels in the instructions for use and to determine the specific purpose in accordance with the choices made. They are also obliged to identify the target population.

Regarding performance of tests in the context of population testing, there are drawbacks both from insufficient diagnostic sensitivity (e.g. leading to missing infected individuals) and insufficient diagnostic specificity (e.g. imposing confinement measures on individuals who are not true positives). This needs to be taken into account along with the stage of the pandemic in a particular population. For example, in the control stage it may be particularly important to identify positive cases with a high level of specificity (i.e. distinguishing COVID-19 from other similar but less dangerous diseases) to avoid unnecessary burden on the healthcare system. In contrast, in the de-escalation stage, sensitivity (detecting all remaining infected individuals) could be more important than specificity to make sure the disease is indeed contained. It is also important to take account of the features of the population in which the test is intended to be used, for example whether the prevalence of infection is expected to be low or high, or whether there are local virus variants.

The Commission, Member States and stakeholders should consider what are the critical aspects of device performance specific to COVID-19 on which a common approach should be taken.

For example, for RT-PCR tests, this could be the identification of stable target sequences (i.e. genetic fragments characteristic of SARS-CoV-2 virus to be detected). With every new patient infected, the virus may change (mutate) and these mutations in turn may render a particular test less effective, or even ineffective. It is therefore important that the mutation profile of the virus is monitored and that on that basis a particular RT-PCR approach is used. For rapid antigen tests, it is important to choose RT-PCR as the comparator method; comparison with e.g. another rapid antigen test is not appropriate, as this approach would result in progressive degradation of references. For antibody tests, if a test specifies that it can be used on blood, serum and plasma samples, evidence should be provided that the test performs as intended on all those types of sample.

Nevertheless, it is important to keep in mind that the field is evolving rapidly and the body of literature is developing. Only the approaches for which a sufficient scientific basis has been established should be considered as best practice.

Finally, good performance of self-tests is particularly important as those are destined to the lay user. A notified body will assess the technical documentation of such tests including design and usability and will issue a certificate (14). At this stage, Member State competent authorities are generally not in favour of the use of COVID-19 self-tests in the current context of the pandemic and some have even prohibited the use of certain types of self-test (15). A reason for this is the possible difficulty for the untrained lay user to make the correct interpretation of the result and its implications (e.g. that the result could be a false positive or false negative which a health professional could identify by taking into account the clinical context of the patient). It should be noted that making devices destined for professional use available to lay users, e.g. via pharmacies or on the web, is not legally permitted.

5.   Validation of test performance

As explained above, the manufacturer evaluates the performance of the device in accordance with the intended purpose before placing the device on the market.

However, especially given the rapid development of the pandemic, the performance of the device may vary in practice in comparison to the performance study the manufacturer has done for the purposes of CE-marking. Therefore, it is highly recommended to carry out additional validation of the clinical performance of tests for COVID-19 by comparison with a reference method in a sufficiently large number of target population subjects before introducing the devices into the clinical routine. Scientific peer-reviewed results for the clinical validation of commercial COVID-19 tests are highly recommended before they can be safely and reliably used for medical or public health decision making. Validation refers to confirmation that the test achieves the performance levels specified by the manufacturer.

Such studies are being done by competent authorities and reference laboratories in Member States. There are obvious benefits of sharing the results of those validations and organising centralised validation studies to make the most efficient use of resources. Fast-track clinical validation studies of rapid diagnostic tests for COVID-19 by hospital laboratories are ongoing in several EU Member States. Cooperation would also be beneficial at international level where the same tests are used in different jurisdictions. Both the WHO and FIND are currently working on validation studies of different devices. (16)

Scarcity of reference methods and materials poses difficulties for these validation studies, and also for the evaluation of device performance by manufacturers. The Commission’s Joint Research Centre has recently developed a positive control material for RT-PCR tests which is available to laboratories in Europe. Seroconversion panels and positive sample panels are examples of further materials that are needed. Another issue currently is the lack of publicly available comparator data, which makes it difficult to compare the performance of devices. External quality assessment schemes could be one way to generate such data. The ECDC (4) and the WHO (17) are already in process of organising an external quality assessment scheme for RT-PCR tests.

6.   Actions undertaken by Commission so far

The Commission has already taken the following steps as regards market access and performance of devices:

—

Facilitating continuous exchange of information between competent authorities for in vitro diagnostics in the framework of the dedicated subgroup of the Medical Device Coordination Group (18). This includes regulatory exchanges on device conformity, availability and reliability, maintaining an inventory of devices and sharing information on national actions, covering also national derogations issued by Member States and justifications for them.

—	Maintaining a dialogue with industry, including topics such as device availability and performance.

—	Regular exchanges with WHO regarding COVID-19-related actions.

—	Establishing cooperation channels with other jurisdictions regarding counterfeit devices.

—	Publication of guidance on medical device conformity assessment (19) and an overview of international recognition of standards (20), which also include standards relevant to in vitro diagnostic devices.

—

Production of a working document containing a literature review on device performance and initial guidance on performance criteria for COVID-19 devices, including RT-PCR tests, antigen and antibody tests, in the context of a project group including the Commission, the ECDC and representatives of experts from in vitro diagnostics competent authorities and health technology assessment bodies.

—	Development of a positive control material which can be used for quality assessment of RT-PCR tests (21) and its distribution to laboratories across the EU.

—

Supporting research and innovation and coordinating European and global research efforts. Several special research and innovation actions have been launched in 2020, addressing, among other topics, the development of COVID-19 diagnostics, treatments and vaccines, as well as the infrastructures and resources that enable this research. Three new projects address this area (22) and several others are expected.

7.   Further actions needed

In order to make sure that tests are used appropriately, that device performance is of highest reasonably possible level, and that approaches to evaluation and validation of device performance are further aligned across the Union, the following actions should be taken:

—

The Commission will facilitate the discussion of national testing strategies in the coming weeks to foster a common EU approach, as far as this is appropriate. National strategies should take account of the intended purpose of each type of test and the importance of the use of the device in a specific context, in particular the specific population groups and the phase of the disease in which the test is supposed to be used. Critical performance parameters such as the diagnostic sensitivity, diagnostic specificity and the limit of detection of the device for the given context and purpose should be considered.

—

The Commission, supported by the ECDC, health technology assessment experts and in vitro diagnostics competent authorities, will assist Member States with a centralised overview of available information on test performance and act as a single point of contact for management of this information. Taking stock of the state of the art on a regular basis will support Member States’ informed decisions on national testing strategies, as well as support the continuous development of devices by manufacturers. Further work will be carried out to collect and analyse evidence on the context(s) in which these tests are used to provide valuable information to inform Member States’ decisions on testing strategy.

—

The Commission, Member States, and stakeholders, with consultation of the ECDC, will discuss best practices for performance evaluation of different types of COVID-19 test in the context of conformity assessment in the coming month and review this topic on a regular basis. If appropriate, the Medical Device Coordination Group will issue corresponding guidance.

—

The Commission in cooperation with Member States will facilitate the placing on the market of safe and reliable test devices. The Commission will discuss with industry and competent authorities what further guidance on conformity assessment is needed. The Commission will assist Member States in their market surveillance activities by putting in place arrangements to share and access information efficiently.

—

A number of counterfeit devices have been identified by national competent authorities that have been placed on the market illegally, for example with falsified proof of national registration, falsified notified body certificate or missing regulatory documentation. Member States have taken actions to remove them from the market. The Commission will continue facilitating maximum collaboration of regulators, also at international level, to detect and remove such devices from the market. It is recommended that the competent authorities cooperate also with importers and in particular distributors, both of whom can help to identify trade with counterfeit devices.

—

The Commission, in full cooperation with Member States and in consultation with the ECDC, will establish a network of COVID-19 reference laboratories across the Union, together with a platform to support them. Activities will include facilitating exchange of information, identification of laboratories’ needs, management and distribution of control samples, external quality assessment, method development, organisation and follow-up of comparative testing, and exchange of knowledge and skills. The Commission will coordinate the work of the network and will provide the necessary analytical support to the Member States in the management and mitigation of the crisis.

—

The Commission, together with Member States, will put efforts into the development of tools to enable evaluation of device performance and align approaches across the Union, such as reference materials and methods for standardised comparison. This will require close cooperation between regulators, health technology assessment bodies (23), the ECDC, the COVID-19 reference laboratory network, research organisations and industry to ensure the most optimal outcome. The Commission will consider which funding opportunities will provide support for these activities.

—

To make sure that sufficient numbers of well-performing tests and related reagents are available where they are needed, the industry and the Member States should make use of instruments at Union level to coordinate supply and demand, such as the clearing house (24), rescEU and joint procurement. Production of sufficient volumes of tests will require enhanced cooperation between companies, which should be done in accordance with the Temporary Framework for assessing antitrust issues related to business cooperation (25).

—

Member States should show solidarity in making arrangements for the fair distribution of available stocks and laboratory equipment to where they are most needed. Coordination of national strategies will be indispensable for this and further guidance should be provided in the context of the exit strategy currently in development by the Commission and Member States.

---

(1)  OJ L 331, 7.12.1998, p. 1.

(2)  OJ L 117, 5.5.2017, p. 176.

(3)  https://www.who.int/publications-detail/laboratory-testing-for-2019-novel-coronavirus-in-suspected-human-cases-20200117

(4)  Rapid risk assessment: Coronavirus disease 2019 (COVID-19) pandemic: increased transmission in the EU/EEA and the UK – eighth update https://www.ecdc.europa.eu/en/publications-data/rapid-risk-assessment-coronavirus-disease-2019-covid-19-pandemic-eighth-update

(5)  For example colour detection versus fluorescence detection – the latter technique is more sensitive.

(6)  Commission Decision 2002/364/EC on common specifications for in vitro diagnostic devices (OJ L 131, 16.5.2002, p. 17).

(7)  From working document of the Commission services produced in the context of a project group including the Commission, the ECDC and representatives of experts from in vitro diagnostics competent authorities and health technology assessment bodies. The information is limited to that publicly available by 6 April 2020 and is not further verified. Document to be published on the Commission website.

(8)  http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-181031-grrp-essential-principles-n47.pdf

(9)  Directive 98/79/EC Annex I A(3)

(10)  Directive 98/79/EC Annex I B 8.5

(11)  Directive 98/79/EC Annex I B 8.7 (d)

(12)  Directive 98/79/EC Annex III (3)

(13)  For reference see Commission Decision 2008/932/EC of 2 December 2008, C(2008)7378, OJ L 333, 11.12.2008 p. 5, on a certain HIV test of the manufacturer M.B.S. In the case in question, the combined data of various national institutes showed a picture according to which all HIV tests could be grouped into three tiers, grossly corresponding to three generations of tests. In that case, the HIV test in question was deemed not to correspond to the “state of the art” because it fell in the lowest performing tier.

(14)  Directive 98/79/EC Annex III(6)

(15)  Belgium has prohibited the making available on the market, putting into service and use of rapid antibody self-tests for 6 months as of 19 March 2020: http://www.ejustice.just.fgov.be/eli/arrete/2020/03/17/2020040686/moniteur Finland, Sweden, Ireland and Germany have made public statements warning against self-tests. The Netherlands published a warning that rapid tests should not be made available as self-tests. Estonia made a public warning not to use tests intended for professional use as self-tests. This is not an exhaustive list.

(16)  Note: The WHO’s emergency use assessment and listing programme (see https://www.who.int/diagnostics_laboratory/EUL/en/) has received 30 applications and finalised listings have been published for three RT-PCR devices to date. For COVID-19, this programme does not include laboratory-based validation.

(17)  https://www.who.int/ihr/training/laboratory_quality/10_b_eqa_contents.pdf

(18)  Set up according to Art. 103 of Regulation (EU) 2017/745 and Art. 98 of Regulation (EU) 2017/746. This group is also responsible for overseeing the implementation of Directive 98/79/EC.

(19)  https://ec.europa.eu/docsroom/documents/40607

(20)  https://ec.europa.eu/docsroom/documents/40606

(21)  https://ec.europa.eu/jrc/en/news/new-control-material-developed-jrc-scientists-help-prevent-coronavirus-test-failures

(22)  Projects CoNVat, CoronaDX and HG nCoV19 test https://ec.europa.eu/info/files/new-research-actions-coronavirus_en

(23)  https://eunethta.eu/

(24)  The Commission has set up a ‘Clearing house for medical equipment’ that facilitates the identification of available supplies, including testing kits, and their matching with demand by the Member States. This also entails collaboration with industry on increasing production by existing manufacturers, as well as facilitating imports and activating alternative ways of producing equipment.

(25)  Communication from the Commission – Temporary Framework for assessing antitrust issues related to business cooperation in response to situations of urgency stemming from the current COVID-19 outbreak C(2020) 3200 (OJ C 116 I, 8.4.2020, p. 7).