Diagnosis from an Epidemiological Point of View. The Example of COVID-19

A diagnostic test is anything that provides data about the patient's health or disease. All such tests can be formally evaluated for their accuracy and precision (sensitivity and specificity). Once a test has been performed, whether positive or negative, sensitivity and specificity are not the priority issues; the physician's dilemma is whether or not the patient has the disease, once the test result is known. This is the positive and negative predictive value. However, the positive predictive value depends on the prevalence: when prevalence is high, the positive predictive value of the test increases and, as a consequence, there are fewer false positives and more false negatives. When prevalence is low, the opposite occurs: the positive predictive value of the test decreases, and there will be more false positives and fewer false negatives. Furthermore, diagnoses are generally not determined and labeled in isolation; rather, diagnoses are applied sequentially (Bayesian inference). Thus, prior knowledge facilitates rapid decision-making that is generally correct by increasing the pre-test diagnostic probability. When tests are performed in series (one after the other), specificity and positive predictive value are maximized, but sensitivity and negative predictive value are reduced. Thus, we have the example of COVID-19. For a low prevalence scenario (such as the current one), assuming a specificity of 98%, a positive test ensures the diagnosis in at least 2/3 of patients. And a negative result practically rules it out. When the incidence of COVID-19 is high. For example, with a prevalence of 30% or in practice when we are in some "COVID-19 wave", a positive result ensures the diagnosis in 94% but a negative result can occur in up to 10% of patients. On the other hand, a second test after the first gives a more reliable result if its results go in the same direction.

Throughout the entire process leading to diagnosis, the physician uses various sources of information. These include the patient's anamnesis, physical examination, epidemiological information, and the results of the so-called diagnostic tests. A diagnostic test is anything that provides data about the patient's health or illness. Diagnostic tests include the usual laboratory blood and urine tests, as well as imaging techniques, physical examination findings, and a medical history such as diet, environment, or travel. All of these tests can be formally evaluated for their accuracy and precision (sensitivity and specificity).

Medicine is a statistical science since it always deals with the probability of becoming ill and with the uncertain timing of diagnosis and treatment. The physician must adapt scientific and technological knowledge to the personal and social clinical situation of the patient he or she is dealing with at that time. Good medical judgment achieves a balance between the risk involved in any diagnostic and/or therapeutic intervention and the expected benefit of said intervention in the specific patient, which requires combining science and art. "Diagnosis" is a term that refers to a "diagnostic-oriented procedure."

Most authors acknowledge that the presence of a disease in an individual often cannot be determined with certainty. Absolute certainty in diagnosis is unattainable, regardless of how much information is obtained, how many observations are made, or how many diagnostic tests are performed in the medical performance. Furthermore, the physician's goal is not to achieve certainty but to reduce the level of uncertainty sufficiently to make the therapeutic decision [1-3].

The evaluation of the performance of a diagnostic test begins with the quantification (estimation, rather) of the magnitude of the errors that can be made or, its inverse, the magnitude of the correct answers that are made when trying to "guess" a diagnosis from the results provided by said procedure. In 1947, the terms "sensitivity" and "specificity" were introduced, which are the traditional and basic measures of the diagnostic value of a test. They measure the diagnostic discrimination of a test in relation to a reference criterion, which is considered the truth [4].

Sensitivity is the probability that the test identifies someone as sick who actually is sick. Specificity is the probability that the test identifies someone as not sick who actually is not sick. Although sensitivity and specificity are considered the fundamental operational characteristics of a diagnostic test, in practice their capacity to quantify medical uncertainty is limited. The physician needs to evaluate the extent to which their results really modify the degree of knowledge that was obtained about the patient's condition. Specifically, he is interested in knowing the probability that an individual for whom a positive result has been obtained is actually sick; and conversely, knowing the probability that an individual with a negative result is actually free of the disease. The measures or indicators that answer these questions are known as predictive values [5]. So, to understand the results of diagnostic tests, it is necessary to determine the positive predictive value of the test, something that most medical professionals find difficult to do [6].

In short, it can be said that "Sensitivity", "specificity,", "true positive rate", and "true negative rate" are misleading terms. A high true positive rate-sensitivity-does not mean that the test is "sensitive" to the presence of the disease, since the rate may be high in the absence of the disease. For the diagnosis of any particular disease, the result of an arbitrarily chosen diagnosis is highly likely to be negative in the absence of that disease, so even an arbitrarily chosen diagnosis may have a high "specificity" for the particular disease.

The physician may focus on the "sensitivity" of, say, a particular question about hemoptysis, or an x-ray test in the diagnosis of lung cancer. By doing so, he or she may realize that the "true positive rate" for each of these diagnoses is highly dependent on how advanced the disease is, in terms of tumor size, for example. Thus, the physician might realize that "sensitivity," with its inherent lack of specificity for any particular subtype of disease, is a seriously misleading concept. And the same goes for "true negative rate" or "specificity" [7].

On the other hand, there are differences in the sensitivity and specificity of diagnostic tests in primary care compared to the hospital setting. In primary care, the negative predictive value – the probability that a certain disease is not the cause of the patient's problem – is more important than the positive predictive value, due to the low prevalence of disease compared to the hospital setting. The sensitivity of the test is the ability to detect all sick patients. Tests must have a high sensitivity to give few false negatives (so that no patient is missed). The specificity of the test is the ability to detect only sick patients. Once a test has been done, whether it was positive or negative, sensitivity and specificity are not priority issues; the doctor's dilemma is whether or not the patient has the disease, once the test result is known. This is the positive and negative predictive value. In diseases with a low probability (for example, a patient with chest pain presenting to the general practitioner for myocardial infarction) the sensitivity and specificity of the test can be confusing in practice, and it is more useful to calculate the positive predictive value of the test. Thus, if the ECG is normal, it does not exclude a heart attack, but its probability is greatly reduced [8].

The influence of prevalence should be discussed in the calculation of sensitivity and specificity. The sensitivities and specificities of diagnostic tests are not the same in all areas. The interpretation of the test changes depending on the context in which it is applied. Sensitivity and specificity are not an intrinsic feature of the test. This principle is not intuitively obvious to many physicians. So, positive predictive value (PPV) of a test = True positives / Test positives [i.e., PPV= True positives / True positives + False positives, or: PPV= sensitivity x prevalence / (sensitivity x prevalence) + (100- specificity) x (100-prevalence) [9].

This mathematical formula is calculated according to the theorem of Bayes. Prevalence of the disease or problem influences the result: when prevalence is high, the positive predictive value of the test increases, and as a consequence, there are few false positives and more false negatives. When the prevalence is low, the opposite occurs: the positive predictive value of the test decreases, and there will be more false positives and fewer false negatives [10].

Furthermore, diagnoses are not generally determined or labelled in isolation. Rather, diagnoses are applied sequentially (Bayesian inference); it is a contextual process and is interpreted as a whole. From the perspective of clinical activity, which is in principle more individualized, the prevalence of a disease corresponds to the estimation of the probability of suffering from the disease before performing the test. Here, the positive predictive value can be considered clinically as the probability of having the disease once a positive (or negative) result is obtained or a posteriori probability [5].

Prior knowledge facilitates rapid decision-making that is generally correct by increasing the pre-test diagnostic probability (application of Bayes' theorem in the sense of modifying probabilities by adding relevant information). That is, the accumulation of knowledge of patients, families, and communities allows for the rapid and low-cost assessment of diagnostic probabilities that are quickly accepted or rejected [11].

When tests are performed in parallel (at the same time) (because a rapid assessment is needed), sensitivity and negative predictive value (compared to each test individually) are increased; that is, it is less likely that the disease will not be found, but false positives and over-diagnosis are also more likely. When tests are performed in series (one after the other), specificity and positive predictive value are maximised, but sensitivity and negative predictive value are lowered (we may miss the disease). When multiple tests are performed, it is assumed that the additional information from each test is independent of that already available from the preceding test. The concept of predictive value rests on this assumption. However, it is unlikely that multiple tests for most diseases are truly independent of each other (this could happen when the manifestation of the disease changes over time). If this assumption that tests are completely independent is false, then the calculation of the probability of disease made from several tests tends to overestimate the value of the test. On the other hand, when the test result is measured as a scale and the 95% range of results is defined as “normal” (as is usual), the more tests are ordered, the greater the risk of false positive results. If enough tests are ordered, an abnormal result will appear in virtually all subjects [12].

We have the example of coronavirus disease 2019 (COVID-19). As clinicians care for patients with contact histories and symptoms that could represent COVID-19, interpretation of diagnostic test results is crucial. To accurately interpret test results, it is necessary to know the positive and negative predictive values of a test in the applied setting, which depend on its sensitivity and specificity, along with the pretest prevalence or probability. The performance of a test depends on two measures: sensitivity and specificity. Sensitive tests generate few false negatives and specific tests lead to few false positives [13].

The unattended Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) rapid antigen test has been reported to have a sensitivity of 70% to 81% during the period when the original Omicron emerged and became predominant, from which the BA.4 and BA.5 variants evolved. It is also reported that 20% to 30% of infections would be missed with a single test. In fact, current guidelines state that people who have symptoms of COVID-19 should take two or three antigen tests spaced 48 hours apart if the initial tests are negative. For a low prevalence scenario (such as the current one), assuming a specificity of 98%, a positive test ensures the diagnosis in at least 2/3 of patients. And a negative result practically rules it out. Exactly the opposite of what happens with high prevalence. That is: a positive test forces a PCR test, and a negative test practically rules out the disease. It is different when the incidence of COVID-19 is high. For example, with a prevalence of 30% or in practice when we are in some "COVID-19 wave", a positive result ensures the diagnosis in 94% but a negative result can occur in up to 10% of patients, requiring a polymerase chain reaction in the latter case [14-16]; and average specificities were high in symptomatic (99.6%) and asymptomatic (99.8%) participants. So a negative result practically rules out infection.

With a prevalence of 5% using data for the most sensitive assays in symptomatic people, positive predictive values (probability that a person is sick after a positive test) were 84% to 90% meaning that between 1 in 10 and 1 in 6 positive results will be a false positive, and between 1 in 4 and 1 in 8 cases will be missed. With a prevalence of 0.5% (low viral circulation) applying the same tests to asymptomatic people would result in a Positive Predictive Value of 11% to 28%, meaning that between 7 in 10 and 9 in 10 positive results will be false positives, and between 1 in 2 and 1 in 3 cases will be missed [17].

A Bayesian approach to illustrate the interpretation of COVID-19 negative tests based on the clinical suspicion of disease probability. A positive test in both high pre-test and low pre-test scenarios most likely represents acute infection. Likewise, a negative test in a low pre-test probability case indicates a low likelihood of acute infection. However, when COVID-19 infection is likely, such as in a healthcare worker with significant exposure, a negative test should not rule out acute infection. In this case, as recommended by the Centers for Disease Control and Prevention, repeat testing or further evaluation should be considered [18,19].

In summary, one important fact to remember: when prevalence is high, the positive predictive value of the test increases and, as a consequence, there are few false positives and more false negatives. When prevalence is low, the opposite occurs: the positive predictive value of the test decreases, and there will be more false positives and fewer false negatives. For example, in a population where the prevalence is 5%, a test with a sensitivity of 90% and a specificity of 95% will produce a positive predictive value of 49%. In other words, less than half of those who test positive will have antibodies. Alternatively, the same test in a population with an antibody prevalence greater than 52% will produce a positive predictive value greater than 95%, meaning that fewer than one in 20 people who test positive will have a false positive result. Even if a test were 98% sensitive and 99% specific, it would still produce a false negative result in 2 out of every 100 infected people. If we test 5 million Americans daily and only 1% of them have COVID-19, a total of 1000 positive cases will be missed, increasing the risk of spread, and another 49,500 people will receive false positive results. False positive results can be a burden on public health officials tasked with contact tracing and other public health activities, and many people may be quarantined unnecessarily. Understanding the positive and negative predictive value of a test should be factored into clinical decision-making and patient counseling [20]. Finally, on the other hand, a second test after the first gives a more reliable result if its results go in the same direction [21].

Medical practice always takes place in a framework of uncertainty. Doctors can never be absolutely sure of anything. What happens is that, in order to make our decisions, we assume certain probabilities of an event as high enough to justify our decision. We continually move between two probability thresholds: the diagnostic and the therapeutic [22].

In conclusion, to accurately interpret test results, it is necessary to know the positive and negative predictive values of this test in the applied setting, which depend on its sensitivity and specificity, together with the prevalence or pre-test probability. When prevalence is high, the positive predictive value of the test increases and, as a consequence, there are fewer false positives and more false negatives. When prevalence is low, the opposite occurs: the positive predictive value of the test decreases, and there will be more false positives and fewer false negatives. The natural statistical framework for evidence-based medicine is a Bayesian approach to decision-making. Greater prior knowledge (individual -of the patient-, or population -of the prevalence) before the test leads to more timely requesting of diagnostic tests and to a more appropriate assessment of their results.

1. Kassirer JP. Our stubborn quest for diagnostic certainty: a cause of excessive testing. N Engl J Med. 1989;320:1489-91. Available from: https://doi.org/10.1056/nejm198906013202211
2. Gaarder KR. Diagnosis. South Med J. 1989;82:1153-4. Available from: https://pubmed.ncbi.nlm.nih.gov/2772686/
3. Hui SL, Walter SD. Estimating the error rates of diagnostic tests. Biometrics. 1980;36:167-71. Available from: https://pubmed.ncbi.nlm.nih.gov/7370371/
4. Yerushalmy J. Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Pub Health Rep. 1947;62:1432-49. Available from: https://pubmed.ncbi.nlm.nih.gov/20340527/
5. Sergas. [Diagnostic tests]. Servicio Gallego de Salud. 2024. Available from: https://www.sergas.es/Saude-publica/Documents/1932/6-Ayuda%20Pruebas%20diagnsticas.pdf
6. Talboy AN, Schneider SL. Improving understanding of diagnostic test outcomes. Med Decis Making. 2018;38(5):573-583. Available from: https://doi.org/10.1177/0272989X18758293
7. Miettinen OS. The modern scientific physician: 4. The useful property of a diagnostic. CMAJ. 2001;165(7):910-1. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC81501/
8. Morrell D. Epidemiology in general practice. Oxford: Oxford Medical Publications; 1988;155. https://www.cabidigitallibrary.org/doi/full/10.5555/19892062330
9. Slater MR. Veterinary epidemiology. St. Louis, Missouri: Butterworth Heinemann; 2003.
10. Leeflang MM, Rutjes AW, Reitsma JB, Hooft L, Bossuyt PM. Variation of a test's sensitivity and specificity with disease prevalence. CMAJ. 2013;185(11): E537–E544. Available from: https://doi.org/10.1503%2Fcmaj.121286
11. Gérvas J, Pérez Fernández M. Adventures and misadventures of solitary sailors on the sea of uncertainty. Aten Primaria. 2005;35(2):95-8. Available from: https://doi.org/10.1157/13071916
12. Fletcher RH, Fletcher SW, Wagner EH. Clinical epidemiology: the essentials. Baltimore: Williams & Wilkins; 1982.
13. Turabian JL. Uncertainties in preliminary seroprevalence estimation of SARS-CoV-2: looking at the whole COVID-19 iceberg. Implications for general medicine. Epidemol Int J. 2020;4(S2):000S2-0. Available from: https://doi.org/10.23880/eij-16000S1-008
14. Schuit E, Venekamp RP, Hooft L, Veldhuijzen IK, van den Bijllaardt W, Pas SD, et al. Diagnostic accuracy of COVID-19 rapid antigen tests with unsupervised self-sampling in people with symptoms in the Omicron period: cross-sectional study. BMJ. 2022;378:e071215. Available from: https://www.bmj.com/content/378/bmj-2022-071215
15. Samsunder N, de Vos M, Ngcapu S, Giandhari J, Lewis L, Kharsany ABM, et al. Clinical evaluation of SARS-CoV-2 rapid antigen tests during the Omicron wave in South Africa. J Infect Dis. 2022; 226(8):1412-1417. Available from: https://doi.org/10.1093/infdis/jiac333
16. Dunn JJ, Brown CA, Jung J. Laboratory diagnostics and testing guidance for COVID-19. Medscape. 2020 Dec 1. Available from: https://emedicine.medscape.com/article/2500114-overview
17. Dinnes J, Deeks JJ, Berhane S, Taylor M, Adriano A, Davenport C, et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database of Syst Rev. 2021; 3(3):CD013705. Available from: https://pubmed.ncbi.nlm.nih.gov/33760236/
18. Good CB, Hernandez I, Smith K. Interpreting COVID-19 test results: a Bayesian approach. J Gen Intern Med. 2020;35(8):2490-1. Available from: https://doi.org/10.1007/s11606-020-05918-8
19. Centers for Disease Control and Prevention. COVID-19 fact sheet for healthcare providers. 2020 Mar 15.
20. Shuren J, Stenzel T. COVID-19 molecular diagnostic testing—lessons learned. N Engl J Med. 2020;383(17):e97. Available from: https://www.nejm.org/doi/full/10.1056/NEJMp2023830?query=TOC
21. Bravo-Grau S, Cruz JP. Diagnostic accuracy studies: tools for its interpretation. Rev Chil Radiol. 2015;21(4):158-64. Available from: https://scielo.conicyt.cl/pdf/rchradiol/v21n4/art07.pdf
22. Molina Arias M. Characteristics of diagnostic tests. Rev Pediatr Aten Primaria. 2013;15(58). Available from: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1139-76322013000200013