By MICHEL ACCAD
Last month marked the 400th anniversary of the birth of John Graunt, commonly regarded as the father of epidemiology. His major published work, Natural and Political Observations Made upon the Bills of Mortality, called attention to the death statistics published weekly in London beginning in the late 16th century. Graunt was skeptical of how causes of death were ascribed, especially in times of plagues. Evidently, 400 years of scientific advances have done little to lessen his doubts!
A few days ago, Fox News reported that Colorado governor Jared Polis had “pushed back against recent coronavirus death counts, including those conducted by the Centers for Disease Control and Prevention.” The Centennial State had previously reported a COVID death count of 1,150 but then revised that number down to 878. That is but one of many reports raising questions about what counts as a COVID case or a COVID death. Beyond the raw numbers, many controversies also rage about derivative statistics such as “case fatality rates” and “infection fatality rates,” not just among the general public but between academics as well.
Of course, a large part of the wrangling is due not only to our unfamiliarity with this new disease but also to profound disagreements about how epidemics should be confronted. I don’t want to get into the weeds of those disputes here. Instead, I’d like to call attention to another problem, namely, the somewhat confused way in which we think about medical diagnosis in general, not just COVID diagnoses.
The way I see it, there are two concepts at play in how physicians view diagnoses and think about them in relation to medical practice. These two concepts—one more in line with the traditional role of the physician, the other adapted to modern healthcare demands—are at odds with one another even though they both shape the cognitive framework of doctors.
The old view: diagnosis as shared conceptual model with therapeutic aims
If we reflect on what doctors traditionally do, we may see that a diagnosis is not so much an objective reality that a doctor uncovers in a patient as it is a kind of tacit convention between physicians about how to refer to an illness that shares common features among those that it affects.
For a new disease to be identified as such, at least a few people have to be affected with it and the pattern of affliction needs to be distinct enough for doctors to recognize it as different from the vast number of other ways people get sick. The illness is then given a name, but that name refers more to a shared conceptual model than to an entity that “resides” in the patient. In a certain sense, when we say “Imelda has COVID-19” we really mean “Dr. Welby has COVID-19 in mind when he thinks about Imelda’s illness.”
The conventional or shared understanding of particular diagnoses invariably changes over time: new knowledge, new testing capabilities, new treatments, new biological theories, and theories about health and disease inevitably modify the meaning of a given diagnosis, even if the specific term itself perdures. In a recent podcast episode, our guest Adam Rodman gave us a flavor of this in regards to “the flu.” Nosology—the classification of diseases—is quite messy.
Depending on how quickly a new disease emerges or how widespread it becomes, the diagnosis can take more or less time to mature in the consciousness of physicians. But here’s the rub: no two patients are sick in the same way. To steal an insight from Tolstoy, all healthy persons are alike, but each patient is a patient in his own way.
So, deciding whether a given patient meets the conventional understanding of a particular diagnosis is ultimately a matter of judgment. The expression “to make a diagnosis” is apt. A diagnosis is made, i.e., applied to a patient in real time, as appropriate to the circumstances.
Consider the following scenarios:
- Imelda is admitted to the hospital for a shortness of breath. She has ground glass opacities on lung CT and tests positive for SARS-CoV2. Does she have COVID-19?
- Imelda is admitted to the hospital for a stroke. She has no fever and her lungs are clear but she tests positive for SARS-CoV2. Does she have COVID-19?
- Imelda is in a nursing home minding her own business peacefully and she tests positive for SARS-CoV2 during a routine screen. Does she have COVID-19?
- Imelda is in hospice care for terminal breast cancer. She dies. A nasal swab is positive for SARS-CoV2. Did she die of COVID-19?
The answer to those questions depends on what we mean by “COVID-19” but, before we can even address that question, we also have to wonder what is a diagnosis for?
If I am Imelda’s physician, I make a diagnosis with the view to act on it in order to help her (as best as I can). The diagnosis is pretty much tied to the prognosis and treatment. It is meant to help me provide the best care to restore her health.
In principle, then, I can also suspend my judgment and withhold making a final diagnosis. If I don’t know what ails Imelda it may be best if I remain circumspect, at least temporarily. Or, if Imelda is without complaints and without signs of illness despite evidence of virus in her nasal passages, I may recommend isolating her from others but I need not feel compelled to render a particular diagnosis.
In other words, if a diagnosis is really shorthand for an idealization of a particular form of illness that is shared among physicians, the making of a diagnosis in a particular instance takes place with a therapeutic goal in mind. In this framework, to make a diagnosis is not so much a pronouncement that is true or false (who knows what Imelda really has?) as it is one that is practically wise or unwise (in the end, things turned out for Imelda as best as could be expected).
Understood properly, then, a diagnosis, is a shared model for “practical reasoning” in the art of medicine. Done well, practical reasoning manifests what Aristotle called practical wisdom, or phronesis, and what the late Herb Fred brilliantly captured as “Mutton’s Law.”
The new view: diagnosis as a formality to be discovered
The other, competing conception of the diagnostic process, has taken prominence with the advent of scientific medicine and with the increasing influence that medical research has had on medical education.
When medical scientists seek to study a disease, they also seek to reduce or eliminate any evaluative steps that are subject to human judgment which is considered a potential source of bias. For that reason, they have a tendency to formalize disease definitions in order to standardize clinical research. These definitions almost always take the form of “diagnostic criteria” that can be part of inclusion or exclusion rules in research protocols.
Over the last many decades, and under the influence of research-oriented faculty, medical education has largely incorporated the formalization of disease definitions into its curriculum. Nowadays, medical textbooks often follow the example of clinical research protocols and highlight criteria of symptoms and signs to define particular diseases. Professional societies which represent various medical specialties often do the same when they promulgate practice guidelines for physicians.
The most glaring example of that formalization is in psychiatry. The Diagnostic and Statistical Manual is precisely a compendium of criteria-defined diagnoses that now rules psychiatric practice. But physical illnesses are also increasingly treated in that way: Infective endocarditis, rheumatoid arthritis, arterial hypertension, Kawasaki disease, POEMS syndrome, and innumerable other illnesses have been codified according to formal diagnostic criteria.
The formal definition of disease entails a tendency to view the diagnosis as an objective reality residing in the patient: If a set of symptoms and signs defines a disease, and those symptoms and signs are in the patient, then the diagnosis itself must also be found in the patient. In turn, this entails viewing the diagnostic process as a task of detection or discovery: to determine whether “Imelda has COVID-19” or not, the physician must put that reality to the test. The diagnosis is “reached” once the criteria have been sufficiently satisfied.
The formal definition of disease also entails a greater separation between diagnosis and therapy. Under the traditional understanding of diagnosis, the diagnostic process is integrated to the therapeutic aim. The physician adjusts or calibrates his diagnostic pronouncements based on the therapeutic options available. With a formal definition of disease, on the other hand, the diagnostic process tends to be circumscribed and is guided by a set of cognitive tools (“decision analysis”) that can be kept apart from the therapeutic decision-making which enjoys its own distinct set of cognitive tools.
To be clear, I am not submitting that doctors are no longer applying diagnostic judgment in the traditional sense described above. No one has openly rejected the old concept. I am mainly proposing that these two distinct conceptions of diagnosis are nowadays conflated: the first one, rooted in the traditional therapeutic activity of the physician, is a conceptual model for practical reasoning which allows a greater degree of judgment and individualization, particularly as it keeps the diagnostic process in close alignment with the therapeutic possibilities. The other, derived from the emphasis on clinical research and scientific medicine, represents disease as a precise, objective reality to be discovered or detected.
As we shall soon see, only one or the other can be correct and, in my view, only the former, traditional concept is suitable for genuine medical care. But at play are also the demands of healthcare systems: Government oversight and insurance payment require that medical disease be formalized and made explicit. The criterion-based definitions of disease, then, serve the bureaucratic needs of the system which can exert its authority and financial power of persuasion on doctors to make them adopt—consciously or unconsciously; enthusiastically or less so—the process of diagnostic standardization that characterizes the scientific approach.
Testing 101 is 2×2
The shift in understanding regarding diagnosis is quite apparent in the current expectation that medical students and residents demonstrate facility with probability concepts and with “decision analysis.” Decision-analysis is a theory initially developed for quality control in manufacturing and engineering, but it also includes tools imported from radar signal-detection science, such as receiver-operating characteristics curves.
Terms such as sensitivity, specificity, and predictive value are now part of common medical jargon and clinicians are increasingly expected to be able to explain the meaning of those terms using a “2×2 table” such as the taken from a recent review:
I won’t get into a detailed explanation of what the 2×2 table purports to indicate (Dr. Koka covered that topic magisterially here), but I would like to draw attention to the fact that 2×2 tables require a “gold standard” in order to have any meaning. This is indicated at the top of the table above. A test result is a “true positive” if it is positive in those in whom we know the disease to be present.
But how do we know that the disease is present or not if a diagnosis is a shared conceptual model of practical reasoning that resides principally in the collective minds of a college of physicians? Do we conduct a poll? If a diagnosis is an objective reality to be discovered, who has access to that reality? What can serve as a gold standard?
In reality, no gold standard for disease can possibly exist. Test sensitivity and specificity can only refer to the detection of isolated patient characteristics. What tests do is identify an elevated chemical concentration, an abnormal cellular architecture, a mass, a density, or a germ that are not supposed to be there, etc. Tests cannot aim to detect a disease per se, even though textbook definitions for test sensitivity and specificity frequently imply or even state that it is the disease itself that is sought by the test. For example, in its entry titled “Evaluating Diagnostic Tests,” the hugely popular and “evidence-based” online clinical encyclopedia UpToDate, edited and curated by well-established academic physicians, states that “Sensitivity is the probability that an individual with the disease will test positive (emphasis mine),” and the mirror opposite for specificity.
To be fair, other sources that define concepts of sensitivity and specificity may be more careful to say that what is being “diagnosed” by a test is a “sign,” a “condition,” or a “disorder.” But what that shows is that the term “diagnosis” has been watered down to refer to any kind of abnormality. It’s no wonder that loose thinking about diagnosis has led to an epidemic of “overdiagnosis.” We have let CT scans “diagnose” pulmonary embolism, biopsies “diagnose” thyroid cancers, angiograms “diagnose” coronary disease when, in truth, diagnoses should only be made by physicians, be about individual patients, and take into account the totality of the available evidence, including the personal life circumstances of the affected man or woman. The cure for overdiagnosis is not to restrict the use of tests but to focus the attention of physicians on a more correct view of what it means to diagnose.
What’s a COVID-19 diagnosis?
And now we have COVID-19.
Unsurprisingly, the “COVID-19 tests” available to us do not test for the actual disease but for fragments of virus (in the case of RT-PCR tests) or for recent SARS-CoV2 positivity on RT-PCR testing (in the case of antibody tests). This also means that, at this stage in the game, estimating the incidence or prevalence of COVID-19 is quite a crude exercise. Not only do the COVID tests not test for the disease, but the collective experience of physicians is still in its infancy. It is still too early to expect a mature “shared model” of practical reasoning.
In a few months or perhaps sooner, I imagine that diagnostic criteria will be proposed for COVID-19, perhaps with some kind of scoring system among “major” and “minor” criteria to render a verdict of COVID “definitive” or “probable” or “excluded” in the manner of the criteria for, say, infective endocarditis. But formal diagnostic criteria will not help objectify the disease and cannot serve as a gold standard either. In fact, diagnostic criteria themselves can be evaluated according to 2×2 and ROC analysis: Do the criteria capture all cases of COVID-19? Do they miss any? If so, how many and according to what other gold standard? Ultimately, there is no escape from the judgment of clinicians.
To be clear, the widespread suspicion regarding COVID mortality statistics is not directed principally at physicians but mostly at public health officials who apply their own concepts of diagnosis to serve their policy needs. Still, I don’t think that many doctors, let alone the public, recognize that when a physician fills out a death certificate he or she is not acting in a true medical capacity: there is no patient interest to serve. A cause of death is not, strictly-speaking, a medical diagnosis.
That only confirms the importance of carefully understanding what a diagnosis means and of safeguarding that meaning for the good of the profession and of society. Contrary to popular academic belief, the shift from practical reasoning to probabilistic thinking is not serving us well.
What do we call a medical education that neglects forming wise physicians in favor of training standardized ones for the sake of clinical research and for assembly-line healthcare practice? At this point, I’ll withhold making a diagnosis.
Michel Accad is a cardiologist based in San Francisco and host of the podcast, The Accad & Koka Report. This post originally appeared on his blog, Alert and Oriented here.
