Sufficient Scruples

Bioethics, healthcare policy, and related issues.

August 8, 2006

Cutting Off the Long Tail

by @ 4:09 PM. Filed under Biotechnology, General, Global/Community Health, Medical Science, Provider Roles, Research Issues, Theory

DB, of the eponymous “Medical Rants”, makes a good point about rare diseases:

The problem with Lemierre’s Disease is that it represents a “long tail” disease. Most sore throats are viral or due to streptococcal disease. At least we thought that until recently. Evidence from 2005 in two articles suggests that the organism thought responsible for most Lemierre’s Disease – Fusobacterium necrophorum – may cause as much as 10% of pharyngitis. . . .

For the past 30 years, the infectious disease community has worked to decrease the use of unnecessary antibiotics. They have assumed that group A beta hemolytic streptococcal infection is the only pharyngitis cause which needs “necessary antibiotics”. They have assumed that group C and group G streptococci do not need antibiotics. They have excluded the possibility of unknown bacterial infections. Now it appears that Fusobacterium necrophorum may indeed be an “unknown bacterial cause” of pharyngitis.

What can we do about the Long Tail?

The phrase “Long Tail” arises from ideas introduced by Clay Shirky and Chris Anderson,  referring to the statistical fact that the asymptotic “tail” of low-frequency events on the outer edge(s) of a standard statistical distribution can often sum to a greater percentage of all events than the seemingly huge bump of high-frequency events near the origin of the curve. That means that most of the impact of the phenomenon in question is accounted for by events that are rarely seen (e.g., Amazon sells more books all together in the form of obscure volumes that sell one or two copies a day than it does of the relative few massive bestsellers that move thousands of copies a day; in medical terms, a great deal of illness – DB reports an estimate of 25% – comes in the form of rare conditions that the treating physician may never have seen, but which occur somewhere every day, as compared with the few dozen common complaints that make up the bulk of any individual physician’s office practice).

The problem with long-tail events is that they are difficult or impossible to prepare for. Assuming these are the sorts of things that require prepration and training (for instance, studying up on medical diagnoses and stockpiling supplies to treat them – as opposed to merely stocking books in a bookstore which any clerk can then sell) you can’t make ready for all possible long-tail events because there are simply too many possible ones, of which only a few will actually occur to you personally. Doctors must concentrate on the diagnoses they are likely to see; it is humanly impossible for them to learn everything necessary about every possible one they could see. (There is a line in William Nolen’s celebrated The Making of a Surgeon in which he comes out with some obscure diagnosis for a difficult case, and the attending physician tells him “Learn all you can about appendicitis, Nolen, and leave the dum-dum fever to the experts.”)

This has two frightening consequences: the first, obviously, is that it almost guarantees sub-optimal treatment for a large percentage of cases, at least until those cases get to the point that specialist help is obviously needed; the second is one DB emphasizes – that some long-tail diseases may actually be increasing in frequency as “evidence-based” treatment becomes the standard. Strictly literal differential diagnosis, based on clinical probabilities, by definition will not initiate treatment for low-probability conditions in most cases, necessitating ineffective treatment until the clinician has worked far enough down the differential list to come upon the low-probability condition; in the meantime, an infectious condition has time to spread. Broader empirical treatment, such as presumptive antibiotics for all likely infectious conditions, may have treated many long-tail conditions effectively during the relatively brief historical window between the advent of truly effective medicine (early 20th Century) and the availability of clear research-based treatment guidelines for many conditions (late 20th Century). More-accurate diagnosis may thus, paradoxically, result in longer-delayed treatment and worse public-health outcomes than the offhand scattergun approach, in some cases at least. (Of course, the informal approach has great dangers in itself, including promoting bacterial resistance and exposing patients to unnecessary treatments.)

One problem with diagnostic uncertainty or delayed treatment is that they infuriate patients, who have as great a desire for instant gratification in their healthcare as they do in their sex lives, politics, or entertainment. Telling patients “I don’t know what you have; let me look it up and come back next week” will only drive them to another physician with greater confidence (though no greater knowledge) and more willingness to be accomodating; the same thing is anecdotally reported about doctors who refuse to prescribe antibiotics for obviously viral conditions. That also augurs for less diagnostic fussiness and more makin’ with the pills and the potions.

This suggests that one possible response to the long-tail phenomenon is simply to ignore it (sort of): to acknowledge that a large fraction of one’s encounters will be with rarely- or never-seen conditions, and that there is no way to prepare for these, and therefore to simply treat everything as presumptively in the fat part of the curve until proven otherwise. But it’s counter-intuitive to suggest that the right way to deal with challenging puzzles of medical knowledge is not to try to master that knowledge.

Suggestions?

2 Responses to “Cutting Off the Long Tail”

  1. Dan Says:

    This seems to pertain directly to your last post about disabilities: Blue, and more pointedly Ballastexistenz, are showing that normals operate in everyday life by observing a few salient features and operating on the assuption that the usual case that fits those features is the correct one.

    In the case of somebody in a wheel chair, the salient features are a passable guide to behavior (duh, they can’t walk), but in the case of a mental deficiency, particularly an emotional one, the salient features turn out to be a fairly poor guide to behavior (Somebody who shows no sympathy to human feeling can be modelled either as (a) incapable of detecting data that’s obvious to others or (b) not valuing other humans as people.)

    Ballastexistenz goes on at some length to explain that others do a very poor job of modelling her mental structure. Surely, this is an indication that her mental structure is very unusual, not that several thousand years of evolution have been bad at matching typcial human mental hardware to… typical human mental hardware.

    Blue’s commentor That Girl notes quite correctly that a perceptible problem is much easier for the observer to handle, since the observer can then actually react to the problem meaningfully. Perhaps now that the handicapped are no longer being institutionalized so much, the average person will have the same dilemma as what you describe here: the modal case is not the mean case, and poor performance is the mathematical requirement.

  2. DB’s Medical Rants » Blog Archive » Long tail, practice time, education time Says:

    [...] Our challenge, as physicians, is to know when we no longer reside in the “short head” of the above curve, i.e., when we have to consider the “long tail”. I believe that long tail excellence requires two factors, adequate time for thinking and adequate time for learning. Earlier this week, another blog commented on this problem – Cutting Off the Long Tail. The author wondered whether: The problem with long-tail events is that they are difficult or impossible to prepare for. Assuming these are the sorts of things that require prepration and training (for instance, studying up on medical diagnoses and stockpiling supplies to treat them – as opposed to merely stocking books in a bookstore which any clerk can then sell) you can’t make ready for all possible long-tail events because there are simply too many possible ones, of which only a few will actually occur to you personally. Doctors must concentrate on the diagnoses they are likely to see; it is humanly impossible for them to learn everything necessary about every possible one they could see. (There is a line in William Nolen’s celebrated The Making of a Surgeon in which he comes out with some obscure diagnosis for a difficult case, and the attending physician tells him “Learn all you can about appendicitis, Nolen, and leave the dum-dum fever to the experts.”) [...]

Leave a Reply

Logged in as . Logout »

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

CommentLuv badge

About:

Search
Sufficient Scruples:

Categories:

Archives:

August 2006
M T W T F S S
« Jul   Sep »
 123456
78910111213
14151617181920
21222324252627
28293031  

Links & Feeds:

RSS 2.0

Comments RSS 2.0

XFN

Follow KTKeith on Twitter

Sources:

Powered by WordPress

Get Firefox!

Theme copyright © 2002–2014Mike Little.

Ask the Ethicist!

Podcasts:

White Papers:

Bioethics Links:

Blogroll: