The Huffington Post today breaks an original story on a long-running scandal at Columbia University Hospital, in New York, regarding lack of informed consent on a risky blood-volume-expander study. Heart-surgery patients were recruited into a study involving a new formulation of a volume expander that had been known to cause severe bleeding in its existing form; minimal information was provided in the consent form, some patients did not speak English, some were recruited in the ER under stressful circumstances, and the hospital IRB apparently did not adequately review the existing literature indicating the level of risk potentially involved. (Frustratingly, the story does not say what stage this experiment was conducted at, but it appears to have been a Phase I trial.)

The consequences:

At least two patients in the study died shortly after receiving the fluid and more than two dozen others required transfusions, according to documents submitted to the federal government by the hospital and obtained by the Huffington Post Investigative Fund.

There were attempts to rein in the problem as it developed:

In November 2000, two Columbia anesthesiologists – Marc Dickstein and Mark Heath- sought out the head of the institutional review board, Paul Papagni, a lawyer. They told Papagni that they had been in the operating room when a number of patients had hemorrhaged. They feared the study’s design virtually guaranteed that there would be more who would suffer hemorrhaging

However, their objections were derailed by internal politics. Columbia did later act decisively to crack down on the lead investigator, Dr. Elliott Bennett-Guerrero and report the breaches of protocol to the HHS, but they also downplayed the severity of the consequences, in part by reporting only a narrow range of outcomes from selected patients, not including the most severe adverse outcome, the bleeding that was the most important known side effect. The hospital also ignored advice from the HHS that it contact patients from the study and apprise them of the truth. Dr. Bennett-Guerrero has now been dismissed from the hospital (he landed in a Directorship and medical-school professorship at Duke – guess they’re OK with all this). Columbia has now been ordered by HHS to contact its former patients.

So, all in all, a serious problem plagued by misbehavior at every level, and an apparent partial coverup.

That’s all very worrisome, but it is the investigator’s attempted defense that particularly caught my attention:

Bennett-Guerrero . . . said in e-mails: “It is hard to imagine that an unbiased expert in cardiac surgery clinical trials could conclude that subjects were harmed in this study, since with only 50 patients per group the study was not designed or powered to prove any differences in major complications including death.”

What he’s saying is that because the sample size was so small, the statistical error in the results is necessarily mathematically too large to be able to show that the deaths or other events that resulted are clearly more numerous that would be expected by chance in such a group of patients.

There are a couple of real problems with this.

First, with a sample size of 50 and, as the article notes, varying dosages of the expander given, up to “three times the level recommended by the manufacturers”, this appears to have been a Phase I Ascending Dose trial. Phase I trials are conducted expressly for the purpose of monitoring safety and adverse side effects of the experimental treatment – they are intended to weed out unsafe treatments before they are tried on large groups of patients. In such trials, safety monitoring is paramount; the first sign of harm to patients should bring the trial to a halt. It’s true that such trial sizes are often too small for statistical significance, but the whole point is to gain confidence before exposing a larger sample size – so empirical monitoring is vital. In this trial, two patients died and numerous others suffered serious hemorrhaging – severe-enough outcomes that experienced clinicians complained directly to the IRB – but the lead investigator never reported a problem or stopped the trial.

(It is possible this was a Phase II or combined Phase I/II trial – though again the facts still seem to suggest Phase I. But if so, the sample size should have been large enough to be likely to return statistically significant results. The trial would be worthless without them.)

Another concern is that no early-Phase trial is supposed to be conducted on patients receiving conventional therapy or in lieu of conventional therapy, where such therapy exists. They are conducted on healthy volunteers. Partly this is to ensure that participation is truly voluntary (i.e., that patients are not being enticed into trials because they see it as a requirement for receiving other therapy), partly precisely to avoid this problem of contaminating apparent adverse consequences of the experiment with the patients’ underlying pathologies.

So it is difficult to see how this trial could have been appropriately designed, aside from the question of informed consent. Either it was a safety trial conducted on patients whose health was already compromised to the point that adverse effects could not be identified as the results of the experimental medication, or it was a dosage-efficacy trial conducted on a sample size too small to provide reliable results, either positive or negative. And in either case, clinical judgment seems to have been dispensed with as patients died but – because of the built-in lack of confirmatory mathematics – no suspicion was entertained about a possible link to the experiment they were participating in.

That raises questions of the investigator’s intent. At this point, I want to step away from this particular incident, and make it clear I am not making insinuations about Dr. Bennett-Guerrero or others from the Columbia trial. Clearly things went badly there but I don’t know what was going through his mind or what his intentions were. I want to use this situation to illustrate ways in which clinical trial design can be (again, I am saying nothing about this particular case) manipulated to  evade ethical protections for subjects.

If a trial is deliberately designed with a sample size too small to return significant results, then by definition no negative results can ever be discerned (nor can any positive results, either, of course). At the Phase I level, where harm is the only reported result, lack of positive results is not a problem, but the impossibility of negative results means that the candidate drug will automatically pass the screening. (Since you can’t find any statistically significant negative results, there will be no statistically significant negative results to report, thus the drug can never be proven to have failed the test. And since, at Phase I, “not failing” is a good-enough result to justify further research, the lack of a robust experimental design can, paradoxically, be a very useful feature.) With a lax IRB focusing only on the mechanics of the informed consent procedure, and not the possible pathways for harm or the mathematical intricacies of the results testing, one can easily get permission to conduct a “drug test” that no drug can possibly fail.

Doing so, of course, requires that you suspend judgment as to the empirical/clinical course demonstrated by the subjects. With no mathematical test for success/failure, an ethical researcher must rely on careful clinical monitoring to detect problems with individual patients or the trial as a whole. Starting with healthy subjects makes such problems obvious, since they aren’t supposed to die at all, but conducting the trial on subjects already sick (or, in fact, pulled directly out of the ER into heart surgery) creates a ready explanation why some of them may die, if in fact they do. So, again, there is an incentive to conduct the trial in what would otherwise be a scientifically invalid manner, essentially building in negative outcomes from the beginning (if the patients already have life-threatening illnesses, you’re going to get some bad outcomes no matter what) as a screen for the negative outcomes that may arise from the experimental procedure. This makes it difficult to honestly answer the question whether the procedure harmed the subjects, but makes it easy to argue that it did not provably harm them.

Thus, for an unscrupulous researcher (and again, this part of the discussion is hypothetical; it is not aimed at a particular individual), it may be possible to design a trial that cannot deliver honest and reliable results, but which also cannot fail to provide the preferred result from the point of view of a drug manufacturer or funding source. Doing so requires conducting a trial that is both scientifically non-decisive by intention and which lacks the ability to identify clear harms to subjects. It may also require deliberately enticing patients into the trial for whom better and safer therapies are available, precisely to use their pathologies as a ready excuse for adverse clinical outcomes which the trial may produce in them. And, it necessarily requires vacating the professional obligation to use vigilance and judgment to monitor and protect patients in all circumstances, and especially the experimental environment – and to instead rationalize patients’ outcomes away in order to avoid public knowledge, and possibly self-knowledge, of the harms inflicted upon them.

Convenient rationalizations are not an acceptable mindset for those who take vulnerable others into their care. The fact that a trial design cannot determine whether its subjects have been harmed is not an acceptable exculpation of those whose obligation was to watch for, detect, and ameliorate such harms. It is a reason why such trials must not be conducted in the first place.