Long before he became famous or infamous (depending on your politics) as FBI Director, James Comey served as US Attorney for the Southern District of New York. That’s the office responsible for prosecuting most major financial crimes in the US. Jesse Eisinger’s book, The Chickens**t Club, recounts a speech Comey made to his staff after assuming that high-profile post. He asked which of his prosecutors had never lost a case at trial, and many proudly raised their hands. Comey then said, “You are all members of what we like to call The Chickens**t Club.” By that he meant: You are too timid to take a case to trial unless you already know you will win.
I worry that our clinical trials too often follow the same pattern as those white-collar criminal trials. When we evaluate new treatments or programs, we may only pursue the trials likely to give the answer we hope for. That might mean testing an intervention only slightly different from one already proven effective. Or testing a treatment in an environment where it’s almost certain to succeed. Our wishes come through in our language. Trials showing that a new treatment is superior are “positive”, while trials finding no advantage for a new treatment or program are “negative.”
Our preference for trials with “positive” results reflects how researchers are rewarded or reinforced. We know that positive trials are more likely to be published than so-called negative trials. Grant review panels prefer positive research that yields good news and gives the appearance of continuous progress.
Looking back on my career, I can see some trials that might have put me in The Chickens**t Club. For example, we probably didn’t need to do several clinical trials of collaborative care for depression in one health system before taking that idea to scale.
But here’s where the analogy between investigators and prosecutors does not apply: A prosecutor in a criminal trial is supposed to take sides. An investigator in a clinical trial shouldn’t have a strong preference for one result or the other. In fact, clinical investigators have the greatest obligation to pursue trials when the outcome is uncertain. “Equipoise” is the term to describe that balanced position.
Greg Simon