"Starter Set" Materials for the Saga

This web page derives from our web site supplement for the manuscript Deriving Chemosensitivity from Cell Lines: Forensic Bioinformatics and Reproducible Research in High-Throughput Biology by Keith A. Baggerly and Kevin R. Coombes. The main page is here.

Particularly since the story was covered by 60 Minutes, we've gotten requests for more details. A (noncomprehensive!) list of documents/links we've found ourselves suggesting frequently is given below. Hope some of these prove useful!

1. A Video of Us Telling the Story

There are a few videos out there of us giving talks on this story. The one that we'd recommend at present is one from Cambridge in late 2010. This is about 35 minutes long, but should convey the gist of the types of problems we were seeing and how we identified them. Fair warning -- one review of this on the web notes "Be warned, Dr. Baggerly is a fast-talking nerdish PhD who thinks you understand what he's saying [which you likely won't totally get], but watch at least some of it to get the flavor of the genre" which is probably fair ;).

2. The 60 Minutes Segment and Transcript

This is certainly how most people have encountered the story. The clip and transcript are available here. In addition to the segment that aired, there's a short (1:30) clip of Paul Goldberg (of the Cancer Letter) discussing the Rhodes scholar angle, which is well worth watching. We included both this clip and selected short bits from the main piece in talks we've given since the segment aired.

3. Slides from some Recent Talks

We try to update at least some of the slides in our talks, so more recent versions will at least mention later developments. The slides linked to here are from a talk I gave on Feb 15, 2012. We included clips from the 60 Minutes segment at the end of slide 27.

The slides linked to here are from a presentation I gave on Jul 9, 2012, where I used now-available documents to clarify more of who knew what as things were going on.

4. Our 2009 Annals of Applied Statistics Paper

This is where we detail the specific problems we encountered. This may be an atypical statistics paper in that we include all of 3 formulas, all of which are wrong. A copy of the paper is available here.

5. A 2011 Editorial from Clinical Chemistry (Subscription Required)

This is a short (2.5pg) piece we wrote after listening to representatives from both NCI (Lisa McShane) and FDA (Robert Becker) give testimony to the Institute of Medicine (IOM). We use extracts from their talks to emphasize just what information should be required to support clinical "omics" publications. The piece (subscription required) is here.

6. Various Notes from the Institute of Medicine Open Sessions

Sparked by this case, the IOM reviewed the level of evidence that should be required before "omics"-based assays are used to guide patient therapy in clinical trials. This committee began meeting in December of 2010, and issued its report on March 23, 2012. Many of the meetings were open and recorded (audio only, but accompanying slides are typically available). Most of these are linked to from here.

We'd probably start with the testimony we gave on March 31, 2011. Be warned, this segment wound up being nearly 3 hours long. The other one we'd recommend listening to early is Lisa McShane's (biostatistician from the NCI) from December 20, 2010, where she laid out much of what the NCI knew and was doing behind the scenes while all of this was going on. Our annotation of the 550 pages (!) of documents the NCI released at this session is available from the link above. Our summary is about 15 pages. The audio of Lisa McShane's presentation (about an hour) is available from The Cancer Letter linked to their Jan 28, 2011 issue.

7. The IOM Omics Report, and Some Subsequent Presentations by IOM Committee Members

We debated a bit about this, because the full Omics report (at 274p!) isn't really starter material. It is, however, a very thorough exploration of how studies should be performed if the goal is to translate the omics-based tests into clinical use. That said, a "report brief" (5p) is here, and if you really understand the figure on the last page, you're essentially there. I suspect, however, that you might not fully understand it. I thought I did, but listening to some of the recorded presentations by committee members -- Gil Omenn, Joe Gray, Dan Hayes and Daniela Witten at AACR (Apr 3), and Daniela Witten and Larry Kessler at a U Washington session on research ethics (Jul 19) -- added further detail for me. If you're familiar with the background now, I think I'd point you to the video of Kessler's summary of the recommendations first and suggest expanding from there.