It seems to me after reading the material sent on this panel, that we have an 
excellent foundation for addressing the questions.  In the case of the first 
two questions, I cannot add much that has not been said before.  I will make a 
few comments though and you will find them below:


The questions are

1.  What is an investigator's personal responsibility with respect to research 
ethics (including reproducibility) in his/her own laboratory? (We don't have 
to use the word "lab" but want to include grad students, programmers, etc. 
who may conduct some analyses directly.)

	It seems worthwhile to me to make a few distinctions for the audience.  
Are all the errors we are talking about of equivalent importance?  Do we care 
as much about negative as (strikingly) positive results?  Is there a hierarchy 
as to what deserves what level of re-review? In an ideal world, everything 
would be replicated - negative and positive work.  But we need to dispense 
with this rapidly because that world does not exist nor is it likely to in the 
foreseeable future.  I will argue that we need a hierarchy of research results 
that would be matched to appropriate checks and balances.  [Think of this the 
way FDA might - the strength of the regulatory controls on products roughly 
corresponds to the risk the products pose to patients and to operators exposed 
to the product.]

This the provides a bit of context within which we can ask and answer this 
first question.

I would also think it worth considering to point out the economics of research 
and the reward system for researchers that have promulgated this system.  
Finally, in this area, is it worth distinguishing between purposeful fraud 
versus poor methodology that has led to results that eventually are not 
reproducible? 


2.  What is the journals' role in ensuring reproducibility?

	I think others should answer this.


3.  What is the institutional role?

	I believe this means the host academic or business (say drug or device 
company) that employs the investigator(s) who have produced these results.  
Possible roles include: procedures to determine chain of accountability; 
environmental support for open dialogue about research results; where possible 
help with infrastructure; encouragement to open up researcher's work and with 
respect to appropriate protection for faculty who have released their data; 
support through the promotion process for the importance of scientific 
reproductions of others' work; and establishment of appropriate oversight 
boards for important clinical and laboratory research.


4.  What is the responsibility of federal agencies with respect to 
reproducibility and research ethics? How does this differ when the agency is 
in the role of the grantor (e.g., NIH) versus the role of a regulator (e.g., 
FDA)?

	Given my 10+ years at NCI and 12+ years at FDA, maybe I can add a bit 
here.  Clearly the roles are different because their missions and legal 
governing statutes are different.  With respect to NIH and grants - grants are 
gifts to investigators in order to carry out planned research.  NIH should not 
be expected to check and reproduce research results in almost all cases - this 
is the responsibility of the investigator and institution.  I see three 
important roles for NIH:  a) given the recent problems, NIH should review and 
adjust its grant (and contract) review systems in order to establish a high 
bar for researchers who apply to NIH - in fact, in most cases of the research 
we are discussing, NIH can adopt review processes explicitly to look for 
research that has the capability of being easily reproduced; b) for selected 
studies (high profile, very risky and with big risk-reward), NIH should set 
aside funds for such reproduction studies and have a mechanism to fund them in 
close proximity in time with the originally research; and c) help develop 
standards for review and consider developing a repository for code and data 
that would allow research results to be easily obtained and reproduced.

	FDA has a very different role - regulatory.  In some of the cases we 
have been discussing, there is an iron-clad requirement of the FDA to protect 
the data from others.  This makes reproducibility more than challenging. Some 
have suggested the FDA do the reproducing work.  This is not likely because of 
the lack of resources (and sometimes the expertise) at the FDA.  However, 
there is one major role FDA can play.  Almost all medical products FDA 
regulates fall under some version of the quality system requirements (QSR).  
For producing a medical product, following the quality systems regulations 
provides a high level of fidelity in the process.  There is a saying at FDA, 
"you can't inspect quality into a product." What this means is that the 
quality of the product depends in large part on the structured and well 
documented systems that are put into place to produce the drug or device.  
Following these systems would avoid a number of the problems we have seen; 
yet, because of the proprietary nature of product discovery, results of the 
type we want to see may not be made available by the FDA.  This suggests 
strengthening the analytic capability of statisticians at FDA.