Message From the Editor: Transformative Research

Transformative research:  a range of endeavors which promise extraordinary outcomes, such as: revolutionizing entire disciplines; creating entirely new fields; or disrupting accepted theories and perspectives — in other words, those endeavors which have the potential to change the way we address challenges in science, engineering, and innovation

–National Science Foundation (3)
 

Most studies are natural extensions of prior work:  measured, incremental advances that either confirm or refute previous research or that extend it in an obvious way.  These incremental studies are essential to the advancement of science.  Every new discovery must be validated to confirm that it is correct.  Furthermore, a finding in isolation may have little value without fleshing out the implications.  Investigators are often drawn to performing incremental studies because the methods are tested and results are more predictable.  From the investigator’s perspective, the risk in doing an incremental study is much lower.  Thus, safe and incremental studies naturally dominate the literature. 

Very rarely, a truly transformative study is published (Box).  These break the prevailing paradigm and launch whole new areas of research.  They represent research at the frontier, on the cutting edge of science.  In evolutionary terms, they are akin to dramatic shifts in species that occur around natural disasters. 

There are no standards for identifying truly transformative discoveries.  Of course, “transformativeness” is not a dichotomous trait, either, so publications may range from extremely to not at all transformative with many in between.   Having no established tools at our disposal, the editors of Annals of Neurology informally agreed on a list of 10 in the neurosciences (Table).  The process was much harder than we thought it would be.  Each of us initially nominated five papers, and only two papers were nominated by more than person.  We then rated these independently and those receiving the highest ratings are listed.  There were disagreements about whether a discovery was distinctly new, whether methodological developments by themselves should qualify, and whether clinical research counted.  Discoveries covering multiple papers were easier to identify than single studies.  We were anxious about missing other more worthy studies.  In the end, we generally agreed that the list included 10 great discoveries but that there were probably 50 more in the neurosciences at least as transformative.

Last year the Annals published a fascinating look back into the decision of the Nobel Committee to honor Enders, Weller, and Robbins for the discovery of tissue culture methods used to grow poliovirus, but not Koprowski, Sabin, and  Salk for the translation of this method into successful vaccination strategies.1  Although revolutionary for human health, the development of a polio vaccine program was not deemed to be a discovery of a “primary nature” deserving of the Prize.  Clearly, opinions will differ on how to define transformative discoveries.  All would agree that stunning intuitive leaps into new, uncharted scientific territory qualify.  However, it seems wrong to exclude from consideration those applications, such as the polio vaccine, that are perhaps less creative but nonetheless change the world.

Identifying transformative studies retrospectively is hard enough, but the challenge is even greater when a finding is recent.  The publication of transformative studies is often jarring, with many questioning the validity of the findings and their interpretation.  Scientists like their paradigms and don’t part with them easily.  Furthermore, surprising findings are less likely to be correct.  Just extending Bayes’ Theorem, the more dramatically a new finding contradicts prevailing belief, the more likely it is incorrect.  So, greater skepticism of potentially transformative findings is appropriate.  If most published research is incorrect, as has been suggested in a statistical analysis of prior scientific publications,2 only a small fraction of potentially transformative studies will be validated.  And of course, a study can’t be transformative if it is wrong.

Even further back in its development, a proposal for a project that is potentially transformative may be particularly difficult to judge.  Many of us have seen Stan Prusiner’s slide with him squirming under the thumb of the NIH, fighting to gain support for his grant applications.  Grant applications for transformative research will often appear startling and risky.  Preliminary research may be limited.  A leap of faith may be required.  There is a broad perception that potentially transformative research does poorly in peer review.  In response to this, the National Science Foundation has worked hard to encourage favorable review of more risky and exciting work.3 It also set up specific granting mechanisms designed to encourage transformative research and to discourage reviewer focus on details of the proposed methods.  Similarly, the Pioneer Awards of the NIH are designed to support investigators with highly innovative ideas.4  Proposals for these are very short and much more attention is given to the creative potential of the applicant.  These programs are highly competitive so they will likely be successful simply on this basis.  Nonetheless, examples of transformative findings from these grants would go a long way in promoting targeted funding mechanisms in a portfolio strained by limited funding.

We have almost no information about what predicts transformation.  Who are these people who go on to produce transformative studies and win prizes like the Nobel and the Lasker?  Are they particularly ambitious, hard-working, smart, creative, or just lucky?  Are they triple threats or do they focus tightly on the mission at hand?  Similarly, do we have any hope of identifying transformative projects in advance or do they really arise from good fortune, hard work, and resourcefulness?  How important is environment?  Do these discoveries come from working in isolation or from applying advances in other areas to a whole new problem?  These are all key questions if we went to accelerate discovery, and should be the focus of university and department administrators, as well as funders.  It seems particularly odd that the predictors of transformative research are completely unstudied. 

Without established methods to predict success, funding more studies attempting transformation will mean funding more failures.  Nonetheless, the value of transformative studies clearly justifies even a very low chance of success.  A search of Pubmed generates over 2 million articles on topics related to neuroscience and neurology.  Most of these are never cited.  The truly transformative studies probably constitute <0.01% of these, assuming there are 200 or so out there.  The progress these few studies have spurred would be difficult to overestimate.  In his book Pioneering Research, Donald Braben argues that transformative research is largely responsible for dramatic increases in global gross domestic product during the past three centuries, with a recent slowing in growth attributable to the additional constraints placed on scientists interested in doing high risk research.5

Of course, a team of sluggers will not win the pennant.  We need good fielders and pitchers, and batters who know how to take a ball.  As we have opined previously, fundamental discoveries must be translated through animal and bedside research before any real benefit to humankind is realized, and this work, perhaps less glamorous, is essential in the portfolio, and we cannot expect industry to take this on.6, 7 Some have argued that the more clinical end of the translational end of the research spectrum belongs in the private sector, but we have little evidence that it is done well there, with many areas left neglected because they are not financially rewarding.8  Thus, we need a balanced portfolio-a carefully selected team of investigators and projects representing a breadth of interests and topics. 

At the Annals of Neurology, we would love to be a journal that publishes transformative research, but we are competing against all other journals for the small number of manuscripts that truly meet criteria.   Furthermore, medically important findings that build upon earlier discoveries are of great interest to all of us even if they are not transformative.  One question we always ask is, does the finding change clinical practice or our fundamental understanding of the disease?  By applying this criterion, we are at least pushing toward the transformative end of the spectrum. 

S. Claiborne Johnston MD, PhD, Stephen L. Hauser, MD

Editors

 

References

1.            Norrby E, Prusiner SB. Polio and Nobel prizes: looking back 50 years. Ann Neurol. 2007;61:385-395

2.            Ioannidis JP. Why most published research findings are false. PLoS medicine. 2005;2:e124

3.            National Science Foundation. Important notice to presidents of universities and colleges and heads of other National Science Foundation Awardee Organizations:  Transformative Research, http://www.nsf.gov/pubs/2007/in130/in130.pdf Accessed March 28, 2008

4.            National Institutes of Health. NIH Director’s Pioneer Award, http://nihroadmap.nih.gov/pioneer/ Accessed March 28, 2008

5.            Braben DW. Pioneering Research:  A Risk Worth Taking. Hoboken: Wiley, 2004:198

6.            Johnston SC, Hauser SL. Basic and clinical research: What is the most appropriate weighting in a public investment portfolio? Ann Neurol. 2006;60:A9-A11

7.            Johnston SC, Hauser SL. A status report on neuroscience research, without grade inflation. Ann Neurol. 2006;60:A9-11

8.            Johnston SC, Hauser SL. Can industry rescue the NIH? Ann Neurol. 2006;60:A11-14