Big names or big ideas: Do peer-review
panels select the best science proposals?
Danielle Li1*† and Leila Agha2,3*†
This paper examines the success of peer-review panels in predicting the future quality of
proposed research.We construct new data to track publication, citation, and patenting
outcomes associated with more than 130,000 research project (R01) grants funded by
the U.S. National Institutes of Health from1980 to 2008.We find that better peer-review scores
are consistently associated with better research outcomes and that this relationship persists
even when we include detailed controls for an investigator’s publication history, grant history,
institutional affiliations, career stage, and degree types. A one–standard deviation worse
peer-review score among awarded grants is associated with 15% fewer citations, 7% fewer
publications, 19% fewer high-impact publications, and 14%fewer follow-on patents.