Repository

This curated collection of documents includes key papers, slide decks, video materials and links to further resources on research evaluation and peer review.

To create your own reading list, you can select items and then click on “Add to Reading List” at the bottom of the page. SciencEval then automatically generates a link to your new reading list, which you can share with colleagues and students.

If you cannot find a document, which you believe should also be included in this repository, please contact us at SciencEval@snf.ch.

Documents
Tennant, J. P. et al.. A multi-disciplinary perspective on emergent and future innovations in peer review F1000Research, 2017, 6, 1151 https://f1000research.com/articles/6-1151/v3
Li, D. & Agha, L. Big names or big ideas: Do peer-review panels select the best science proposals? Science, 2015, 348, 434-438 https://www.sciencemag.org/lookup/doi/10.1126/science.aaa0185
Luukkonen, T. Conservatism and risk-taking in peer review: Emerging ERC practices Research Evaluation, 2012, 21, 48-60 https://academic.oup.com/rev/article/21/1/48/1643858
Rijcke, S. d.; Wouters, P. F.; Rushforth, A. D.; Franssen, T. P. & Hammarfelt, B. Evaluation practices and effects of indicator use—a literature review Research Evaluation, 2016, 25, 161-169 https://academic.oup.com/rev/article/25/2/161/2362680
Bollen, J.; Crandall, D.; Junk, D.; Ding, Y. & Börner, K. From funding agencies to scientific agency: Collective allocation of science funding as an alternative to peer review EMBO Reports, 2014, 15, 131-133 https://www.ncbi.nlm.nih.gov/pmc/articles/pmc3989857/
Ioannidis, J. P. A. Fund people not projects Nature, 2011, 477, 529-531 https://www.nature.com/articles/477529a
Roy, R. Funding Science: The Real Defects of Peer Review and An Alternative To It: Science, Technology, & Human Values, 2016 https://journals.sagepub.com/doi/10.1177/016224398501000309
Severin, A.; Martins, J.; Heyard, R.; Delavy, F.; Jorstad, A. & Egger, M. Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports BMJ Open, 2020, 10, e035058 http://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2019-035058
Marsh, H. W.; Jayasinghe, U. W. & Bond, N. W. Gender differences in peer reviews of grant applications: A substantive-methodological synergy in support of the null hypothesis model Journal of Informetrics, 2011, 5, 167-180 https://www.sciencedirect.com/science/article/abs/pii/s175115771000091x
Marsh, H. W.; Jayasinghe, U. W. & Bond, N. W. Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability American Psychologist, , 63, 160-168 https://doi.apa.org/doilanding?doi=10.1037%2f0003-066x.63.3.160
Gilbert, J. R.; Williams, E. S. & Lundberg, G. D. Is There Gender Bias in JAMA's Peer Review Process? JAMA, 1994, 272, 139-142 https://jamanetwork.com/journals/jama/fullarticle/376219
Guthrie, S.; Rodriguez Rincon, D.; McInroy, G.; Ioppolo, B. & Gunashekar, S. Measuring bias, burden and conservatism in research funding processes F1000Research, 2019, 8, 851 https://f1000research.com/articles/8-851/v1
Fang, F. C.; Bowen, A. & Casadevall, A. NIH peer review percentile scores are poorly predictive of grant productivity eLife, 2016, 5, e13323 https://elifesciences.org/articles/13323
Csiszar, A. Peer review: Troubled from the start Nature News, 2016, 532, 306 https://www.timeshighereducation.com/features/peer-review-not-old-you-might-think
Fyfe, A. Peer review: not as old as you might think Times Higher Education (THE), 2015 https://www.nature.com/news/peer-review-troubled-from-the-start-1.19763
Danthi Narasimhan; Wu Colin O.; Shi Peibei & Lauer Michael Percentile Ranking and Citation Impact of a Large Cohort of National Heart, Lung, and Blood Institute–Funded Cardiovascular R01 Grants Circulation Research, 2014, 114, 600-606 https://www.ahajournals.org/doi/10.1161/circresaha.114.302656
Casadevall, A. & Fang, F. C. Reforming Science: Methodological and Cultural Reforms Infection and Immunity, 2012, 80, 891-896 https://iai.asm.org/content/80/3/891
Hutchins, B. I.; Yuan, X.; Anderson, J. M. & Santangelo, G. M. Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level PLOS Biology, 2016, 14, e1002541 https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002541
Fang, F. C. & Casadevall, A. Research Funding: the Case for a Modified Lottery mBio, 2016, 7 https://mbio.asm.org/content/7/2/e00422-16
Ernst, E.; Resch, K. & Uher, E. Reviewer Bias Annals of Internal Medicine, 1992, 116, 958-958 https://www.acpjournals.org/doi/abs/10.7326/0003-4819-116-11-958_2
Crane, D. Reward Systems in Art, Science, and Religion: American Behavioral Scientist, 2016 https://journals.sagepub.com/doi/10.1177/000276427601900604
Langfeldt, L. The Decision-Making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome: Social Studies of Science, 2016 https://journals.sagepub.com/doi/10.1177/030631201031006002
Blank, R. M. The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review The American Economic Review, 1991, 81, 1041-1067 https://www.jstor.org/stable/2006906?seq=1
Bol, T.; Vaan, M. d. & Rijt, A. v. d. The Matthew effect in science funding Proceedings of the National Academy of Sciences, 2018, 115, 4887-4890 https://www.pnas.org/content/115/19/4887
Guthrie, S.; Ghiga, I. & Wooding, S. What do we know about grant peer review in the health sciences? F1000Research, 2017, 6, 1335 https://f1000research.com/articles/6-1335/v1
Davis, M. L. E. S.; Conner, T. R.; Miller-Bains, K. & Shapard, L. What makes an effective grants peer reviewer? An exploratory study of the necessary skills PLOS ONE, 2020, 15, e0232327 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232327
Women in neuroscience: a numbers game Nature Neuroscience, 2006, 9, 853-853 https://www.nature.com/articles/nn0706-853
Videos
SNSF Evaluation Workshop 1 - Applicant 1
SNSF Evaluation Workshop 2 - Applicant 2
SNSF Evaluation Workshop 3 - Applicant 3
SNSF Evaluation Workshop 4 - Project 1
SNSF Evaluation Workshop 5 - Project 2
SNSF Evaluation Workshop 6 - Project 3
SNSF Evaluation Workshop 7 - Discussion 1
SNSF Evaluation Workshop 8 - Discussion 2
SNSF Evaluation Workshop 9 - Discussion 3
Social Thinking: Crash Course Psychology #37
Social Influence: Crash Course Psychology #38
Prejudice and Discrimination: Crash Course Psychology #39
Links
Slide Decks
Hill, M. Historical Introduction to Research Evaluation and Peer Review. SciencEval Workshop, Swiss National Science Foundation. 11.12.2020. https://zenodo.org/record/4322690#.x9iw3y2zn24
De Rijcke, S. The State of the Art in Research on Research Evaluation and Peer Review. SciencEval Workshop, Swiss National Science Foundation. 11.12.2020. https://zenodo.org/record/4314239#.x9ixxs2zn25
Add to Reading List

Log In

Forgot password?

Finish session

Your current session finishes after 1m 60s.

Press OK if you want stay logged.

Repair password

Enter the email address associated with your account and we'll send you a link to reset your password.

Create your own Reading List

Sign in or Register

Forgot password?

Are you sure you want to delete ?

Message have sent

Enter the email address associated with your account and we'll send you a link to reset your password.

ОК

Delete account

When you delete your account all your personal and account information as well as all your reading lists will be delete permanently. All links to your reading lists will not work anymore. This action cannot be undone.

Select at least one time from the repository for your reading list ?