A randomized trial of fellowships for early career researchers finds a high reliability in funding decisions

Clarke, Philip, Herbert, Danielle, Graves, Nick, & Barnett, Adrian G. (2016) A randomized trial of fellowships for early career researchers finds a high reliability in funding decisions. Journal of Clinical Epidemiology, 69, pp. 147-151.

[img] Accepted Version (PDF 280kB)
Administrators only until December 2016 | Request a copy from author

View at publisher



Funding for early career researchers in Australia's largest medical research funding scheme is determined by a competitive peer-review process using a panel of four reviewers. The purpose of this experiment was to appraise the reliability of funding by duplicating applications that were considered by separate grant review panels.

Study Design and Methods

Sixty duplicate applications were considered by two independent grant review panels that were awarding funding for Australia's National Health and Medical Research Council. Panel members were blinded to which applications were included in the experiment and to whether it was the original or duplicate application. Scores were compared across panels using Bland–Altman plots to determine measures of agreement, including whether agreement would have impacted on actual funding.


Twenty-three percent of the applicants were funded by both panels and 60 percent were not funded by both, giving an overall agreement of 83 percent [95% confidence interval (CI): 73%, 92%]. The chance-adjusted agreement was 0.75 (95% CI: 0.58, 0.92).


There was a comparatively high level of agreement when compared with other types of funding schemes. Further experimental research could be used to determine if this higher agreement is due to nature of the application, the composition of the assessment panel, or the characteristics of the applicants.

Impact and interest:

0 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 90245
Item Type: Journal Article
Refereed: Yes
Keywords: Funding, Peer review, Reliability, Research, Fellowship, Career
DOI: 10.1016/j.jclinepi.2015.04.010
ISSN: 0895-4356
Subjects: Australian and New Zealand Standard Research Classification > MEDICAL AND HEALTH SCIENCES (110000) > PUBLIC HEALTH AND HEALTH SERVICES (111700) > Epidemiology (111706)
Divisions: Current > QUT Faculties and Divisions > Faculty of Health
Current > Institutes > Institute of Health and Biomedical Innovation
Current > Schools > School of Public Health & Social Work
Copyright Owner: Copyright 2015 Elsevier
Deposited On: 16 Nov 2015 23:54
Last Modified: 04 Mar 2016 17:12

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page