Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • November 9, 2019
  • Article
  • BMJ: British Medical Journal

Effect of Revealing Authors' Conflicts of Interests in Peer Review: Randomized Controlled Trial

By: Leslie K. John, George Loewenstein, Andrew Marder and Michael Callaham
  • Format:Electronic
ShareBar

Abstract

Objective: To assess the impact of disclosing authors’ conflict of interest declarations to peer reviewers at a medical journal.
Design: Randomised controlled trial.

Setting: The study was conducted within the manuscript review process at the Annals of Emergency Medicine.

Participants: Reviewers (n=885 reviewers) who reviewed manuscripts submitted between June 2, 2014 and January 23, 2018 inclusive (n=1480 manuscripts).

Intervention: Reviewers were randomised to either receive (treatment) or to not receive (control) authors’ full ICMJE-format conflict of interest disclosures before performing their review. Reviewers rated the manuscripts as usual on eight quality ratings and were then surveyed to obtain “counterfactual scores”—i.e., the scores they believed they would have given had they been assigned to the opposite arm—as well as attitudes toward conflicts of interest. .

Main outcome measures: The primary outcome measure was the overall quality score that reviewers assigned to the manuscript upon submitting their review (1 to 5 scale). The trial had 99% power to detect a .4 point difference, determined previously by editors to be the minimal significance threshold for a difference in score that would influence their editorial decision. Secondary outcomes looked at the scores the reviewers submitted for the seven more specific quality ratings as well as counterfactual scores elicited in the follow-up survey.

Results: Providing authors’ conflict of interest disclosures did not affect reviewers’ ratings of manuscript quality (Mcontrol=2.70 out of 5, SD=1.11; Mtreatment=2.74 out of 5, SD=1.13; mean difference=0.04, 95% CI=-0.05 to 0.14), even for manuscripts with disclosed conflicts (Mcontrol=2.85 out of 5, SD=1.12; Mtreatment=2.96 out of 5, SD=1.16; mean difference=0.11, 95% CI=-0.05 to 0.26). Similarly, there was no effect of the treatment on any of the other seven quality ratings the reviewers assigned. Reviewers acknowledged conflicts of interest as an important issue and believed they could correct for them when disclosed. Yet, their counterfactual scores did not differ from actual scores (Mactual=2.69, Mcounterfactual=2.67; difference in means=0.02, 95% CI=-0.02 to -0.01). When conflicts were reported, a comparison of different source types (e.g., government, for-profit corporation, etc.) found no difference in impact.

Conclusions: Current ethical standards require conflict of interest disclosure for all scientific reports. As currently implemented, this practice demonstrated no impact on any quality ratings of real manuscripts being evaluated for publication by real peer reviewers.

Keywords

Conflicts Of Interest; Peer Review; Randomized Controlled Trial; Scientific Publication; Conflict of Interests; Journals and Magazines; Science

Citation

John, Leslie K., George Loewenstein, Andrew Marder, and Michael Callaham. "Effect of Revealing Authors' Conflicts of Interests in Peer Review: Randomized Controlled Trial." BMJ: British Medical Journal 367, no. 8221 (November 9, 2019).
  • Find it at Harvard
  • Read Now

About The Author

Leslie K. John

Negotiation, Organizations & Markets
→More Publications

More from the Authors

    • 2024
    • Faculty Research

    Measuring the Prevalence of Sensitive Behaviors

    By: Tamar Krishnamurti and Leslie John
    • 2024
    • Faculty Research

    The Agreeable Revealer: Personality Correlates of Self-Disclosure

    By: Elinora Pentcheva and Leslie John
    • 2024
    • Faculty Research

    Should I Stay or Should I Disclose? How Omission Bias Guides Our Disclosure Decisions

    By: Elinora Pentcheva and Leslie John
More from the Authors
  • Measuring the Prevalence of Sensitive Behaviors By: Tamar Krishnamurti and Leslie John
  • The Agreeable Revealer: Personality Correlates of Self-Disclosure By: Elinora Pentcheva and Leslie John
  • Should I Stay or Should I Disclose? How Omission Bias Guides Our Disclosure Decisions By: Elinora Pentcheva and Leslie John
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College.