Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2022
  • Working Paper

Improving Human-Algorithm Collaboration: Causes and Mitigation of Over- and Under-Adherence

By: Maya Balakrishnan, Kris Ferreira and Jordan Tong
  • Format:Print
  • | Language:English
  • | Pages:74
ShareBar

Abstract

Even if algorithms make better predictions than humans on average, humans may sometimes have “private” information which an algorithm does not have access to that can improve performance. How can we help humans effectively use and adjust recommendations made by algorithms in such situations? When deciding whether and how to override an algorithm’s recommendations, we hypothesize that people are biased towards following a naïve advice weighting (NAW) heuristic: they take a weighted average between their own prediction and the algorithm’s, with a constant weight across prediction instances, regardless of whether they have valuable private information. This leads to humans over-adhering to the algorithm’s predictions when their private information is valuable and under-adhering when it is not. In a lab experiment where participants are tasked with making demand predictions for 20 products while having access to an algorithm’s recommendations, we confirm this bias towards NAW and find that it leads to a 20-61% increase in prediction error. In a follow-up experiment, we find that feature transparency—even when the underlying algorithm is a black box—helps users more effectively discriminate when and how to deviate from algorithms, resulting in a 25% reduction in prediction error.

Keywords

Cognitive Biases; Algorithm Transparency; Forecasting and Prediction; Behavior; AI and Machine Learning; Analytics and Data Science; Cognition and Thinking

Citation

Balakrishnan, Maya, Kris Ferreira, and Jordan Tong. "Improving Human-Algorithm Collaboration: Causes and Mitigation of Over- and Under-Adherence." Working Paper, December 2022.
  • Read Now

About The Author

Kris Johnson Ferreira

Technology and Operations Management
→More Publications

More from the Authors

    • 10 Nov 2022 - 13 Nov 2022
    • Faculty Research

    Timely Statements: Swift Brand Activism Is the Most Effective and Memorable

    By: Julian De Freitas, Jimin Nam, M. Balakrishnan and Alison Wood Brooks
    • 2022
    • Faculty Research

    Speedy Activists: Firm Reaction Time to Sociopolitical Events Influences Consumer Behavior

    By: Jimin Nam, M. Balakrishnan, Julian De Freitas and Alison Wood Brooks
    • March 2022
    • Faculty Research

    JOANN: Joannalytics Inventory Allocation Tool

    By: Kris Ferreira
More from the Authors
  • Timely Statements: Swift Brand Activism Is the Most Effective and Memorable By: Julian De Freitas, Jimin Nam, M. Balakrishnan and Alison Wood Brooks
  • Speedy Activists: Firm Reaction Time to Sociopolitical Events Influences Consumer Behavior By: Jimin Nam, M. Balakrishnan, Julian De Freitas and Alison Wood Brooks
  • JOANN: Joannalytics Inventory Allocation Tool By: Kris Ferreira
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College