Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • March 2022
  • Article
  • Computational and Mathematical Organization Theory

Sensitivity Analysis of Agent-based Models: A New Protocol

By: Emanuele Borgonovo, Marco Pangallo, Jan Rivkin, Leonardo Rizzo and Nicolaj Siggelkow
  • Format:Print
ShareBar

Abstract

Agent-based models (ABMs) are increasingly used in the management sciences. Though useful, ABMs are often critiqued: it is hard to discern why they produce the results they do and whether other assumptions would yield similar results. To help researchers address such critiques, we propose a systematic approach to conducting sensitivity analyses of ABMs. Our approach deals with a feature that can complicate sensitivity analyses: most ABMs include important non-parametric elements, while most sensitivity analysis methods are designed for parametric elements only. The approach moves from charting out the elements of an ABM through identifying the goal of the sensitivity analysis to specifying a method for the analysis. We focus on four common goals of sensitivity analysis: determining whether results are robust, which elements have the greatest impact on outcomes, how elements interact to shape outcomes, and which direction outcomes move when elements change. For the first three goals, we suggest a combination of randomized finite change indices calculation through a factorial design. For direction of change, we propose a modification of individual conditional expectation (ICE) plots to account for the stochastic nature of the ABM response. We illustrate our approach using the Garbage Can Model, a classic ABM that examines how organizations make decisions.

Keywords

Agent-based Modeling; Sensitivity Analysis; Design Of Experiments; Total Order Sensitivity Indices; Organizations; Behavior; Decision Making; Mathematical Methods

Citation

Borgonovo, Emanuele, Marco Pangallo, Jan Rivkin, Leonardo Rizzo, and Nicolaj Siggelkow. "Sensitivity Analysis of Agent-based Models: A New Protocol." Computational and Mathematical Organization Theory 28, no. 1 (March 2022): 52–94.
  • Read Now

About The Author

Jan W. Rivkin

Strategy
→More Publications

More from the Authors

    • April 2022
    • Faculty Research

    Connecting Students in Chattanooga (B)

    By: Jan W. Rivkin and Manjari Raman
    • April 2022
    • Faculty Research

    Connecting Students in Chattanooga (A)

    By: Jan W. Rivkin and Manjari Raman
    • May 2021
    • Faculty Research

    Hurtigruten: Sailing into Warm Water?

    By: Jan Rivkin and Kerry Herman
More from the Authors
  • Connecting Students in Chattanooga (B) By: Jan W. Rivkin and Manjari Raman
  • Connecting Students in Chattanooga (A) By: Jan W. Rivkin and Manjari Raman
  • Hurtigruten: Sailing into Warm Water? By: Jan Rivkin and Kerry Herman
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College