Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • February 2021
  • Tutorial

Assessing Prediction Accuracy of Machine Learning Models

By: Michael Toffel and Natalie Epstein
  • Format:Video
  • | Language:English
ShareBar

Abstract

This video describes how to assess the accuracy of machine learning prediction models, primarily in the context of machine learning models that predict binary outcomes, such as logistic regression, random forest, or nearest neighbor models. After introducing and differentiating the concepts of training and testing data, the video presents the confusion matrix and uses it to describe a series of accuracy metrics including true/false positives/negatives, true positive rate (sensitivity or recall), false negative rate (Type II error rate), precision, true negative rate (specificity), and false positive rate (Type I error rate). It also addresses the impact of setting thresholds to convert continuous predictions to binary classifications, and describes the receiver operating characteristic curve (ROC curve) and area under the curve (AUC). Several examples are provided, including one where during the middle of a course, an instructor is viewing students’ interim grades, attendance, and participation, and needs to decide which students should seek tutoring to avoid a poor final grade. This video can be assigned in conjunction with the “Assessing Prediction Accuracy of Machine Learning Models” technical note (HBS No. 621045).

Keywords

Statistics; Experiments; Forecasting and Prediction; Performance Evaluation; AI and Machine Learning

Citation

Toffel, Michael, and Natalie Epstein. Assessing Prediction Accuracy of Machine Learning Models. Harvard Business School Tutorial 621-706, February 2021. (Click here to access this tutorial.)
  • Purchase

About The Author

Michael W. Toffel

Technology and Operations Management
→More Publications

More from the Authors

    • June 2025
    • Faculty Research

    Arla Foods: Data-Driven Decarbonization

    By: Michael Parzen, Michael W. Toffel and Susan Pinckney
    • 2025
    • Faculty Research

    How Firms Respond to Worker Activism: Evidence from Global Supply Chains

    By: Yanhua Bird, Jodi L. Short and Michael W. Toffel
    • May 2025
    • Faculty Research

    Calyx Global: Rating Carbon Credits

    By: Michael W. Toffel and Adam Chen
More from the Authors
  • Arla Foods: Data-Driven Decarbonization By: Michael Parzen, Michael W. Toffel and Susan Pinckney
  • How Firms Respond to Worker Activism: Evidence from Global Supply Chains By: Yanhua Bird, Jodi L. Short and Michael W. Toffel
  • Calyx Global: Rating Carbon Credits By: Michael W. Toffel and Adam Chen
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College.