Publications
Publications
- 2020
- HBS Working Paper Series
Machine Learning for Pattern Discovery in Management Research
Abstract
Supervised machine learning (ML) methods are a powerful toolkit for discovering robust patterns in quantitative data. The patterns identified by ML could be used as an observation for further inductive or abductive research, but should not be treated as the result of a deductive causal test. To demonstrate the application of ML for pattern discovery, we implement ML algorithms to study employee turnover at a large technology company. We interpret the relationships between variables using partial dependence plots, which uncover surprising patterns that may have gone unnoticed using traditional methods. To guide readers evaluating such pattern discovery, we provide standards for evaluating model performance, illustrate human decisions in the process, and warn of common misinterpretation pitfalls. An online appendix provides code and data to implement the algorithms demonstrated in the paper.
Keywords
Machine Learning; Theory Building; Induction; Decision Trees; Random Forests; K-nearest Neighbors; Neural Network; P-hacking; Analytics and Data Science; Analysis
Citation
Choudhury, Prithwiraj, Ryan Allen, and Michael G. Endres. "Machine Learning for Pattern Discovery in Management Research." Harvard Business School Working Paper, No. 19-032, September 2018. (Revised June 2020.)