Publications
Publications
- 2025
- HBS Working Paper Series
Evaluations Amid Measurement Error: Determining the Optimal Timing for Workplace Interventions
By: Matthew DosSantos DiSorbo, Iavor I. Bojinov and Fiammetta Menchetti
Abstract
Researchers have embraced factorial experiments to simultaneously evaluate multiple treatments, each with different levels. Typically, in large-scale factorial experiments, the primary objective is identifying the treatment with the largest causal effect, especially when evaluations are hampered by measurement error, attrition, and non-compliance. Point estimates are unreliable, but—as we show—asymmetry in the largest treatment effect allows identification of the most impactful intervention. To exploit this asymmetry, we propose a Fisher randomization test as a general non-parametric approach for inference, which we apply to an existing field experiment that evaluated interns following workplace programs at a large financial firm. We show that the earliest intervention has an immediate and enduring impact: performance improves in the week of the intervention and in future weeks, sometimes even to a greater extent than interventions in those future weeks. The takeaway—intervene early—has important consequences across the many contexts of workplace programs.
Keywords
Factorial Designs; Fisher Randomizations; Rank Estimators; Employer Interventions; Causal Inference; Mathematical Methods; Performance Improvement
Citation
DosSantos DiSorbo, Matthew, Iavor I. Bojinov, and Fiammetta Menchetti. "Evaluations Amid Measurement Error: Determining the Optimal Timing for Workplace Interventions." Harvard Business School Working Paper, No. 24-075, June 2024. (Revised May 2025.)