Publications
Publications
- 2024
- HBS Working Paper Series
Using LLMs for Market Research
By: James Brand, Ayelet Israeli and Donald Ngwe
Abstract
Large language models (LLMs) have rapidly gained popularity as labor-augmenting
tools for programming, writing, and many other processes that benefit from quick text
generation. In this paper we explore the uses and benefits of LLMs for researchers and
practitioners who aim to understand consumer preferences. We focus on the distributional
nature of LLM responses, and query the Generative Pre-trained Transformer 3.5 Turbo
(GPT-3.5 Turbo) model to generate dozens of responses to each survey question. We offer
two sets of results to illustrate and assess our approach. First, we show that estimates of
willingness-to-pay for products and features derived from GPT responses are realistic and
comparable to estimates from human studies. Second, we demonstrate a practical method
for market researchers to enhance GPT’s responses by incorporating previous survey data
from similar contexts via fine-tuning. This method improves the alignment of GPT’s responses
with human responses for existing and, importantly, new product features. We do
not find similar improvements in the alignment for new product categories or for differences
between customer segments.
Keywords
Large Language Model; Research; AI and Machine Learning; Analysis; Customers; Consumer Behavior; Technology Industry; Information Technology Industry
Citation
Brand, James, Ayelet Israeli, and Donald Ngwe. "Using LLMs for Market Research." Harvard Business School Working Paper, No. 23-062, April 2023. (Revised July 2024.)