Estimating very large demand systems

Working paper

We present a discrete choice, random utility model and a new estimation technique for analyzing consumer demand for large numbers of products. We allow the consumer to purchase multiple units of any product and to purchase multiple products at once (think of a consumer selecting a bundle of goods in a supermarket). In our model each product has an associated unobservable vector of attributes from which the consumer derives utility. Our model allows for heterogeneous utility functions across consumers, complex patterns of substitution and complementarity across products, and nonlinear price effects. The dimension of the attribute space is, by assumption, much smaller than the number of products, which effectively reduces the size of the consumption space and simplifies estimation. Nonetheless, because the number of bundles available is massive, a new estimation technique, which is based on the practice of negative sampling in machine learning, is needed to sidestep an intractable likelihood function. We prove consistency of our estimator, validate the consistency result through simulation exercises, and estimate our model using supermarket scanner data.

Authors: Lanier, J; Large, J; Quah, J

• Publication status: Published
• Publisher: University of Oxford
• Series: Department of Economics Discussion Paper Series
• Place of publication: Oxford
• Publication date: 2023-02-14
• ISSN: 1471-0498
• Paper number: 998
• Language: English
• Keywords: scanner data; machine learning; discrete choice; demand estimation; negative sampling