Abstract: Experimentation serves as the foundation of scientific decision-making. Adaptive allocation of measurement effort can significantly improve statistical power. However, implementing standard bandit algorithms, which assume continual reallocation of measurement effort, is challenging due to delayed feedback and infrastructural or organizational difficulties. To address this, we introduce a new framework for adaptive experimentation, motivated by practical instances involving a limited number of reallocation epochs in which outcomes are measured in batches. Our framework diverges from the traditional theory-driven paradigm by utilizing computational tools for algorithmic design.

We observe that normal approximations, which are universal in statistical inference, can also guide the design of scalable adaptive designs. By deriving an asymptotic sequential experiment, we formulate a dynamic program that can leverage prior information on average rewards. We propose a simple iterative planning method called Residual Horizon Optimization, which selects sampling allocations by optimizing a planning objective. Our method significantly improves statistical power over standard adaptive policies, even when compared to Bayesian bandit algorithms (e.g., Thompson sampling) that require full distributional knowledge of individual rewards. Overall, we expand the scope of adaptive experimentation to settings that pose difficulties for standard adaptive policies, including problems with a small number of reallocation epochs, low signal-to-noise ratio, and unknown reward distributions.

This work was led by Ethan Che. Paper link: https://arxiv.org/abs/2303.11582.

Bio: Hongseok Namkoong is an Assistant Professor in the Decision, Risk, and Operations division at Columbia Business School and a member of the Columbia Data Science Institute. His research interests lie at the interface of machine learning, operations research, and causal inference, with a particular emphasis on developing reliable learning methods for decision-making problems. Hong's research has been recognized by several awards, including paper awards at Neural Information Processing Systems, International Conference on Machine Learning, INFORMS Applied Probability Society, and Conference on Computer Vision and Pattern Recognition, and the Amazon Research Award. He received his Ph.D. from Stanford University and worked as a research scientist at Facebook Core Data Science before joining Columbia. Outside of academia, he serves as a LinkedIn Scholar at LinkedIn's Trust & Responsible AI team.