IT Forum presents "Scaling Wasserstein distances to high dimensions via smoothing"

Scaling Wasserstein distances to high dimensions via smoothing
Friday, February 12, 2021 - 1:00pm
Prof Ziv Goldfeld (Cornell)
Abstract / Description: 

Wasserstein distances has recently seen a surge of applications in statistics and machine learning. This stems from many advantageous properties they possess, such as metric structure (they metrize weak convergence), robustness to support mismatch, compatibility to gradient-based optimization, and rich geometric properties. In practice, we rarely have access to the actual distribution and only get data from it, which necessitates estimating the distance from samples. A central issue is that such estimators suffer from the curse of dimensionality: their empirical convergence rate scales as n^{-1/d} for d-dimensional distributions. This rate deteriorates exponentially fast with dimension, making it impossible to obtain meaningful accuracy guarantees, considering the dimensionality of real-world data.

This talk will present a novel framework of smooth Wasserstein distances, that inherits the properties of their classic counterparts while alleviating the empirical curse of dimensionality. Specifically, we will show that the empirical approximation error of the smooth distance decays as n^{-1/2}, in all dimensions. For the special case of the smooth 1-Wasserstein distance, we will also derive a high-dimensional limit distribution, further highlighting the favorable statistical behavior of the smooth framework. Applications to implicit generative modeling will be considered, leveraging the statistical efficiency results to establish n^{-1/2} generalization bounds in any dimension.

Bio: Ziv Goldfeld is an assistant professor in the School of Electrical and Computer Engineering, and a graduate field member in Computer Science and the Center of Applied Mathematics, at Cornell University. Before joining Cornell, he was a postdoctoral research fellow in LIDS at MIT, hosted by Yury Polyanskiy. Ziv graduated with a B.Sc., M.Sc., and Ph.D. (all summa cum laude) in Electrical and Computer Engineering from Ben Gurion University, Israel, in 2012, 2015 and 2018, respectively. His graduate advisor was Haim Permuter.

Ziv's research interests include optimal transport theory, statistical learning theory, information theory, and high-dimensional and nonparametric statistics. He seeks to understand and design engineering systems by formulating and solving mathematical models. A main focus is working towards a comprehensive statistical learning theory to obtain better understanding and strong performance guarantees for modern machine learning methods that operate on real-world high-dimensional data.

Honors include the NSF CAREER Award, NSF CRII Award, the IBM University Award, and the Rothschild Postdoctoral Fellowship.