Abstract: Recent advances in visual generative models have led to the generation of high quality, diverse images of nearly any imaginable concept, thanks to increasingly large models and huge training datasets. Looking at the quality of images that these models are able to produce, one may wonder: “are we done?”. In this talk, I argue that we’ve only just scratched the surface. These models have seen billions of training images—what other knowledge have they amassed along the way? And what else can they be used for, beyond the image generation task they were trained for? My talk will cover my recent explorations in answering these questions: I’ll demonstrate how the outputs of large text-to-image models can replace curated datasets for downstream supervised tasks, like image editing and 3D reconstruction. Delving deeper, I’ll show that by probing the internal representations of these models, one can extract underlying properties of the generated content, such as scene geometry, point correspondence, and more.

Bio: Aleksander Holynski is a postdoctoral scholar at UC Berkeley, working with Alyosha Efros and Angjoo Kanazawa, and concurrently a Research Scientist at Google Research. He completed his PhD at the University of Washington, advised by Steve Seitz, Brian Curless, and Rick Szeliski, and he received his B.S. at the University of Illinois at Urbana-Champaign. His co-authored work has received a best student paper award at ICCV 2023.