Talk Abstract: The analysis of nighttime driving images presents challenges distinct from those encountered with daytime images. Nighttime images often contain large regions of low intensity, punctuated by very bright light sources, such as car headlights and street lamps. These factors limit the performance of computer vision systems. First, the high dynamic range of the scene cannot be accurately captured in a single frame; the limited pixel well capacity saturates the sensor data in large regions near the light source. Second, the optical blur, including lens flare, spreads photons from these intense lights, greatly reducing the image contrast in nearby dark regions. A particular problem for supervised learning is that these factors make image labeling both costly and prone to errors. To address these challenges, we have implemented an end-to-end image system simulation to synthesize physically realistic nighttime spectral radiance images. These are then fed into a physically accurate model of the imaging system (digital twin), encompassing both optics and sensors. We describe the simulation system and introduce a synthetic nighttime dataset with pixel-level labels, designed to facilitate the training of networks for a variety of computer vision applications. Finally, we simulate the performance of several types of image systems (lens, sensor and acquisition policy) under nighttime driving conditions.

Speaker Biography: Zhenyi Liu is a postdoctoral fellow at Stanford University, working with Brian Wandell and Joyce Farrell. His research interests lie in image system simulation, including scene, optics, and sensor modeling, as well as downstream computer vision tasks. He obtained his Ph.D. degree from Jilin University in 2021 and was a visiting student researcher at Stanford between 2017 and 2019. He worked at Tsinghua University as a visiting scholar from 2019 to 2022, prior to returning to Stanford. He has collaborated on projects with various companies like Ford, BMW, Meta, Xiaomi, and VIVO, focusing on image system simulation and automotive application research.