Gordon Wetzstein and PhD candidate David Lindell create algorithm that reconstructs hidden scenes
Using a new algorithm, Stanford researchers have reconstructed the movements of individual particles of light to see through clouds, fog and other obstructions.
Professor Gordon Wetzstein and EE PhD candidate David Lindell, have created a system that reconstructs shapes obscured by 1-inch-thick foam. Their tests are detailed in, "Three-dimensional imaging through scattering media based on confocal diffuse tomography", published in Nature Communications.
Gordon Wetzstein reports, "A lot of imaging techniques make images look a little bit better, a little bit less noisy, but this is really something where we make the invisible visible. This is really pushing the frontier of what may be possible with any kind of sensing system. It's like superhuman vision."
"We were interested in being able to image through scattering media without these assumptions and to collect all the photons that have been scattered to reconstruct the image," said David Lindell, EE PhD candidate and lead author of the paper. "This makes our system especially useful for large-scale applications, where there would be very few ballistic photons."
In order to make their algorithm amenable to the complexities of scattering, the researchers had to closely co-design their hardware and software, although the hardware components they used are only slightly more advanced than what is currently found in autonomous cars. Depending on the brightness of the hidden objects, scanning in their tests took anywhere from one minute to one hour, but the algorithm reconstructed the obscured scene in real-time and could be run on a laptop.
"You couldn't see through the foam with your own eyes, and even just looking at the photon measurements from the detector, you really don't see anything," said David. "But, with just a handful of photons, the reconstruction algorithm can expose these objects – and you can see not only what they look like, but where they are in 3D space."
Excerpted from Stanford News, "Stanford researchers devise way to see through clouds and fog", September 2020.