SCIEN Talk

In this talk I will describe our system for capturing, reconstructing, compressing, and rendering high quality immersive light field video. We record immersive light fields using a custom array of 46 time-synchronized cameras distributed on the surface of a hemispherical, 92cm diameter dome. From this data we produce 6DOF volumetric videos with a wide 80-cm viewing baseline, 10 pixels per degree angular resolution, and a wide field of view (>220 degrees), at 30fps video frame rates. Even though the cameras are placed 18cm apart on average, our system can reconstruct objects as close as 20cm to the camera rig. We accomplish this by leveraging the recently introduced DeepView view interpolation algorithm, replacing its underlying multi-plane image (MPI) scene representation with a collection of spherical shells which are better suited for representing panoramic light field content. We further process this data to reduce the large number of shell layers to a small, fixed number of RGBA+depth layers without significant loss in visual quality. The resulting RGB, alpha, and depth channels in these layers are then compressed using conventional texture atlasing and video compression techniques. The final, compressed representation is lightweight and can be rendered on mobile VR/AR platforms or in a web browser.

Label-free optical imaging of living biological systems offers rich information that can be of immense value for a variety of biomedical tasks. Despite the exceptional theoretical potential, current label-free microscopy platforms are challenging for real-world clinical and biological applications. The major obstacles include the lack of flexible laser sources, limited contrast, and the challenge of acquiring and interpreting the high-dimensional dataset.

In this talk, I will present new optical imaging platforms and methodologies that will address these challenges. By generating and tailoring coherent supercontinuum from photonic crystal fibers, simultaneous metabolic and structural imaging can be achieved without aids of stains, enabling perturbation-free exploration of living systems. These capabilities further motivate development of analytical tools for image-based segmentation and diagnosis, showing broad potential of this label-free imaging technology in discovering new metabolic biomarkers and enabling real-time point-of-procedure applications.

After registering, you will receive a confirmation email containing information about joining the meeting.

The encapsulation of the retina inside the eye has always challenged our ability to study the anatomy and physiology of retinal neurons in their native state. Our group is developing new tools using adaptive optics that allow not only structural imaging but also functional recording and control of retinal neurons at a cellular spatial scale. By combining adaptive optics with calcium imaging, we can optically record from hundreds of ganglion cells in the nonhuman primate eye over periods as along as years. This approach is especially well-suited for recording from cells serving the central fovea, which has been difficult to access with microelectrodes. Using optogenetics, we can also directly excite these same ganglion cells with light in the living animal. These capabilities together establish a two-way communication link with retinal ganglion cells. I will discuss the advantages and the current limitations of these approaches, as well as speculate about possible future applications for vision restoration and understanding the role of the retina in perception.

My attempts to photograph the Earth from space using the moon as a camera, and several computational imaging projects resulting from those attempts.

Single-photon avalanche diodes (SPADs) are an emerging sensor technology capable of detecting and time-tagging individual photons with picosecond precision. Despite (or perhaps, due to) these capabilities, SPADs are considered specialized devices suitable only for photon-starved scenarios, and restricted to a limited set of niche applications. This raises the following questions: Can SPADs operate not just in low light, but in bright scenes as well? Can SPADs be used not just with precisely controlled active light sources such as pulsed lasers, but under passive, uncontrolled illumination like cellphone or machine vision cameras?
I will describe our recent work on designing computational imaging techniques that (a) enable single-photon sensors to operate across the entire gamut of imaging conditions including high-flux scenes, and (b) leverages SPADs as passive imaging devices for ultra-low light photography. The overall goal is to transform SPADs into all-weather, general-purpose sensors capable of both active and passive imaging, across photon-starved and photon-flooded environments.

With over 466 million people suffering from disabling hearing loss globally according to the World Health Organization, and the number expected to rise to 900 million people by 2050, hearing aids are crucially important medical wearable devices. Untreated hearing loss has been linked to increased risks of social isolation, depression, dementia, fall injuries, and other health issues. In this talk, we will present a new class of in-ear devices with embedded sensors and artificial intelligence, which are shaped to fit an individual with 3D-imaging of the ear geometry. In addition to providing frequency-dependent amplification of sound to compensate for hearing loss, these devices serve as a continuous monitor for important physiological parameters, an automatic fall detection and alert system, as well as a personal assistant with connectivity to the cloud. Furthermore, Bluetooth-paired with a vision aid, these devices present exciting possibilities for multisensory perceptual augmentation of hearing, balance, vision, and memory, helping people live better and more productive lives.

Retinal topography is affected by pathology such as drusen and tumors, and it may be useful to determine topography with fundus imaging when three-dimensional imaging is not available. In this talk, I will present a novel method of retinal topography scanning using the stripe projection technology of the CLARUSTM 700 (ZEISS, Dublin, CA) wide-field fundus camera. The camera projects stripes onto the retina and records images of the returned light while maintaining a small angle between illumination and imaging. We make use of this structured illumination, analyzing neighboring stripes to determine depth -i.e. the retinal topography – from both relative defocus and stripe displacements. The resulting topography maps are finally compared to three-dimensional data from optical coherence tomography imaging.

Single Photon Avalanche Detectors (SPADs) can detect single photon arrival events and in doing so, record a click that can be used to also determine the photon arrival time on the detector with picosecond temporal resolution. This timing capability is finding many uses such as LIDAR and fluorescence lifetime imaging and provides an opportunity for revisiting fundamental imaging concepts by combining SPAD data with computational image retrieval techniques. The computational techniques can in general resort to inverse retrieval approaches or machine learning, with the choice depending on the specific nature of the data and imaging problem at hand. I will overview some of our work, starting from the first attempts to capture light-in-flight using SPAD cameras and covering the topics of non-line-of-sight imaging, imaging through diffusion, extraction of 3D images from time-of-light data only, fluorescence lifetime imaging and coincidence counting for quantum imaging applications.

Additive manufacturing printing, i.e., 3-D printing, is one of the most important technological innovations in the past few decades. Among the various techniques, two-photon polymerization (TPP) is the most precise 3-D printing process that has been used to create many complex structures for advanced photonic and nanoscale applications, e.g., microrobots, optical memories, metamaterials, photonic crystals, and bio-scaffolds etc. However, to date the technology still remains a laboratory tool due to its high operation cost and limited fabrication rate, i.e., serial laser scanning process. In this seminar, I will present our recent work on the parallelization of the TPP process based on (1) temporal focusing and (2) binary holography, where programmable femtosecond light sheets or tens to hundreds of shaped laser beams are used to substantially improve the rate without sacrificing resolution. In addition, the engineered laser foci can improve the strength and structural integrity of the printed structures. Our experiments demonstrate arbitrarily complex structures can be fabricated at a record-breaking resolution and speed, i.e., lateral/axial resolution: 140 nm/175 nm at 10s mm3/min, which is 3-4 orders of magnitude higher than any existing fabrication methods. Our new methods provide an effective and low-cost solution to scale-up the fabrication of functional micro- and nano-structures (~$1.5/mm3). This means our technology may play a large role in fields such as healthcare, clean energy and water, computing, and telecommunications.

With multiple mechanisms of contrast, high sensitivity, high resolution, and the possibility to create miniature, inexpensive devices, light-based techniques have tremendous potential to positively impact cancer detection and survival. Many organs of the body can be reached in a minimally-invasive fashion with small flexible endoscopes. Some organs, such as the fallopian tubes and ovaries, require extremely miniature (sub-mm) and flexible endoscopes to avoid tissue cutting. Additionally, some modalities, such as side-viewing optical coherence tomography, are naturally suited to miniature endoscopes, whereas others like forward-viewing reflectance or fluorescence imaging, may require performance tradeoffs. The development of small, robust and fiber-delivered advanced light sources, miniature fiber bundles, and sensitive detectors has aided the development of novel miniature endoscopes. In this talk, I will discuss our recent advancements in endoscope design for multimodality optical early detection of ovarian cancer.