SCIEN Talk

The availability of academic and commercial light field camera systems has spurred significant research into the use of light fields and multi-view imagery in computer vision and computer graphics. In this talk, we discuss our results over the past few years, focusing on a few themes. First, we describe our work on a unified formulation of shape from light field cameras, combining cues such as defocus, correspondence, and shading. Then, we go beyond photoconsistency, addressing non-Lambertian objects, occlusions, and an SVBRDF-invariant shape recovery algorithm. Finally, we show that advances in machine learning can be used to interpolate light fields from very sparse angular samples, in the limit a single 2D image, and create light field videos from sparse temporal samples. We also discuss recent work on combining machine learning with plenoptic sampling theory to create virtual explorations of real scenes from a very sparse set of input images captured on a handheld mobile phone.

The Zeiss Group develops, produces and distributes measuring technology, microscopes, medical technology, eyeglass lenses, camera and cinema lenses, binoculars and semiconductor manufacturing equipment. In this talk, I will present a novel webcam based optical 3D scanning method that allows independent surface mesh generation inside X-ray microscopes. These surface models can be used for collision avoidance and improved ease of use.

Autonomous mobile robots (e.g., self-driving cars, delivery trucks) are emerging and reshaping the world. However, there is one critical problem blocking the entire industry: how can mobile robots drive safely and naturally in the complex 3D world, and how do we know? In this talk, I will discuss our solution, called "training academy", to the aforementioned problem. We apply 3D vision, deep learning, and reinforcement learning techniques to generate real-world, high-fidelity driving scenarios and train the autonomous systems to develop human-like intelligence in a simulated environment.

Rendering refers to a process of creating digital images of an object or a scene from 3D data using computers and algorithms. Inverse rendering is the inverse process of rendering, i.e., reconstructing 3D data from 2D images. The 3D data to be recovered can be 3D geometry, reflectance of a surface, camera viewpoints, or lighting conditions.
In this talk, we will discuss three inverse rendering problems. First, inverse rendering using flash photography captures 3D geometry and reflectance of a static object using a single camera and a flashlight attached to the camera. An alternating and iterative optimization framework is proposed to jointly solve for several unknown properties. Second, inverse rendering at microscale reconstructs 3D normals and reflectance of a surface at microscale. A specially designed acquisition system, as well as an inverse rendering algorithm for microscale material appearance, are proposed. Lastly, inverse rendering for human hair describes a novel 3D reconstruction algorithm for modeling high-quality human hair geometry. We hope that our work on these advanced inverse rendering problems boosts hyper-realism in computer graphics

Observing large-scale three-dimensional subcellular dynamics in vivo at high spatiotemporal resolution has long been a pursuit for biology. However, both the signal-to-noise ratio and resolution degradation in multicellular organisms pose great challenges. In this talk, I will discuss our recent work in in vivo aberration-free 3D fluorescence imaging at millisecond scale by scanning light-field microscopy with digital adaptive optics. Specifically, we propose scanning light-field microscopy to achieve diffraction-limited 3D synthetic aperture for incoherent conditions, which facilitates real-time digital adaptive optics for every pixel in post-processing. Various fast subcellular processes are observed, including mitochondrial dynamics in cultured neurons, membrane dynamics in zebrafish embryos, and calcium propagations in cardiac cells, human cerebral organoids, and Drosophila larval neurons, enabling simultaneous in vivo studies of morphological and functional dynamics in 3D.

In this talk, I will present an overview of our most recent advancements in Robotic Imaging, Machine Leaning and Medical Augmented Reality. I will first discuss the particular requirements for intra-operative imaging and visualization. I will then present some of our latest results in intra-operative multimodal robotic imaging and its translation to clinical applications. I will then discuss the impact of research advancement in machine learning on medical imaging and computer assisted intervention. I will finally present some applications of virtual and augmented reality in the medical domain. Starting by the current deployment of AR and VR technology within medical education, I discuss its current and future impact on surgical education and training. I will then review the first deployment of augmented reality into operating rooms in the last two decades and present some of our latest achievements in this field (see also: www.medicalaugmentedreality.org.

Combining photography and spectroscopy, spectral imaging enables us to see what no traditional color camera has seen before. The current trend is to miniaturize the technology and bring it towards industry. In this talk, I will first give a general introduction to the most common pitfalls of spectral imaging and the challenges that come with miniaturization. Major pitfalls include balancing cross-talk, quantum efficiency, illumination and the optics. Miniaturization has become possible thanks to the monolithic per-pixel integration of thin-film Fabry-Pérot filters on CMOS imaging sensors. I will explain the difficulty of using these cameras with non- telecentric lenses. This is a major concern because of the angular dependency of the thin-film filters. I will demonstrate how this important issue can be solved using a model-based approach.

Deep learning algorithms offer a powerful means to automatically analyze the content of biomedical images. However, many biological samples of interest are difficult to resolve with a standard optical microscope. Either they are too large to fit within the microscope's field-of-view, or too thick, or are quickly moving around. In this talk, I will discuss our recent work in addressing these challenges by using deep learning algorithms to design new experimental strategies for microscopic imaging. Specifically, we use deep neural networks to jointly optimize the physical parameters of our computational microscopes - their illumination settings, lens layouts and data transfer pipelines, for example - for specific tasks. Examples include learning specific illumination patterns that can improve classification of the malaria parasite by up to 15%, and establishing fast methods to automatically track moving specimens across gigapixel-sized images.

Individuals of all genders invited to be a part of:
#StanfordToo: A Conversation about Sexual Harassment in Our Academic Spaces, where we will feature real stories of harassment at Stanford academic STEM in a conversation with Provost Drell, Dean Minor (SoM), and Dean Graham (SE3). We will have plenty of time for audience discussion on how we can take concrete action to dismantle this culture and actively work towards a more inclusive Stanford for everyone. While our emphasis is on STEM fields, we welcome and encourage participation from students, postdocs, staff, and faculty of all academic disciplines and backgrounds.

Handbooks are an essential requirement for understanding and using many artifacts found in our daily life. We use handbooks to understand how things work and how to maintain them. Most handbooks still exist on paper relying on graphical illustrations and accompanying textual explanations to convey the relevant information to the reader. With the success of video sharing platforms a large body of video tutorials available for nearly every aspect of life became available. Video tutorials can often expand printed handbooks with the demonstrations of actions required to solve certain tasks. However, interpreting printed manuals and video tutorials often requires a certain mental effort since users have to match printed images or video frames with the physical object in their environment.

Augmented Reality (AR) has been demonstrated to be effective of presenting information traditionally provided in printed handbooks and video tutorials. However, creating interactive illustrative graphics for AR is costly and requires specially trained authors. In this this talk, I will present research towards the automation of the authoring process of AR handbooks by interactively retargeting conventional, two-dimensional image and video data into three-dimensional AR handbooks. In addition, I will present interaction, visualization and rendering techniques tailored for AR handbooks.