SCIEN Talk

Neural networks have surpassed the performance of virtually any traditional computer vision algorithm thanks to their ability to learn priors directly from the data. The common encoder/decoder with skip connections architecture, for instance, has been successfully employed in a number of tasks, from optical flow estimation, to image deblurring, image denoising, and even higher level tasks, such as image-to-image translation.

To improve the results further, one must leverage the constraints of the specific problem at hand, in particular when the domain is fairly well understood, such as the case of computational imaging.

In this talk I will describe a few of my recent projects that build on this observation, ranging from reflection removal, to novel view synthesis, and deblurring.

Disney Research has been actively pushing the state-of-the-art in digitizing humans over the past decade, impacting both academia and industry. In this talk I will give an overview of a selected few projects in this area, from research into production. I will be talking about photogrammetric shape acquisition and dense performance capture for faces, eye and teeth scanning and parameterization, as well as physically based capture and modelling for hair and volumetric tissues.

Precise determination of the position of a single point source (e.g. fluorescent molecule/protein, quantum dot) is at the heart of microscopy methods such as single particle tracking and super-resolution localization microscopy ((F)PALM, STORM). Localizing a point source in all three dimensions, i.e. including depth, poses a significant challenge; the depth of field of a standard high-NA microscope is fundamentally limited, and its pointspread-function (PSF), namely, the shape that a point source creates in the image plane, contains little information about the emitter's depth. Various techniques exist that enable 3D localization, prominent among them being PSF engineering, in which the PSF of a microscope is modified to encode the depth of the source. This is achieved by shaping the wavefront of the light emitted from the sample, using a phase mask in the pupil (Fourier) plane of the microscope.

In this talk, I will describe how our search for the optimal PSF for 3D localization, using tools from estimation theory, led to the development of microscopy systems with unprecedented capabilities in terms of depth of field and spectral discrimination. Such methods enable fast, precise, non-destructive localization in thick samples and in multicolor. Applications of these novel advances will be demonstrated, including super-resolution imaging, tracking biomolecules in living cells and microfluidic flow profiling. I will also present our most recent results: 1. Application of deep learning for solving difficult localization problems (high density, low SNR, multicolor imaging), and 2. Precise refractometry from minute volumes by super-critical-angle fluorescence.

Exploiting to see beyond the diffraction-limit of optical microscopies is of great significance. State-of-the-art solutions of super-resolution microscopy, like STED and STORM approaches, rely on the fluorescent effect of labeling samples. It is still challenging to obtain the super-resolution for unlabeling samples without fluorescent effect. To this end, we have developed a novel super-resolution method, called Spatial Frequency Shift (SFS), to realize the deep super-resolution with or without fluorescent effect in wide field imaging. The principle and the applications of this SFS technique will be presented.

Accurate and reliable 3D perception is the key remaining bottleneck to making self-driving vehicles safe and ubiquitous. Today, it's relatively easy to get an autonomous car to work 99% of the time, but it's the incredibly long tail of edge cases that's preventing them from reaching real-world deployment without a backup driver constantly watching over. All of this comes down to how well the autonomous car can see and understand the world around it. The key to achieving accurate, safer-than-human level 3D perception all starts with the LiDAR. That said, both legacy LiDAR solutions and newer upstarts, which largely leverage off-the-shelf components, have still struggled to meet the stringent performance requirements needed to solve key edge cases encountered in everyday driving scenarios.
Luminar, founded in 2012 by Austin Russell, has taken an entirely new approach to LiDAR, building its' system from the ground up at the component level for over 5 years. The result was the first and only solution that meets and exceeds all of the key performance requirements demanded by Car/Truck OEM's and technology leaders to achieve safe autonomy, in addition to unit economics that can enable widespread adoption across even mainstream consumer vehicle platforms. This has culminated with last years' release of their first scalable product for autonomous test and development fleets, which has subsequently led to rapidly accelerating adoption in the market. During this talk, raw Luminar LiDAR data from autonomous test vehicles will be presented to the audience, demonstrating real world examples of life-threatening edge cases and how they can now be avoided.

Encoded microparticles have become a powerful tool for a wide array of applications, including high-throughput sample tracking and massively parallel biological multiplexing. Spectral encoding, where particles are encoded with distinct luminescence spectra, provides a particularly appealing encoding strategy because of the ease of reading codes and assay flexibility. We recently developed a microfluidic method for producing microparticles with > 1,100 spectral codes by ratiometrically embedding different amounts of lanthanide nanophosphorsn within them, which we term MRBLEs (Microspheres with Ratiometric Barcode Lanthanide Encoding). We are now applying these MRBLEs towards a wide variety of biological problems, from high-throughput and quantitative profiling of protein-peptide interactions to specific and sensitive detection of multiple bacterial species from blood for fast diagnosis of sepsis.

We start to lack the processing power and bandwidth to drive 8K and high-resolution head-mounted displays. However, as the human eye and visual system have their own limitations, the relevant question is what spatial and temporal resolution is the ultimate limit for any technology. In this talk, I will review the visual models of spatio-temporal and chromatic contrast sensitivity which can explain such limitations. Then I will show how they can be used to reduce rendering cost in VR-applications, find more efficient encoding of high dynamic range images and compress images in a visually lossless manner.

Image domain transfer includes methods that transform an image based on an example, commonly used in photorealistic and artistic style transfer, as well as learning-based methods that learn a transfer function based on a training set. These are usually based on generative adversarial networks (GANs), and can be supervised or unsupervised as well as unimodal or multimodal. I will present a number of our recent methods in this space that can be used to translate, for instance, a label map to a realistic street image, a day time street image to a night time street image, a dog to different cat breeds, and many more.

In this presentation, I would like to discuss with you how to establish a sustainable and smart society through the internet of energy for connecting at any time and any place. I suspect that you have heard the phrase, "Internet of Energy" less often. The meaning of this phrase is simple. Because of a ubiquitous energy transmission system, you do not need to worry about a shortage of electric power. One of the most important items for establishing a sustainable society is [...]

"Inaugural Linvill Distinguished Seminar on Electronic Systems Technology," EE News, July 2018

SCIEN Industry Affiliates Meeting gives you the opportunity to meet new SCIEN faculty and the postdocs and graduate students who are working in image systems engineering, with expertise in optics, computational imaging, human vision and machine learning. Read Poster Abstracts and bios.

Registration is required