Graduate

IT-Forum presents Estimating the Information Flow in Deep Neural Networks

Topic: 
Estimating the Information Flow in Deep Neural Networks
Abstract / Description: 

This talk will discuss the flow of information and the evolution of internal representations during deep neural network (DNN) training, aiming to demystify the compression aspect of the information bottleneck theory. The theory suggests that DNN training comprises a rapid fitting phase followed by a slower compression phase, in which the mutual information I(X;T) between the input X and internal representations T decreases. Several papers observe compression of estimated mutual information on different DNN models, but the true I(X;T) over these networks is provably either constant (discrete X) or infinite (continuous X). We will explain this discrepancy between theory and experiments, and explain what was actually measured by these past works.

To this end, an auxiliary (noisy) DNN framework will be introduced, in which I(X;T) is a meaningful quantity that depends on the network's parameters. We will show that this noisy framework is a good proxy for the original (deterministic) system both in terms of performance and the learned representations. To accurately track I(X;T) over noisy DNNs, a differential entropy estimator tailor to exploit the DNN's layered structure will be developed and theoretical guarantees on the associated minimax risk will be provided. Using this estimator along with a certain analogy to an information-theoretic communication problem, we will elucidate the geometric mechanism that drives compression of I(X;T) in noisy DNNs. Based on these findings, we will circle back to deterministic networks and explain what the past observations of compression were in fact showing. Future research directions inspired by this study aiming to facilitate a comprehensive information-theoretic understanding of deep learning will also be discussed.

Date and Time: 
Wednesday, October 31, 2018 - 1:15pm
Venue: 
Packard 202

EE380 Computer Systems Colloquium presents "Partisan Gerrymandering and the Supreme Court: The Role of Social Science"

Topic: 
Partisan Gerrymandering and the Supreme Court: The Role of Social Science
Abstract / Description: 

***The talk for October 31, 2018 is drawn from our back list of videos and will not be a live presentation. This talk was originally given November 1, 2017. ***
We have been planning to have a speaker for this slot to address the issues of elections in a technological state. Despite many discussions and invitations, we have been unable to find anyone willing to take on speak about the current juncture of politics, technology, and economics.

The U.S. Supreme Court is considering a case this term, Gill v Whitford, that might lead to the first constitutional constraints on partisanship in redistricting. Eric McGhee is the inventor of the efficiency gap, a measure of gerrymandering that the court is considering in the case. He will describe the case's legal background, discuss some of the metrics that have been proposed for measuring gerrymandering, and reflect on the role of social science in the litigation.

Related NPR Science Friday Talk (Nov 3):

Does Math Have A Place In The Courtroom. Audio is 17 minutes.
So is it possible that these Ivy League-educated Supreme Court justices really don't understand the math of this case? Oliver Roeder, senior writer for FiveThirtyEight joins Ira to discuss whether the Supreme Court is allergic to math, and what that means for future cases. And Moon Duchin, associate professor of mathematics at Tufts University, returns to discuss the best math to use for rooting out gerrymandering.

Date and Time: 
Wednesday, October 31, 2018 - 4:30pm
Venue: 
Gates B03

AP483, Ginzton Lab, & AMO Seminar Series presents Impact of Structural Correlation and Monomer Heterogeneity in the Phase Behavior of Soft Materials and Chromosomal DNA

Topic: 
Impact of Structural Correlation and Monomer Heterogeneity in the Phase Behavior of Soft Materials and Chromosomal DNA
Abstract / Description: 

Polymer self-assembly plays a critical role in a range of soft-material applications and in the organization of chromosomal DNA in living cells. In many cases, the polymer chains are composed of incompatible monomers that are not regularly arranged along the chains. The resulting phase segregation exhibits considerable heterogeneity in the microstructures, and the size scale of these morphologies can be comparable to the statistical correlation that arises from the molecular rigidity of the polymer chains. To establish a predictive understanding of these effects, molecular models must retain sufficient detail to capture molecular elasticity and sequence heterogeneity. This talk highlights efforts to capture these effects using analytical theory and computational modeling. First, we demonstrate the impact of structural rigidity on the phase segregation of copolymer chain in the melt phase, resulting in non-universal phase phenomena due to the interplay of concentration fluctuations and structural correlation. We then demonstrate how these effects impact the phase behavior in statistical random copolymers and in heterogeneous copolymers based on chromosomal DNA properties. With these results, we demonstrate that the spatial segregation of DNA in living cells can be predicted using a heterogeneous copolymer model of microphase segregation.

Date and Time: 
Monday, November 5, 2018 - 4:15pm
Venue: 
Spilker 232

ISL Colloquium presents Taming the Devil of Gradient-based Optimization Methods with the Angel of Differential Equations

Topic: 
Taming the Devil of Gradient-based Optimization Methods with the Angel of Differential Equations
Abstract / Description: 

In this talk, we use ordinary differential equations to model, analyze, and interpret gradient-based optimization methods. In the first part of the talk, we derive a second-order ODE that is the limit of Nesterov's accelerated gradient method for non-strongly objectives (NAG-C). The continuous-time ODE is shown to allow for a better understanding of NAG-C and, as a byproduct, we obtain a family of accelerated methods with similar convergence rates. In the second part, we begin by recognizing that existing ODEs in the literature are inadequate to distinguish between two fundamentally different methods, Nesterov's accelerated gradient method for strongly convex functions (NAG-SC) and Polyak's heavy-ball method. In response, we derive high-resolution ODEs as more accurate surrogates for the three aforementioned methods. These novel ODEs can be integrated into a general framework that allows for a fine-grained analysis of the discrete optimization algorithms through translating properties of the amenable ODEs into those of their discrete counterparts. As the first application of this framework, we identify the effect of a term referred to as 'gradient correction' in NAG-SC but not in the heavy-ball method, shedding insight into why the former achieves acceleration while the latter does not. Moreover, in this high-resolution ODE framework, NAG-C is shown to boost the squared gradient norm minimization at the inverse cubic rate, which is the sharpest known rate concerning NAG-C itself. Finally, by modifying the high-resolution ODE of NAG-C, we obtain a family of new optimization methods that are shown to maintain the accelerated convergence rates as NAG-C for smooth convex functions. This is based on joint work with Stephen Boyd, Emmanuel Candes, Simon Du, Michael Jordan, and Bin Shi.

Date and Time: 
Thursday, November 1, 2018 - 4:15pm
Venue: 
Packard 101

SystemX Seminar presents Robot Reality Check

Topic: 
Robot Reality Check
Abstract / Description: 

Rich will provide an overview of the general status of the robotics industry and its impact in various market segments. He will also discuss his experiences in early stage robotics in Silicon Valley and review the landscape of emerging robotics companies. Finally, Rich will discuss his journey with Seismic and share insights on the company strategy and the new category of apparel called Powered Clothing.

Date and Time: 
Thursday, November 1, 2018 - 4:30pm
Venue: 
Huang 018

Nanoscale cascaded plasmonic logic gates for non-boolean wave computation

Topic: 
Nanoscale cascaded plasmonic logic gates for non-boolean wave computation
Abstract / Description: 

Several Beyond CMOS options for logic computing have been explored in the last decade. It is clear that beating ''ultimate CMOS'' is extremely hard when viewed at the functional level, where an algorithm with loops, control and arithmetic operations has to be executed in a given amount of time. Both the total area and total energy should be very reduced to improve beyond ultimate scaled CMOS. In this talk, wave based computing based on plasmonics technology will be discussed and evaluated. Based on some of the characteristics it may have some clear potential, though many challenges remain.

Date and Time: 
Monday, October 29, 2018 - 3:00pm
Venue: 
Gates 304

OSA/SPIE Seminar: Entanglement across disciplines

Topic: 
Entanglement across disciplines
Abstract / Description: 

As physicists or engineers we may be aware that philosophers and historians have long been interested in quantum theory and its potential ontological implications. Over the past few decades, diverse new branches of the humanities and social sciences have begun to grapple with aspects of quantum physics and to offer radical interpretive approaches. In this talk I'll briefly introduce some of these developments and then invite the audience to participate in an open discussion. The presentation will be non-technical in nature but I'll assume that everyone is familiar with the structure and application of quantum theory.

Date and Time: 
Wednesday, October 31, 2018 - 4:00pm
Venue: 
Spilker 232

IT-Forum presents Uncoupled isotonic regression and Wasserstein deconvolution

Topic: 
Uncoupled isotonic regression and Wasserstein deconvolution
Abstract / Description: 

Isotonic regression is a standard problem in shape-constrained estimation where the goal is to estimate an unknown nondecreasing regression function f from independent pairs (x_i,y_i) where 𝔼[y_i]=f(x_i), i=1,...n. While this problem is well understood both statistically and computationally, much less is known about its uncoupled counterpart where one is given only the unordered sets {x_1,...,x_n} and {y_1,...,y_n}. In this work, we leverage tools from optimal transport theory to derive minimax rates under weak moments conditions on y_i and to give an efficient algorithm achieving optimal rates. Both upper and lower bounds employ moment-matching arguments that are also pertinent to learning mixtures of distributions and deconvolution.

Date and Time: 
Friday, October 26, 2018 - 1:15pm
Venue: 
Packard 202

EE380 Computer Systems Colloquium presents "Ten Arguments for Deleting Your Social Media Accounts Right Now and other thoughts about Internet"

Topic: 
Ten Arguments for Deleting Your Social Media Accounts Right Now and other thoughts about Internet
Abstract / Description: 

TBA

Date and Time: 
Wednesday, October 24, 2018 - 4:30pm
Venue: 
Gates B03

Pages

Subscribe to RSS - Graduate