EE Student Information

The Department of Electrical Engineering supports Black Lives Matter. Read more.

• • • • 

 EE Student Information, Spring & Summer Quarters 19-20: FAQs and Updated EE Course List.

Updates will be posted on this page, as well as emailed to the EE student mail list.

Please see Stanford University Health Alerts for course and travel updates.

As always, use your best judgement and consider your own and others' well-being at all times.

IT-Forum

IT Forum welcomes Paul Cuff, Renaissance Technologies

Topic: 
TBA
Abstract / Description: 

Soft covering is a phenomenon whereby an i.i.d. distribution on sequences of a given length is approximately produced from a very structured generation process. Specifically, a sequence is drawn uniformly at random from a "codebook" of sequences and then corrupted by memoryless noise (i.e. a discrete memoryless channel, or DMC). Among other things, soft covering means that the codebook is not recognizable in the resulting distribution. The soft covering phenomenon occurs when the codebook itself is constructed randomly, with a correspondence between the codebook distribution, the DMC, and the target i.i.d. distribution, and when the codebook is large enough. Mutual information is the minimum exponential rate for the codebook size. We show the exact exponential rate of convergence of the approximating distribution to the target, as measured by total variation distance, as a function of the excess codebook rate above mutual information. The proof involves a novel Poisson approximation step in the converse.

Soft covering is a crucial tool for secrecy capacity proofs and has applications broadly in network information theory for analysis of encoder performance. Wyner invented this tool for the purpose of solving a problem he named "common information." The quantum analogue of Wyner's problem is "entanglement of purification," which is an important open problem with physical significance: What is the minimum entanglement needed to produce a desired quantum state spanning two locations? The literature on this problem identifies sufficient asymptotic rates for this question that are generally excessively high. I will make brief mention of how the soft covering principle might be utilized for a more efficient design in this quantum setting.

Date and Time: 
Friday, January 25, 2019 - 1:15pm
Venue: 
Packard 202

IT Forum presents "Understanding the limitations of AI: When Algorithms Fail"

Topic: 
Understanding the limitations of AI: When Algorithms Fail
Abstract / Description: 

Automated decision making tools are currently used in high stakes scenarios. From natural language processing tools used to automatically determine one's suitability for a job, to health diagnostic systems trained to determine a patient's outcome, machine learning models are used to make decisions that can have serious consequences on people's lives. In spite of the consequential nature of these use cases, vendors of such models are not required to perform specific tests showing the suitability of their models for a given task. Nor are they required to provide documentation describing the characteristics of their models, or disclose the results of algorithmic audits to ensure that certain groups are not unfairly treated. I will show some examples to examine the dire consequences of basing decisions entirely on machine learning based systems, and discuss recent work on auditing and exposing the gender and skin tone bias found in commercial gender classification systems. I will end with the concept of an AI datasheet to standardize information for datasets and pre-trained models, in order to push the field as a whole towards transparency and accountability.

Date and Time: 
Friday, January 18, 2019 - 1:15pm
Venue: 
Packard 202

IT Forum & ISL presents Functional interfaces to compression (or: Down With Streams!)

Topic: 
Functional interfaces to compression (or: Down With Streams!)
Abstract / Description: 

From a computer-science perspective, the world of compression can seem like an amazing country glimpsed through a narrow straw. The problem isn't the compression itself, but the typical interface: a stream of symbols (or audio samples, pixels, video frames, nucleotides, or Lidar points...) goes in, an opaque bitstream comes out, and on the other side, the bitstream is translated back into some approximation of the input. The coding and decoding modules maintain an internal state that evolves over time. In practice, these "streaming" interfaces with inaccessible mutable state have limited the kinds of applications that can be built.

In this talk, I'll discuss my group's experience with what can happen when we build applications around compression systems that expose a "functional" interface, one that makes state explicit and visible. We've found it's possible to achieve tighter couplings between coding and the rest of an application, improving performance and allowing compression algorithms to be used in settings where they were previously infeasible. In Lepton (NSDI 2017), we implemented a Huffman and a JPEG encoder in purely functional style, allowing the system to compress images in parallel across a distributed network filesystem with arbitrary block boundaries (e.g., in the middle of a Huffman symbol or JPEG block). This free-software system is is in production at Dropbox and has compressed, by 23%, hundreds of petabytes of user files. ExCamera (NSDI 2017) uses a purely functional video codec to parallelize video encoding into thousands of tiny tasks, each handling a fraction of a second of video, much shorter than the interval between key frames, and executing with 4,000-way parallelism on AWS Lambda. Salsify (NSDI 2018) uses a purely functional video codec to explore execution paths of the encoder without committing to them, letting it match the capacity estimates from a transport protocol. This architecture outperforms "streaming"-oriented video applications -- Skype, Facetime, Hangouts, WebRTC -- in delay and visual quality. I'll briefly discuss some of our ongoing work in trying to compress the communication between a pair of neural networks jointly trained to accomplish some goal, e.g. to support efficient evaluation when data and compute live in different places. In general, our findings suggest that while, in some contexts, improvements in codecs may have reached a point of diminishing returns, compression *systems* still have plenty of low-hanging fruit.

Date and Time: 
Friday, January 11, 2019 - 1:15pm
Venue: 
Packard 202

John G. Linvill Distinguished Seminar on Electronic Systems Technology

Topic: 
Internet of Things and Internet of Energy for Connecting at Any Time and Any Place
Abstract / Description: 

In this presentation, I would like to discuss with you how to establish a sustainable and smart society through the internet of energy for connecting at any time and any place. I suspect that you have heard the phrase, "Internet of Energy" less often. The meaning of this phrase is simple. Because of a ubiquitous energy transmission system, you do not need to worry about a shortage of electric power. One of the most important items for establishing a sustainable society is [...]


"Inaugural Linvill Distinguished Seminar on Electronic Systems Technology," EE News, July 2018

 

Date and Time: 
Monday, January 14, 2019 - 4:30pm
Venue: 
Hewlett 200

CANCELLED! ISL & IT Forum present "Bayesian Suffix Trees: Learning and Using Discrete Time Series"

Topic: 
CANCELLED! Bayesian Suffix Trees: Learning and Using Discrete Time Series
Abstract / Description: 

CANCELLED!  We apologize for any inconvenience.

One of the main obstacles in the development of effective algorithms for inference and learning from discrete time series data, is the difficulty encountered in the identification of useful temporal structure. We will discuss a class of novel methodological tools for effective Bayesian inference and model selection for general discrete time series, which offer promising results on both small and big data. Our starting point is the development of a rich class of Bayesian hierarchical models for variable-memory Markov chains. The particular prior structure we adopt makes it possible to design effective, linear-time algorithms that can compute most of the important features of the resulting posterior and predictive distributions without resorting to MCMC. We have applied the resulting tools to numerous application-specific tasks, including on-line prediction, segmentation, classification, anomaly detection, entropy estimation, and causality testing, on data sets from different areas of application, including data compression, neuroscience, finance, genetics, and animal communication. Results on both simulated and real data will be presented.

Date and Time: 
Wednesday, December 12, 2018 - 3:00pm
Venue: 
Packard 202

IT Forum presents "Perceptual Engineering"

Topic: 
Perceptual Engineering
Abstract / Description: 

The distance between the real and the digital is clearest at the interface layer. The ways that our bodies interact with the physical world are rich and elaborate while digital interactions are far more limited. Through an increased level of direct and intuitive interaction, my work aims to raise computing devices from external systems that require deliberate usage to those that are truly an extension of us, advancing both the state of research and human ability. My approach is to use the entire body for input and output, to allow for implicit and natural interactions. I call my concept "perceptual engineering," i.e., a method to alter the user's perception (or more specifically the input signals to their perception) and manipulate it in subtle ways. For example, modifying a user's sense of space, place, balance and orientation or manipulating their visual attention, all without the user's explicit input, and in order to assist or guide their interactive experience in an effortless way.

I build devices and immersive systems that explore the use of cognitive illusions to manage attention, physiological signals for interaction, deep learning for automatic VR generation, embodiment for remote collaborative learning, tangible interaction for augmenting play, haptics for enhancing immersion, and vestibular stimulation to mitigate motion sickness in VR. My "perceptual engineering" approach has been shown to, (1) support implicit and natural interactions with haptic feedback, (2) induce believable physical sensations of motion in VR, (3) provide a novel way to communicate with the user through proprioception and kinesthesia, and (4) serve as a platform to question the boundaries of our sense of agency and trust. For decades, interaction design has been driven to answer the question: how can new technologies allow users to interact with digital content in the most natural way? If we look at the evolution of computing over the last 50 years, interaction has gone from punch cards to mouse and keyboard to touch and voice. Similarly, devices have become smaller and closer to the user's body. With every transition, the things people can do have become more personal. The main question that drives my research is: what is the next logical step?

Date and Time: 
Friday, December 7, 2018 - 1:15pm
Venue: 
Packard 202

IT-Forum presents Estimating the Information Flow in Deep Neural Networks

Topic: 
Estimating the Information Flow in Deep Neural Networks
Abstract / Description: 

This talk will discuss the flow of information and the evolution of internal representations during deep neural network (DNN) training, aiming to demystify the compression aspect of the information bottleneck theory. The theory suggests that DNN training comprises a rapid fitting phase followed by a slower compression phase, in which the mutual information I(X;T) between the input X and internal representations T decreases. Several papers observe compression of estimated mutual information on different DNN models, but the true I(X;T) over these networks is provably either constant (discrete X) or infinite (continuous X). We will explain this discrepancy between theory and experiments, and explain what was actually measured by these past works.

To this end, an auxiliary (noisy) DNN framework will be introduced, in which I(X;T) is a meaningful quantity that depends on the network's parameters. We will show that this noisy framework is a good proxy for the original (deterministic) system both in terms of performance and the learned representations. To accurately track I(X;T) over noisy DNNs, a differential entropy estimator tailor to exploit the DNN's layered structure will be developed and theoretical guarantees on the associated minimax risk will be provided. Using this estimator along with a certain analogy to an information-theoretic communication problem, we will elucidate the geometric mechanism that drives compression of I(X;T) in noisy DNNs. Based on these findings, we will circle back to deterministic networks and explain what the past observations of compression were in fact showing. Future research directions inspired by this study aiming to facilitate a comprehensive information-theoretic understanding of deep learning will also be discussed.

Date and Time: 
Wednesday, October 31, 2018 - 1:15pm
Venue: 
Packard 202

IT-Forum presents Uncoupled isotonic regression and Wasserstein deconvolution

Topic: 
Uncoupled isotonic regression and Wasserstein deconvolution
Abstract / Description: 

Isotonic regression is a standard problem in shape-constrained estimation where the goal is to estimate an unknown nondecreasing regression function f from independent pairs (x_i,y_i) where 𝔼[y_i]=f(x_i), i=1,...n. While this problem is well understood both statistically and computationally, much less is known about its uncoupled counterpart where one is given only the unordered sets {x_1,...,x_n} and {y_1,...,y_n}. In this work, we leverage tools from optimal transport theory to derive minimax rates under weak moments conditions on y_i and to give an efficient algorithm achieving optimal rates. Both upper and lower bounds employ moment-matching arguments that are also pertinent to learning mixtures of distributions and deconvolution.

Date and Time: 
Friday, October 26, 2018 - 1:15pm
Venue: 
Packard 202

IT-Forum: Arbitrarily Varying Broadcast and Relay Channels

Topic: 
Arbitrarily Varying Broadcast and Relay Channels
Abstract / Description: 

Two models of an arbitrarily varying channel (AVC) are studied; both are relevant to modern networks under jamming attacks by an adversary or a hacker. The arbitrarily varying broadcast channel is considered when state information is available at the transmitter in a causal manner. Inner and outer bounds are established, on both the random code capacity region and the deterministic code capacity region with degraded message sets. The form of the bounds raises the question whether the minimax theorem can be generalized to rate regions, i.e. whether the order of the intersection over state distributions and the union over Shannon strategies can be interchanged. A sufficient condition is given, under which this assertion holds and the random code capacity region is determined. As an example, the arbitrarily varying binary symmetric broadcast channel is examined, showing that there are cases where the condition holds, hence the capacity region is determined, and other cases where there is a gap between the bounds. The gap implies that the minimax theorem does not always hold for rate regions.

In the second part of the talk, a new model is introduced, namely, the arbitrarily varying relay channel. The results include the cutset bound, decode-forward bound and partial decode-forward bound on the random code capacity, which require modification of the usual methods for the AVC to fit the block Markov coding scheme. The random code capacity is further determined for special cases. Then, deterministic coding schemes are considered, and the deterministic code capacity is derived under certain conditions, for the degraded and reversely degraded relay channel, and the case of orthogonal sender components. The following question is addressed: If the encoder-decoder and encoder-relay marginals are both symmetrizable, does that necessarily imply zero capacity? We show and explain why the answer is no. The random code capacity is determined for the arbitrarily varying Gaussian relay channel with sender frequency division, and the deterministic code capacity is bounded using the techniques of Csisz\'ar and Narayan's 1991 paper on the Gaussian AVC. It is observed that the gap vanishes as the input becomes less constrained. It is commonly believed that the primitive relay channel "captures most essential features and challenges of relaying, and thus serves as a good testbed for new relay coding techniques" (Kim, 2007). It is observed that in the arbitrarily varying case, this may no longer be true.

This work is part of a Ph.D. thesis under the supervision of Yossef Steinberg.

Date and Time: 
Friday, October 12, 2018 - 1:15pm
Venue: 
Packard 202

IT-Forum: Structured Cooperation for Channels with Feedback and beyond

Topic: 
Structured Cooperation for Channels with Feedback and beyond
Abstract / Description: 

The capacities of fundamental communication problems such as channels with feedback and two-way communications channels are characterized with multi-letter expressions. The challenge in simplifying these expressions is their exhaustive dependence on all information that is accumulated throughout the communication. In this talk, we aim to simplify such capacities by imposing a structure on the accumulated data via a new sequential quantization technique on a directed graph.

First application of this method is for channels with memory and feedback. We will show upper and lower single-letter bounds on the capacity. The bounds are expressed with structured auxiliary random variable (r.v.), a notion that suits problems of sequential nature. For all cases where the capacity is known, the bounds are tight (with small cardinality of the structured auxiliary r.v.). This reveals a simple capacity formula that captures the major role of structure in feedback problems. We will also present a simple and sequential coding scheme, which is based on the posterior matching principle, and achieves the lower bound (and the capacity in many cases).

As time permits, we will show that structure is beneficial for other communication scenarios such as two-way communication channels with common outputs and the energy harvesting model.

The talk is based on a joint work with Prof. Henry Pfister (Duke Univeristy), Prof. Haim Permuter (BGU) and Prof. Navin Kashyap (IISc).

Date and Time: 
Monday, October 8, 2018 - 4:15pm
Venue: 
Packard 202

Pages

Subscribe to RSS - IT-Forum