EE Student Information

EE Student Information, Spring Quarter 19-20: FAQs and Updated EE Course List.

Updates will be posted on this page, as well as emailed to the EE student mail list.

Please see Stanford University Health Alerts for course and travel updates.

As always, use your best judgement and consider your own and others' well-being at all times.

IT-Forum

IT-Forum

Topic: 
Rate-distortion of sub-Nyquist sampled processes
Abstract / Description: 

Consider the task of analog to digital conversion in which a continuous time random process is mapped into a stream of bits. The optimal trade-off between the bitrate and the minimal average distortion in recovering the waveform from its bit representation is described by the Shannon rate-distortion function of the continuous-time source. Traditionally, in solving for the optimal mapping and the rate-distortion function we assume that the analog waveform has a discrete time version, as in the case of a band-limited signal sampled above its Nyquist frequency. Such assumption, however, may not hold in many scenarios due to wideband signaling and A/D technology limitations. A more relevant assumption in such scenarios is that only a sub-Nyquist sampled version of the source can be observed, and that the error in analog to digital conversion is due to both sub-sampling and finite bit representation. This assumption gives rise to a combined sampling and source coding problem, in which the quantities of merit are the sampling frequency, the bitrate and the average distortion.

In this talk we will characterize the optimal trade-off among these three parameters. The resulting rate-distortion-samplingFrequency function can be seen as a generalization of the classical Shannon-Kotelnikov-Whittaker sampling theorem to the case where finite bitrate representation is required. This characterization also provides us with a new critical sampling rate: the minimal sampling rate required to achieve the rate-distortion function of a Gaussian stationary process for a given rate-distortion pair. That is, although the Nyquist rate is the minimal sampling frequency that allows perfect reconstruction of a bandlimited signal from its samples, relaxing perfect reconstruction to a prescribed distortion allows sampling below the Nyquist rate while achieving the same rate-distortion trade-off.

Date and Time: 
Friday, October 23, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
Learning (from) networks: fundamental limits, algorithms, and applications
Abstract / Description: 

Network models provide a unifying framework for understanding dependencies among variables in medical, biological, and other sciences. Networks can be used to reveal underlying data structures, infer functional modules, and facilitate experiment design. In practice, however, size, uncertainty and complexity of the underlying associations render these applications challenging.

In this talk, we illustrate the use of spectral, combinatorial, and statistical inference techniques in several significant network science problems. First, we consider the problem of network alignment where the goal is to find a bijective mapping between nodes of two networks to maximize their overlapping edges while minimizing mismatches. To solve this combinatorial problem, we present a new scalable spectral algorithm, and establish its efficiency theoretically and experimentally over several synthetic and real networks. Next, we introduce network maximal correlation (NMC) as an essential measure to capture nonlinear associations in networks. We characterize NMC using geometric properties of Hilbert spaces and illustrate its application in learning network topology when variables have unknown nonlinear dependencies. Finally, we discuss the problem of learning low dimensional structures (such as clusters) in large networks, where we introduce logistic Random Dot Product Graphs, a new class of networks which includes most stochastic block models as well as other low dimensional structures. Using this model, we propose a spectral network clustering algorithm that possesses robust performance under different clustering setups. In all of these problems, we examine underlying fundamental limits and present efficient algorithms for solving them. We also highlight applications of the proposed algorithms to data-driven problems such as functional and regulatory genomics of human diseases, and cancer.

Date and Time: 
Thursday, October 15, 2015 - 3:00pm to 4:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
Communication in Strategic Environments: Crawford-Sobel Meet Shannon
Abstract / Description: 

Over thirty years ago, economists Vincent Crawford and Joel Sobel introduced the concepts of strategic information transmission (SIT) and cheap talk in their seminal Econometrica paper, as a way of understanding how information is strategically revealed (or not) by agents whose interests are only partially aligned. This theory has had tremendous success in explaining situations ranging from advertising to expert advice sharing, and many extensions of the original SIT model and the broader "principal- agent" class of problems have been extensively studied in the economics literature since. However, despite its name and even superficially obvious connection with information theory (IT), SIT has so far received very little attention from the IT community.

In this talk, I will present approaches to address such strategic communication problems from the lens of information and game theories. Specifically, I will focus on a strategic communication paradigm where the better-informed transmitter communicates with a receiver who makes the ultimate decision concerning both agents. While the economists have extensively studied the Nash equilibrium variant of this problem, the more relevant Stackelberg equilibrium enables the use of Shannon theoretic tools. I will present the fundamental limits of strategic compression and communication problems in the SIT context. Particularly, three problem settings will be considered, focusing on the quadratic distortion measures and jointly Gaussian variables: compression, communication, and the simple equilibrium conditions without any compression or communication. The analysis will then be extended to the receiver side information setting, where the strategic aspect of the problem yields rather surprising results regarding optimality of uncoded communication. Finally, several applications of the results within the broader context of decision theory will be presented.

Date and Time: 
Friday, October 9, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
Recursive Estimation and Application of Multivariate Markov Processes
Abstract / Description: 

A multivariate Markov process is a vector of random processes which are jointly, but not necessarily individually, Markov. Multivariate Markov processes form a rich family of mathematically tractable models, which have been found useful in numerous applications. Examples include the hidden Markov process in discrete-time, and the Markov modulated Poisson process in continuous-time. Multivariate Markov processes lend themselves to the mathematical tractability of Markov processes while allowing nonMarkovian features for the individual process components. For example, while the distribution of the sojourn time of a multivariate Markov chain in each of its states is geometric or exponential, the distribution of the sojourn time of an individual process component in each of its states is phase-type. In this talk we present forward recursions for some statistics of discrete and continuous-time multivariate Markov processes, which are relevant to their parameter estimation. We shall also briefly review applications in cognitive radio and network tomography.

This presentation is based on joint work with Brian L. Mark. This work was supported in part by the National Science Foundation under grant CCF- 0916568.

Date and Time: 
Friday, July 24, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
Deep Speech, a Deep Learning based speech recognition system
Abstract / Description: 

Speech recognition is still an unsolved problem in AI. Humans transcribe speech substantially better than machines, particularly when the speech is noisy, accented or spoken in a natural, unaffected manner. Over the past half-century slow yet steady progress has been made in speech recognition punctuated with rare breakthroughs including the Hidden Markov Model in the 70s and, more recently, Deep Learning.

In this talk I will briefly overview the current state of speech recognition technology and discuss the challenges that must be overcome in order to make progress and eventually approach human level performance. I will also focus on Deep Speech, a Deep Learning based speech recognition system built at Baidu Research's Silicon Valley AI lab. Key to the success of Deep Speech is a significantly simpler architecture than traditional speech systems, a well-optimized recurrent neural network training system that uses multiple GPUs and a set of novel data synthesis techniques. These ingredients taken together have shown great potential for rapid progress in speech recognition.

Date and Time: 
Friday, June 5, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
Capacity of Energy Harvesting Communication
Abstract / Description: 

Recent advances in energy harvesting technologies enable wireless devices to harvest the energy they need from the natural resources in their environment. This development opens the exciting new possibility to build wireless networks that are self-powered, self-sustainable and which have lifetimes limited by their hardware and not the size of their batteries.

However, energy harvesting also brings a fundamental shift in communication system design principles. In conventional systems, energy (or power) is a deterministic quantity continuously available to the transmitter and communication is typically constrained only in terms of average power. In harvesting systems, the energy available for communication is a stochastic rather than a deterministic process which has memory and is input-dependent. In this talk, we will investigate the information-theoretic capacity of this non-traditional communication system. We will characterize the capacity as an n-letter mutual information rate under various assumptions on the availability of energy arrival information and use these n-letter expressions to approximate the capacity in terms of a power control problem that has been extensively studied in the recent communication theory literature. We will then proceed to deriving an approximate solution to this power control problem which will allow us to approximate the capacity with a simple and insightful formula within a constant gap independent of system parameters.

Joint work with Dor Shaviv, Phan-Minh Nguyen and Yishun Dong.

Date and Time: 
Friday, May 29, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
A Topic Modeling Approach to Learning Preference-Behavior from Pairwise Comparisons
Abstract / Description: 

The recent explosion of web analytics tools has enabled us to collect an immense amount of partial preferences for large sets of items such as products from Amazon, movies from Netflix, or restaurants from Yelp, from a large and diverse population of users through transactions, clicks, etc. Modeling, learning, and ultimately predicting the preference behavior of users from pairwise comparisons has been extensively studied since the 1927 work of Thurstone. Yet, almost all models to date have been founded on a clustering-perspective in which users are grouped by their preference behavior.

We take a fundamentally different decomposition-perspective and propose a new class of generative models for pairwise comparisons in which user preference behavior can be decomposed into contributions from multiple shared latent "causes" (partial orders) that are prevalent in the population. We show how the estimation of shared latent partial orders in the new generative model can be formally reduced to the estimation of topics in a statistically equivalent topic modeling problem in which causes correspond to topics and item-pairs to words. We show that an inevitable consequence of having a relatively small number of shared latent causes in a world of large number of item-pairs is the presence of "novel" item-pairs for each latent cause. We then leverage recent advances in the topic modeling literature and develop an algorithm based on extreme-point identification of convex polytopes to learn the shared latent partial orders. Our algorithm is provably consistent and comes with polynomial sample and computational complexity guarantees. We demonstrate that our new model is empirically competitive with the current state-of-the-art approaches in predicting preferences on semi-synthetic and real world datasets.

This talk is based on joint work with Weicong Ding and Venkatesh Saligrama at Boston University.

Date and Time: 
Friday, May 22, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
Social Learning in Decision-Making Groups
Abstract / Description: 

People have always been influenced by the opinions of their acquaintances. Increasingly, through recommendations and ratings provided on all sorts of goods and services, people are also influenced by the opinions of people that are not even acquaintances. This ubiquity of the sharing of opinions has intensified the interest is the concept of herding (or informational cascades) introduced in 1992. While agents in most previous works have only individualistic goals, this talk focuses on social influence among agents in two collaborative settings.

We consider agents that perform Bayesian binary hypothesis testing and, in addition to their private signals, observe the decisions of earlier-acting agents. In the first setting, each decision has its own corresponding Bayes risk. Each agent affects the minimum possible Bayes risk for subsequent agents, so an agent may have a mixed objective including her own Bayes risk and the Bayes risks of subsequent agents; we demonstrate her tension between being informative to other agents and being right in her own decisions, and we show that she is more informative to others when she is open minded. In the second setting, opinions are aggregated by voting, and all agents aim to minimize the Bayes risk of the team's decision. We show that social learning is futile when the agents observe conditionally independent and identically distributed private signals (but not merely conditionally independent private signals) or when the agents require unanimity to make a decision. Our experiments with human subjects suggest that when opinions of people with equal qualities of information are aggregated by voting, the ballots should be secret. They have also raised questions about rationality and trust.

Date and Time: 
Friday, May 1, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

IT-Forum

Topic: 
Sub-optimality of the Han-Kobayashi region for the interference channel: Remarks on computing regions expressed by auxiliaries
Abstract / Description: 

A computable characterization of the capacity region for the two receiver interference channel, along with that of the two receiver broadcast channel, constitute two of the most fundamental open questions in network information theory. A computable achievable region proposed by Han and Kobayashi (1981) was the best-known achievable region and its optimality or sub-optimality had not been established prior to this work. In this work, we show that coding across time-slots can improve on Han-Kobayashi region (also known as multi-letter inner bounds).

The talk will be mainly about the various ideas and developments that eventually led to this result. It touches upon various computational aspects of computable regions or in other words, the study of extremal auxiliaries.

A similar result about Marton's achievable region for broadcast channels has not been found yet; and the difficulty lies precisely in the narrowing of extremal auxliaries, although significant progress has been made in this direction in the past few years.

This will be a whiteboard talk and though there is a rough outline, the talk can be steered by the audience.

Date and Time: 
Friday, May 15, 2015 - 1:00pm to 2:00pm
Venue: 
Packard 202

Pages

Subscribe to RSS - IT-Forum