EE Student Information

EE Student Information, Spring Quarter 19-20: FAQs and Updated EE Course List.

Updates will be posted on this page, as well as emailed to the EE student mail list.

As always, use your best judgement and consider your own and others' well-being at all times.

IT-Forum

IT-Forum

Topic:
From Wiener process to Bits (and back)
Abstract / Description:

What is the minimal mean squared error in recovering a path of the Brownian motion from any finite bit per time lag encoded version of it? Is it possible to attain a bounded mean squared error in this case?

Berger's PhD thesis answers these two question by providing a coding theorem for the Brownian motion, showing that its optimal distortion-rate performance are described by Shannon's distortion-rate function. That is, Berger's result implies that for any positive bitrate, there exists a coding scheme that allows recovering of the Brownian path with mean squared error given by Shannon's distortion-rate function (first evaluated by Iaglom).

Berger's result, however, does not take into account practical considerations in encoding analog process. Indeed, hardware and other implementation constraints restrict access to samples of the continuous-time path, taken at a finite time resolution. The question that we ask in thistalk is what is the minimal mean squared error in recovering the original Brownian path from a code that is only a function of these samples. This question is answered by deriving the distortion-rate function as a function of the sampling rate. It can be shown that the ratio between this function and the optimal distortion-rate performance without sampling constraint of Berger, are only a function of the number of bits per sample. For example, our result implies that using 1 bits per sample, one can attain distortion which is no more than 1.12 times the distortion-rate function.

Another interesting question arises in the following case: assume that the source encoder receives the discrete-time samples but is unaware of their underlying continuous-time origin. As a result, the encoder applies an optimal source code with respect to the empirical distribution of these samples, namely with respect to a discrete-time Brownian motion. We show that the maximal loss in this case, compared to an encoder that considers the continuous-time origin, does not exceed 1/6 times the intensity of the process.

The Information Theory Forum (IT-Forum) at Stanford ISL is an interdisciplinary academic forum which focuses on mathematical aspects of information processing. With a primary emphasis on information theory, we also welcome researchers from signal processing, learning and statistical inference, control and optimization to deliver talks at our forum. We also warmly welcome industrial affiliates in the above fields. The forum is typically held in Packard 202 every Friday at 1:00 pm during the academic year.

The Information Theory Forum is organized by graduate students Jiantao Jiao and Yanjun Han. To suggest speakers, please contact any of the students.

Date and Time:
Friday, October 21, 2016 - 1:00pm to 2:15pm
Venue:
Packard 202

IT-Forum

Topic:
Erasure coding for big-data systems: Theory and Practice
Abstract / Description:

Erasure codes are being increasingly deployed as an alternative to data replication in large-scale distributed storage systems to achieve fault tolerance in a storage-efficient manner. This paradigm shift has opened up exciting challenges and opportunities both on the theoretical as well as the system-design fronts. Specifically, while traditional codes are optimal in utilizing storage space, they significantly increase the usage of other important cluster resources such as network and device I/O. Furthermore, the usage of codes has primarily been limited to achieving space-efficient fault tolerance for storing "cold'' (less-frequently accessed) data, beyond which the potential of codes in big-data systems is largely unexplored.

We present new code constructions, and design and build erasure-coded storage systems, which provably reduce the usage of network and device I/O by a significant amount while not compromising on storage efficiency. Our codes have been evaluated on Facebook's data warehouse cluster in production, and will be a part of the next release of Apache Hadoop. Furthermore, we explore new avenues for coding in big-data systems, in particular for "hot'' (frequently accessed) data, by showing how codes can be employed to achieve significant benefits in load balancing and reducing latency in data-intensive cluster caches.

The Information Theory Forum (IT-Forum) at Stanford ISL is an interdisciplinary academic forum which focuses on mathematical aspects of information processing. With a primary emphasis on information theory, we also welcome researchers from signal processing, learning and statistical inference, control and optimization to deliver talks at our forum. We also warmly welcome industrial affiliates in the above fields. The forum is typically held in Packard 202 every Friday at 1:00 pm during the academic year.

The Information Theory Forum is organized by graduate students Jiantao Jiao and Yanjun Han. To suggest speakers, please contact any of the students.

Date and Time:
Friday, October 7, 2016 - 1:15pm to 2:15pm
Venue:
Packard 202

IT-Forum

Topic:
Part I: Wiretap Channels with Random States; Part II: Differential Privacy as a Mutual Information Constraint
Abstract / Description:

In the first part, we propose a new encoding scheme for the wiretap channel setting where a random state is known non-causally to the encoder but not necessarily to the decoder (the Gelfand-Pinsker setting). This setting combines two scenarios with celebrated results in information theory--the wiretap channel and the Gelfand-Pinsker channel--and it so happens that essentially the same encoding scheme is optimal for both scenarios individually. A decade ago, Chen and Han Vinck analyzed the secrecy rates achieved by that same encoding scheme. Quite surprisingly, Chia and El Gamal more recently showed that a different encoding scheme which explicitly extracts a key from the state to be used for encryption can sometimes outperform that scheme. In this work, we show a simple superposition code design which performs at least as well as both of these competing schemes, accomplishing the key agreement concept in a more subtle way through the selection of the auxiliary random variables.

In the second part, an information theoretic notion of differential privacy will be presented, which establishes an equivalence between differential privacy and a mutual information constraint. This intuitive definition in terms of mutual information admits very simple proofs of several important properties of differential privacy.

The Information Theory Forum (IT-Forum) at Stanford ISL is an interdisciplinary academic forum which focuses on mathematical aspects of information processing. With a primary emphasis on information theory, we also welcome researchers from signal processing, learning and statistical inference, control and optimization to deliver talks at our forum. We also warmly welcome industrial affiliates in the above fields. The forum is typically held in Packard 202 every Friday at 1:00 pm during the academic year.

The Information Theory Forum is organized by graduate students Jiantao Jiao and Yanjun Han. To suggest speakers, please contact any of the students.

Date and Time:
Friday, September 30, 2016 - 1:15pm to 2:15pm
Venue:
Packard 202

IT-Forum

Topic:
Deniable/Covert/Stealthy/LPD communication
Abstract / Description:

The urge to communicate, to speak and be heard, is a fundamental human need. However, embedded within our increasingly sophisticated communication networks, Big Brother is often watching. There are situations where even the fact that communication is happening (not just the content of that communication), can have real-world consequences. For instance, if you are a politically active citizen in an authoritarian society with broad censorship powers, the mere fact that you are communicating with the outside world can be construed by those authorities as sufficient justification for reprisals.

There has been a flurry of recent work in the information theory community dealing with this problem. In general, this problem encompasses models of communication with dual goals. Firstly, all communication from the source (Alice) to the destination (Bob) should be reliable, i.e., the communication protocol should be resilient to random noise, and perhaps even active jamming. Secondly, if the communication is overheard by a third party (Eve), it should be deniable from Eve. That is, not only should Eve should not learn anything about the source's message, in fact Eve should not even be able to reliably decide whether or not the source is indeed communicating to Bob. To be able to instantiate such deniability in communication, there need to be asymmetries that might exist between Bob and Eve (for instance, perhaps the noise on the channel to Eve is higher than the noise on the channel to Bob, or perhaps Eve observes only a subset of the transmissions that Bob does).

The tools used are typically information-theoretic and/or coding-theoretic in nature. Typically, deniability is formally defined in terms of a hypothesis-testing metric, and then one demands that the communication protocol that is reliable to Bob also have "high deniability", regardless of Eve's estimation strategy. Recently, in various communication settings (wired, wireless and quantum channels/networks), fundamental bounds on optimal rates for deniable communication, and also complementary schemes that are close to optimal.

Date and Time:
Friday, June 3, 2016 - 1:00pm to 2:00pm
Venue:
Packard 202

IT-Forum

Topic:
Cooperation Costs and Consequences
Abstract / Description:

In communication networks, cooperation is a strategy in which communicators exchange information not for the purpose of sending messages one to the other but instead for the purpose of enabling improved communication performance somewhere else in the network. For example, the transmitters of a multiple access channel may exchange information not because either has a message intended for the other but instead to increase the total rate at which both can communicate to the receiver. Measuring the cost of cooperation as the amount of information exchanged and the benefit of cooperation as the increase in sum-capacity to the intended receiver, we can ask simple yet surprisingly rich questions about when cooperation is worth its cost and how big the gains can be.

Date and Time:
Friday, May 20, 2016 - 1:00pm to 2:00pm
Venue:
Packard 202

Claude E. Shannon's 100th Birthday

Topic:
Centennial year of the 'Father of the Information Age'
Abstract / Description:

From UCLA Shannon Centennial Celebration website:

Claude Shannon was an American mathematician, electrical engineer, and cryptographer known as "the father of information theory". Shannon founded information theory and is perhaps equally well known for founding both digital computer and digital circuit design theory. Shannon also laid the foundations of cryptography and did basic work on code breaking and secure telecommunications.

Events taking place around the world are listed at IEEE Information Theory Society.

Date and Time:
Saturday, April 30, 2016 - 12:00pm
Venue:
N/A

IT-Forum

Topic:
On choice and communication in random matching markets
Abstract / Description:

We study the structure of stable matchings in two-sided random matching markets. First we show that even the slightest imbalance yields an essentially unique stable matching explaining why unique stable matching are empirically ubiquitous. This raises the question of how the composition of the market determines the set of potential matches. Towards this goal we study the communication requirements for reaching a stable matching in tiered random markets. We find that in such markets, a small amount of communication is needed from each agent to reach a stable matching with high probability. In particular, we construct a communication protocol that finds a stable matching with $O(\log^2 n)$ bits of communication on average per agent, and $O(\sqrt{n}\polylog (n))$ bits in worst case for an agent. Our results are tight in the sense that any communication protocol requires at least these costs, both on average and for the worst case agent.

Our construction reveals an illuminating structure of stable matchings. Agents ''apply'' to their favorite agents in a ''target tier'', which is the worst tier they can guarantee to be matched into. On the other hand, each agent maybe reached out by other agents from her top potential tier(s). Our results suggest that agents do not need to know their complete preferences, but only their favorite potential matches in their target tier and their preferences among agents who reach out to them.

Based on joint works with (i) Yash Kanoria and Jacob Leshno, and (ii) with Mark Braverman, Yash Kanoria and Peng Shi.

The Information Theory Forum (IT-Forum) at Stanford ISL is an interdisciplinary academic forum which focuses on mathematical aspects of information processing. With a primary emphasis on information theory, we also welcome researchers from signal processing, learning and statistical inference, control and optimization to deliver talks at our forum. We also warmly welcome industrial affiliates in the above fields. The forum is typically held in Packard 202 every Friday at 1:00 pm during the academic year.

The Information Theory Forum is organized by graduate students Jiantao Jiao and Yanjun Han. To suggest speakers, please contact any of the students.

Date and Time:
Friday, May 6, 2016 - 1:00pm to 2:00pm
Venue:
Packard 202

IT-Forum

Topic:
TBA
Abstract / Description:

Data in the form of noisy pairwise comparisons arises in many domains such as sports, crowdsourcing and operations research. Some fundamental inferential questions include estimating a total or partial order of the items, or estimating the set of underlying pairwise-comparison probabilities. Prior works on these topics often rely on restrictive "parametric" models, that often provide poor fits to the data. Instead, we consider much richer models that rely on "permutations" rather than parameters. We establish tight statistical (information-theoretic) guarantees on estimation under these models, showing that this increased flexibility in the models yields a significantly higher robustness to the estimation procedures, while remarkably, increasing the worst case risk by only logarithmic factors. We also discuss various associated computational challenges.

Joint work with Sivaraman Balakrishnan, Adityanand Guntuboyina and Martin J. Wainwright

The Information Theory Forum (IT-Forum) at Stanford ISL is an interdisciplinary academic forum which focuses on mathematical aspects of information processing. With a primary emphasis on information theory, we also welcome researchers from signal processing, learning and statistical inference, control and optimization to deliver talks at our forum. We also warmly welcome industrial affiliates in the above fields. The forum is typically held in Packard 202 every Friday at 1:00 pm during the academic year.

The Information Theory Forum is organized by graduate students Jiantao Jiao and Yanjun Han. To suggest speakers, please contact any of the students.

Date and Time:
Friday, April 29, 2016 - 1:00pm to 2:00pm
Venue:
Packard 202

IT-Forum

Topic:
Neural universal discrete denoising
Abstract / Description:

In this talk, I will present a novel framework of applying deep neural network (DNN) to the universal discrete denoising problem. DNN has recently shown remarkable performance improvements in diverse applications, and most of the success are based on the supervised learning framework. While successful in many applications, it is not straightforward to apply such framework to the universal discrete denoising problem, in which a denoiser tries to estimate an unknown finite-valued clean data based on its noisy observation. The reason is because the ground-truth label for a denoiser is the clean data subject to the estimation and is clearly not available for training a denoiser. In this work, I follow the framework of DUDE (Discrete Universal DEnoiser) and devise a novel way of training a DNN as a discrete denoiser solely based on the given noisy observation. The key idea is to utilize an unbiased estimate of the true loss of a denoiser and define a novel objective function for DNN based on the "pseudo-labels". The resulting scheme is dubbed as Neural DUDE, and the experimental results show that Neural DUDE significantly outperforms the original DUDE, which is the state-of-the-art on several discrete denoising problems. Furthermore, we show that Neural DUDE overcomes the critical limitation of DUDE, namely, it is much more robust to the choice of the hyper-parameter and has a concrete way of choosing the best hyper-parameter for given data. Such property makes Neural DUDE an attractive choice in practice. Finally, I will conclude with some potential future research directions, such as extending the framework to the denoising of continuous-valued data.

The Information Theory Forum (IT-Forum) at Stanford ISL is an interdisciplinary academic forum which focuses on mathematical aspects of information processing. With a primary emphasis on information theory, we also welcome researchers from signal processing, learning and statistical inference, control and optimization to deliver talks at our forum. We also warmly welcome industrial affiliates in the above fields. The forum is typically held in Packard 202 every Friday at 1:00 pm during the academic year.

The Information Theory Forum is organized by graduate students Jiantao Jiao and Yanjun Han. To suggest speakers, please contact any of the students.

Date and Time:
Friday, April 22, 2016 - 1:00pm to 2:00pm
Venue:
Packard 202

IT-Forum

Topic:
Securing massive MIMO at the Physical Layer
Abstract / Description:

Massive MIMO is one of the highlights of the envisioned 5G communication systems. In the massive MIMO paradigm, the base station is equipped with a number of antennas, typically much larger than the number of users served. While many issues behind the design of multicellular massive MIMO systems have been studied extensively, security of massive MIMO has not been addressed in most part. In this talk, we provide a brief introduction to physical layer security and massive MIMO before we discuss major vulnerabilities of massive MIMO as well as potential defense strategies.

To that end, we consider a single-cell downlink massive MIMO system in the presence of attackers capable of simultaneous jamming and eavesdropping. We show that, classical attacks of data jamming and data eavesdropping can be simultaneously rendered useless: as the number of antennas grow, with proper precoding and power control, mobiles can simultaneously achieve (1) full equivocation without the need to use wiretap encoding under data eavesdropping and (2) no-attack achievable rates under data jamming. However, we introduce a new attack strategy that involves jamming pilot signals and eavesdropping in succession and show significant reductions in the achievable secrecy rate, even in the asymptotic regime in the number of antennas. To counter this attack, we develop a defense strategy in which we use a secret key to encrypt the pilot sequence assignments, rather than encrypt the data. We show that hiding the training signal assignments from the attacker enables the mobiles to achieve secure degrees of freedom, identical to the achievable degrees of freedom under no attack.

Date and Time:
Friday, April 15, 2016 - 1:00pm to 2:00pm
Venue:
Packard 202