IT-Forum

ISL & IT Forum present "Feedback capacity of channels with memory via Reinforcement Learning and Graph-based auxiliary random variable""

Topic: 
Feedback capacity of channels with memory via Reinforcement Learning and Graph-based auxiliary random variable
Abstract / Description: 

In this talk we present two novel ideas: the first is novel method to compute the feedback capacity of channels with memory using reinforcement learning (RL). The second is a new technique of using Graph-based auxiliary random variable to convert a multi-letter expression of feedback capacity formula into a single letter expression.

In RL, one seeks to maximize cumulative rewards collected in a sequential decision-making environment. This is done by collecting samples of the underlying environment and using them to learn the optimal decision rule. The main advantage of this approach is its computational efficiency, even in high dimensional problems. Hence, RL can be used to estimate numerically the feedback capacity of unifilar finite state channels (FSCs) with large alphabet size. The outcome of the RL algorithm sheds light on the properties of the optimal decision rule, which in our case, is the optimal input distribution of the channel.

The insights gained from the RL computation can be converted into analytic, single-letter capacity expressions by solving corresponding lower and upper bounds. The bounds are based on another novel idea of using Graph-based auxiliary random variable

We demonstrate the efficiency of this method by analytically solving the feedback capacity of the well-known Ising channel with a large alphabet. We also provide a simple coding scheme that achieves the feedback capacity.

 

Date and Time: 
Friday, August 2, 2019 - 3:00pm
Venue: 
Packard 202

ISL & IT Forum present "Resource-efficient quantized deep learning"

Topic: 
Resource-efficient quantized deep learning
Abstract / Description: 

Reducing the numerical precision of neural networks is one of the simplest, most effective and most common methods to improve resource efficiency (e.g. by reducing the memory and power requirements). Much research has been invested in finding how to quantize neural nets without significantly degrading performance. I will describe the main bottlenecks and solutions in various settings:
1) 1bit inference (1bit weights and activations), NIPS 2016 - link
2) 8bit training (8bit weights, activations, gradients, and batch-norm), NeurIPS 2018 - link
3) 4bit inference when quantization is done only post-training, Arxiv 2019 - link
4) Calculating the maximum trainable depth as a function of the numerical precision, Arxiv 2019 - link

Date and Time: 
Friday, July 26, 2019 - 2:00pm
Venue: 
Packard 202

ISL & IT Forum present "Towards Achieving Secure Consensus and Trusted Data Exchange for Multi-Robot Teams""

Topic: 
Towards Achieving Secure Consensus and Trusted Data Exchange for Multi-Robot Teams
Abstract / Description: 

As physical robot networks become more pervasive all around us, in the form of teams of autonomous vehicles, fleets of delivery drones, and smart and mobile IoT, it becomes increasingly critical to question the robustness of their coordination algorithms to security threats and/or corrupted data. Indeed, it has been shown that many multi-robot tasks easily fail in the presence of erroneous or hacked data. We investigate the vulnerabilities of important multi-robot algorithms such as consensus, coverage, and distributed mapping to malicious or erroneous data and we demonstrate the potential of communication to thwart certain attacks, such as the Sybil Attack, on these algorithms. Our key insight is that coordinated mobility can be combined with signal processing of communication signals to allow agents to learn important information about the environment and the nature of other agents in the network (for example the presence of cooperative versus adversarial agents). Along these lines, we will present a theoretical and experimental framework for provably securing multi-robot distributed algorithms through careful use of communication. We will present both theoretical results and experimental results on actual hardware implementations for bounding the influence of a Sybil Attack on consensus and on coverage by using observations over the wireless channels. In some cases, we show that the effect of a Sybil Attack can be nearly eliminated with high probability by deriving the appropriate switching function using a sufficient number of observations over the wireless network. Finally, we will briefly describe promising results on new methods for outlier rejection and active rendezvous in a pose graph optimization framework that exploits feedback gathered from communication channels to arrive at improved accuracy.

Date and Time: 
Wednesday, July 24, 2019 - 2:00pm
Venue: 
Packard 202

ISL & IT Forum present "Building a DNA information storage system from the bottom up"

Topic: 
Building a DNA information storage system from the bottom up
Abstract / Description: 

DNA has emerged as a compelling data storage medium due to its density, longevity, and eternal relevance compared to current memory technologies. However, the high price of synthesizing DNA (list price $3,500 per megabyte) remains a major bottleneck for adoption of this promising storage solution. In this talk, I will present our work towards breaking down this barrier using enzymatic DNA synthesize and a tailored codec for robust data retrieval. I will also touch upon some fundamental considerations when designing DNA information storage systems.

Date and Time: 
Friday, June 21, 2019 - 1:15pm
Venue: 
Packard 202

ISL & IT Forum present "Tensor networks and quantum Markov chain"

Topic: 
Tensor networks and quantum Markov chain
Abstract / Description: 

Tensor networks can often accurately approximate various systems that are of interest to physicists, but the computational cost of contracting the network is often prohibitively demanding. Quantum computer can resolve this bottleneck, but because the size of the computation scales with the size of the network, the existing quantum computers may appear to be far too noisy for this task. We prove a nontrivial error bound on the outcome of the computation that does not scale with the size of the network. This is possible because certain tensor networks are secretly implementing a quantum analogue of a (classical) Markovian dynamics. The long-time stability of the Markovian dynamics gives rise to a nontrivial error bound on the outcome of the computation. This suggests that there may be practical problems of interest that are amenable to relatively noisy quantum computers.

Date and Time: 
Friday, June 7, 2019 - 1:15pm
Venue: 
Packard 202

IT-Forum Student Talks

Topic: 
Student Talks
Abstract / Description: 

Jesse Gibson
Title: Information Theory in Next-Generation DNA Sequencing Technologies
The completion of the first human genome sequence in the early 2000's represented a major success for the scientific community. It also represented the return on a significant investment - billions of dollars and a little over a decade's worth of work. DNA sequencing costs have since fallen at a rate even faster than the exponential improvements predicted by Moore's Law in computing. Today, a complete human genome can be sequenced for closer to a few thousand dollars in about a week. A major factor in this improvement was the shift from the serial reads used in the human genome project to massively parallel sequencing technologies. These new technologies however present new problems - how can we understand a genome given millions of short reads drawn randomly from the underlying sequence and potentially degraded by noise in the process? Bioinformatics boasts a plethora of algorithms that meet this challenge in practice but it's not immediately obvious what 'optimal' analyses might look like. This talk will focus on the work of David Tse and others that have used the perspective of information theory to establish fundamental bounds on our ability to interpret next generation sequencing output. In particular, I will focus on the problems of assembling an unknown sequence de novo and of variant calling when the population of molecules being sequenced is not homogenous.

Tony Ginart
Title: Towards a Compression Theory for Metric Embeddings
Embedding matrices are widely used in diverse domains such as NLP, recommendation systems, information retrieval, and computer vision. For large-scale datasets, embedding matrices can consume large amounts of memory, and compression for these objects is desirable. However, the relevant distortion metrics applicable to embeddings make them significantly more compressible than classical sources considered in information theory. Furthermore, related results such as the Johnson-Lindenstrauss theorem are formulated in terms of reduction in dimension rather than codelength. Additionally, embeddings come with certain query-time constraints, such as in the sense of a succinct data structure. In this talk, we formulate the problem of metric embedding compression from the information theoretic lense and review related literature in information theory, signal processing, and dimensionality reduction. We adapt pre-existing results to establish some lower and upper bound in some regimes. Finally, we cover some of the state-of-the-art compression algorithms used to compress embeddings in practice.

Date and Time: 
Tuesday, June 4, 2019 - 10:00am
Venue: 
Packard 214

ISL & IT Forum present "String reconstruction problems inspired by problems in multiomics data processing"

Topic: 
String reconstruction problems inspired by problems in multiomics data processing
Abstract / Description: 

String reconstruction problems frequently arise in many areas of genomic data processing molecular storage, and synthetic biology. In the most general setting, they may be described as follows: one is given a single or multiple copies of a coded or uncoded string, and the copies are subsequently subjected to some form of (random) processing such as fragmentation or repeated transmission through a noise-inducing channel. The goal of the reconstruction method is to obtain an exact or approximate version of the string based on the processed outputs. Examples of string reconstruction questions include reconstruction from noisy traces, reconstruction from substrings and k-decks and reconstruction from compositional substring information. We review the above and some related problems and then proceed to describe coding methods that lead to strings that can be reconstructed exactly from their noisy traces, substrings and compositions. In particular, we focus on DNA profile codes, hybrid reconstruction from traces and uniquely reconstructable code designs. In the process, we introduce new questions in the areas of restricted de Bruin graphs, counting of rational points in polytopes, and string replacement methods.

This is a joint work with Ryan Gabrys, Han Mao Kiah, Srilakshmi Pattabiraman and Gregory Puleo.

Date and Time: 
Friday, May 31, 2019 - 1:15pm
Venue: 
Packard 202

IT Forum & ISL Colloquium present "Extracting information from cells"

Topic: 
Extracting information from cells
Abstract / Description: 

Genomic approaches to studying the molecular biology of the cell have advanced considerably over the pas few years, and it is now possible to make high-throughput measurements of molecules in individual cells. I will discuss some of the considerations that must be taken into account in designing experiments, and subsequently in extracting the maximum information from them (optimally).

Date and Time: 
Friday, May 24, 2019 - 1:15pm
Venue: 
Packard 202

IT Forum welcomes Jiantao Jiao (UC Berkeley)

Topic: 
Deconstructing Generative Adversarial Networks
Abstract / Description: 

We deconstruct the Generative Adversarial Networks (GANs) to its three fundamental problems to study: formulation, generalization, and optimization. We propose systematic principles to formulate the population goals of GANs (when infinite samples are available), and reveal and further develop connections between GANs and robust statistics. We provide principled methods to achieve the population formulations of GANs given finite samples with small generalization error, and demonstrate the intricacy of moving from infinite samples to finite samples in statistical error. We show through examples the importance of solving the inner maximization problem before the outer minimization problem, and demonstrate embedding the knowledge of the solution of the inner maximization problem could make a locally unstable algorithm globally stable. Joint work with Banghua Zhu and David Tse.

Date and Time: 
Tuesday, May 21, 2019 - 10:00am
Venue: 
Packard 214

Pages

Subscribe to RSS - IT-Forum