EE380 Computer Systems Colloquium

Python is a dynamically typed language, and some of its appeal derives from this. Nevertheless, especially for large code bases, it would be nice if a compiler could find type errors before the code is even run. Optional static type checking promises exactly this, and over the past four years we have successfully introduced this feature into Python 3. This talk introduces the type system we've adopted and the syntax used for type annotations, some tips on how to get started with a large existing code base, and our experience using the 'mypy' type checker at Dropbox. The entire system is open source, and has also been adopted by other companies such as Lyft, Quora and Facebook.

Artificial intelligence has advanced rapidly in the last five years. This talk intends to provide high level answers to questions like:

• What can the evolution of intelligence in the animal kingdom teach us about the evolution of AI?
• How should people who are not AI researchers view the societal transformation that is now underway? What are some of the social, economic, and political implications of this technology as it exists now?
• What will future AI systems likely be capable of, and what are the largest expected impacts of these systems?

The talk will be understandable for non-computer scientists.

Neuroscience has entered a golden age in which experimental technologies now allow us to record thousands of neurons, over many trials during complex behaviors, yielding large-scale, high dimensional datasets. However, while we can record thousands of neurons, mammalian circuits controlling complex behaviors can contain tens of millions of behaviorally relevant neurons. Thus, despite significant experimental advances, neuroscience remains in a vastly undersampled measurement regime. Nevertheless, a wide array of statistical procedures for dimensionality reduction of multineuronal recordings uncover remarkably insightful, low dimensional neural state space dynamics whose geometry reveals how behavior and cognition emerge from neural circuits. What theoretical principles explain this remarkable success; in essence, how is it that we can understand anything about the brain while recording an infinitesimal fraction of its degrees of freedom?

We present a theory that addresses this question, and test it using neural data recorded from reaching monkeys. Overall, this theory yields a picture of the neural measurement process as a random projection of neural dynamics, conceptual insights into how we can reliably recover neural state space dynamics in such under-sampled measurement regimes, and quantitative guidelines for the design of future experiments. Moreover, it reveals the existence of phase transition boundaries in our ability to successfully decode cognition and behavior on single trials as a function of the number of recorded neurons, the complexity of the task, and the smoothness of neural dynamics. We will also discuss non-negative tensor analysis methods to perform multi-timescale dimensionality reduction and demixing of neural dynamics that reveal how rapid neural dynamics within single trials mediate perception, cognition and action, and how slow changes in these dynamics mediate learning.

A growing proportion of human activities such as social interactions, entertainment, shopping, and gathering information are now mediated by digital devices and services. Such digitally mediated activities can be easily recorded, offering an unprecedented opportunity to study and measure intimate psycho-demographic traits using actual--rather than self-reported--behavior. Our research shows that digital records of behavior, such as samples of text, Tweets, Facebook Likes, web-browsing logs, or even facial images can be used to accurately measure a wide range of traits including personality, intelligence, and political views. Such Big Data assessment has a number of advantages: it does not require participants' active involvement; it can be easily and inexpensively applied to large populations; and it is relatively immune to cheating or misrepresentation. If used ethically, it could revolutionize psychological assessment, marketing, recruitment, insurance, and many other industries. In the wrong hands, however, such methods pose significant privacy risks. In this talk, we will discuss how to reap the benefits of Big Data assessment while avoiding the pitfalls.

Circuit and design division at CEA LETI is focusing on innovative architectures and circuits dedicated to digital, imagers, wireless, sensors, power management and embedded software. After a brief overview of adaptive circuits for low power multi-processors and IoT architectures, the talk will detail new technologies opportunities for more flexibility. Digital and mixed-signal architectures using 3D technologies will be presented in the scope of multi-processors activity as well as imagers and neuro-inspired circuits. Also, the integration of non-volatile memories will be shown in the perspective of new architectures for computing. Finally, embedding learning will be addressed to solve power challenges at the edge and in end-devices: some new design approaches will be discussed.

Wireless communications are ubiquitous in the 21 st century--we use them to read the newspaper, talk to our colleagues or children, watch sporting events or other forms of entertainment, and to monitor and control the environment we live ins-- among just a few. This exponentiation of demand for wireless capacity has driven a new era of innovation in this space because spectrum and energy are expensive and constrained resources.

The future of wireless communications will demand leaps in spectrum efficiency, bandwidth efficiency, and power efficiency for successful technology deployments. Key applications that will fundamentally change how we interact with wireless systems and the demands we place on wireless technologies include Dynamic Spectrum Access Networks, massive MIMO, and the evasive unicorn of the "universal handset". While each of these breakthrough "system" capabilities make simultaneous demands of spectrum efficiency, bandwidth efficiency, and power efficiency, the current suite of legacy technologies forces system designers to make undesirable trade-offs because of the limitations of linear amplifier technology.

Eridan's solution is the antithesis of "linear". The Switch Mode Mixer Modulator (SMs3 ) technology emphasizes precision and flexibility, and simultaneously delivers spectrum efficiency, bandwidth efficiency, and power efficiency. The resulting capabilities dramatically increase total wireless capacity with minimum need for expanding operations into extended regions of the wireless spectrum.

This presentation will discuss the driving forces behind wireless system performance, the physics of linear amplifiers and SM3, measured performance of SM3 systems, and the implications for wireless system capabilities in the near future.

I will give introduction to blockchain technology, current state of the industry and its challenges and the Einsteinium Foundation that is embarking on a truly ambitious path likely to change how cryptocurrency is viewed and used in everyday life. Scientific research is a long-term investment in our future, and the future of our planet. Funding "big ideasi" has fallen dramatically around the world for in recent years. The defining characteristic of Einsteinium is its ongoing commitment to research and charitable missions. Einsteinium coin is a Bitcoin-like currency with a philanthropic objective of funding scientific, cutting edge IT and cryptocurrency projects.

A serious discussion of the role of the engineer as an art world serf in a world of dwindling practical applications.

Advances in information technology have provided many benefits for people with disabilities, including wide availability of textual content via text to speech, flexible control of motor wheelchairs, captioned video, and much more. People with cognitive disabilities benefit from easier communication, and better tools for scheduling and reminders. Will advances in machine learning enhance this impact? Progress in natural language processing, autonomous vehicles, and emotion detection, all driven by machine learning, may deliver important benefits soon. Further out, can we look for systems that can help people with cognitive challenges understand our complex world more easily, work more effectively, stay safe, and interact more comfortably in social situations? What are the technical barriers to overcome in pursuing these goals, and what are the theoretical developments in machine learning that may overcome them?

I will present a novel comprehensive theory of large scale learning with Deep Neural Networks, based on the correspondence between Deep Learning and the Information Bottleneck framework. The new theory has the following components:

1. rethinking Learning theory; I will prove a new generalization bound, the input-compression bound, which shows that compression of the representation of input variable is far more important for good generalization than the dimension of the network hypothesis class, an ill defined notion for deep learning.
2. I will prove that for large scale Deep Neural Networks the mutual information on the input and the output variables, for the last hidden layer, provide a complete characterization of the sample complexity and accuracy of the network. This makes the information Bottleneck bound for the problem as the optimal trade-off between sample complexity and accuracy with ANY learning algorithm.
3. I will show how Stochastic Gradient Descent, as used in Deep Learning, achieves this optimal bound. In that sense, Deep Learning is a method for solving the Information Bottleneck problem for large scale supervised learning problems. The theory provide a new computational understating of the benefit of the hidden layers, and gives concrete predictions for the structure of the layers of Deep Neural Networks and their design principles. These turn out to depend solely on the joint distribution of the input and output and on the sample size.

Based partly on works with Ravid Shwartz-Ziv and Noga Zaslavsky.