Special Seminar: Outlier-aware optimizations of neural network design

Topic: 
Outlier-aware optimizations of neural network design
Wednesday, July 18, 2018 - 4:00pm
Venue: 
Gates 104
Speaker: 
Prof. Sungjoo Yoo (Seoul National University)
Abstract / Description: 

Outliers in weights and activations often prevent us from benefiting from very low precision, e.g., 2 or 4 bits. In this talk, we will present our recent works on hardware accelerator and low memory-cost training where we handle a very small amount of outliers, e.g., large values occupying only 1% of total data, in high precision and the majority of data in very low precision.

The accelerator, based on 4-bit computation, offers average 30% better performance and energy efficiency compared with the zero-skipping 8-bit accelerator. The low memory-cost training solution applies 3-bit precision to 98% of activations to be stored during training, which finally leads to 9X reduction in the memory cost of ResNet-152 while keeping the training accuracy.

Bio:

Professor Sungjoo Yoo received HIS Ph.D. at Seoul National University in 2000. From 2000-2004, he was a researcher at TIMA laboratory in Grenoble, France. In 2004, he joined Samsung Electronics, joining POSTECH (Pohang university of Science and TECHnology) in 2008. In 2015, he joined Seoul National University and is now full professor. Currently, he is on sabbatical at Facebook, Menlo Park as research scientist. His research interests include software/hardware co-design of machine learning applications.