EE Student Information

The Department of Electrical Engineering supports Black Lives Matter. Read more.

• • • • •

EE Student Information, Spring Quarter through Academic Year 2020-2021: FAQs and Updated EE Course List.

Updates will be posted on this page, as well as emailed to the EE student mail list.

Please see Stanford University Health Alerts for course and travel updates.

As always, use your best judgement and consider your own and others' well-being at all times.

Statistics Department Seminar presents "Prevalence of neural collapse during terminal phase of deep learning training"

Prevalence of neural collapse during terminal phase of deep learning training
Tuesday, September 15, 2020 - 4:30pm
Meeting ID: 973 5368 4241 (+password)
David Donoho (Stanford)
Abstract / Description: 

Modern deep neural networks for image classification have achieved superhuman performance. Yet, the complex details of trained networks have forced most practitioners and researchers to regard them as blackboxes with little that could be understood. This talk considers in detail a now-standard training methodology: driving the cross-entropy loss to zero, continuing long after the classification error is already zero. Applying this methodology to an authoritative collection of standard deepnets and datasets, we observe the emergence of a simple and highly symmetric geometry of the deepnet features and of the deepnet classifier; and we document important benefits that the geometry conveys, thereby helping us understand an important component of the modern deep learning training paradigm.

This is joint work with Vardan Papyan, University of Toronto, and XY Han, Cornell.