EE Student Information

ISL Colloquium: GLOM: Representing part-whole hierarchies in a neural network

GLOM: Representing part-whole hierarchies in a neural network
Monday, November 8, 2021 - 10:00am
Prof. Geoffrey Hinton (Univ of Toronto)
Abstract / Description: 

I will present a single idea about representation which allows advances made by several different groups to be combined into an imaginary system called GLOM. The advances include transformers, neural fields, contrastive representation learning, distillation and capsules. GLOM answers the question: How can a neural network with a fixed architecture parse an image into a part-whole hierarchy which has a different structure for each image? The idea is simply to use islands of identical vectors to represent the nodes in the parse tree. The talk will discuss the many ramifications of this idea. If GLOM can be made to work, it should significantly improve the interpretability of the representations produced by transformer-like systems when applied to vision or language.

Bio: Geoffrey Hinton is known by many to be the godfather of deep learning and was the recipient of the Turing award in 2018. Aside from his seminal 1986 paper on backpropagation, Hinton has invented several foundational deep learning techniques throughout his decades-long career. Hinton currently splits his time between the University of Toronto and Google Brain. He is a fellow of the UK Royal Society and has received numerous awards to acknowledge his work.