EE Student Information

The Department of Electrical Engineering supports Black Lives Matter. Read more.

• • • • •

EE Student Information, Spring Quarter through Academic Year 2020-2021: FAQs and Updated EE Course List.

Updates will be posted on this page, as well as emailed to the EE student mail list.

Please see Stanford University Health Alerts for course and travel updates.

As always, use your best judgement and consider your own and others' well-being at all times.

ISL Colloquium presents "Optimization Aspects of Temporal Abstraction in Reinforcement Learning"

Optimization Aspects of Temporal Abstraction in Reinforcement Learning
Thursday, May 2, 2019 - 4:15pm
Packard 101
Pierre-Luc Bacon (Stanford)
Abstract / Description: 

Temporal abstraction refers to the idea that complicated sequential decision making problems can sometimes be simplified by considering the "big picture" first. In this talk, I will give an overview of some of my work on learning such temporal abstractions end-to-end within the "option-critic" architecture (Bacon et al., 2017). I will then explain how other related hierarchical RL frameworks, such as Feudal RL by Dayan and Hinton (1993), can also be approached under the same option-critic architecture. However, we will see that that this formulation leads to a so-called "bilevel" optimization problem. While this is a more difficult problem, the good news is that the literature on bilevel optimization is rich and many of its tools have yet to be re-discovered by our community. I will finally show how "iterative differentiation" techniques (Griewankand Walther, 2008) can be applied to our problem while providing a new interpretation to the "inverse RL" approach of Rust (1988).


Pierre-Luc Bacon is a postdoc in Emma Brunskill's group. He completed his PhD with Doina Precup at McGill University in 2018. His research focuses on temporal abstraction and representation learning in RL.