Abstract: Deep learning has enabled rapid advances in perception, planning, and natural language understanding for robots. However, current learning-based systems lack any formal assurances when required to generalize to novel scenarios. For example, perception systems can fail to identify or localize unfamiliar objects, and large language model (LLM)-based planners can hallucinate outputs that lead to unsafe outcomes when executed by robots. How can we rigorously quantify the uncertainty of machine learning components such that robots know when they don't know and can act accordingly?

In this talk, I will present our group's work on developing principled
theoretical and algorithmic techniques for providing formal assurances on learning-enabled robots that act based on rich sensory inputs (e.g.,
vision) and natural language instructions. The key technical insight is to
leverage and extend powerful methods from conformal prediction and
generalization theory for rigorous uncertainty quantification in a way that complements and scales with the growing capabilities of foundation models. I will present experimental validation of our methods for providing strong statistical guarantees on LLM planners that ask for help when they are uncertain, and for vision-based navigation and manipulation.

Bio: Anirudha Majumdar is an Assistant Professor at Princeton University in the Mechanical and Aerospace Engineering department and Associated Faculty in the Computer Science department. He also holds a part-time position as a Visiting Research Scientist at the Google AI Lab in Princeton. Majumdar received a Ph.D. in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology in 2016, and a B.S.E. in Mechanical Engineering and Mathematics from the University of Pennsylvania in 2011. Subsequently, he was a postdoctoral scholar at Stanford University from 2016 to 2017 at the Autonomous Systems Lab in the Aeronautics and Astronautics department.

He is a recipient of the Sloan Fellowship, ONR Young Investigator Program (YIP) award, the NSF CAREER award, the Google Faculty Research Award (twice), the Amazon Research Award (twice), the Young Faculty Researcher Award from the Toyota Research Institute, the Paper of the Year Award from the International Journal of Robotics Research (IJRR), the Best Conference Paper Award at the International Conference on Robotics and Automation (ICRA), and the Excellence in Teaching Award (Princeton SEAS).