Lagunita Theme

Lecture Zoom

Zoom link and password can be found on canvas.

Useful Resources

• Piazza: link
• iGibson: website
• Google Cloud tutorial: repo
• Jupyter notebook tutorial (Wed 04/07 lecture): link

Course Description

This course provides a research survey of advanced methods for robot learning in simulation, analyzing the simulation techniques and recent research results enabled by advances in physics and virtual sensing simulation. The course covers two main components: agent-environment interactions and domains for multi-agent and human-robot interaction. First, we cover agent-environment interactions by studying novel simulation environments for robotics, imitation and reinforcement learning methods, simulation for navigation and manipulation and ‘sim2real’ techniques. In the second part, we explore models and algorithms for simulation and robot learning in multi-agent domains and human-robot interaction, studying the principles of learning for interactive tasks in which each agent collaborates to accomplish tasks. The topics include domains of social navigation, human-robot collaborative manipulation and multi-agent settings.
This a project-based seminar class. Projects will leverage the state-of-the-art simulation environment iGibson, in which students will develop simulations to explore learning and planning methods for diverse domains. We will provide a list of suggested projects but students might also propose an original idea. The course will cover a set of research papers with presentations by students. This is a research field in rapid transformation with exciting research lines. The goal of the class is to provide practical experience and understanding of the main research lines to enable students to conduct innovative research in this field.

Syllabus

Date	Presenter	Theme	Resources
Introduction
Mon 3/29		No Class
Wed 3/31	Savarese/Martín-Martín/D’Arpino	Introduction	Slides: Part [1] [2] [3]
Mon 04/05	Li	iGibson Introduction and Tutorial	Slides
Wed 04/07	Xia	iGibson Tutorial and Demos	Slides Tutorial
Part I: Agent-Environment Interaction
Mon 04/12	Martín-Martín	Simulation Environments for Robot Interaction
Reading: Habitat: A Platform for Embodied AI Research SAPIEN: A SimulAted Part-based Interactive ENvironment ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation AI2-THOR: An Interactive 3D Environment for Visual AI MuJoCo: A physics engine for model-based control
Wed 04/14	Students	Reinforcement learning and Imitation learning in Simulation
Reading: Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations Reinforcement and Imitation Learning for Diverse Visuomotor Skills AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers Human-in-the-Loop Imitation Learning using Remote Teleoperation
Mon 04/19	Students	Simulation for Manipulation and Mobile Manipulation
Reading: Learning Dexterous In-Hand Manipulation Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics 6-DOF GraspNet: Variational Grasp Generation for Object Manipulation ReLMoGen: Integrating Motion Generation in Reinforcement Learning for Mobile Manipulation
Wed 04/21	Students	Sim2Real techniques
Reading: RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Real Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World Sim-to-Real Transfer of Robotic Control with Dynamics Randomization Sim2Real Transfer for Reinforcement Learning without Dynamics Randomization Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
Mon 04/26	‪Erwin Coumans (Google)	Guest Lecture TBA
Wed 04/28	Xiangyun Meng (University of Washington)	Guest Lecture TBA
Part II: Multi-Agent and Human-Robot Interactions
Mon 05/03	D’Arpino	Robot Learning in Simulation for HRI & Social Navigation	Slides
Reading: VirtualHome: Simulating Household Activities via Programs Assistive VR Gym: Interactions with Real People to Improve Virtual Assistive Robots SEAN-EP: A Platform for Collecting Human Feedback for Social Robot Navigation at Scale Visual Navigation Among Humans with Optimal Control as a Supervisor Deep Local Trajectory Replanning and Control for Robot Navigation Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learning
Wed 05/05	Students	Agent Simulation and Interactive Tasks
Reading: Learning to Collaborate from Simulation for Robot-Assisted Dressing DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills Character Controllers Using Motion VAEs Learning Robust Control Policies for End-to-End
Mon 05/10	Roozbeh Mottaghi (AI2, University of Washington)	Guest Lecture TBA
Wed 05/12	Marynel Vázquez (Yale)	Guest Lecture TBA
Mon 05/17	Chuang Gan (MIT-IBM Watson AI Lab)	Guest Lecture TBA
Wed 05/19	Students	Robot Learning for Collaborative Manipulation
Reading: Human-centered collaborative robots with deep reinforcement learning Collaborative Behavior Models for Optimized Human-Robot Teamwork Reactive Human-to-Robot Handovers of Arbitrary Objects Helping Robots Learn: A Human-Robot Master-Apprentice Model Using Demonstrations via Virtual Reality Teleoperation
Mon 05/24	Nick Haber (Stanford)	Guest Lecture TBA
Course Project Presentation
Wed 05/26	Everyone	Final Project Office Hours
Mon 05/31		No Class (Memorial Day)
Wed 06/02	Students	Final Project Presentations

Evaluation:

• 30% Paper Presentation
• 40% Project
• 30% Class Participation