Header menu link for other important links
X
RAIL: Risk-Averse Imitation Learning
Published in International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
2018
Volume: 3
   
Pages: 2062 - 2063
Abstract
Imitation learning algorithms learn viable policies by imitating an expert's behavior when reward signals are not available. Generative Adversarial Imitation Learning (GALL) is a state-of-the-art algorithm for learning policies when the expert's behavior is available as a fixed set of trajectories. We evaluate in terms of the expert's cost function and observe that the distribution of trajectory-costs is often more heavy-tailed for GAIL-agents than the expert at a number of benchmark continuous-control tasks. Thus, high-cost trajectories, corresponding to tail-end events of catastrophic failure, are more likely to be encountered by the GAIL-agents than the expert. This makes the reliability of GAIL-agents questionable when it comes to deployment in risk-sensitive applications like robotic surgery and autonomous driving. In this work, we aim to minimize the occurrence of tail-end events by minimizing tail risk within the GALL framework. We quantify tail risk by the Conditional-Value-at-Risk (CVaR) of trajectories and develop the Risk-Averse Imitation Learning (RALL) algorithm. We observe that the policies learned with RAIL show lower tail-end risk than those of vanilla GAIL. Thus, the proposed RALL algorithm appears as a potent alternative to GALL for improved reliability in risk-sensitive applications. © 2018 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.
About the journal
JournalProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
ISSN15488403
Open AccessNo
Concepts (18)
  •  related image
    Autonomous agents
  •  related image
    Cost functions
  •  related image
    Multi agent systems
  •  related image
    Reinforcement learning
  •  related image
    Reliability
  •  related image
    Risk analysis
  •  related image
    ROBOTIC SURGERY
  •  related image
    Trajectories
  •  related image
    Value engineering
  •  related image
    AUTONOMOUS DRIVING
  •  related image
    Catastrophic failures
  •  related image
    CONDITIONAL VALUE-AT-RISK
  •  related image
    Continuous control
  •  related image
    IMITATION LEARNING
  •  related image
    Risk minimization
  •  related image
    SENSITIVE APPLICATION
  •  related image
    State-of-the-art algorithms
  •  related image
    Learning algorithms