Workshop on Relational Reinforcement Learning

July 8, 2004
to be held in conjunction with

International Conference on Machine Learning
http://eecs.orst.edu/research/rrl/index.html
Banff, Alberta, Canada

 

  Program

  Accepted Papers

  Participants

  Proceedings

  Registration

  Organizing Committee

  Program Commitee

Relational Reinforcement Learning

Reinforcement learning has shown a lot of promise in recent years as a highly effective approach for building autonomous learning agents. Most work on reinforcement learning (and closely related decision-theoretic planning) is based on propositional or attribute-value representations of the state and actions and is inapplicable to domains with relational structure without extensive feature engineering. Nevertheless, good performance in such domains appears to be a critical feature of human intelligence. There is justifiable hope that relational representations allow reinforcement learning to converge faster to more general and robust policies.

Recently there have been a few successful attempts to generalize reinforcement learning to relational setting under the name of Relational Reinforcement Learning (RRL). A number of related efforts have begun to extend decision-theoretic planning techniques to relational domains---such work can also be viewed as indirectly applicable to relational reinforcement learning problems. The goal of this workshop is to foster more research in this area and to strengthen its connections to a number of different but related fields such as inductive logic programming, speedup learning, probabilistic relational modeling, and decision theoretic planning. Given the renewed interest in relational representations in machine learning and AI, this appears to be an appropriate time to encourage efforts to integrate these approaches into a comprehensive framework that includes expressive representations, inference, and action execution.

The issues the presentations might address include but are not limited to:

• What might we do with RRL that likely cannot be done with propositional reinforcement learning? What are some challenging test domains for RRL?
• How can we combine inference and planning with learning and acting to achieve both reactivity and deliberation?
• How can RRL systems best exploit decades of prior work on knowledge representation?
• When should a relational RL system use primarily propositional or explicit state-space methods as subroutines?
• Can RRL be effective in domains as structured as classical deterministic planning domains and their stochastic variants?

• How do value function and policy-search approaches compare?
• How can we relax the goal of optimality when it is intractable?
• How can we develop hierarchical methods?
• What kinds of function approximators work well?
• What are good methods for automatically constructing macro actions?
• What are good algorithms for tackling relational planning problems?
• What sort of world models are useful to learn?
• How can we best incorporate prior knowledge about the domain?
• How can RRL make progress in cases where the reward is sparse?
• Do techniques generalize to multi-agent domains such as games or internet trading?
• Can convergence and other theoretical guarantees be proven, and under what sort of assumptions?

School of Electrical Engineering and Computer Science, 1148 Kelley Engineering Center
Oregon State University, Corvallis, OR 97331-5501
Send a comment about this web site | This page was last modified on Friday, July 02, 2004
Copyright © 2009 | Disclaimer | Committed to Diversity