Shetty, S., Xue, T. and Calinon, S. (2024)
Generalized Policy Iteration using Tensor Approximation for Hybrid Control
In Proc. Intl Conf. on Learning Representations (ICLR).
Abstract
Approximate Reinforcement Learning (ARL) has demonstrated its scalability by solving some highly challenging problems in robotics by relying on Neural Networks (NN) for function approximation. However, the existing methods are primarily tailored for systems with either discrete or continuous action space, failing to consider hybrid action space. To address this, we present a novel algorithm called Generalized Policy Iteration using Tensor Train (TTPI) that belongs to the class of Approximate Dynamic Programming (ADP). We use a low-rank tensor approximation technique called Tensor Train (TT) to approximate the state-value and advantage function which enables us to efficiently handle hybrid systems. We demonstrate the superiority of our approach over previous baselines for some benchmark problems with hybrid action spaces. Additionally, the robustness and generalization of the policy for hybrid systems are showcased through a real-world robotics experiment involving a non-prehensile manipulation task which is considered to be a highly challenging control problem.
Bibtex reference
@inproceedings{Shetty24ICLR,
author="Shetty, S. and Xue, T. and Calinon, S.",
title="Generalized Policy Iteration using Tensor Approximation for Hybrid Control",
booktitle="Proc.\ Intl Conf.\ on Learning Representations ({ICLR})",
year="2024",
pages=""
}