d'Apolito, Francesco (AIT- Austrian Institute of Technology), Sulzbachner, Christoph (AIT Austrian Institute of Technology)

Three-Dimensional Waypoint Navigation of Multicopters by Attitude and Throttle Commands Using Off-Policy Reinforcement Learning

Scheduled for presentation during the Regular Session "Navigation" (FrB4), Friday, June 24, 2022, 13:10−13:30, Divona-2

2022 International Conference on Unmanned Aircraft Systems (ICUAS), June 21-24, 2022, Dubrovnik, Croatia

This information is tentative and subject to change. Compiled on April 26, 2024

Abstract

Artificial intelligence, in particular machine learning, is becoming increasingly important in automation and robotics. Machine learning approaches are also becoming more and more accepted in aviation. In particular, Reinforcement Learning is gaining more attention in navigation and control problems, for example in training flight manoeuvres. This paper aims to investigate the use of Off-Policy Reinforcement Learning techniques for three-dimensional waypoint navigation of multicopters by providing roll, pitch and throttle commands. It describes and compare the trainings performed using two well-known Off-Policy algorithms, namely the Deep Deterministic Policy Gradient (DDPG) and the Soft Actor Critic (SAC). Furthermore, we investigate the impact of the reward definition on the training outcome. For each of the used algorithm, two agents are trained with two different reward definitions. Finally, the paper shows the validations performed to evaluate the performance of the four trained agents under different known and unknown conditions. Their performances are evaluated and compared with respect to the training algorithm and the reward definition used.