Asarkaya, Ahmet (University of Minnesota), Aksaray, Derya (University of Minnesota), Yazicioglu, Yasin (University of Minnesota)

Temporal-Logic-Constrained Hybrid Reinforcement Learning to Perform Optimal Aerial Monitoring with Delivery Drones

Scheduled for presentation during the Regular Session "Learning Methods II" (WeB2), Wednesday, June 16, 2021, 15:20−15:40, Kozani

2021 International Conference on Unmanned Aircraft Systems (ICUAS), June 15-18, 2021, Athens, Greece

This information is tentative and subject to change. Compiled on April 20, 2024

Abstract

In this paper, we consider a package delivery drone that is desired to simultaneously perform aerial monitoring as a secondary mission. To integrate this secondary mission, we utilize a reward function representing the value of information gathered via aerial monitoring. We use time window temporal logic (TWTL) specifications to define the pickup and delivery tasks while utilizing reinforcement learning (RL) to maximize the expected sum of rewards. The high-level decision-making of the drone is modeled as a Markov decision process (MDP). In this regard, we extend the previous work where a model-free RL algorithm was used to solve this optimization problem. We propose a modified Dyna-Q algorithm to address the shortage of online samples. We provide extensive simulation results to compare the performance of the model-free and hybrid RL algorithms in this application and investigate the effect of the different system parameters on the overall performance.