Autonomous Flight Path Optimization for UAV under Management of Battery System via Reinforcement learning with Model Predictive Control

Abstract

Unmanned aerial vehicles (UAVs) have the potential to significantly reduce labor and risk by gathering information during disasters, serving as airborne base stations during emergencies, and as last-mile delivery vehicles. However, autonomously controlled UAVs consume large amounts of battery power, and energy constraints may limit their use. This study focuses on the energy constraint problem of UAVs and aims to obtain autonomous battery management for UAVs, which is difficult to model accurately because the battery depletion of UAVs is greatly influenced by external factors, making model-based control difficult. Therefore, this paper proposes a control model for UAVs that combines reinforcement learning and model predictive control, which do not require nominal models for optimization. Specifically, by introducing reinforcement learning into the UAV guidance system, the UAV's internal environment, i.e., the battery depletion function, is implicitly estimated, and destination directions are given in response to battery depletion. The UAV's control system uses model predictive control to accurately follow destination instructions from the guidance system. Experiments confirmed the acquisition of battery-aware autonomous flight with the proposed control model and the effectiveness of combining reinforcement learning and model-based learning.