Vamvakas, Dimitrios (Democritus University of Thrace), Korkas, Christos (Center for Research & Technology Hellas), Tsaknakis, Christos (Democritus University of Thrace), Kosmatopoulos, Elias (Democritus University of Thrace and CERTH, Greece)

Hierarchical Reinforcement Learning for Optimal EV Charging: A Multi-Level Framework for Dynamic Pricing and Load Scheduling

Scheduled for presentation during the Regular Session "Networked systems" (WeCC), Wednesday, June 11, 2025, 17:50−18:10,

33rd Mediterranean Conference on Control and Automation, June 10-13, 2025, Tangier, Morocco

This information is tentative and subject to change. Compiled on May 9, 2025

Abstract

This paper explores the application of Hierarchical Reinforcement Learning (HRL) in optimizing electric vehicle (EV) charging, addressing challenges in load scheduling, energy cost management, and real-time dynamic pricing. We propose a hierarchical environment framework with two interconnected levels designed to efficiently manage charging demands across multiple Charging Stations (Chargym). The upper level focuses on dynamic pricing optimization, while the lower level handles load distribution among EVs. The Deep Deterministic Policy Gradient (DDPG) algorithm is implemented within this framework and evaluated against a baseline to assess its performance. Experimental results demonstrate that HRL effectively decomposes the complex EV charging problem into manageable subtasks, achieving improved efficiency in scheduling and pricing while ensuring cost-effective energy distribution. The upper-level DDPG agent in our formulation has shown a 35.58% improvement, and the overall DSO profit has shown a 101.04% improvement, when compared to the RBC baseline.