ТРЁХФАЗОВОЕ ОБУЧЕНИЕ ИНТЕЛЛЕКТУАЛЬНОГО ЭНЕРГОМЕНЕДЖМЕНТА ГИБРИДНОЙ ЭНЕРГОСИСТЕМЫ С КЛИМАТО-ОСВЕДОМЛЁННЫМ ПРЕДСТАВЛЕНИЕМ СОСТОЯНИЯ

Abstract

A training methodology is proposed for the intelligent energy management system of a hybrid renewable energy system (HRES) with storage that overcomes the two principal obstacles to the direct application of reinforcement learning (RL) on operating equipment — sample inefficiency and the inadmissibility of unsafe exploration. The methodology is built on a three-phase cycle: offline – shadow mode – online. In the first phase, training is performed solely on synthetic telemetry of a digital twin: the reward function is recovered by maximum-entropy inverse reinforcement learning from expert trajectories of a stochastic predictive model, followed by behavior cloning and conservative offline learning. The second phase is a shadow mode on the operating plant without applying control signals, with divergence monitoring and twin adaptation. The third is cautious online fine-tuning under an external safety shield. A distinctive feature of the state representation is the inclusion of meteorological forecast uncertainty descriptors alongside point values, which renders the agent risk-sensitive. The problem is formulated as a constrained Markov decision process with a conditional value-at-risk cone for tail risks of imbalance. The object is an industrial HRES (250 kW, 60 kW, 100 kWh). The agent architecture, control-iteration pseudocode and a set of evaluation indicators are presented. The relevance of the methodology is determined by the need for pre-deployment training of intelligent control systems for the distributed energy of Kazakhstan under a sparse ground meteorological network.