A Strategic Day-ahead Bidding Strategy and Operation for Battery Energy Storage System by Reinforcement Learning

The Battery Energy Storage System (BESS) plays an essential role in the smart grid, and the ancillary market offers a high revenue. It is important for BESS owners to maximise their profit by deciding how to balance between the different offers and bidding with the rivals. Therefore, this paper formulates the BESS bidding problem as a Markov Decision Process (MDP) to maximise the total profit from the Automation Generation Control (AGC) market and the energy market, considering the factors such as charging/discharging losses and the lifetime of the BESS. In the proposed algorithm, function approximation technology is introduced to handle the continuous massive bidding scales and avoid the dimension curse. As a model-free approach, the proposed algorithm can learn from the stochastic and dynamic environment of a power market, so as to help the BESS owners to decide their bidding and operational schedules profitably. Several case studies illustrate the effectiveness and validity of the proposed algorithm.