1.School of Automation, Southeast University, Nanjing 210096, China;2.Key Laboratory of Measurement and Control of Complex System of Engineering,Southeast University, Nanjing 210096, China;3.School of Electrical Engineering, Southeast University, Nanjing 210096, China

An environmental modeling method suitable for reinforcement learning is proposed for the charging path planning problem of electric vehicles. Based on the actual situation of urban road network and geographical distribution of charging stations, this method divides the basic driving path of electric vehicles into three segments for representation. Based on the three-segment expression method, the design scheme of state space, action space, state transition, and reward function is proposed. The charging path planning is modeled as a Markov decision process, and solved by the Q learning method and the deep Q network (DQN) method. The experimental results show that the design scheme of the reinforcement learning environment based on the three-segment expression method is solvable and portable. It takes into account the realistic scenarios such as the deceleration and turning of electric vehicles in the process of driving from the road to the charging station, and simplifies the charging action into a driving direction choice, which improves the efficiency of the reinforcement learning algorithm based on Q learning and DQN.