1. bookVolume 31 (2021): Issue 3 (September 2021)
Journal Details
License
Format
Journal
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
access type Open Access

An Automated Driving Strategy Generating Method Based on WGAIL–DDPG

Published Online: 27 Sep 2021
Page range: 461 - 470
Received: 22 Jan 2021
Accepted: 12 Jul 2021
Journal Details
License
Format
Journal
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
Abstract

Reliability, efficiency and generalization are basic evaluation criteria for a vehicle automated driving system. This paper proposes an automated driving decision-making method based on the Wasserstein generative adversarial imitation learning–deep deterministic policy gradient (WGAIL–DDPG(λ)). Here the exact reward function is designed based on the requirements of a vehicle’s driving performance, i.e., safety, dynamic and ride comfort performance. The model’s training efficiency is improved through the proposed imitation learning strategy, and a gain regulator is designed to smooth the transition from imitation to reinforcement phases. Test results show that the proposed decision-making model can generate actions quickly and accurately according to the surrounding environment. Meanwhile, the imitation learning strategy based on expert experience and the gain regulator can effectively improve the training efficiency for the reinforcement learning model. Additionally, an extended test also proves its good adaptability for different driving conditions.

Keywords

Anderson, C.W., Lee, M. and Elliott, D.L. (2015). Faster reinforcement learning after pretraining deep networks to predict state dynamics, 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland, pp. 1–7. Search in Google Scholar

Bai, Z., Shangguan, W., Cai, B. and Chai, L. (2019). Deep reinforcement learning based high-level driving behavior decision-making model in heterogeneous traffic, 2019 Chinese Control Conference (CCC), Guangzhou, China, pp. 8600–8605. Search in Google Scholar

Chang, B.-J., Hwang, R.-H., Tsai, Y.-L., Yu, B.-H. and Liang, Y.-H. (2019). Cooperative adaptive driving for platooning autonomous self driving based on edge computing, International Journal of Applied Mathematics and Computer Science 29(2): 213–225, DOI: 10.2478/amcs-2019-0016. Search in Google Scholar

Chen, X., Tian, G., Miao, Y. and Gong, J.-w. (2017). Driving rule acquisition and decision algorithm to unmanned vehicle in urban traffic, Transactions of Beijing Institute of Technology 37(5): 491–496. Search in Google Scholar

Dossa, R.F., Lian, X., Nomoto, H., Matsubara, T. and Uehara, K. (2020). Hybrid of reinforcement and imitation learning for human-like agents, IEICE Transactions on Information and Systems 103(9): 1960–1970. Search in Google Scholar

Gao, H., Shi, G., Wang, K., Xie, G. and Liu, Y. (2019). Research on decision-making of autonomous vehicle following based on reinforcement learning method, Industrial Robot: The International Journal of Robotics Research and Application 46(3): 444–452. Search in Google Scholar

Gu, X., Han, Y. and Yu, J. (2020). Vehicle lane-changing decision model based on decision mechanism and support vector machine, Journal of Harbin Institute of Technology 52(07): 111–121. Search in Google Scholar

Hedjar, R. and Bounkhel, M. (2019). An automatic collision avoidance algorithm for multiple marine surface vehicles, International Journal of Applied Mathematics and Computer Science 29(4): 759–768, DOI: 10.2478/amcs-2019-0056. Search in Google Scholar

Ho, J. and Ermon, S. (2016). Generative adversarial imitation learning, Advances in Neural Information Processing Systems 29: 4572–4580. Search in Google Scholar

Pomerleau, D. (1998). ALVINN: An autonomous land vehicle in a neural network, in D.S. Touretzky (Ed), Advances in Neural Information Processing Systems, Morgan Kaufmann Publishers, Burlington. Search in Google Scholar

Xia, W. and Li, H. (2017). Training method of automatic driving strategy based on deep reinforcement learning, Journal of Integration Technology 6(3): 29–40. Search in Google Scholar

Xiong, G.-m., Li, Y. and Wang, S.-y. (2015). A behavior prediction and control method based on FSM for intelligent vehicles in an intersection, Transactions of Beijing Institute of Technology 35(1): 7. Search in Google Scholar

Xu, H., Gao, Y., Yu, F. and Darrell, T. (2017). End-to-end learning of driving models from large-scale video datasets, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 2174–2182. Search in Google Scholar

Zhu, M., Wang, X. and Wang, Y. (2018). Human-like autonomous car-following model with deep reinforcement learning, Transportation Research C: Emerging Technologies 97: 348–368. Search in Google Scholar

Ziegler, J., Bender, P., Schreiber, M., Lategahn, H., Strauss, T., Stiller, C., Dang, T., Franke, U., Appenrodt, N., Keller, C.G., Kaus, E., Herrtwich, R.G., Rabe, C., Pfeiffer, D., Lindner, F., Stein, F., Erbs, F., Enzweiler, M., Knoppel, C., Hipp, J., Haueis, M., Trepte, M., Brenk, C., Tamke, A., Ghanaat, M., Braun, M., Joos, A., Fritz, H., Mock, H., Hein, M. and Zeeb, E. (2014). Making Bertha drive—An autonomous journey on a historic route, IEEE Intelligent Transportation Systems Magazine 6(2): 8–20. Search in Google Scholar

Zong, X., Xu, G., Yu, G., Su, H. and Hu, C. (2017). Obstacle avoidance for self-driving vehicle with reinforcement learning, SAE International Journal of Passenger Cars—Electronic and Electrical Systems 11(07-11-01-0003): 30–39. Search in Google Scholar

Zou, Q., Xiong, K. and Hou, Y. (2020). An end-to-end learning of driving strategies based on DDPG and imitation learning, 2020 Chinese Control And Decision Conference (CCDC), Hefei, China, pp. 3190–3195. Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo