Goal-LBP: Goal-Based Local Behavior Guided Trajectory Prediction for Autonomous Driving
Authors: 
Zhen Yao, Xin Li, Bo Lang, Mooi Choo ChuahAuthors Info & Claims
Zhen Yao, Xin Li, Bo Lang, Mooi Choo ChuahAuthors Info & ClaimsPages 6770 - 6779
Abstract
In recent years, the design of models for performing the trajectory prediction task, one of the critical tasks in autonomous driving, has received great attention from researchers. However, accurately predicting future locations is challenging due to the difficulty of learning accurate intentions and modeling multimodality. Historical paths at a certain location can help predict the future trajectory of an agent currently located in that position and address these limitations. In this work, we propose a goal-based local behavior guided model, Goal-LBP, using such information (referred to as local behavior data) to generate potential goals and guide the prediction of trajectories conditioned on such goals. Goal-LBP uses Transformer encoders to extract homogeneous features and attention mechanism to represent the heterogeneous interactions and subsequently uses an encoder-decoder Gated Recurrent Unit (GRU) model to generate predictions. We evaluate our Goal-LBP using two large-scale real-world autonomous driving datasets, namely nuScenes and Argoverse. Our results show that compared to several SOTA models, Goal-LBP achieves the best ADE/FDE performance and it ranked #2 on the leaderboard of the nuScenes trajectory benchmark in June 2023. In addition, we also demonstrate that our local behavior estimator block can be easily added to two existing SOTA methods, namely AgentFormer and LaPred. Adding this LBE block improves the original AgentFormer and LaPred performance by at least 10%.
References
[1]
J. Xiaoet al., “Vehicle trajectory interpolation based on ensemble transfer regression,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 7680–7691, Jul. 2022.
[2]
Y. Huang, Z. Xiao, D. Wang, H. Jiang, and D. Wu, “Exploring individual travel patterns across private car trajectory data,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 12, pp. 5036–5050, Dec. 2020.
[3]
Z. Tanet al., “Human–machine interaction in intelligent and connected vehicles: A review of status quo, issues, and opportunities,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 13954–13975, Sep. 2022.
[4]
H. Georgiouet al., “Moving objects analytics: Survey on future location & trajectory prediction methods,” 2018, arXiv:1807.04639.
[5]
A. Rudenko, L. Palmieri, M. Herman, K. M. Kitani, D. M. Gavrila, and K. O. Arras, “Human motion trajectory prediction: A survey,” Int. J. Robot. Res., vol. 39, no. 8, pp. 895–935, Jun. 2020.
[6]
Z. Zheng, X. Ying, Z. Yao, and M. C. Chuah, “Robustness of trajectory prediction models under map-based attacks,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2023, pp. 4530–4539.
[7]
T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone, “Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,” in Proc. 16th Eur. Conf. Comput. Vis. (ECCV), Glasgow, U.K. Cham, Switzerland: Springer, Aug. 2020, 2020, pp. 683–700.
[8]
M. Morzy, “Mining frequent trajectories of moving objects for location prediction,” in Proc. 5th Int. Conf. Mach. Learn. Data Mining Pattern Recognit. (MLDM), Leipzig, Germany. Berlin, Germany: Springer, Jul. 2007, pp. 667–680.
[9]
H. Jeung, Q. Liu, H. T. Shen, and X. Zhou, “A hybrid prediction model for moving objects,” in Proc. IEEE 24th Int. Conf. Data Eng., Apr. 2008, pp. 70–79.
[10]
A. Monreale, F. Pinelli, R. Trasarti, and F. Giannotti, “WhereNext: A location predictor on trajectory pattern mining,” in Proc. 15th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Jun. 2009, pp. 637–646.
[11]
J. Wiest, M. Höffken, U. Kreßel, and K. Dietmayer, “Probabilistic trajectory prediction with Gaussian mixture models,” in Proc. IEEE Intell. Vehicles Symp., Jun. 2012, pp. 141–146.
[12]
J. Li, H. Ma, and M. Tomizuka, “Conditional generative neural system for probabilistic trajectory prediction,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Nov. 2019, pp. 6150–6156.
[13]
H. Kim, D. Kim, G. Kim, J. Cho, and K. Huh, “Multi-head attention based probabilistic vehicle trajectory prediction,” in Proc. IEEE Intell. Vehicles Symp. (IV), Oct. 2020, pp. 1720–1725.
[14]
N. Deo, E. Wolff, and O. Beijbom, “Multimodal trajectory prediction conditioned on lane-graph traversals,” in Proc. Conf. Robot Learn., 2022, pp. 203–212.
[15]
J. Hong, B. Sapp, and J. Philbin, “Rules of the road: Predicting driving behavior with a convolutional model of semantic interactions,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 8446–8454.
[16]
T. Gilles, S. Sabatini, D. Tsishkou, B. Stanciulescu, and F. Moutarde, “THOMAS: Trajectory heatmap output with learned multi-agent sampling,” 2021, arXiv:2110.06607.
[17]
K. Messaoud, N. Deo, M. M. Trivedi, and F. Nashashibi, “Trajectory prediction for autonomous driving based on multi-head attention with joint agent-map representation,” in Proc. IEEE Intell. Vehicles Symp. (IV), Jul. 2021, pp. 165–170.
[18]
T. Gilles, S. Sabatini, D. Tsishkou, B. Stanciulescu, and F. Moutarde, “GOHOME: Graph-oriented heatmap output for future motion estimation,” in Proc. Int. Conf. Robot. Autom. (ICRA), May 2022, pp. 9107–9114.
[19]
X. Li, X. Ying, and M. C. Chuah, “GRIP: Graph-based interaction-aware trajectory prediction,” in Proc. IEEE Intell. Transp. Syst. Conf. (ITSC), Oct. 2019, pp. 3960–3966.
[20]
X. Li, X. Ying, and M. C. Chuah, “GRIP++: Enhanced graph-based interaction-aware trajectory prediction for autonomous driving,” 2019, arXiv:1907.07792.
[21]
A. Vaswaniet al., “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017.
[22]
Y. Yuan, X. Weng, Y. Ou, and K. Kitani, “AgentFormer: Agent-aware transformers for socio-temporal multi-agent forecasting,” 2021, arXiv:2103.14023.
[23]
H. Caesaret al., “NuScenes: A multimodal dataset for autonomous driving,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 11618–11628.
[24]
M.-F. Changet al., “Argoverse: 3D tracking and forecasting with rich maps,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 8740–8749.
[25]
P. Sunet al., “Scalability in perception for autonomous driving: Waymo open dataset,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 2443–2451.
[26]
A. Ess, B. Leibe, K. Schindler, and L. Van Gool, “A mobile vision system for robust multi-person tracking,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1–8.
[27]
C. Yu, X. Ma, J. Ren, H. Zhao, and S. Yi, “Spatio-temporal graph transformer networks for pedestrian trajectory prediction,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2020, pp. 507–523.
[28]
B. Kimet al., “LaPred: Lane-aware prediction of multi-modal future trajectories of dynamic agents,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 14631–14640.
[29]
C. Wang, Y. Wang, M. Xu, and D. J. Crandall, “Stepwise goal-driven networks for trajectory prediction,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 2716–2723, Apr. 2022.
[30]
E. Rehder and H. Kloeden, “Goal-directed pedestrian prediction,” in Proc. IEEE Int. Conf. Comput. Vis. Workshop (ICCVW), Dec. 2015, pp. 139–147.
[31]
H. Zhaoet al., “TNT: Target-driven trajectory prediction,” in Proc. Conf. Robot Learn., 2021, pp. 895–904.
[32]
W. Zeng, M. Liang, R. Liao, and R. Urtasun, “LaneRCNN: Distributed representations for graph-centric motion forecasting,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Sep. 2021, pp. 532–539.
[33]
M. Lianget al., “Learning lane graph representations for motion forecasting,” in Proc. 16th Eur. Conf. Comput. Vis. (ECCV), Glasgow, U.K. Cham, Switzerland: Springer, Aug. 2020, pp. 541–556.
[34]
J. Wang, T. Ye, Z. Gu, and J. Chen, “LTP: Lane-based trajectory prediction for autonomous driving,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 17134–17142.
[35]
J. Gu, C. Sun, and H. Zhao, “DenseTNT: End-to-end trajectory prediction from dense goal sets,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2021, pp. 15283–15292.
[36]
T. Zhaoet al., “Multi-agent tensor fusion for contextual trajectory prediction,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 12118–12126.
[37]
Y. Chai, B. Sapp, M. Bansal, and D. Anguelov, “MultiPath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction,” 2019, arXiv:1910.05449.
[38]
H. Cuiet al., “Multimodal trajectory predictions for autonomous driving using deep convolutional networks,” in Proc. Int. Conf. Robot. Autom. (ICRA), May 2019, pp. 2090–2096.
[39]
N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. S. Torr, and M. Chandraker, “DESIRE: Distant future prediction in dynamic scenes with interacting agents,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 2165–2174.
[40]
Y. LeCunet al., “Backpropagation applied to handwritten zip code recognition,” Neural Comput., vol. 1, no. 4, pp. 541–551, Dec. 1989.
[41]
J. Wang, C. Guo, M. Guo, and J. Chen, “Jointly learning agent and lane information for multimodal trajectory prediction,” 2021, arXiv:2111.13350.
[42]
Y. Liu, J. Zhang, L. Fang, Q. Jiang, and B. Zhou, “Multimodal motion prediction with stacked transformers,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 7573–7582.
[43]
Y. Zhong, Z. Ni, S. Chen, and U. Neumann, “Aware of the history: Trajectory forecasting with the local behavior data,” in Proc. Eur. Conf. Comput. Vis. Cham, Switzerland: Springer, 2022, pp. 393–409.
[44]
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” 2017, arXiv:1711.05101.
Recommendations
Transformer based composite network for autonomous driving trajectory prediction on multi-lane highways
AbstractIn order to navigate through complex traffic scenarios safely and efficiently, the autonomous vehicle (AV) predicts its own behavior and future trajectory based on the predicted trajectories of surrounding vehicles to avoid potential collisions. ...
Neural network based lane change trajectory prediction in autonomous vehicles
Transactions on computational science XIIIDuring a lane change, vehicle collision warning systems detect the likelihood of collision and time to collision to warn vehicles of an imminent collision. In autonomous systems, a vehicle utilizes data obtained by its own sensors to predict future ...
ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Driving
KDD '24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data MiningTrajectory prediction of moving traffic agents is crucial for the safety of autonomous vehicles, whereas previous approaches usually rely on sufficiently long-observed trajectory (e.g., 2 seconds) to predict the future trajectory of the agents. However, ...
Comments
Information & Contributors
Information
Published In
1524-9050 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 20 December 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in