APSIPA Transactions on Signal and Information Processing > Vol 13 > Issue 5

Multi-Modal Pedestrian Crossing Intention Prediction with Transformer-Based Model

Ting-Wei Wang, National Tsing Hua University, Taiwan, Shang-Hong Lai, National Tsing Hua University, Taiwan, lai@cs.nthu.edu.tw
 
Suggested Citation
Ting-Wei Wang and Shang-Hong Lai (2024), "Multi-Modal Pedestrian Crossing Intention Prediction with Transformer-Based Model", APSIPA Transactions on Signal and Information Processing: Vol. 13: No. 5, e401. http://dx.doi.org/10.1561/116.20240019

Publication Date: 07 Oct 2024
© 2024 T.-W. Wang and S.-H. Lai
 
Subjects
Deep learning,  Classification and prediction,  Activity and gesture recognition,  Video analysis and event recognition,  Pattern recognition and learning,  Automotive industries
 
Keywords
Pedestrian crossing intention predictionmulti-modal learningtransformer modelhuman posture
 

Share

Open Access

This is published under the terms of CC BY-NC.

Downloaded: 124 times

In this article:
Introduction 
Related Work 
Proposed Method 
Experimental Results 
Conclusions 
References 

Abstract

Pedestrian crossing intention prediction based on computer vision plays a pivotal role in enhancing the safety of autonomous driving and advanced driver assistance systems. In this paper, we present a novel multi-modal pedestrian crossing intention prediction framework leveraging the transformer model. By integrating diverse sources of information and leveraging the transformer’s sequential modeling and parallelization capabilities, our system accurately predicts pedestrian crossing intentions. We introduce a novel representation of traffic environment data and incorporate lifted 3D human pose and head orientation data to enhance the model’s understanding of pedestrian behavior. Experimental results demonstrate the state-of-the-art accuracy of our proposed system on benchmark datasets.

DOI:10.1561/116.20240019

Companion

APSIPA Transactions on Signal and Information Processing Special Issue - Invited Papers from APSIPA ASC 2023
See the other articles that are part of this special issue.