Ego-Centric Pedestrian Trajectory Prediction Considering Camera Motion Parameters

Accurate pedestrian trajectory prediction is crucial for enhancing pedestrian and autonomous vehicle safety. In comparison to the bird-eye view, ego-centric trajectory prediction is more challenging because the ego-motion of the camera can dynamically alter the scale of pedestrian bounding boxes. As the distance between the vehicle and pedestrians decreases, pedestrian bounding boxes are magnified, subsequently influencing the predicted pedestrian trajectories within the image plane. To address this challenge, the paper proposes a deep learning network based on LSTM encoder-decoder architecture that leverages trajectory, ego-vehicle motion information, and local-visual context as the additional input information. Specifically, the motion parameters of the in-vehicle camera are introduced to represent self-vehicle motion information at a coarse granularity level, reducing the impact of noise when representing self-vehicle speed by optical flow. Meanwhile, due to the disproportionate impact of pedestrians at different distances on the optimization process, the loss function is normalized using the actual width and height of the pedestrian’s bounding box, effectively reducing the weight assigned to pedestrians in proximity during optimization. The attention mechanism is utilized to better capture the long-term temporal variations. Several typical pedestrian trajectory prediction benchmark datasets are used for the algorithm validation, such as JAAD. According to the results, the proposed method outperformed the selected state-of-the-art algorithms with about 19% improvements in trajectory prediction accuracy.