Object Detection Using Multi-Sensor Fusion Based on Deep Learning

Autonomous driving is an important part of intelligent traffic. It requires accurate and effective environmental perception. Fusion of multi-sensor perception information is an effective way to achieve environmental perception. This paper studies information fusion of millimeter-wave radar and camera, mainly using deep learning method to achieve traffic object detection. The detection result is directly used to make driving decisions. We constructed a perception data collection system and used spatial-temporal synchronization to associate multi-sensor data. Then we design a type of deep fusion algorithm. Based on YOLOv2 (you only look once V2) algorithm, we modified the neural network to implement deep fusion. We applied a method to input multi-sensor data into CNN (convolutional neural networks) using a KITTI dataset. The model can provide useful object information such as category and location. By comparing our results with the result of classical post-fusion algorithm, we found that deep fusion is more efficient.