Vision-Based 6-DoF Pose Estimation of Forklift and Load for On-Site Stability Check

Heavy equipment contributes to accidents in construction as a major category of causes. Various studies develop simulation models to investigate the correlation between equipment’s state variables (e.g., center of mass, orientation, velocity, and acceleration) during movement and the risk of accidents such as tip-over. However, the impact of material attached to equipment is rarely incorporated as the dynamic state variables of the material are not readily available onsite. To address this issue, this research streamlined the process of preparing the training data, which enables the implementation of existing vision-based 6-DoF object pose estimation models. An evaluation test is performed to demonstrate the potential of the proposed approach. This research also develops the method to convert the estimated pose from the image’s coordinate system to a world coordinate system, which enables estimating state variables in a dynamics simulation. Finally, a simple stability check using the stability triangle is demonstrated using the vision-based approach.