Mixed Reality — Enhanced Operator Interface for Teleoperation Systems in Unstructured Environment

Conventional teleoperation systems have been developed as an efficient solution to explore unstructured environments where hazardous materials have been accidently spilled or released. Due to inevitable restrictions in the field of view of the remote camera(s) and in the quality of the visual displays fed back from the remote worksite, the operator is often unable to maintain a level of situational awareness sufficient for safe and efficient task execution. To address some of the challenges of this nature, Mixed Reality (MR) technology that combines real environment with virtual information together on a single display is envisaged to enhance the current teleoperation visual interface for supporting decision making and judging while operating in unstructured environment. The paper presents an intuitive concept of applying techniques of MR to increase the operator's situational awareness and spatial cognitive skills critical to teleoperation, with a focus on remote navigation and excavation.