KraNavi

The KraNavi research project aims to develop an augmented reality (AR)-based assistance system for autonomous transport using overhead cranes. Intuitive control via voice and gestures is intended to significantly simplify operation. Another goal is to prevent accidents caused by operating errors. An environment recognition system identifies and classifies objects, people, and obstacles in real time and provides the operator with all relevant information using AR.

The project focuses on researching a multimodal operating concept. The combination of gesture- and voice-based control with AI-supported evaluation enables intuitive communication between humans and machines. Principles of human interaction, such as pointing with the index finger, are transferred directly to crane controls. This creates a previously unexplored control paradigm for stationary transport systems that significantly increases safety and productivity.

The solution comprises an AR device and several sensors mounted on the crane for environmental recognition. A central control unit handles route planning and guidance. The crane operator uses voice commands and gestures to select the load, confirm the start of transport, and automatically approve the planned route. The path is visualized in the field of vision, hazards are highlighted, and the subsequent transport is carried out autonomously. KraNavi thus addresses a significant gap in the state of the art, as current systems do not enable comparable human-machine interaction and environmental perception.