Teleoperating a 40-Ton Material Handler
The Problem: Scarce Expertise, Locked Into a Single Cab
Running a material handler has always meant putting a skilled person in the cab, on site, in whatever conditions the job demands: scrap yards, demolition sites, timber terminals and inland ports, with dust, noise, vibration and hours of physical strain. That constraint has become one of the industry’s biggest bottlenecks. Skilled operators are scarce and expensive, and tying one of them to a single physical machine wastes their expertise and limits how flexibly a fleet can be scheduled.
The obvious answer everyone points to is automation. But jumping straight to full autonomy is too risky and too unproven for safety-critical heavy machinery, and no operator, site manager or insurer is ready to accept it today. What the industry needs is a credible path there, not a leap
The Solution: Remote Control With Real Depth Perception
Together with Sennebogen, we built that path on an 830 G material handler. Teleoperation is the key building block: it lets the operator control the machine from anywhere, safely and precisely, without sitting in the cab.
Making remote control genuinely usable for precise material handling is harder than streaming video. Flat camera feeds strip away the depth cues an operator relies on to position a grapple over a scrap pile or place a load on a truck bed, and without them every movement becomes a guess, made worse by a 17 metre reach where small errors at the joystick translate into large ones at the tool. So we gave the operator a virtual shadow: a live 3D reconstruction of the implement and its surroundings, built from stereo cameras and AI-based depth and object detection running on edge compute mounted directly on the machine. The operator sees exactly where the grapple sits in real space, and can aim with the same confidence as if they were on the machine.
None of this works if it is slow. Handling cycles are fast and continuous, so any lag between joystick input and machine response breaks the operator’s sense of control and turns precision work into overcorrection. We engineered the full chain to stay under 100 milliseconds end to end, from camera capture through AI inference, encoding, the slip ring, industrial Ethernet and a 5G link with automatic VPN failover. That solution includes the live depth and object detection, not just the video. Commands travel the same path in reverse through CANopen and CAN over IP into the machine kinematics, while a two-channel emergency stop and safety relay chain runs independently of the network link.
The compute platform is proven, cost-efficient hardware from the autonomous driving world. That choice was deliberate: it keeps the bill of materials realistic and makes the architecture scale beyond a single prototype machine.
Scaling was a design constraint from the start, not an afterthought. The system is built as a machine-independent stack: sensors and edge compute in a self-contained cabinet, a standard fieldbus interface to the machine controls, and a remote station that is agnostic to what sits at the other end of the link. The material handler was the first application, but the same architecture carries over to crawler-mounted machines and to cranes, where the case for getting the operator out of the cab is often even stronger. What changes per machine type is the sensor placement, the calibration and the kinematic model, not the underlying system.
The Impact: A Working Machine and a Path Beyond It
The immediate effect is a different working reality. The operator steps out of the hazard zone and into a safe, comfortable environment while keeping full control. And because control is no longer bound to a physical seat, one qualified operator can run equipment across multiple sites in a single shift instead of traveling to each one, turning a scarce skill into a far more productive resource.
The strategic effect matters more. Teleoperation is the first stepping stone toward full autonomy, and it is the only one that pays for itself while you build it. Every machine equipped this way is networked, sensorized and AI-assisted from day one. The camera calibration, the depth models, the latency budget, the safety architecture and the recorded operator behaviour are exactly the foundation autonomous operation will need, and they accumulate with every hour the system is used in the field. Instead of betting on a single autonomy milestone years out, the fleet advances step by step: remote control today, assistance functions next, semi-autonomous handling cycles after that. And because the stack is machine-independent, that progress is not locked to one product line. Every improvement made on the material handler applies to crawlers and cranes as well.
At bauma 2025, Sennebogen put the system in front of the entire heavy machinery industry and demonstrated live operation of a real machine. That is the part we are proudest of: not a concept video or a simulation, but a system that works, built and integrated end to end, running in public under real conditions. It is also the first milestone on a longer road, and the road to autonomy in heavy machinery is one an operator can walk with the machine rather than wait for.