Case Study · Teleoperation

Taking the operator out of the cab 

On a 40-ton Sennebogen material handler, we built a remote control system precise enough for real material handling — and turned it into the first step toward autonomy.

·4 min read
Live at bauma 2025: Sennebogen ran the full system on a real machine, in public.

Scarce expertise, locked into a single cab

Running a material handler has always meant putting a skilled person in the cab, on site, in whatever the job demands,  scrap yards, demolition sites, timber terminals, inland ports, with dust, noise, vibration and hours of physical strain. That constraint has become one of the industry’s tightest bottlenecks: skilled operators are scarce and expensive, and tying one of them to a single physical machine wastes their expertise and limits how flexibly a fleet can be scheduled.

The reflex answer is automation. But jumping straight to full autonomy is too risky and too unproven for safety-critical heavy machinery. No operator, site manager or insurer is ready to accept it today. What the industry needs is a credible path there, not a leap.

Remote control with real depth perception

We built that path on a Sennebogen 830 G material handler: our solver and our engineers designed and built the teleoperation system end to end; Sennebogen brought the machine and the domain expertise. 

Making remote control usable for precise material handling is harder than streaming video. Flat camera feeds strip away the depth cues an operator relies on to position a grapple over a scrap pile or place a load on a truck bed. Without them, every movement is a guess, and a 17-meter reach turns small joystick errors into large ones at the grapple.

So we gave the operator a virtual shadow: a live 3D reconstruction of the implement and its surroundings, built from stereo cameras and AI-based depth and object detection running on edge compute mounted directly on the machine. 

The operator sees exactly where the grapple sits in real space, and aims with the same confidence as from inside the cab.

Under 100 milliseconds, end to end

Handling cycles are fast and continuous, so any lag between joystick input and machine response breaks the operator’s sense of control and turns precision work into overcorrection. 

We engineered the full chain to stay under 100 milliseconds end to end, from camera capture through AI inference, encoding, the slip ring, industrial Ethernet and a 5G link with automatic VPN failover. The implementation includes the live depth and object detection, not just the video. Commands travel the same signal path - in reverse - into the machine kinematics, while a two-channel emergency stop and safety-relay chain runs independently of the network link.

The compute platform reuses proven, cost-efficient hardware from the autonomous-driving world. But the depth reconstruction, the latency budget and the safety architecture on top are ours: our solver together with our engineers created them from the ground up. Reusing proven hardware is a deliberate choice that keeps the bill of materials realistic and lets the architecture scale beyond a single prototype machine.

Built to scale across machine types

Scaling was a design constraint from the start. The system is a machine-independent stack: sensors and edge compute in a self-contained cabinet, a standard fieldbus interface to the machine controls, and a remote station agnostic to what sits at the other end of the link. The material handler was the first application, but the same architecture carries over to crawler-mounted machines and to cranes, where the case for getting the operator out of the cab is often even stronger. 

What changes per machine type is sensor placement, calibration and the kinematic model, not the underlying system.

Live depth and object detection on the machine’s edge compute — the operator’s virtual shadow of the grapple and its surroundings.
The Sennebogen 830 G at bauma 2025, rigged for teleoperation.

A working machine - and the first step toward autonomy

The immediate effect is a different working reality. The operator steps out of the hazard zone and into a safe, comfortable environment while keeping full control. And because control is no longer bound to a physical seat, one qualified operator can run equipment across multiple sites in a single shift instead of traveling to each one, turning a scarce skill into a far more productive resource.

The strategic effect matters more. Teleoperation is the first stepping stone toward full autonomy. Every machine equipped this way is networked, sensorized and AI-assisted from day one. 

The camera calibration, the depth models, the latency budget, the safety architecture and the recorded operator behavior are exactly the foundation autonomous operation will need, and they accumulate with every hour the system runs in the field. Instead of betting on a single autonomy milestone years out, the fleet advances step by step: remote control today, assistance functions next, semi-autonomous handling cycles after that. 
Because the stack is machine-independent, that progress isn’t locked to one product line. Every improvement made on the material handler applies to crawlers and cranes as well.

The near future of autonomy isn't the empty cab.
It's one operator running a whole fleet.