Envisioning The Autonomous Teleoperation Experience

How we designed for the safe and remote control of autonomous vehicles

Remote vehicle teleoperation serves as a critical backup to ensure the safety of passengers and cargo in autonomous vehicles. When AI systems encounter challenges they cannot handle, teleoperators step in to control the vehicles remotely. They are also essential for managing the "last mile" of trips where unpredictable conditions are most complex. We designed a comprehensive interface experience for teleoperators to communicate with passengers and manage scheduled and emergency pick-ups in driverless autonomous vehicles.

Services
Stakeholder interviews
Rapid concepting
Competitive and domain research
UX/UI Design
Design workshop
Motion Design
Envisioning
Client
Designated Driver
Industry
Automotive
Timeline
5 months
The Challenge
Designing for safety and urgency
Teleoperators manage one vehicle after another from an off-site cockpit equipped with a steering wheel and a grid of six video screens. Regardless of the situation, the user experience must provide teleoperators with comprehensive information to quickly and safely assist passengers. The cockpit replicates the vehicle's environment, including peripheral views, audio, dashboard elements, and visibility of passengers. This gives teleoperators a comprehensive understanding of current situations and historical elements which may contribute to passenger safety.
Our Approach
Envisioning a brand-new “driving” experience
Our process ensures that all design decisions are user-centered and aligned with our clients' goals. We began with stakeholder interviews to understand our client's vision. Next, we conducted competitive and inspirational research to explore the autonomous vehicle space, including military standards for cockpit UI. This discovery phase culminated in an in-person workshop where we presented and prioritized ideas collaboratively with the project team.

In the concepting phase, we created a set of written user scenarios covering all teleoperators' tasks. These scenarios directly informed user flows, diagramming which screens display what information at any given time. This detailed planning allows us to focus on the specifics when designing wireframes and prototypes.

Throughout the design process, we maintain alignment with our clients, and our methodical approach incorporates feedback at every stage. This approach resulted in a working demo for CES, the Consumer Electronics Show, showcasing our commitment to thorough and user-focused design.

“Love the combination of design and research that Dupla brings. Not a lot of companies offer that. And the work they created for us was a true innovation for our product and the entire industry. They gave us something we had not seen anywhere else.”

Manuela Papadopol

Chief Executive Officer, Designated Driver

The Solution
Providing context for an emergency pickup
To address how a teleoperator might assess the situation of a stopped vehicle before deciding to take over, we designed Peek Mode. Using the vehicle’s sensors, Peek Mode offers a visual of what the car sees at the moment it needs assistance, along with all relevant sensor information.

After a teleoperator takes control, the next challenge is understanding the context leading up to the emergency request. To meet this need, we envisioned Playback Mode. Playback Mode enables the teleoperator to review the last 30 seconds of video and sensor data, offering a comprehensive view of the events leading up to the vehicle's request for help.
Communicating with the autonomous vehicle passengers
Imagine being in a driverless autonomous vehicle that suddenly comes to a stop. Just as the teleoperator needs context, so do the passengers. We designed a flow that keeps passengers informed throughout the process. It shows when the vehicle is in autonomous mode, when it has identified an obstacle and is requesting teleoperator assistance, and when the teleoperator has taken over and is communicating live with the passengers. This ensures that passengers feel reassured and informed at every step.
Handling scheduled pickups
A more common scenario for a teleoperator involves handling scheduled pickups, where they are pre-informed about when to take over a vehicle for "last mile" tasks, such as package deliveries. We designed a scenario where the teleoperator monitors upcoming pickups, takes control of the vehicle to drive it to its destination, triggers a drop-off alert, and waits for the recipient to remove the package from the vehicle. This ensures a smooth and efficient delivery process.
Impact
Sharing a vision with potential partners
The work Dupla did leading up to Designated Driver’s inaugural demo at CES 2019 prompted a series of post-conference discussions and partnerships which facilitated the growth of the company moving forward.
Next Case Study