Envisioning The Autonomous Teleoperation Experience

How we designed for the safe and remote control of autonomous vehicles

Remote vehicle teleoperation is a crucial last line of defense to make sure passengers and cargo in autonomous vehicles are safe. Teleoperators remotely take control of vehicles from afar, closing gaps that AI can’t yet close. Teleoperators can also conveniently solve for that “last mile” for pick-ups and drop-off trips of both passengers and deliveries where unknown conditions are at their most complex. Dupla designed the end-to-end experience for how teleoperators handle scheduled or emergency pick-ups and communicate with the passengers inside a driverless autonomous vehicle.

Services
Stakeholder interviews
Rapid concepting
Competitive and domain research
UX/UI Design
Design workshop
Motion Design
Envisioning
Client
Designated Driver
Industry
Automotive
Timeline
5 months
The Challenge
Designing for safety and urgency
Teleoperators are virtually called into car after car from behind a cockpit with a steering wheel and a grid of six video screens. Sometimes teleoperators are called in for scheduled drives, but for others, they’re called in for situations where the autonomous vehicle has stopped for a wide range of unexpected reasons. Whatever the reason, the cockpit UX needs to bring the teleoperator up to speed as quickly as possible. This means that the cockpit needs to mirror the environment of the car (peripheral view, audio, dashboard elements, viewing passengers). It also needs to show the moments leading up to the halt.
Our Approach
Envisioning a brand-new “driving” experience
Our process ensures that all our design decisions are rooted in the needs of our users and aligned with our clients. Discovery began with stakeholder interviews to understand our client’s vision. Interviews were followed by competitive and inspirational research to understand the autonomous vehicle space, including military standards for cockpit UI. Discovery was followed by an in-person workshop where we presented ideas and prioritized them collaboratively with the project team.

The concepting phase started with the creation of a set of written user scenarios that collectively covered all of the teleoperators' tasks. These scenarios directly informed user flows. These essentially diagram out which screens show what at any given time. In this way, by the time we start designing wireframes and then prototypes, we’re basically filling in the colors to a very detailed drawing. At each stage of the design process, we ensure we’re aligned with our clients, and our methodical approach means that feedback at any stage is carried through to the final output, which in this case was a working demo for CES.

“Love the combination of design and research that Dupla brings. Not a lot of companies offer that. And the work they created for us was a true innovation for our product and the entire industry. They gave us something we had not seen anywhere else.”

Manuela Papadopol

Chief Executive Officer, Designated Driver

The Solution
Providing context for an emergency pickup
How might a teleoperator assess the situation of a stopped vehicle before deciding to take over? This question led us to design Peek Mode — a way for the remote driver to look through the vehicle’s eyes to see what the car sees at the moment it needs help, with the support of all the pertinent sensor information.

Once the driver decides to take over, the next challenge is to assess the vehicle’s context leading up to the emergency request. To address this need, we designed Playback Mode — the ability for the teleoperator to review the last 30 seconds of video from the cameras and sensor information leading up to the vehicle’s request for pickup.
Communicating with the autonomous vehicle passengers
Imagine being driven by a driverless autonomous vehicle, and the car suddenly comes to a stop. The same way the teleoperator needs context, so do the passengers. We designed a flow that allows passengers to understand when the vehicle is in autonomous mode, when it has identified an obstacle and is requesting the teleoperator, up until the teleoperator takes over and is communicating live with the passengers—so they feel reassured through it all.
Handling scheduled pickups
A more common scenario for a teleoperator is to handle scheduled pickups, where they know ahead of time when a vehicle needs to be taken over for “last mile” situations, such as package deliveries. We designed a scenario where the teleoperator monitors upcoming pickups, takes over the vehicle to its destination, triggers a dropoff alert, and waits for the recipient to remove a package from the vehicle.
Impact
Sharing a vision with potential partners
The work Dupla did leading up to Designated Driver’s inaugural demo at CES 2019 prompted a series of post-conference discussions and partnerships which facilitated the growth of the company moving forward.
Next Case Study