Zoox CTO: Why Self-Driving Cars Are So Hard to Get Right

The road to autonomous vehicles is long and arduous, and it has already taken one high-profile victim: GM’s Cruise. Elon Musk has been promising autonomy in Teslas for years. It’s still not here.

Jesse Levinson, co-founder and CTO at Zoox, Amazon’s robotaxi subsidiary, will be the first to tell you that autonomy is very hard to do.

“In the olden days of 20 years ago, AI wasn’t very popular, and most people thought it was a stupid thing to work on,” Levinson said. He thought inventing a car that could drive itself was “cool,” adding that “I was pretty convinced that someday in my lifetime, cars would drive themselves.”

Today, 11 years after co-founding Zoox, Levinson discovered that autonomy is “a super-hard problem. If you really want to remove the driver, it has to be just profoundly good. It can’t just be like, ‘Oh, if you drive it for an hour, sometimes it works.’ That’s cool if it’s a driver assistance system. But if there’s no driver, it has to work … 99.9% [of the time] — a lot of nines.”

This week, Zoox agreed to recall the software on 258 vehicles due to unexpected braking issues following two accidents involving its test vehicles in May 2024, according to Reuters and the National Highway Traffic Safety Administration. These vehicles, which had human safety drivers, braked and caused motorcycles to rear-end them. Two motorcyclists had minor injuries.

Not only is autonomy a hard problem to solve, it’s also costly, Levinson said.

“It’s taken longer [to get to self-driving vehicles] than we thought,” Levinson said during a session at last week’s HumanX conference in Las Vegas. Also, “it’s much more expensive relative to what we thought than it is time-consuming. … We’ve been spending a lot of money.”

Part of the problem is self-inflicted. Zoox decided to re-architect the traditional car by removing the steering wheel and braking and acceleration pedals, which became the cause of an intense internal debate.

“A lot of really good engineers wanted to put a steering wheel and pedals in the robotaxi and keep that for a while until we can show that we don’t need to disengage very often,” he said. “It was like a little bit of a comfort blanket kind of a situation.”

But Zoox wanted to differentiate itself from other carmakers that say their vehicles are autonomous but in reality are merely driver-assistance systems. So it decided to go all in and change the concept of a car itself.

“We started with a blank sheet of paper and we were able to do some things that in retrospect feel pretty obvious, but nobody was thinking about them because they were so constrained by the legacy of cars,” Levinson said.

Read more: Amazon-Owned Zoox Expands Operation of Autonomous Cars in Las Vegas

Zoox is currently testing its vehicles in the San Francisco Bay Area, Seattle, Austin and Miami. In Las Vegas, it plans to launch a paid service to the public later this year, according to Las Vegas Weekly.

Zoox’s autonomous vehicle has no front or back — a journalist who took a test ride said it looked like a toaster— using LED lights to indicate the direction it’s going. Instead of a horn, it has a 30-channel speaker array that can shoot targeted audio directly at someone or something.

“It’s much less obnoxious than honking your horn and bothering everybody,” Levinson said.

Inside, seats face each other. Each seat has four-zone heating and cooling. There is a high-end sound system and little lights on the roof. There are USB-C and wireless chargers, a touchscreen display and other features.

Levinson said it’s been “fun” to see people’s reactions when they first see the robotaxi. “’That’s not a car I recognize. What is that?’ Then they get a little bit closer and they’re like, ‘Where’s the driver?’ And there isn’t one.’”

Zoox borrows from aviation technology to build in redundancies in the vehicle, just like a plane has backup engines so it can still land even if there is engine failure, according to Levinson.

The vehicle uses cameras, LiDAR and radar to see 500 feet around it in all directions and even around corners, with a 360-degree view. It can perceive the color of traffic lights and even facial expressions of pedestrians, according to Zoox. The computing system also has backups.

One thing people don’t realize is that it’s very hard to get the robotaxi to drive smoothly since it sees a lot of objects around, Levinson said.

“If you’re not careful, it’s really easy to overreact because you can look at 300 things and think maybe one of them is going to run into us,” causing the car to brake. “Then you get a really jerky ride,” Levinson said. “So making it feel kind of seamless and magical, there’s a lot more work that goes into that than you might assume.”

These and other technical challenges, as well as the cost of developing autonomous vehicles, took a toll. Six years after its founding in 2014, Zoox started to struggle. Levinson realized that to continue, it would need the kind of capital that was hard to raise as a private company.

In 2020, the founders sold the company to Amazon. “Not only do they have the capital to buy Zoox, but they also have the capital to take it all the way,” he said.

The post Zoox CTO: Why Self-Driving Cars Are So Hard to Get Right appeared first on PYMNTS.com.