:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/53736381/Self_Driving_4.0.jpg)
Some of Uber’s self-driving cars aren’t driving as smoothly as the company hoped they would. Documents circulated throughout the company’s self-driving group, which Recode obtained, gives us a first look at the progress of the ride-hail company’s robot cars in Pennsylvania, Arizona and California.
The top line: Uber’s robot cars are steadily increasing the number of miles driven autonomously. But the figures on rider experience — defined as a combination of how many times drivers have to take over and how smoothly the car drives — are still showing little progress.
It’s still early in Uber’s testing. Alphabet’s self-driving company, Waymo, has been working on its own autonomous tech since 2009, for instance, and while it has vastly improved year over year, the system still sees a handful of what are known as “disengagements”: Basically, when a driver has to take over for the computer.
We’ve reached out to Uber for comment.
Successfully creating self-driving technology has become a crucial factor to Uber’s profitability. It would allow Uber to generate higher sales per ride since it would keep all of the fare. Uber has currently suffered losses in some markets partly because of having to offer subsidies to attract drivers. Computers are cheaper in the long run.
As a whole, Uber’s self-driving system is putting on many more miles than it did in January. Last week, the company’s 43 active cars drove 20,354 miles autonomously, according to the documents. This is only the second time since late December 2016 that its cars have driven more than 20,000 miles in a week.
In January, the cars only drove 5,000 miles. At that point, however, the company only had about 20 active vehicles, mainly in Pittsburgh. By February, the company’s cars were driving themselves around 18,000 miles a week.
Uber passengers took around 930 rides in these autonomous cars in Pittsburgh last week and around 150 rides in Phoenix. To be clear, these vehicles still had a driver at the wheel to take over if needed.
In Pittsburgh, where Uber launched its commercial self-driving pilot in September, the company has been performing around 800 or more UberX trips per week in semi-autonomous mode since the middle of February.
Though Uber has more cars on the road this month than last month and has increased its weekly autonomous mileage — both good developments — the company’s human drivers are still taking over the system more times than they did in January.
Uber uses several different metrics to determine how its systems have progressed. Those include:
- The average number of miles a car drives itself before a driver has to take over for any reason
- The average number of miles between “critical” interventions — when a driver has to avoid causing harm, such as hitting pedestrians or causing material property damage
- The average number of autonomous miles between “bad experiences” — things like jerky motions or hard braking, which are more likely to cause discomfort than damage
For example: During the week ending March 8, the 43 active cars on the road only drove an average of close to 0.8 miles before the safety driver had to take over for one reason or another.
This metric, called miles per intervention, includes all the times drivers have had to take back control from the system over the course of a week.
The reasons for these interventions can vary, but that can include navigating unclear lane markings, the system overshooting a turn or driving in inclement weather. The stat excludes “accidental disengagements, end-of-route disengagements and early takeovers.”
That’s a modest decline from earlier this year. At the end of January, the driver had to take over closer to once per 0.9 miles; Uber hit the one-mile mark the first week of February.
Then there’s what the company calls “critical” interventions. This is how many times a safety driver had to take over to avoid harmful events — incidents that would have resulted in a car hitting a person or causing more than around $5,000 in property damage if a driver didn’t take over.
The good news is the number of miles between these “critical” interventions has recently improved. Last week, the company’s cars drove an average of approximately 200 miles between those types of incidents that required a driver to take over.
While that’s an improvement from the prior week, which was about 114 miles between critical interventions, that progress hasn’t been steady. That means the cars aren’t yet necessarily getting increasingly safer.
At the end of January, drivers only needed to take over after an average of 125 miles driven, but that dropped to about once per 50 miles the first week of February. Those numbers then increased over the following two weeks but dropped again in the first week of March.
Part of that can be attributed to the cars being introduced in new places and on new routes, like parts of Arizona, or having to navigate around objects or road markings they don’t recognize. In short: The systems are still learning.
The cars have also had more “bad experiences” during the week ending on March 8 than in January. The miles driven between things like auto-detected hard decelerations or abrupt car jerks and movement has dropped from more than four miles in January to less than two miles as of last week.
In other words, cars only went an average of two miles without some sort of sudden movement or otherwise bad experience. These incidents have less to do with safety and more to do with the total rider experience and how smoothly the car is able to drive itself. Still.
In another document that specifically analyzed cars in Arizona, Uber’s self-driving team wrote that the rider experience in one part of the state, Scottsdale Road, was “not great.” The cars only drove 0.67 miles between interventions and two miles between so-called bad events.
The company has yet to let riders into vehicles along that route and said they would have to re-evaluate whether that route is ready after the autonomy team worked on some of the bugs, according to the email.
While Uber has yet to publicly report its self-driving miles and disengagements, all the other companies operating in California have had to submit their reports for the last two years.
However, it’s not exactly fruitful to compare the reports side by side. That’s primarily because each company is at a different stage in its self-driving development and has performed different levels of testing on public roads.
This article originally appeared on Recode.net.
