Way back when you took your driving test and got your licence, someone somewhere gave you a lecture about responsibility. Maybe it was your parents, the driving instructor, or the RMS tester. You were in charge of the car, and therefore it was your responsibility to not crash it.
You probably haven’t gotten that lecture in a while, but it’s still true. Someone must always be responsible for any moving vehicle. It is a basic tenet of driving, flying aeroplanes, operating forklifts, riding a bicycle, or, hell, even walking. If you bump into someone else, you’re responsible.
Some people like to pretend they present some vastly more complex responsibility matrix about Who Is Responsible when something goes wrong and invoke barely relevant philosophical thought experiments in the process to make it all seem like a different problem than it is. But whether the operator of the vehicle is a person or computer, someone is still in charge.
You wouldn’t know that, though, from Wednesday’s U.S. National Transportation Safety Board’s hearing on the death of Elaine Herzberg, the woman struck and killed by Uber’s self-driving test vehicle in March 2018. According to their findings, everyone is responsible.
But it also pinned the crash on Uber’s safety driver for not paying attention—a separate issue from how the self-driving cars were programmed and how management designed the testing process to prioritise “metrics” like kilometres driven to impress the new boss before safety—and on U.S. government agencies such as the Arizona Department of Transportation and the National Highway Traffic Safety Administration for not enacting more stringent regulations or mandatory safety reporting.
Most gallingly, the NTSB even found Herzberg herself was partly responsible because she did not cross the street at a crosswalk—never you mind that the bike path she was on spit her out in the middle of the block—and having drugs in her system at the time of the crash, a fact which, in its most charitable interpretation, means Herzberg was not alert enough to dive out of the way of the Uber SUV prior to impact.
Here is the NTSB’s entire “Probable Cause” statement:
The National Transportation Safety Board determines that the probable cause of the crash in Tempe, Arizona, was the failure of the vehicle operator to monitor the driving environment and the operation of the automated driving system because she was visually distracted throughout the trip by her personal cell phone. Contributing to the crash were the Uber Advanced Technologies Group’s (1) inadequate safety risk assessment procedures, (2) ineffective oversight of vehicle operators, and (3) lack of adequate mechanisms for addressing operators’ automation complacency—all a consequence of its inadequate safety culture. Further factors contributing to the crash were (1) the impaired pedestrian’s crossing of N. Mill Avenue outside a crosswalk, and (2) the Arizona Department of Transportation’s insufficient oversight of automated vehicle testing
In sum, the NTSB concluded that the blame first and foremost rested on the safety driver, a contractor hired by Uber to supervise the software driving the car. Uber’s inadequate safety culture was relegated to a “contributing” factor to the crash. Herzberg’s behaviour and the Arizona Department of Transportation’s lack of oversight in AV tasting are also name-checked.
This attitude that there is plenty of blame to go around was in keeping with the hearing’s overall tone, one of deference and occasionally even praise for Uber. This may sound bizarre given the NTSB investigation concluded the company’s workers were a cause of the crash. But on multiple occasions, NTSB members not only implicitly forgave Uber, but praised the company for its actions after the crash, as if the company prior to Hertzberg’s death was completely separate from the one after. They were awfully good sports about the whole thing.
The summary remarks from the NTSB Chairman, Robert Sumwalt, characterised his general approach. “Uber ATG has really embraced the lessons from this event, from this tragic event,” Sumwalt said. “Uber has truly embraced those lessons and we want to encourage them to continue on that journey and we want others to learn from this as well.”
You could almost hear the underlying sentiment, the one he so obviously wanted to say, that we all make mistakes.
Meanwhile, the NTSB was never very clear on precisely What Uber’s journey consisted of or what cultural changes the company made other than requiring two safety drivers instead of one (would that have made a difference during the 5.6 seconds the driver had to save Herzberg’s life?). Despite the fact Sumwalt stated during his opening remarks that he hopes other AV companies learn from this, it wasn’t at all clear what he hopes they learn.
Certainly, the hearing synopsis includes no hints. It includes recommendations to NHTSA, the state of Arizona, the American Association of Motor Vehicle Administrators, and, lastly, to Uber, which are below in full:
Complete the implementation of a safety management system for automated driving system testing that, at a minimum, includes safety policy, safety risk management, safety assurance, and safety promotion.
These “recommendations,” bland to the point of banality, are Homer Simpson-esque in their lack of substance, as if repeating the word “safety” enough will do the job.
To be clear, the NTSB has no power to actually punish anyone. It conducts investigations and issues recommendations, although they’re certainly within their rights to make better ones.
But the NTSB’s coziness with Uber—in a not-so-veiled jab at Elon Musk, Sumwalt thanked the Uber CEO for not hanging up on him—would merely be an odd curiosity if it wasn’t emblematic of the structural failing to hold Uber accountable in any meaningful way for Herzberg’s death. Prosecutors declined to charge Uber for any criminal wrongdoing. Herzberg’s family reached an undisclosed settlement with the company less than two weeks after the crash before many of the underlying facts of the case came to light.
In the end, the person who may end up being held most accountable in all of this is the safety driver. Prosecutors have not ruled out charging her, and she was the only person mentioned in the NTSB’s probable cause statement.
To be sure, the safety driver is far from blameless. She was intermittently watching clips from The Voice on her smartphone propped below the steering wheel in the minutes leading up to the crash. She was not paying attention to the road. Doing so may have saved someone’s life.
But that is exactly it; the role of the safety driver is to save lives before they are taken, not to drive the car. It is a thankless and inevitably doomed task when sitting in a car driven by a bad computer program. To conflate the safety driver as the operator of the vehicle, as the NTSB does when it refers to her as the “operator,” is a categorical error. She was not driving the car, the computer was. That was not an error, but the purpose.
The core wrong here, aside from Uber’s series of decisions to put cars driven by computers with unacceptably high failure rates on the road, was Uber’s ignorance—or willful dismissal of—decades of research that shows humans don’t share responsibility with computers well, research its rival Waymo had embraced and learned from years prior.
Waymo, armed with much of the same information and technology as Uber, determined it would not take the risk. Uber did.
Someone must always be in control of the car. When that car is driven by a computer program, the company that makes that program is in control. When that computer program is very bad at driving cars, then no one is in control. Even a human backup driver cannot make up for that.
The most cynical interpretation of the safety driver’s role in all this is one of the fall guy. In this interpretation, they are there precisely to be blamed in case something goes wrong.
I don’t know if I’m willing to go quite that far, but if that was the plan all along, then the Uber crash suggests it was a good one. After all, Uber is in the legal clear, having paid next to no price both legally and financially for this catastrophe. They even got an attaboy from the crash investigators for having said all the right things. The safety driver still might go to prison.