Silicon Valley’s Patagonia-wearing elites aren’t the only ones interested in autonomous vehicles.
The U.S. Department of Defence, once a pioneering incubator for early AV development, is now interacting with top AV startups to imagine the next era of potentially autonomous military vehicles. Though Defence officials signalled to Gizmodo the department’s bullishness on the emerging technology, they’re simultaneously concerned over potential hacking and cybersecurity risks.
Gizmodo joined Deputy Secretary of Defence Kathleen Hicks and a group of officials as they observed an autonomous vehicle presentation this week at the SLAC National Accelerator Laboratory in Menlo Park California. When asked about the state of the AV industry immediately following the event, one of Hicks’ Special Assistants spoke positively but said there’s “still some kinks that needed to be worked out.” Foremost among those were concerns around potential hacking and security, the official said.
A quick note: This particular autonomous carmaker, which is associated with one of the nation’s largest tech firms and is widely known for its testing on the SLAC grounds, asked Gizmodo not mention its name in this story.
DSD Hicks, who spoke with the unnamed carmaker about AV cybersecurity during the demonstration, also highlighted potential security concerns in an interview with Gizmodo later that day.
“I think it just shows you how long a path this [autonomous vehicles] is, even in the commercial sector, to get all the trained information, the data you need, the security environment, and then the cybersecurity.”
Those security concerns aren’t necessarily unwarranted. Autonomous vehicles, like any other piece of computerised technology, are by definition susceptible to some degree of would-be attackers. Those attacks can potentially occur by tricking a vehicle’s cameras and sensors into veering into the wrong lane (as was demonstrated by Keen Security Labs researchers in 2019) or by targeting software vulnerabilities much like one would on any other computer. Just last month, a teen security researcher named David Colombo was able to remotely hack into 25 Tesla’s spread out across 13 countries in about an hour due to a security vulnerability in an open-source logging tool. In that case, Colombo was able to remotely start the car, blast the stereo, and unlock doors all potentially without the vehicle’s owner knowing. In other cases, reports have shown the ability to force a car buzzing along a highway at 113 km/h to come to a grinding halt by initiating an exploit.
As it turns out, the military’s not the only one with AV safety concerns. Just 34% of U.S. adults surveyed in a new Morning Consult survey said they trusted driverless vehicle technology, with only 9% going out even further on a limb and saying they trust the tech “a lot.” A third simply said they don’t trust it at all.
“Officials at the DoD should be concerned with the hacking of AVs,” Jane A. LeClair said in an interview with Gizmodo. LeClair, a professor and Chief Operating Officer at the Washington Centre for Cybersecurity Research & Development also told us that the researchers who have already demonstrated their ability to hack AVs were likely less sophisticated than a potential U.S. military adversary looking to do damage. “Much has to be done to insure the cybersecurity of AV systems,” LeClair said, “And it should be integrated into the system, not added on as an afterthought.”
AV development has taken “longer than I ever imagined”
When speaking with Gizmodo, Hicks said she believed more advanced autonomous use cases for the DoD were by no means out of reach, but was quick to acknowledge the technology’s current limitations.
“It’s more elusive than I think it should have been,” Hicks said. In the near term, Hicks highlighted what she saw as the importance of secure testing of autonomous systems in specific environments.
“The easiest use cases are the cases that are in the least complex environments and where we have a lot of data to create a trusted set of data and an environment in which to operate that,” Hicks said. The Deputy Secretary of Defence pointed out what she viewed as the importance of training these still budding autonomous systems in “relatively safe environments,” where data can be securely collected and inferences made. Whenever DoD does accelerate its autonomous vehicle testing, Hicks said the Department would do so while adhering to principles of a “human-in-the-loop” system.
While the Pentagon holds fast to its human-in-the-loop principle, certain generals and influential voices on U.S. AI policy — such as former Google CEO Eric Schmidt — have called that principle into question. Schmidt, who was tasked by the Trump administration to co-head the National Security Commission on AI, recently wrote a book with Henry Kissinger arguing for a less restrictive, more hawkish U.S. AI approach.
“I can appreciate the concern of DoD officials in leaning towards closed testing areas,” LeClair said. “One can’t have an Abrams tank running over a soccer mum in her minivan can we!” Jokes aside, LeClair acknowledges real-world tests of these AI systems are inevitable at some point but said “that may be a bit later.”
Commenting on the maturity of the AV industry generally, Hicks said progress has taken “longer than I ever imagined.”
“Trying to figure out why it [AV development] keeps pushing [back], I think is a key question that I think many senior leaders in the department have.”
A Defence official told Gizmodo the Department is also interested in sensors and data collection related to AVs. The official said the DoD has been keeping an eye on autonomous advancements from companies like John Deere which are developing autonomous systems for more off-road applications. John Deere, known for its iconic green-framed giant yellow-wheeled agricultural workhorses earlier this year revealed a fully autonomous tractor that pulls together a GPS system with 12 cameras that enable 360-degree obstacle detection.
In theory, the company argues farmers of the future could simply program their tractors and walk away as it autonomously plows, plants, and sprays its fields. DoD isn’t exactly getting ready to hang up its tank cannons for farmers picks, but it does see an opportunity to learn from private industry to improve autonomous in rugged, off-road scenarios military vehicles are more likely to face. In other words, an autonomous military truck may actually have more in common with an industrial tractor than it does a Tesla cruising down a San Francisco highway.
Defence industry has deep ties with autonomous tech
The AV industry of today likely wouldn’t exist if it weren’t for the DoD. Like the internet and GPS before it, driverless technology got a huge early boost from DARPA, the Pentagon’s anything goes, gonzo research arm. Nearly two decades ago in 2004, the agency held its first “Grand Challenge” which awarded millions in prize money to scrappy competitors who could design vehicles able to autonomously navigate over long distances. None of the challengers’ vehicles were able to complete DARPA’s 229 km course. Nonetheless, the agency followed that challenge up with a pair of follow-up challenges, one in 2005 and another in 2007. All of this, DARPA claims, was meant to help spur innovation in what was then a large unknown.
Things have changed considerably since then. AV technology accelerated far past the confines of remote desert and into the Silicon Valley spotlight. Alphabet-owned Waymo, General Motors backed Cruise, and Intel Subsidiary Mobileye have all ramped up autonomous testing in the past year with hopes to deliver real consumer products in the near future. And then there’s Tesla, which despite offering comparably worse driverless capabilities than those other competitors, has complicated the landscape by allowing its Autopilot Beta testers to test its capabilities on public roads, safety concerns be damned.
Despite years (and I mean years) of broken promises from AV charlatans, LeClair says the auto industry in 2022 has seen “significant advances in electric and autonomous vehicles.” Ditto for AI writ large. According to Stanford’s recently released annual AI Index Report, global private investment in 2021 totaled $US93.5 ($130) billion. That’s more than double the investment in 2020.
The military’s been busy, too. Not settling for mere four wheel autonomy, DARPA recently completed a 30 minute autonomous test flight of a UH-60A Black Hawk helicopter without a human on-board. That autonomous flight was made possible thanks to an experimental Aircrew Labour In-Cockpit Automation System (ALIAS) system. AI controlled F-16 have meanwhile already beaten human pilots in simulated dogfights. DARPA’s even hoping to showcase four AI-powered L-39 jets engaged in a live dogfight over Lake Ontario by 2024.
Still, all those tests are just that — tests. Limited proof of concepts are one thing, but taking those and applying them to an actual shooting war is something else entirely. “Autonomous devices are showing great promise in the air and at sea,” LeClair said, “But the complexity of a land battlefield is a whole different environment for an AV to operate in. If the system of a Tesla has a hiccup, problems are limited but on the battlefield an error can be devastating.”
All of that’s either mind-shatteringly cool or shit-your-pants-scary depending on your level of comfort with future battlefields ripe with an assortment of AI armour. In the best-case scenario, all these autonomous systems could mean fewer soldiers in harm’s way. Despite voicing concerns, LeClair said that potential means autonomy may be worth it.
“I’m all for taking the man out of the machine and not putting our troops in undo danger, “LeClair said. “Perhaps the DoD should concentrate on the transport of supplies and equipment via AV…in a support role not on the battlefield yet.”