Autonomous vehicles will require lots of sensors, vast computing power and AI software to manage multiple complex systems.
Autonomous vehicle development continues to attract large investments in transportation and other industrial sectors. Those bets are needed as a host of tough technical problems remain far from being solved.
As I see it, here are three key questions:
To answer these questions, a presentation summarized in three charts aims to provide some perspective for novices and experts alike.
AV complexity problem
The fundamental problem with AVs is the tremendous complexity involved in developing safe, reliable autonomous vehicle for SAE Level 4 capabilities. The first chart summarizes those challenges.
The AV problem is divided into three groups in the red blocks as shown in the above figure. The potential solutions are listed in 12 black boxes—four boxes for each problem group. Note that three boxes have blue text that are the same—because software platforms, AI software including machine learning and neural networks are required to solve the three problem categories.
The first is knowing the exact location of the AV with centimeter-level accuracy in most scenarios. The next step is classifying all road users and objects, including what, if anything, they are doing, and predicting what they may do in the next few seconds.
The solution to these problems is lots of sensors, vast computing power and the platforms and AI software required to manage multiple complex systems. For example, a typical robotaxi requires more than 30 sensors, including cameras, radars and lidars. For exsmplr, Zoox’s recent robotaxi announcement listed 64 sensors: 28 cameras, 20 radars and 16 lidars.
AI-based vision software also is required to process sensor data. High-definition maps are needed for most AVs for accurate location determination.
All software and hardware systems require extensive cybersecurity protection. Software must also be updated regularly with built-in, over-the-air (OTA) software update capabilities.
The second problem is ensuring AV hardware and software reliability with no single-point failures. In the event of a failure, a so-called “limp home” capability is needed that can at least guide a vehicle to the side of the road.
As AV regulations are introduced, safety and operational rules must be included as part of the system and reliability design.
Also needed are hardware redundancy within the system design. At least three AV systems require redundancy: driving controls (steering, braking, speed); vision sensor capabilities (three categories); and computing.
The system architecture must use technologies that can simplify software platform cooperation, allowing strong cybersecurity and OTA updates. A recent column covered these topics.
These systems remain quite expensive and will require significant cost reductions. Fortunately, chip-based technologies are providing substantial cost savings, especially for the most expensive component: lidar.
Simulation of AV components is critical, including software and hardware as well as all types of testing and modelling.
AV event data recorders will be required to gain insights into crashes and what can be done to improve safety. Teleoperation also is becoming standard in AV regulation, and can be the key to limp-home capabilities while solving edge cases.
Problem No. 3 is developing a software driver that exceeds human drivers. How much better is still being debated. It’s clear that the AV developers must continue to test and improve their systems. Development time will depend on the use cases.
Edge case testing is used extensively, and basically means finding new driving situations the software driver has not seen before and may not know how to handle. Adding new edge cases to the software driver capability is considered perhaps the highest priority.
Another hard problem is confirming that AV software drivers can outperform human drivers. It’s unclear how AV regulations and future AV type-approval will handle this important problem.
Solutions mostly involve testing, analysis of vast amounts of test data to identify software driver weaknesses, then more testing. Fortunately, much of this testing can be simulated at a much higher rate than road testing—up to 100 times more miles per day in simulation mode versus road testing. Those simulation are focused on edge cases and similar scenarios.
Testing must include different weather and lighting conditions. Most historical AV testing has been done in ideal weather conditions. Hence, the need for greater real-world simulations.
AV use cases
The complexities described above will vary considerably depending on the AV use case. AV complexity is mostly decided by driving complexity. The figure below is an overview of AV use case complexity, focused on SAE L4 deployments. Many variations of these scenarios are not included.
The chart above shows how various AV use cases fit into a two-dimensional space, with AV complexity increasing on the y-axis and driving complexity growing on the x-axis. Driving complexity includes route obstacles, driving speed, traffic density, road user variety (cars, bikes, pedestrians, etc.) and weather conditions. Fatality risks are also listed, mostly determined by speed. Some AV use cases have very low fatality risks.
Low AV complexity
Low AV complexity refers to simple routes, low speed and low user or traffic variety. At the simplest level, operation is restricted to closed areas such as a campus, office park or a military base. Sidewalk delivery vehicles are furthest along with multiple players. Starship, the sidewalk AV leader, surpassed 1.5 million deliveries in May 2021 and will soon surpass 2 million deliveries.
Fixed-route AVs also have low AV complexity, and the market niche includes multiple players. Deployment has been slow due to a high AV prices, but have been undergoing testing in hundreds of cities. Applications include low-complexity bus routes and/or closed environments.
Fixed-route AVs are also likely to be used on flexible trips such as on-demand pickups. The recent ISO 22737 low-speed autonomous driving (LSAD) regulation released in July 2021 should have positive impact on fixed-route AV deployment.
Goods-only AVs for last-mile deliveries bring with them more traffic complexity, navigating roads at higher speeds than sidewalk AVs. Vans and small trucks can also be retrofitted as AV delivery vehicles. They are undergoing testing using safety drivers.
Medium AV complexity
This category includes several AV scenarios. Low-speed goods AVs without a safety driver are in this category. Autonomous trucks with hub-to-hub routes may also be included, but for now require a safety driver. This category is also called middle-mile trucking.
If and when safety drivers are removed, teleoperation monitoring could be used for hub-to-hub trucking and robotaxis. Most AV regulations require teleoperation as a last resort for managing AVs if they are stuck. Teleoperation may also become a more pervasive technology, ultimately replacing of safety drivers.
High AV complexity
The figure above includes three use cases with high AV complexity. The hub-to-hub trucking use case has the lowest in this category, followed by robotaxis. Personal AVs, still on the drawing board, would also be categorized as high complexity. Personal AVs are likely to benefit from experience gleaned from robotaxi deployments across metro areas.
AV use case deployment
AV deployments will transition from simple to complex. The slightly modified use case chart below changes the x-axis to represent a timeline, affixing green labels indicating a rearrangement of the use case blocks. The AV use cases are positioned to reflect when on the timeline they are likely to see meaningful usage.
In this scenario, sidewalk AVs have the highest deployment rate, delivering meals, groceries and other small packages in many cities. Sidewalk AVs are also the least expensive products due to fewer sensors, less weight and pedestrian speed. Bumping into someone or something is relatively low risk.
Goods-only AVs are represented by the Nuro delivery vehicle, which is mostly in a testing mode. Current advertising indicates that Nuro may be poised for wider deployment.
Robotaxis remain primarily in the testing stage with a safety driver. Waymo has removed the safety driver in most of its Phoenix area testing. Several robotaxi operators have been allowed to charge for their services in a few U.S. and Chinese cities.
Goods AVs with safety driver are also delivering packages for last- or middle-mile operations between stores and/or warehouses.
Fixed-route AVs such as EasyMile, Local Motors and Navya have undergone extensive testing in several countries. The pandemic halted most testing that had focused on transporting up to 12 passengers per ride. The recent ISO LSAD regulation covers this use case and should kickstart fixed-route AV usage in the next few years.
Hub-to-hub autonomous truck usage with safety driver is seeing growing testing. Much of it includes transporting goods to paying customers.
The remaining categories are much harder to deploy and will arrive later than shown in the chart above. Hub-to-hub autonomous truck deployment may occur around 2025. Volume deployment of robotaxis is a few years later, but could happen in a few cities, according to some robotaxi hopefuls. Personal AVs will be significantly later than robotaxis.
AI addresses complexity
AV technology remains hard to do, but some use cases have less complexity and are being deployed in limited volumes. Regulations are emerging for simpler AV scenarios, and many companies will eventually deploy what is allowed by regulation.
Teleoperation will be required in all AV regulation, but it can also be used to remove safety drivers for earlier deployment of some use cases.
AV system cost is currently dominated by pricey lidar, which will decline rapidly over the next five years. That means excessive AV system cost will not be a showstopper after 2025.
Early deployment of complex AV systems ultimately depends on AI technology breakthroughs, which is impossible to forecast. If such innovation occurs, potential users may not have to wait until the 2030s for personal AVs.
This article was originally published on EE Times.
Egil Juliussen has over 35 years’ experience in the high-tech and automotive industries. Most recently he was director of research at the automotive technology group of IHS Markit. His latest research was focused on autonomous vehicles and mobility-as-a-service. He was co-founder of Telematics Research Group, which was acquired by iSuppli (IHS acquired iSuppli in 2010); before that he co-founded Future Computing and Computer Industry Almanac. Previously, Dr. Juliussen was with Texas Instruments where he was a strategic and product planner for microprocessors and PCs. He is the author of over 700 papers, reports and conference presentations. He received B.S., M.S., and Ph.D. degrees in electrical engineering from Purdue University, and is a member of SAE and IEEE.