AI workloads are pushing automotive sensors harder, forcing engineers to rethink how long safety-critical systems can be trusted.
Key Takeaways
The automotive industry is racing to integrate edge AI technology into autonomous and assisted driving, but it’s not clear yet how those systems will age under massive compute demands and harsh road conditions, which can vary greatly from one region to the next, and from one user to the next.
Safety-critical systems in vehicles are expected to last a decade or more, but workloads will differ depending on how often, how aggressively, and where a vehicle is driven, as well as how many corner cases and unexpected stresses it encounters along the way. Automotive companies and their suppliers are pouring tens of billions of dollars into autonomous driving. Nevertheless, questions remain about how and when the automotive sensors that supply critical data to those systems will degrade, how to ensure AI models effectively manage aging — especially with software upgrades and ongoing learning by AI systems — and how to design whole systems capable of meeting the future demands of rapidly scaling AI.
AI must perform flawlessly in safety-critical systems, and there is evidence that it is effective in avoiding accidents. A just-released report by the Insurance Institute for Highway Safety showed that Waymo’s L4 vehicles experienced 68% fewer reportable crashes than human drivers, and a 91% lower rate of rear-ending another vehicle. But the technology is so new that no solid data exists for how and where those rates might shift over the course of a decade.
AI impact on sensor health, aging
The integration of AI into cars and trucks is fundamentally changing what’s expected of automotive systems. Data-heavy AI workloads only add to the pressure, causing sensors and other electronic components to age more rapidly. “As AI is ‘always on,’ it contributes to the gradual decline of the hardware within automotive systems,” said Paul Karazuba, vice president of marketing, Silicon IP at Rambus. “Traditional automotive electronics operate intermittently, whereas modern software-defined vehicles run continuous AI inference for perception, sensor fusion, driver monitoring, and autonomous functions. Sensors like cameras, radar, lidar, and the chips processing their data remain active throughout vehicle operation. The result of this is several conditions that negatively affect the lifespan of those sensors — higher average temperatures from sustained compute workloads, continuous electrical stress on transistors, reduced thermal recovery periods because processors and sensors are rarely idle, and long-term drift in analog components and sensor calibration.”
This requires a whole different kind of AI chip. “As AI is responsible for so many aspects of the automotive lifecycle now, it can age the system faster, meaning the hardware and software must be built to last,” Karazuba continued. “Because of this, chipmakers are adjusting how vehicles are designed to ensure they can withstand the strain of AI workloads, along with the realities of harsh physical environments.”
These automotive AI workloads are only going to get more strenuous as more autonomy is added to vehicles. Engineers developing AI models that will manage these vehicles may not have a clear picture of how these models will impact the hardware, especially as these models are updated and learn on their own. “When you’re looking at AI computation for things like autonomy processing, it becomes a lot harder, and the shift to LLMs and transformer models that have cropped up in the last three to five years has pushed the performance requirements of processing these large models through the roof,” said Rob Fisher, senior director of product management at Imagination Technologies. “That’s a very hard thing to account for because there’s only so much more processing you can put into an SoC speculatively before the requirements are there, because it becomes cost-prohibitive. It’s very much a balance of trying to understand where the applications are going to be in the future, deriving the performance you need, and then determining how much you’re willing to bet on that performance being a requirement in the future.”
How exactly these new workloads will cause a specific sensor to age is difficult to predict, so systems must be designed to confront a wide variety of sensor degradation challenges.
“Each sensor model or each sensor will age differently,” said Lukas Klose, head of the AES AI Solution Center at Keysight Technologies. “There won’t be two sensors aging the same way. So the real problem we are facing is that, from the current understanding, there won’t be an individually trained AI model for each vehicle. There will be one AI model for the fleet of vehicles with the same software version, and this AI model must deal with a lot of different ways of sensor degeneration. This leads to the challenge of understanding broadly and making sure that we catch all corner cases in the design process.”
Even low-level sensor system degradation can radically change the output of a vehicle’s key AI systems. “With AI, the system predicts the most likely outcome rather than computing an exact answer from predefined rules. You get a close approximation — hopefully — of what the actual value is,” Klose said. “But there’s always a margin of error, and this margin of error needs to be determined. The problem is that for different situations, you may get very different outcomes with only slight variations on the input. It might be one pixel changed on an on-camera image, leading to the effect that a 30 miles-per-hour sign is detected as 80 miles-per-hour. You must start by understanding how a sensor can fail.”
Tracking sensor health
While the integration of AI is driving faster component aging, AI will also play a major role in how sensor health is tracked. “AI can provide continuous analysis of telemetry, including vibrations, temperature, and output fluctuations to detect micro-anomalies, allowing systems to flag degradation ahead of sensor failure,” Karazuba said. “The typical practice incorporates on-chip aging monitors such as ring oscillators and similar structures that track how circuit timing shifts over life. SDVs are also taking this a step further with multiple additional methods. SDVs can assess how a sensor’s calibration and output drift over time and account for variations. Moving forward, AI will likely be able to understand how sensors should perform, and model that against how they are expected to decline over time and what early-failure warning mechanisms or signals could be used to generate health indexes and remaining useful life data.”
Developing a system that can quantify sensor health is easier said than done, but such advancements could open pathways to significantly reducing the negative effects of component aging.
“The challenge comes from the fact that degradation does not always show up as a linear function in the sensor readings, and the profile can be very different depending on what is being measured and where in the design,” noted Lee Harrison, director of product marketing, Tessent Yield Learning, at Siemens EDA. “That means to give an accurate health score, for example, where the device has X remaining days of operation before it fails, a very complex mathematical model is required. This is an area where AI can be used, as it could potentially learn from a deployment of, say, several hundred thousand devices in a data center to build its own accurate model of the silicon lifecycle. This data can also be used to aid the deployment of compute resources. For instance, instead of retiring all silicon that is running at, say, 25% health, the data could be used to ensure that the degrading silicon is not used to run any high-load critical tasks, but is reserved for smaller, less-intensive tasks, extending the life of the silicon still further.”
With thin margins for error and so much to consider, the more sensor health data a system can collect, the better. “A sensor may measure active circuitry directly to understand aging or mimic the device to demonstrate an aging effect. “More data points are better because one also needs to understand if the impact is local versus global on a device,” said Marc Hutner, director of product management for Tessent Yield Learning at Siemens EDA. “There are many causes of aging, which can be traced to thermal variation, high power, high switch factor, etc., and all of these types of sensors lead to better operation of the end system and minimize downtime for the end user.”
Edge AI has also expanded the data tracking capabilities that developers can build into automotive components. “Integrating artificial intelligence presents us with an opportunity to connect the data in ways we may not have thought possible,” said Hutner. “Previously, we would focus on a single sensor readout with a small number of analysis types that were obvious for the end user. We now have the option to connect various sensor types for deeper insights that can lead to a more holistic view of the operation of a car. This could be within a chip, such as a module with multiple sensors, or across the entire car. Initially, the industry is focused on connecting sensors that are close to each other to focus the compute to small regions, but we see possibilities for much greater insights.”
Keeping sensor data protected
Regardless of how much data a system can track, ADAS systems must be capable of determining if data inputs are faulty, because a wrong decision could mean disaster. “Safety systems are only as good as the data that feeds them,” Rambus’ Karazuba said. “SDVs today depend on highly accurate sensors for safety-critical decisions. Any sensor with premature aging could produce inaccurate data in the vehicle’s safety and driving systems that could result in degraded object detection, incorrect sensor fusion, or unsafe vehicle behavior. A drifting sensor can lead to incorrect — and potentially dangerous — AI decisions.”
Sensor degradation must be modeled into AI systems during the development phase for safety-critical systems. “We have to model this degeneration over time, and we have to take it into consideration in our safety analysis and build our operational design domain around this consideration,” Keysight’s Klose said. “We must define what happens if certain pixels die. What happens if certain properties of the sensor degenerate? Can we model this? Can we create data to train our system on it? If yes, can we understand how this impacts overall safety? Most importantly, how does the degeneration change the safe interval of trust? Are there things happening that we cannot explain?”
That uncertainty is where sensor aging stops being a maintenance issue and becomes a safety threat. If an AI system cannot tell when inputs are drifting, every decision it makes rests on a weaker foundation. But ensuring everything is working properly throughout its projected lifetime comes at a cost.
“Safety is expensive, so implementing a safety system is ultimately going to be much more expensive than implementing a consumer-grade system,” Imagination’s Fisher said. “Automotive sensor aging is going to increase because of the high utilization of AI and the compute workloads that are occurring, so you need better ways of detecting faults because they’re going to occur soon. You need ways of creating redundant systems around that. That’s going to increase the necessity of safe components within the SoC. On the other hand, as electrification drives centralization of the vehicle architecture, more processing is done on fewer devices in the car, so those devices are becoming single points of failure. You’re also not going to be able to say, ‘I only need the CPU and NPU to be functionally safe because I’m never going to run anything functionally safe on the GPU.’ That’s not the case anymore. Increasingly, more of the SoC is going to be safety certified, maybe up to ASIL-D, maybe even higher.”
This is complicated by the fact that AI safety processes are opaque, despite efforts to make them more explainable. “Previously, you had a clear decision path, and you understood the algorithm,” Klose said. “Now it’s abstracted to a black box, and this is why we have to go through the hassle of safety cases and statistical analyses to make sure we understand this black box and get close enough to having a confidence level of how safe the system really is.”
Getting inside that black box calls for interdisciplinary research and collaboration across ecosystems.
“Semiconductor manufacturer engineers, software manufacturers, and AI engineers have to work together to share how models degrade,” Klose said. “This is something we learned from the past. We can calculate the risk analysis and threat analysis based on those models and consider them in the way we train AI, and how we test AI to still be safe in those situations. So even for perception tasks, this is still a very complex thing to solve as a community of researchers and engineers.”
Keeping pace with AI
However, planning alone isn’t sufficient. Engineering teams must build their systems with the ability to be updated as AI continues to evolve. “In automotive, the challenge is no longer just how silicon ages over a vehicle lifetime, but how AI models, software, and workloads evolve over that same period,” said Robert Day, go-to-market director for physical AI at Arm. “Hardware must be designed to support continuous software and AI updates long after a vehicle leaves the factory. As AI models become more capable and more demanding, vehicles need flexible compute that can adapt to new workloads without requiring new hardware. General-purpose CPUs play an important role by providing a stable execution platform that can continue running AI workloads directly, or serve as a fallback alongside dedicated AI accelerators.”
Companies will also need to collect and evaluate telemetry data from end users to ensure that their AI is functioning to standard as sensors age. “In the case of Tesla, if you read through the fine print, they’re getting a lot of telemetry. It can’t be personally identifying telemetry, but it can be telemetry in general,” said Reed Hinkel, director of strategic programs at Synopsys. “They collect it all, then they upgrade their algorithm. It’s almost like Google Maps or Waze. They’re collecting all that information to establish trends, but here they’re just feeding it directly into the models.”
To use collected data to improve AI systems, the technology inside the vehicle must meet a new standard of flexibility.
“A lot of our customers are thinking they need a more flexible, programmable way of computing this type of workload, and they turn to the GPU, which is massively parallel and designed to be an efficient processing engine to do this,” Imagination’s Fisher said. “While a lot of our customers still use large amounts of NPUs, they’re adding GPU processing to cover some of this AI compute, or signal processing type of compute, so they can get sufficient performance even when they don’t know exactly what the workloads are going to be.”
There is still much work to be done. “Trust needs to be built up by technology that holds promise. That means we need to improve on the sensor, and on the AI to drive perception and understand where the car is and what it should do,” said Peter Schaefer, chief sales officer for automotive at Infineon Technologies, during a recent presentation. “On one hand, we have to make it very dependable. On the other hand, we need to drive innovation forward, because the cars we have today are not yet those cars we really want to have in the future.”
Systems need to function as expected, but as the amount of integration increases, rapid recovery is essential. So even though battery functionality may seem a long way from AI sensors, just shutting everything down with a fast-acting pyrofuse that disconnects high-voltage circuits in electric vehicles can cause safety issues. Unlike a gas-powered vehicle, in a BEV or PHEV, the car will stop dead if there’s a problem in the battery management system, potentially disabling any safety-critical systems. “Typically, the battery protection is a pyrofuse in a car,” Schaefer said. “The problem is, if it blows up, you have to bring your car to the garage to fix it,” he said, explaining why that fuse needs to be replaced by a circuit breaker. “A circuit breaker provides the kind of protection that doesn’t destroy the fuse.”
AI and sensor aging impact on automotive security
Sensor degradation also poses security concerns for software-defined vehicles in the automotive sector. “AI introduces some new concepts, including the non-deterministic type of aging, meaning there is some accuracy degradation that may become a security vulnerability, not just performance loss,” said Dana Neustadter, senior director of product management for Security IP and Solutions at Synopsys. “AI systems make it more important than ever to move from projecting at design time to adapting over a lifetime. For the AI system, it’s not just about making sure the model works correctly. The behavior needs to be consistent over time. You need to have some form of more complex cross-layer trust correlation as AI introduces some new concepts that need to be considered for security.”
Widespread adoption of ADAS and AV systems in consumer vehicles should put developers on high alert for both digital and perception attacks.
“If the analog signal is maliciously tampered with from the digital side, you can really amplify the effect,” Neustadter said. “Are there improved methods for more attack-aware facts and sensor fusion so you don’t only focus on the redundancy, but the fusion that can also assume some adversarial attacks may be possible? Security for analog is very complex. We’ve been talking about securing from the digital side, where we talk about cyberattacks. With analog, it’s more of a perception kind of attack that needs to be addressed. If an analog signal is maliciously modified or tampered with, the downstream digital processing may amplify that type of bad behavior. Security considerations need to be investigated from the analog side better than what is being done today.”
Securing these systems is another challenge that will likely require cross-company collaboration. “We expect there will be a lot more golden data that’s pre-vetted and maybe even shared between companies,” said Hinkel. “That golden data helps you measure. And if you’re using those models on things that you check into golden data models, that gives you an ability to maybe detect where somebody’s trying to inject false data and determine any kind of drift that might be happening that is material to the operation.”
Like preparing for sensor degradation in general, securing this technology through the future also calls for a blend of advanced planning and quick-turn adaptability.
“If they have methods for monitoring the drift in AI models, or the training data and assumptions for it, or the inference stack aging, they have means to do refreshes, where they also have signed updates and can leverage secure boot, code review, and vulnerability management,” Neustadter said. “There needs to be some improvements there, where you continuously have ways of reviewing it and making sure that you attest that there are no drifts in the model, and for training data, that there is room for better and improved mechanisms for tracking the data provenance and the lifecycle security of it. There are methods today, but I don’t think that they’re enough for moving forward.”
Conclusion
With edge AI pushing the limits of what’s possible in vehicle technology, engineering teams need to incorporate advanced testing and modeling to ensure that the technology progresses safely. The path forward will require automakers, chipmakers, and AI developers to treat aging, adaptability, and security as interconnected design challenges. That means validating systems not just for how they perform on day one, but for how safely they can evolve over a vehicle’s lifetime.
Related Articles
From Highways To Health Care: Portability Proves Key To Physical AI
Success at the edge takes advantage of common threads — lithography, imaging, chiplets, and AI.
Alternative Materials For Hybrid Bonding
Nanotwinned copper, SiCN, BCB, and passivating metals are all possibilities.
AI-Defined Vehicles Increase Pressure On Auto Ethernet Reliability
Time-sensitive networking and MACsec allow Auto Ethernet to handle safety-critical functions and enable real-time processing for AI-enabled features and agents.
Leave a Reply