Model Compression Techniques for Scalable Deployment of Generative AI in Automotive Systems

Abstract

The deployment of generative AI models within automotive systems is fundamentally constrained by the limited computational resources, memory capacity, and energy budgets of in-vehicle hardware. In a companion article, we examined the scalability challenges, current applications, and evolving E/E architectures that frame this problem. The present paper builds on that foundation by providing an in-depth survey of model compression and optimization techniques that enable the deployment of large neural networks on resource-constrained automotive platforms. We first present an overview of deployment strategies including edge computing, hardware selection, optimized inference frameworks, AI compilers, dedicated accelerators, and knowledge distillation. We then conduct a detailed review of three core compression methodologies: pruning (both with and without retraining), quantization (post-training and quantization-aware approaches for both vision and language models), and low-rank tensor decomposition (including Canonical Polyadic, Tucker, Tensor Train, and Tensor Ring methods, as well as Neural Architecture Search-based compression). The techniques were evaluated by analyzing the underlying principles, reviewing the current available methods, and discussing how the techniques can be used for particular situations in an automotive environment. By using a comparative analysis, trade-offs between compression ratio, speedup for inference, accuracy maintained, and hardware compatibility were all evaluated together so that some conclusions could be drawn. The results of this evaluation provide evidence that multiple compression techniques should be used in combination to achieve the best opportunity for deploying generative AI in real time within an automotive environment. We identify this as a open research direction for the automotive AI community.