TLDR: This paper evaluates the performance of Large Language Models (LLMs) within a confidential computing environment using Intel Trust Domain Extensions (TDX) for System-on-Chip (SoC) design tasks. It finds that distilled and quantized LLMs, particularly smaller ones like DeepSeek-r1-1.5B, perform efficiently and securely within TEEs, often outperforming traditional CPU-only setups. While GPUs offer the highest speed, current TEEs lack full GPU support, making lightweight, optimized LLMs a practical solution for secure hardware design applications.
Large Language Models (LLMs) are becoming increasingly vital in various fields, including the complex world of circuit design. However, the sensitive nature of the data and the models themselves in System-on-Chip (SoC) design raises significant concerns about intellectual property protection and data breaches. This is where confidential computing steps in, offering a promising solution through Trusted Execution Environments (TEEs).
TEEs, like Intel’s Trust Domain Extensions (TDX), create secure enclaves where data and computations are isolated and protected from unauthorized access. While TEEs provide robust security, they traditionally haven’t been optimized for the resource-intensive demands of LLMs. This research paper explores how to efficiently deploy LLMs within these secure environments, particularly for SoC design tasks.
Optimizing LLMs for Confidentiality
The study highlights a crucial observation: ‘distilled’ LLMs, which are smaller, more efficient versions of larger models, are particularly well-suited for resource-constrained secure environments. Models like DeepSeek-r1-1.5B, a distilled variant, demonstrate superior performance due to their reduced parameter count. This makes them ideal for deployment on devices with limited computational resources, common in secure settings.
Another key optimization technique explored is ‘quantization’. This process reduces the numerical precision of model weights, converting them from floating-point values to integers (e.g., 4-bit or 8-bit quantization). The research found that quantized models can achieve significant performance gains—up to 3 times faster for 4-bit quantization compared to standard 16-bit models—while also drastically reducing their storage size. For instance, 4-bit quantization can shrink a model to about 30% of its original size, which is vital for TEEs with limited memory.
Performance Across Different Environments
The researchers conducted experiments across three environments: a TEE-based setup (using Intel TDX), a CPU-only environment, and a CPU-GPU hybrid setup. They measured performance in terms of ‘tokens per second’, indicating inference speed.
For smaller, lightweight LLMs (under 3 billion parameters), the TDX environment surprisingly outperformed the traditional CPU-only setup, sometimes by more than double the speed. This suggests that TDX’s optimized CPU instruction set architecture (ISA) can minimize inefficiencies for these smaller models within a secure enclave. However, as model sizes increased (e.g., 14 billion parameters), the performance advantage of TDX narrowed, and in some cases, the CPU-only environment became faster, indicating that larger models demand more memory bandwidth and computational resources that current TDX optimizations might not fully address.
Unsurprisingly, GPU acceleration provided the highest inference speeds, often 20 times faster than CPU execution and 9 times faster than TDX for smaller models. However, a significant limitation is that current TEE implementations, including Intel TDX, do not yet fully support GPU utilization within the secure enclave. This means that while GPUs offer speed, they compromise the confidentiality guarantees of the TEE, as data is processed in plaintext outside the secure domain.
Also Read:
- Unlocking Efficient Privacy in Machine Learning: A Deep Dive into Cross-Level Optimizations
- Unlocking Performance: How AI Models Tackle Specialized SIMD Code Generation
Implications for SoC Design and Future Directions
The findings have significant implications for the use of LLMs in SoC design, where sensitive information like RTL designs and circuit specifications must be protected. The study evaluated domain-specific LLMs, fine-tuned for hardware and security tasks, and found that these specialized models also benefit from TDX’s secure deployment, often exceeding raw CPU-only performance while maintaining confidentiality.
In conclusion, this research demonstrates the feasibility of running LLM inference securely within TEEs for SoC design workflows. It emphasizes the critical balance between security and computational efficiency. While GPU acceleration remains the fastest, the paper highlights that for lightweight, optimized LLMs, a TEE-only approach offers a viable trade-off between performance and robust security. Future research will likely focus on integrating GPU acceleration more securely within TEEs to achieve both high security and computational efficiency for even larger models. For more details, you can read the full research paper here.
