TLDR: Uber has launched PerfInsights, a new generative AI-powered system designed to automatically identify and recommend performance optimization opportunities in Go backend services, significantly reducing manual effort and operational costs. The tool leverages runtime profiles and GenAI-driven static analysis, validated by LLM juries and LLMCheck, to deliver precise and trustworthy optimization suggestions, transforming performance tuning into a scalable and continuous process.
At Uber, the efficiency of backend services directly impacts operational costs and user experience. In March 2024, the top 10 Go services alone accounted for multi-million dollar compute expenditures, underscoring a critical need for systematic performance tuning. Historically, optimizing Go services demanded deep expertise and considerable manual effort, with profiling, benchmarking, and code analysis often taking days or even weeks, making it a prohibitively expensive and non-trivial task for most teams.
In response, Uber developed PerfInsights, a system that originated as a finalist in Uber’s Hackdayz 2024 and has since evolved into a production-ready solution. Its core purpose is to automatically detect performance antipatterns in Go services. PerfInsights achieves this by combining runtime CPU and memory profiles with generative AI (GenAI)-powered static analysis to pinpoint expensive ‘hotpath’ functions and recommend specific optimizations. This approach has yielded transformative results, reducing the time required for detection from days to mere hours and enabling engineers to implement high-impact performance improvements without needing specialized knowledge of compilers or runtimes.
A distinguishing feature of PerfInsights is its emphasis on precision and fostering developer trust. Beyond merely identifying optimization opportunities, the tool validates its suggestions using ‘large language model (LLM) juries’ to mitigate ‘hallucinations’ and enhance confidence in its recommendations. This rigorous validation process has led to hundreds of code changes (diffs) generated by PerfInsights being successfully merged into Uber’s Go monorepo, effectively transforming optimization from a specialist-driven endeavor into a scalable and repeatable practice.
PerfInsights’ optimization pipeline operates in two primary stages: profiling-based function filtering and GenAI-driven antipattern detection. For filtering hotpath functions, it leverages CPU and memory profiles from production services, collected by Uber’s daily fleet-wide profiler during peak traffic. It identifies the top 30 most CPU-intensive functions. If `runtime.mallocgc` (Go’s memory allocation function) consumes over 15% of CPU time, PerfInsights also analyzes memory profiles to uncover potential allocation inefficiencies. A static filter then excludes open-source dependencies and internal runtime functions, ensuring the analysis focuses solely on service-owned code most likely to benefit from optimization.
At the heart of its detection engine is a curated catalog of performance antipatterns, informed by the Go Foundations team’s extensive optimization work and Uber’s Go style guide. These patterns encompass common inefficiencies such as unbounded memory allocations, redundant loop computations, and inefficient string operations. PerfInsights employs a two-stage detection process: the full source code of hotpath functions and a list of antipatterns are passed to an LLM for analysis, allowing it to precisely identify inefficient constructs, such as appending to a slice without preallocating capacity.
To ensure high confidence in its findings, PerfInsights incorporates two robust validation methods: LLM juries and LLMCheck. LLM juries involve multiple LLMs independently assessing the presence of an antipattern and the validity of the suggested optimization, thereby mitigating common LLM hallucinations. LLMCheck provides a second layer of verification through domain-specific rule-based validators designed to catch false positives (e.g., confusing maps with slices, incorrect loop invariant detection). This dual-validation strategy has dramatically improved precision, reducing false positives from over 80% to the low teens.
Uber’s team also emphasized the importance of prompt engineering for reliable LLM results. Key strategies include few-shot prompting (providing illustrative examples), tailoring the model’s role (e.g., as a ‘Go expert’), ensuring output quality (prompting the model to test suggested fixes for runnability), and writing clear, focused prompts (using specific, positive instructions, one prompt per antipattern, and separating detection and validation prompts). Asking for confidence levels in the LLM’s response also encourages deeper model reasoning.
Since its launch, PerfInsights has significantly impacted performance improvements at Uber. It has accelerated engineering velocity and driven scalable compute cost reductions across Go services. The tool seamlessly integrates with downstream tools like Optix, Uber’s continuous code optimization tool, resulting in hundreds of merged code changes and measurable cost savings. Furthermore, it has notably boosted code health; validated detections averaged 265 daily in February, peaking at 500, and by June, this figure dropped to 176—a 33.5% reduction in just four months, indicating a cleaner codebase.
The engineering effort saved is substantial. Previously, uncovering performance antipatterns demanded immense, specialized effort. For instance, addressing 5 critical antipatterns once required two engineers for a full month (approximately 320 hours), and 11 unique antipatterns consumed a four-person Go expert team for a full week (approximately 160 hours). A dedicated Go expert spent six months full-time on related optimization projects (approximately 960 hours). These manual efforts alone totaled over 1,400 hours for just a handful of cases. PerfInsights has reduced the time required to detect and fix an issue from 14.5 hours to approximately 1 hour of tool runtime, representing a remarkable 93.10% time savings.
Beyond monetary savings, PerfInsights has elevated engineering rigor. LLMCheck dashboards provide teams with visibility into detection accuracy and error patterns, cutting hallucination rates by over 80% and fostering greater trust in AI-assisted tooling. Most importantly, PerfInsights has redefined performance tuning as a continuous, data-driven discipline. Its integration into CI/CD pipelines and daily developer workflows ensures that optimization opportunities are surfaced regularly, shifting from reactive troubleshooting to proactive, embedded optimization across Uber’s engineering lifecycle.
Building PerfInsights into a production-ready system provided valuable lessons, particularly in integrating GenAI tooling into a complex, high-scale engineering ecosystem. Key takeaways include the critical importance of prompt engineering and model selection, the ‘unsung hero’ role of aggressive static filtering using CPU and allocation profiles to constrain the search space, and the necessity of robust validation pipelines to build developer trust. The initial success of demonstrating tangible savings and improvements (e.g., the ‘first 5 digits saving diff’) was crucial in driving adoption and making PerfInsights an integral part of Uber’s engineering toolkit.
Also Read:
- Harness Revolutionizes DevOps with Expanded AI-Powered Pipeline Generation
- Meta’s AI Assistant Streamlines Code Reviews
In conclusion, PerfInsights marks a pivotal moment in Uber’s approach to performance engineering, transitioning from slow, expert-led investigations to scalable, GenAI-assisted optimization. By fusing real-world production data with targeted static analysis and validated LLM workflows, Uber has built a system that delivers meaningful performance wins quickly and reliably. This shift has already paid dividends by freeing up developer time, lowering compute costs, and expanding access to performance best practices across teams, showcasing the power of applying GenAI with precision, domain knowledge, and a strong bias for automation.
