TLDR: AnalogSeeker is an open-source foundation language model designed to assist with analog circuit design. It addresses data scarcity by curating textbooks and distills complex knowledge into Q&A pairs. Using a custom training method (NSC-SFT), it significantly improves accuracy on analog circuit benchmarks and demonstrates capabilities in operational amplifier design, making it a valuable tool for the field.
A new open-source initiative, AnalogSeeker, aims to bridge the gap between advanced language models and the intricate world of analog circuit design. Developed by a team of researchers, AnalogSeeker is a foundation language model specifically tailored to integrate domain knowledge and provide design assistance for analog circuits.
Addressing Data Scarcity and Knowledge Complexity
Analog circuit design is a highly specialized field, and unlike software development, it suffers from a significant scarcity of publicly available, high-quality textual data. To overcome this, the AnalogSeeker team devised a unique corpus collection strategy. They systematically curated and cleaned accessible textbooks across various subfields of analog circuits, creating a high-quality textual domain corpus. This includes over twenty core textbooks covering everything from basic circuit theory to advanced integrated circuit design, encompassing at least twelve major circuit types.
Beyond data collection, the complexity of analog circuit knowledge presented another hurdle. This knowledge is often hierarchical and deeply embedded within unstructured text. AnalogSeeker tackles this by introducing a granular domain knowledge distillation method. The raw corpus is broken down into typical, granular learning nodes, and a multi-agent framework is then used to extract implicit knowledge. This process transforms unstructured text into structured question-answer data pairs, complete with detailed reasoning processes. This fine-grained, learnable dataset is crucial for fine-tuning the language model.
Customized Training for Analog Circuit Expertise
Training a foundation model for analog circuits presented unexplored challenges. The researchers explored and shared their training methods, combining theoretical analysis with experimental validation. They established a fine-tuning-centric training paradigm, customizing and implementing a novel algorithm called Neighborhood Self-Constrained Supervised Fine-Tuning (NSC-SFT). This approach enhances training outcomes by ensuring that the model’s output distributions remain consistent before and after training, helping to prevent ‘forgetting’ previously learned information while integrating new domain knowledge.
The AnalogSeeker model was trained based on the Qwen2.5-32B-Instruct model. Through rigorous experimentation, the team found that instruct models are more suitable starting points for domain adaptation than highly specialized reasoning models, which can be fragile when fine-tuned for new domains. They also discovered that, given their dataset’s characteristics, supervised fine-tuning (SFT) alone was more effective than combining it with continual pre-training (CPT), due to the relatively smaller size of the CPT dataset.
Also Read:
- Active Reading: A New Approach to Teaching LLMs to Master Facts
- Advancing Open-Source AI for Complex Web Research
Performance and Practical Application
AnalogSeeker achieved an impressive 85.04% accuracy on AMSBench-TQA, a benchmark specifically designed for evaluating analog circuit knowledge. This represents a significant 15.67% point improvement over the original Qwen2.5-32B-Instruct model and demonstrates competitive performance against mainstream commercial models like DeepSeek-v3 and GPT-4o.
Furthermore, AnalogSeeker has shown its effectiveness in practical applications, specifically in operational amplifier (opamp) design tasks. Integrated into the Atelier framework, AnalogSeeker acts as an agent, assisting with topology design, modification, and circuit analysis. It can propose rational circuit architectures, refine them into SPICE netlists, and identify tunable parameters, demonstrating its potential to provide valuable design assistance to engineers.
The AnalogSeeker project is open-sourced and available for research use, marking a significant step towards more automated and intelligent analog circuit design. The researchers hope their work and shared experiences will benefit the broader research community in this specialized field. You can find more details about this work in their paper: AnalogSeeker: An Open-source Foundation Language Model for Analog Circuit Design.
