TLDR: CoLLM-NAS is a novel framework that integrates collaborative Large Language Models (LLMs) into two-stage Neural Architecture Search (NAS) to efficiently design high-performing neural networks. It features a Navigator LLM for strategic guidance, a Generator LLM for synthesizing architectures, and a Coordinator module for managing their interaction and evaluation. By leveraging LLMs’ inherent architectural knowledge and progressive feedback, CoLLM-NAS significantly reduces computational costs and outperforms existing NAS methods, achieving state-of-the-art results on benchmarks like ImageNet and NAS-Bench-201.
The field of deep learning constantly seeks more efficient and powerful neural network designs. Traditionally, this has been a labor-intensive process, often relying on expert intuition. Neural Architecture Search (NAS) emerged as a way to automate this design, but existing methods often struggle with high computational costs, generating invalid architectures, or failing to outperform human-designed networks.
Recently, Large Language Models (LLMs) have entered the NAS arena, promising to leverage their reasoning and code generation abilities. However, many LLM-based approaches have faced their own set of challenges, such as creating architectures that don’t work or requiring extensive training for each candidate, leading to significant computational overhead.
Introducing CoLLM-NAS: A Collaborative Approach
A new framework called Collaborative LLM-based NAS (CoLLM-NAS) aims to overcome these limitations by integrating the strengths of LLMs with established two-stage NAS methods. This innovative system employs a knowledge-guided search process driven by two specialized LLMs working together, managed by a dedicated Coordinator module. The goal is to make neural architecture design more effective and efficient.
CoLLM-NAS operates in an iterative loop, combining the LLMs’ inherent understanding of neural architectures with new insights gained from ongoing feedback and historical search paths. This allows it to intelligently navigate the vast landscape of possible network designs.
How CoLLM-NAS Works
The framework consists of three main components:
-
Navigator LLM: This is a ‘stateful’ LLM, meaning it remembers past interactions. It acts as the strategic guide, analyzing performance patterns from evaluated architectures and dynamically refining search strategies. Initially, it encourages diverse exploration, then gradually focuses on more promising areas as it learns.
-
Generator LLM: In contrast, this LLM is ‘stateless,’ focusing solely on the current strategy provided by the Navigator. Its job is to translate these strategies into concrete, high-quality candidate architectures that fit within the defined search space constraints.
-
Coordinator Module: This module oversees the entire process. It manages communication between the two LLMs, validates whether generated architectures are legal, evaluates their performance using inherited weights from a pre-trained ‘supernet’ (which speeds up assessment), and keeps an archive of visited architectures to avoid redundant work.
This collaborative setup allows CoLLM-NAS to perform a directed, learning-based optimization. Instead of relying on random mutations, the Navigator LLM maps the optimization history to an abstract strategy, which the Generator LLM then uses to create specific solutions. This ‘trajectory → strategy → solution’ pipeline encourages more structured exploration and improves search robustness.
Impressive Results and Efficiency
CoLLM-NAS has been rigorously tested on various search spaces, including MobileNet, ShuffleNet, AutoFormer, and NAS-Bench-201. The experimental results are compelling:
-
On ImageNet, CoLLM-NAS consistently outperforms existing NAS methods and conventional search algorithms, achieving new state-of-the-art results. For instance, it achieved up to 0.6% accuracy improvements while reducing search costs by 3 to 10 times compared to baselines.
-
In the AutoFormer search space, it discovered better architectures with only 10% of the baseline’s search cost and 25% of the number of evaluations.
-
On NAS-Bench-201, CoLLM-NAS maintained superior performance across all datasets, even with a significantly smaller budget of explored architectures (at most 100 architectures).
The framework’s robustness was also demonstrated through ablation studies, showing consistent performance even with different prompt wordings and various open-source LLMs. This indicates that the improvements come from the collaborative framework itself, rather than specific linguistic formulations or LLM implementations.
Also Read:
- Interactive Learning: How LLMs Can Enhance Reasoning Through Peer Interaction
- Automating Algorithm Creation with LLMs: The EvoPH Framework
A New Era for Neural Architecture Design
CoLLM-NAS represents a significant step forward in automating neural architecture design. By intelligently combining the reasoning capabilities of LLMs with the efficiency of two-stage NAS, it offers a powerful and generalizable framework for discovering high-performing architectures with reduced computational costs. This approach not only sets new benchmarks in NAS but also opens up exciting possibilities for applying collaborative LLM-based optimization to other complex design problems. You can read the full research paper here: CoLLM-NAS Research Paper.
