TLDR: NeuroAda is a new Parameter-Efficient Fine-Tuning (PEFT) method for large language models (LLMs) that combines the precision of selective adaptation with the memory efficiency of addition-based techniques. It works by identifying the most important connections within each neuron and introducing small, trainable ‘bypass connections’ for them, while keeping the original model frozen. This mask-free approach significantly reduces GPU memory usage (up to 60%) and speeds up training, achieving state-of-the-art performance on over 23 tasks with as little as 0.02% trainable parameters, without adding inference-time overhead.
Fine-tuning large language models (LLMs) to excel at specific tasks is a common practice, but it comes with a significant challenge: the immense computational and memory resources required. As LLMs grow larger, the traditional method of fine-tuning every single parameter becomes increasingly impractical. For instance, fine-tuning a LLaMA 2-13B model can demand over 100 GB of memory, far exceeding the capacity of standard GPUs.
To address this, researchers have developed Parameter-Efficient Fine-Tuning (PEFT) methods. These generally fall into two main categories: addition-based and selective in-situ adaptation. Addition-based methods, like LoRA, introduce small, additional modules to the model, offering good memory efficiency. However, their ability to represent complex adaptations can be limited, making them less suitable for very fine-grained adjustments. On the other hand, selective in-situ adaptation directly fine-tunes a carefully chosen subset of the original model’s parameters. This allows for more precise and effective adaptation but often at the cost of significantly increased memory consumption, as the system still needs to compute and store gradients for many parameters, even if they aren’t updated.
A new approach called NeuroAda, developed by Zhi Zhang, Yixian Shen, Congfeng Cao, and Ekaterina Shutova from the University of Amsterdam, aims to bridge this gap. NeuroAda offers a novel PEFT method that enables fine-grained model fine-tuning while maintaining high memory efficiency. The core idea is to identify the most important connections (parameters) within each neuron of the network. Instead of directly fine-tuning these selected original parameters, NeuroAda introduces tiny ‘bypass connections’ for them. During the fine-tuning process, only these newly introduced bypass connections are updated, leaving the vast majority of the original model parameters frozen.
How NeuroAda Works
NeuroAda operates in a clever, three-phase process. First, it performs an offline selection. For every neuron in the network, it identifies the ‘top-k’ input connections that have the highest magnitude (absolute value) in the pre-trained model. This selection is static and doesn’t require any task-specific signals or complex calculations during training, making it highly efficient.
Next, during the training phase, only the parameters associated with these selected top-k bypass connections are updated. The crucial innovation here is a ‘mask-free’ implementation. Instead of using a large binary mask to indicate which parameters to update (which still consumes significant memory), NeuroAda stores only a compact list of indices for the selected connections and their corresponding small, trainable delta values. This dramatically reduces memory overhead. For example, in a LLaMA-2 13B model, NeuroAda with k=1 uses only 0.02 MB of memory for a single layer, compared to over 3 MB for a mask-based approach – a saving of over 156 times.
Finally, after training, these small, learned delta values from the bypass connections are merged directly into the original frozen weights. This ‘one-shot merge’ means that at inference time, the model retains its original structure and incurs zero additional overhead, making it fully compatible with existing inference systems.
Key Advantages of NeuroAda
NeuroAda brings several significant benefits to the table:
- Highly Efficient Computation: By eliminating the need for binary masks and full gradient computation, NeuroAda streamlines the training process.
- Exceptional GPU Memory Usage: Only the newly added bypass parameters are updated, drastically reducing memory consumption by avoiding the need to track optimizer states for the entire model. This can lead to up to 60% memory savings on large models like LLaMA3-8B.
- Task-Agnostic and Generalizable: The parameter selection is based on the inherent magnitudes of the pre-trained weights, making the method consistently applicable across diverse tasks without requiring task-specific warm-up or heuristics.
- Fine-Grained, Neuron-Level Adaptation: A core design principle is that every neuron has at least one input connection selected for update. This ensures that all neurons can potentially modify their activation states, maximizing the representational expressiveness and adaptability of individual neurons.
Also Read:
- Efficient LLM Acceleration: AdaSPEC’s Targeted Distillation Approach
- Capsule Prompt Tuning: Efficiently Guiding Large Language Models
Empirical Validation
The researchers put NeuroAda to the test on over 23 tasks, covering natural language generation (commonsense and arithmetic reasoning) and natural language understanding (GLUE benchmark). They used various models, including RoBERTa-base, LLaMA-7B/13B, LLaMA2-7B, and LLaMA3-8B.
The results were impressive. NeuroAda consistently achieved state-of-the-art performance across different parameter budgets, from relatively high to extremely low (as little as ≤0.02% trainable parameters). For instance, on commonsense reasoning tasks, NeuroAda outperformed the second-best baseline, SMT, by 4% in average accuracy while using fewer trainable parameters. In arithmetic reasoning, it surpassed LoRA by 6% with even fewer parameters. On the GLUE benchmark, it improved the average score by +0.7 compared to LoRA and showed strong generalization, achieving the best score on 6 out of 8 tasks in a low-budget setting.
Furthermore, NeuroAda demonstrated significant improvements in training efficiency. For larger models like LLaMA3-8B, it processed 16.6 samples per second, a substantial increase compared to 1.1 samples per second with mask-based methods.
This research highlights NeuroAda as a promising solution for efficient and effective fine-tuning of large language models, offering a scalable and practical approach for adapting these powerful models to a wide array of downstream tasks. You can find the full research paper here: NeuroAda: Activating Each Neuron’s Potential for Parameter-Efficient Fine-Tuning.
