TLDR: CLAQS is a compact, fully quantum token mixer for text classification that uses only 8 data qubits and shallow circuits. It achieves high accuracy (91.64% on SST-2, 87.08% on IMDB) by jointly learning complex-valued mixing and non-linear transformations within a unified quantum circuit, outperforming classical Transformers and hybrid quantum-classical models with significantly fewer parameters.
In the rapidly evolving landscape of natural language processing (NLP), Transformer-based models have set new benchmarks, largely due to their self-attention mechanism. However, this power comes with significant costs: massive parameter counts and a computational complexity that grows quadratically with sequence length. This has driven researchers to seek more efficient ways to process language, exploring alternatives to traditional attention.
Simultaneously, quantum computing is advancing at a remarkable pace, with quantum processing units (QPUs) and high-throughput GPU simulators becoming more accessible. This opens up exciting possibilities for quantum NLP, moving beyond theoretical concepts to practical applications. Yet, current quantum hardware, often referred to as Noisy Intermediate-Scale Quantum (NISQ) devices, still faces limitations in the number of qubits and circuit depth. Training quantum models can also be unstable, posing challenges for real-world deployment.
Addressing these challenges, a new research paper introduces CLAQS, a novel approach to text classification. CLAQS stands for “Compact Learnable All-Quantum Token Mixer with Shared-ansatz for Text Classification.” This innovative model is designed to be compact, phase-aware, and capable of stabilizing quantum amplitudes while scaling to longer text sequences. It represents a significant step towards making quantum NLP more practical and efficient.
What is CLAQS and How Does It Work?
CLAQS is a fully quantum token mixer that learns complex-valued mixing and non-linear transformations within a single, unified quantum circuit. It aims to replace the computationally intensive dot-product attention mechanism found in classical Transformers with a more efficient quantum alternative. The model operates under a sliding-window regime, aggregating information at the document level.
The core innovations of CLAQS are threefold:
First, it features learnable quantum token mixing. Unlike previous approaches that might rely on fixed coefficients, CLAQS parameterizes both the quantum mixing weights and the non-linear transformations as complex-valued parameters. This allows the model to discover optimal token interactions directly from the data.
Second, CLAQS ensures stabilized training through normalization. It applies an L1 normalization constraint to regulate amplitude scaling, which is crucial for stable end-to-end optimization and prevents issues like coefficient explosion or vanishing that have plagued earlier quantum designs.
Third, it employs a dual-stage quantum circuit architecture. This design decouples shared token embeddings from a window-level quantum feed-forward module. This mirrors the structure of Transformer layers but operates on quantum representations, enhancing modeling capacity without increasing token-wise parameters.
In practice, CLAQS processes text in five main stages: it maps token embeddings to a unitary circuit, performs a complex-weighted unitary superposition (LCU mixer), applies a trainable polynomial transformation (QSVT nonlinearity), refines the aggregated state with an independent quantum feed-forward module, and finally uses multi-axis measurements to produce classical features for classification.
Impressive Performance with Minimal Resources
One of the most striking aspects of CLAQS is its efficiency. The model requires only eight data qubits and shallow circuits, yet it achieves remarkable accuracy on standard sentiment classification benchmarks. On the SST-2 dataset, CLAQS reached 91.64% accuracy, significantly outperforming both classical Transformer baselines (79.36%) and strong hybrid quantum-classical counterparts like AQCF (81.88%). For the IMDB dataset, CLAQS achieved 87.08% accuracy, again surpassing AQCF (86.30%) and classical Transformers (82.18%).
These results are particularly noteworthy because CLAQS operates under a strict 8-qubit budget. To put this into perspective, a typical classical Transformer attention block with 256 dimensions and 8 heads can have over 263,000 parameters. In contrast, CLAQS uses only 454 parameters for IMDB and 326 for SST-2, making it roughly 580 to 807 times smaller in terms of attention parameters. This demonstrates an incredible resource efficiency, bridging the gap between classical attention and quantum superposition under tight resource constraints.
Why CLAQS Delivers Strong Results
The success of CLAQS stems from its ability to treat the attention-like mixer as a fully learnable quantum operator. The complex LCU coefficients, combined with L1 normalization, enable phase-sensitive weighted mixing. Furthermore, the Quantum Singular Value Transformation (QSVT) provides a learnable polynomial nonlinearity that allows token unitaries to compose into higher-order interactions. This is all refined by a window-level quantum feed-forward module, adapting the entire system for classification tasks.
The research highlights that making the LCU and QSVT components learnable is crucial. This learnable quantum mixing consistently outperforms classical window aggregation methods like mean or attention pooling. The separation between the token-shared embedding and the window-level quantum feed-forward also improves task adaptation without increasing token-wise parameters.
Also Read:
- QGraphLIME: Bringing Clarity to Quantum Graph Neural Network Decisions
- Quantum Fusion Layer: A New Approach to Multimodal Learning Scalability
The Future of Quantum NLP
CLAQS represents a significant step towards practical quantum components in NLP. It shows that compact, qubit-based mixers can serve as effective, learnable alternatives to classical attention within modern language processing pipelines. The inherent efficiency of quantum encoding, where a small set of qubits can span an informational space equivalent to much larger classical representations, is a key factor in these gains.
Future work will involve validating CLAQS on actual quantum hardware, expanding its application beyond sentiment analysis to tasks like natural language inference, and exploring scaling through multi-head and multi-layer variants. This research paves the way for quantum-classical co-design, where compact quantum mixers can offer powerful, learnable solutions under tight resource constraints. You can read the full research paper for more technical details here: CLAQS: Compact Learnable All-Quantum Token Mixer with Shared-ansatz for Text Classification.
