TLDR: FactorHD is a new Hyperdimensional Computing (HDC) model that significantly improves the representation and factorization of complex multi-object, multi-class, and multi-subclass relationships in neuro-symbolic AI. It introduces a novel symbolic encoding method with an extra memorization clause and an efficient factorization algorithm that selectively identifies items of interest. This approach overcomes limitations of existing HDC models like “superposition catastrophe” and “the problem of 2,” achieving up to 5667x speedup and maintaining high accuracy (e.g., 92.48% on Cifar-10 when integrated with ResNet-18).
Neuro-symbolic Artificial Intelligence (AI) is an exciting field that combines the logical reasoning capabilities of symbolic AI with the pattern recognition strengths of neural networks. At its core, a promising computational model known as Hyperdimensional Computing (HDC) plays a crucial role. HDC, inspired by how the human brain processes information, uses high-dimensional vectors (HVs) to represent and manipulate data, offering benefits like high computational efficiency and resilience to noise.
While existing HDC models have been effective in representing simpler relationships, such as class-instance (e.g., ‘dog is an animal’) and class-class relations, they face significant hurdles when dealing with more complex structures. Imagine trying to represent multiple objects, each with several layers of classification – like ‘animals-dogs-spaniels-Fido’. This is known as the class-subclass relation, and it presents challenges, particularly in a process called factorization, which is vital for neuro-symbolic AI systems to reason and extract information.
One major issue is the “superposition catastrophe,” where information about different subclasses from multiple objects gets mixed up and becomes indistinguishable. Another is “the problem of 2,” leading to information loss when identical objects are represented simultaneously. Furthermore, as the complexity of these hierarchical representations grows, existing HDC models become highly inefficient in factorization, often requiring numerous repetitive calculations to extract specific information.
Introducing FactorHD: A New Approach to Complex Representations
To overcome these limitations, researchers have proposed FactorHD, a novel HDC model designed specifically for representing and factorizing complex multi-object, multi-class, and multi-subclass relationships efficiently. FactorHD introduces several key innovations:
-
Enhanced Symbolic Encoding: Unlike previous models, FactorHD uses a unique symbolic encoding method. It embeds an extra ‘memorization clause’ within its representation, which helps preserve more detailed information about multiple objects and their various class and subclass levels. This new ‘bundling-binding-bundling’ structure prevents issues like ‘superposition catastrophe’ and ‘the problem of 2’.
-
Efficient Factorization Algorithm: FactorHD employs a smart factorization algorithm that significantly boosts computational efficiency and accuracy. Instead of exhaustively searching through all possible combinations, it selectively eliminates redundant classes and identifies relevant information by focusing on the memorization clause of the target class. This allows for ‘partial factorization,’ meaning it can extract only the specific subclass items of interest without needing to process the entire representation, saving considerable time and computation.
How FactorHD Works (Simplified)
At its core, FactorHD represents information by bundling different subclass levels within the same class and binding different classes together. For multiple objects, it bundles these combined class representations. When it comes to factorization, FactorHD intelligently sifts through this information. For a single object, it can easily identify subclass items. For multiple objects, it uses a ‘threshold similarity’ to pick out all relevant subclass items, then combines them to find the specific objects present in the representation. This method drastically reduces the number of comparisons needed, making it much faster and more scalable.
Also Read:
- Unlocking Clarity: A New Approach to Interpretable Neuro-Symbolic AI
- A Unified Framework for Understanding Neurosymbolic AI
Impressive Performance and Real-World Applications
Evaluations of FactorHD demonstrate remarkable improvements over existing HDC models. For common representation problems, FactorHD achieves an astonishing speedup of approximately 5667 times at a representation size of 10^9, while consistently maintaining over 99% accuracy. Even at smaller problem sizes, it shows a minimum speedup of 18.5 times. Its factorization time complexity is significantly lower, making it highly scalable as the problem size increases.
Beyond theoretical improvements, FactorHD also proves its mettle in practical applications. When integrated with the ResNet-18 neural network, a popular model for image recognition, FactorHD achieved an impressive 92.48% factorization accuracy on the Cifar-10 dataset. It also performed well on the RA VEN and Cifar-100 datasets, demonstrating its ability to handle real-world data with complex hierarchical structures. This integration highlights FactorHD’s potential to enhance neuro-symbolic AI systems by providing a robust and efficient way to represent and reason about complex, multi-layered information.
In conclusion, FactorHD represents a significant leap forward in Hyperdimensional Computing, addressing long-standing challenges in representing and factorizing complex multi-object, multi-class, and multi-subclass relations. Its novel encoding and efficient factorization algorithm pave the way for more powerful and practical neuro-symbolic AI systems. For more in-depth details, you can read the full research paper here.
