TLDR: A new computer vision system uses laser speckle vibrometry to detect tiny surface vibrations on opaque liquid containers. A deep learning model, the “Vibration Transformer,” analyzes these vibrations to accurately determine the hidden liquid level and container type remotely, without physical contact, and even generalizes to new container instances and ambient sounds.
Computer vision has made incredible strides in understanding the visible world, from identifying objects to tracking events. However, a fundamental limitation remains: conventional cameras can only ‘see’ the surface of objects. This means a system can recognize a soda can, but it can’t tell if it’s full or empty. This challenge is particularly significant in industrial settings, where knowing the internal contents of sealed, opaque containers is crucial for inventory, quality control, or safety.
A groundbreaking new research paper, “Learning to See Inside Opaque Liquid Containers using Speckle Vibrometry”, introduces a novel approach to overcome this limitation. Authored by Matan Kichler, Shai Bagon, and Mark Sheinin from the Weizmann Institute of Science, Israel, this work expands the scope of computer vision by enabling it to infer hidden liquid levels in opaque containers by sensing their tiny surface vibrations.
How It Works
The core of this innovation lies in a technique called speckle vibrometry. Imagine shining a laser onto an object’s surface. Due to the laser’s coherent light, the reflected light creates a random interference pattern known as ‘speckle’. Even the slightest vibration or tilt on the object’s surface causes this speckle pattern to shift. By precisely measuring these shifts, the system can detect minute vibrations that are imperceptible to the human eye.
The researchers developed a unique speckle-based vibration sensing system that can simultaneously capture these tiny vibrations across a 2D grid of points on multiple containers at once. This is a significant improvement over previous methods that could only sense a single point or required scanning. The containers are subtly vibrated, either by a nearby speaker playing a specific sound or even by ambient noise in the environment.
Once the vibrations are captured, a sophisticated deep learning model, dubbed the ‘Vibration Transformer’, takes over. Unlike traditional methods that struggle to interpret the complex vibrational responses of everyday containers, this transformer-based architecture is designed to analyze the frequency components of these multi-point surface vibrations. It learns the non-trivial relationship between the unique vibration patterns and the container’s type, as well as its hidden liquid level, expressed as a percentage of its total capacity.
Key Innovations and Benefits
This method offers several compelling advantages. Firstly, it provides a first-of-its-kind way to inspect the fill level of multiple sealed containers remotely and simultaneously, eliminating the need for physical manipulation or manual weighing. This is particularly beneficial for large warehouses storing consumer liquid products like soda or shampoo, heavy containers that are impractical to weigh, or containers holding hazardous, toxic, flammable, or radioactive liquids where physical contact should be minimized.
Secondly, the Vibration Transformer is remarkably robust. It’s invariant to the vibration source, meaning it can accurately estimate liquid levels whether the containers are excited by a controlled sound from a speaker or by general ambient noise. Furthermore, the model demonstrates impressive generalization capabilities, accurately predicting liquid levels for unseen container instances within known classes (e.g., a sixth Coke can after training on five others) and even for fluid levels not explicitly seen during training.
Real-World Applications and Performance
To validate their approach, the researchers built an experimental rig and collected a comprehensive dataset of various everyday liquid containers, including a large 100-liter industrial container weighing about 100 kg when full. The model was trained on 23 unique container types and tested across six increasingly complex scenarios, including predicting levels for containers with novel speaker positions, entirely new instances of known container types, unseen liquid levels, and under ambient sound conditions.
The results were highly promising, with the model achieving very low mean absolute errors (as low as 1% in within-distribution tests) and high accuracy in classifying container types. Even in challenging scenarios like unseen instances or ambient sound, the model performed reasonably well, demonstrating its practical viability. The study also showed that using multiple vibration measurement points per container significantly improved performance in more difficult generalization tasks.
Also Read:
- Faster, Sharper Images: A New Era for Single-Pixel Fluorescence Microscopy
- Enhancing Robotic Grasping with Symmetry-Aware Volumetric Models
Looking Ahead
This research represents a significant leap forward in computer vision, opening up new possibilities for inferring hidden properties of objects. While the current work focuses on liquid levels, the authors envision future research exploring broader questions, such as generalizing to entirely new container classes, classifying liquid types (e.g., water, soda, oil), extending to granular materials like sand, or even using vibrations as a ‘container fingerprint’ for identification. This novel approach could inspire further advancements in detecting contents of closed packages, assessing fruit ripeness, identifying spoilage in sealed foods, and determining chemical composition, truly allowing machines to ‘see’ beyond the surface.
