TLDR: WebEyeTrack is a new open-source framework that brings accurate, real-time eye-tracking to web browsers and mobile devices. It uses lightweight AI models, on-device few-shot learning for quick personalization, and model-based head pose estimation to achieve state-of-the-art accuracy (2.32 cm error) and fast inference (2.4 ms on iPhone 14), addressing issues of expensive hardware, privacy, and head movement in existing solutions.
Eye-tracking technology has long been a powerful tool for understanding how people interact with computers, revealing subtle shifts in visual attention. However, its widespread use has been limited by the need for expensive, specialized hardware like EyeLink and Tobii Pro Fusion, often confining research to controlled lab environments. Even newer virtual reality solutions, such as the Apple Vision Pro, remain financially out of reach for many, hindering the broader application of gaze-enhanced technologies.
Researchers have been working on webcam-based eye-tracking solutions that use built-in cameras on everyday devices to overcome these limitations. While these methods have improved, they often struggle with accuracy, especially when users move their heads. Furthermore, many advanced AI models for gaze estimation, while achieving impressive benchmarks, don’t always consider practical deployment issues like model size, processing speed, and user privacy.
Addressing these challenges, a new framework called WebEyeTrack has been introduced. This innovative solution integrates lightweight, state-of-the-art gaze estimation models directly into web browsers. It incorporates model-based head pose estimation and a clever technique called on-device few-shot learning, which means it can adapt to new users with as few as nine calibration samples. This approach allows WebEyeTrack to achieve top-tier performance with an error margin of just 2.32 cm on the GazeCapture dataset, all while maintaining real-time inference speeds of 2.4 milliseconds on a device like an iPhone 14.
WebEyeTrack is designed to bridge the gap between highly accurate, appearance-based gaze estimation methods and scalable, webcam-based eye-tracking solutions. It combines a model-based head pose estimation, which uses 3D face reconstruction, with BlazeGaze, a lightweight Convolutional Neural Network (CNN) model optimized for real-time processing. The system also detects eye state (open/closed) to prevent predictions during blinks, offers continuous clickstream calibration, and provides online fixation detection, creating a comprehensive and web-friendly eye-tracking system.
A key aspect of WebEyeTrack is its commitment to privacy. The personalization and inference processes happen directly on the user’s device, ensuring that sensitive gaze data never leaves the local environment. This is a significant advantage over cloud-based solutions, which often raise concerns about data privacy and latency.
The framework’s core contributions include its open-source nature, a novel metric head pose estimation method, and the introduction of BlazeGaze. BlazeGaze is a remarkably small CNN model, weighing in at only 670KB, yet it achieves real-time inference on mobile CPUs and GPUs. This efficiency makes it highly suitable for deployment on consumer-grade hardware like smartphones, tablets, and laptops.
In evaluations, WebEyeTrack demonstrated comparable accuracy to more computationally intensive models while significantly improving runtime efficiency. For instance, it outperformed the previous fastest model (Mnist) with a 256% increase in frames per second. When tested for temporal consistency over a 20-minute session, WebEyeTrack showed significantly better robustness to gaze drift compared to other browser-based solutions like WebGazer, whose accuracy declined much more over time due to its lack of head pose awareness.
Also Read:
- Gaze-Controlled Virtual Keyboard Enhances Communication Accessibility with Standard Webcams
- SwiftF0: Advancing Real-time Pitch Detection for Noisy Environments
This development marks a significant step forward for eye-tracking, making it more accessible, accurate, and private for a wider range of applications. The open-source code for WebEyeTrack is available for researchers and developers to explore and integrate into their projects. You can find more details about this research in the full paper: WEBEYETRACK : Scalable Eye-Tracking for the Browser via On-Device Few-Shot Personalization.
