TLDR: A new system called FlightAxis combines Large Language Models (LLMs) with Electrical Muscle Stimulation (EMS) to provide kinesthetic (physical) feedback for learning operational skills like flying. Using an “Align-Analyze-Adjust” strategy, the LLM analyzes flight data and translates guidance into muscle stimulations, helping trainees improve performance, especially in altitude maintenance. User feedback was positive, highlighting comfort and improved awareness, though some concerns about over-restriction for experienced users were noted. This research demonstrates a novel way for AI to offer hands-on training.
Learning complex operational skills like flying, driving, or performing surgery traditionally relies heavily on hands-on practice and the development of muscle memory through kinesthetic feedback. While Large Language Models (LLMs) have emerged as powerful tools for personalized learning, their assistance has primarily been limited to textual or vocal feedback, leaving the crucial kinesthetic aspect largely unexplored. This gap is significant because physical skills inherently require direct physical guidance and immediate, tangible feedback.
Researchers have introduced a groundbreaking approach to bridge this divide with a system called FlightAxis. This innovative tool integrates LLMs with Electrical Muscle Stimulation (EMS) to provide direct kinesthetic guidance for operational skill learning, specifically focusing on flight training. FlightAxis represents a pioneering effort, being the first study to combine the analytical power of LLMs with the physical feedback mechanism of EMS for this purpose.
The “Align-Analyze-Adjust” Strategy
At the heart of FlightAxis is a three-stage workflow designed to ensure professional and precise “hand-by-hand” guidance:
- Align: To ensure accuracy, the LLM is equipped with a comprehensive flight knowledge base. This includes basic flight principles, aircraft-specific information, and mission-specific procedures. Using a technique called Retrieval Augmented Generation (RAG), the LLM can access this vast repository of expertise, allowing it to generate highly accurate and contextually relevant responses.
- Analyze: The system continuously monitors the trainee’s flight status using key metrics such as flight position, attitude, speed, and dynamic flight metrics. The LLM analyzes this real-time data to identify any deviations and generate precise guidance tailored to the current situation.
- Adjust: This is where the kinesthetic feedback comes into play. The LLM’s guidance is translated into specific EMS patterns that guide the trainee’s forearm movements. Through pre-experiments, researchers identified two optimal EMS modes: a “weak-strong-weak” pattern for initiating operations, which offers a good balance of action, feedback, and user autonomy, and a “weak-strong” pattern for correcting abnormal operations, providing clear directional guidance. Alongside EMS, voice prompts are used to direct the trainee’s attention to relevant instruments.
Putting FlightAxis to the Test
To validate the system’s effectiveness, the researchers conducted several experiments. First, they tested the LLM’s ability to generate correct professional guidance using the RAG-augmented system. Across various flight tasks, including Straight and Level Flight, Normal Takeoff and Climb, Steep Turn, and Deadstick Landing, the LLM achieved an impressive overall accuracy rate of 93.2%. While slightly lower under abnormal flight conditions (92.8%), this still demonstrates the model’s robust capability to process complex flight data and provide reliable outputs.
An empirical study was then conducted with 36 participants, none of whom had prior flight experience. They were divided into two groups: a Solo Training Group and an Assisted Training Group, which used FlightAxis. The training focused on “Steep Turns” within Microsoft Flight Simulator. The results showed that the Assisted Training Group achieved significant improvement in altitude maintenance, a critical aspect of flight control. While there were no significant differences in other metrics like speed control or task completion time, the user experience feedback was overwhelmingly positive.
Also Read:
- A Visual Test for AI: How Well Do Large Models Animate Biological Motion?
- New Strategies for Preventing Unintended AI Behavior During Training
User Experience and Future Outlook
Participants in the Assisted Training Group reported high levels of comfort, process memory assistance, operational learning support, and enhanced situational awareness. They found the combined auditory cues and muscle stimulation to be intuitive and explicit, greatly facilitating the acquisition of flight skills. Many novices particularly highlighted how FlightAxis provided clear guidance and helped them identify issues they might have otherwise overlooked.
However, the study also identified areas for improvement. Some participants noted a need for enhanced accuracy in the system’s directional control to prevent overcorrections. Additionally, while beneficial for novices, the kinesthetic assistance could potentially be too restrictive for more experienced users, potentially hindering their autonomous control. These insights are crucial for refining future iterations of such systems.
FlightAxis demonstrates the immense potential of integrating LLMs with kinesthetic feedback for operational skill learning. This approach aligns with the principles of embodied cognition, where physical interaction is fundamental to learning. By offering a co-adaptive control mechanism that guides movements while preserving user autonomy, FlightAxis opens new avenues for AI systems in complex tasks like surgery and remote control. Future work will focus on further enhancing the accuracy and adaptability of the system, exploring other forms of haptic feedback, and ensuring its applicability across a wider range of users and scenarios. You can find more details about this research at the research paper.
