Making invisible scan patterns, visible

Increasing pilot training effectiveness by visualising scan patterns through AR

An essential aspect of flying is the pilot’s visual scan behaviour. Scanning behaviour can be hard to learn, especially for pilots that are training for their first type rating. Instructor assessment of scan behaviour is often limited, as it is based on visual observation of pilot movements, flight simulation results, and questions about flight parameters during flight.

Insights into a pilot’s visual scan behaviour are often mostly subjective. Contemporary tools and techniques used during pilot training are not able to provide instructors with objective scan behaviour information.

Augmented Eye – NLR’s augmented reality-based training technology – provides instructors with the means to perceive the pilot’s visual scan behaviour in real-time, allowing instructors to provide informed feedback and improve training results. Our innovative technology even allows for a wide selection of visualisations for scan pattern behaviour, further improving pilot analysis and assessment.


Best Paper ITEC

The first paper written on this instructor tool received the best paper award at the ITEC 2019 conference. This paper describes the development and evaluation of the first prototype created for Augmented Eye. Eight months later, this paper was presented at the IITSEC as well.


Our vision regarding instructor tools and why these should be aligned with the scenario trained, instead of merely the eye tracking information itself.
Do we want to overload the instructor with a lot of extra data, or do we want to support him with relevant information?

International workshop

In November 2020, NLR hosted the first workshop on “Eye Tracking for Performance Based Training” with guests from all over the world, who are already working with or interested in this topic. Attendees included pilot training organizations, simulator companies, eye tracking manufacturers and research institutes.

Why use Augmented Eye for your pilot training?

Using the Augmented Eye in your training can result in more efficiency (students learn faster) and effectiveness (improved scan patterns). Experienced instructors rated Augmented Eye highly and found it to be of significant added value to their training and feedback.

  • Make otherwise invisible information, such as scanning behaviour, visible
  • Objective visual scan pattern information supports the instructor
  • The presentation of the data can be tailored to the training scenario by using Learning Analytics

We are working on various options to extend the application and tailor it to your needs and preferences, e.g. tracking the eyes of both pilot and co-pilot, connect EEG to measure the pilot’s alertness, data processing, a debrief tool, connection with sim data, etc.

Interested in our approach regarding the learning analytics?

Read the paper presented at AIAA 2020: