Overview

This document describes the operation of the AdHawk MindLink glasses. The AdHawk MindLink glasses kit contains a pair of eye tracking glasses capable of streaming gaze from AdHawk’s camera-free eye tracking sensors and video from the front-facing camera to a PC or the included Android mobile device. Using the eye tracking glasses and the AdHawk Hub software on either PC or Android, eye tracking experience can be viewed in real-time or recorded for later.

Product Description

The AdHawk MindLink kit includes all of the necessary hardware and software required to record and review eye tracking data with the AdHawk MindLink glasses.

Kit Overview

 

Smartphone

The included smartphone comes pre-loaded with the AdHawk Hub software and allows users to record eye tracking sessions in a mobile form factor. The included smartphone has been tested for interoperability and is the recommended device to be used with the AdHawk system. Other Android devices have not been tested and are therefore not recommended.

Calibration Fixture

A calibration case is included in the package. The calibration case can be used to calibrate the glasses from time to time. The AdHawk software will alert the user when a calibration is required.

Nosepieces

The kit ships with a set of replaceable nosepieces. The default nose piece comes pre-installed in the glasses and accomodates the largest number of users. The nosepiece selection guide should be reviewed to select the optimal nosepiece for each user.

Ear Hooks

Ear hooks are included for a more secure fit. The ear hooks are recommended in cases where the glasses may fall off or slip drastically from the user’s face.

Cables

Several cables are included and can be used to connect the AdHawk MindLink glasses to both the included Android device as well as a computer. The AdHawk MindLink glasses use a Micro-USB connector while the smartphone uses USB-C and most computers use USB-A. A Micro-USB to USB-A adapter has been included in order to transfer data from the smartphone to a PC.

Anti-Fog Lens Cleaning Cloth

The included cloth can be used to clean the lenses as well as the area inside the plastic frames. The cleaning cloth is reusable.

For additional information on how to use the various components included with the AdHawk MindLink kit, please review the additional documentation provided or see the AdHawk Microsystems Website.

Specifications

The following table describes the specification of the evaluation kit hardware. These are the default values for the evaluation kit hardware and software, if you would like to discuss additional configurations or customization, contact AdHawk to discuss setting up an integration project.

Specification Value Unit Comments
Output Data Rate 60-500 Hz Pupil position and size, gaze vector, and vergence angle
Tracking Field of View 40×25 degrees  
Accuracy 1 degree Mean Absolute Error
Front-Facing Camera Frame Rate 30 FPS  
Front-Facing Camera Resolution 1280×720 pixels Configurable; default is given
Calibration Sequence 9,5,3 points  
Slip Compensation yes    
Battery Life with Video 4+ hours Recording video with the included mobile device
Battery Life Eye Tracking Only 18 hours Front facing video and audio recording disabled
Glasses Weight 32 g  

Setup

System requirements

  • Desktop PC
  • Intel Core i5 or equivalent
  • 8 GB of RAM
  • 1 GB of storage

Windows

Software requirements

Installation

Hardware Setup

The AdHawk glasses ship with several nose pieces which help adjust the seating of the glasses to each user’s facial structure. The instructions on selecting the correct nose piece can be found in Nosepiece Selection Guide.

The device contains a single micro USB port on the right arm of the glasses. Plug the included cable into the micro USB port on the glasses and then connect the other end of the cable into your PC USB port.

The operating system should detect the device and load the necessary drivers. Once the OS has completed setting up the device, it is ready to use.

Getting Started

Backend Service

The AdHawk Backend Service communicates with AdHawk’s eye tracking hardware, and relays the eye tracking signal to one or more clients including AdHawk Hub.

  1. Launch AdHawk Backend from the desktop. Upon launch, the AdHawk Backend icon should appear in the system tray. Keep the AdHawk Backend Service running in the background while eye tracking.

    The service can be shutdown by right clicking on the system tray icon, and selecting Quit AdHawk Backend.

Hub

The AdHawk Hub software provides all you need to interact with AdHawk’s eye tracking system.

  1. Launch AdHawk Hub from the desktop.

  2. Select a profile:

    Profiles provide the ability to customize the eye tracking experience for different users.

    • On the first launch, you will be immediately presented with a window to set up your profile. Follow through the wizard to set up your profile.

    • On subsequent launches, you will be presented with a dialog to select from one of the profiles on the system.

      Profile selection

       

  3. Once a profile is selected, the main application will open up.

    The primary functions of the AdHawk Hub is divided into two tabs:

    • Track is used to calibrate the eye tracker, and contains tools to evaluate and display gaze output in real time.
    • Replay provides the tools to review eye tracking data previously recorded in the Track tab.
  4. Select Track from the top right corner of the window.

  5. Connect your AdHawk evaluation kit glasses to your PC via USB, and wait until Device connected appears on the right side of the status bar.

    Device connected

     

  6. Ensure that the camera preview shows up on the screen, and that the camera feed is from the front facing camera on the glasses. It may take a second for the camera preview to appear.

    Camera preview

     

    If the camera feed is not from the front facing camera on the glasses, change Camera index in the configuration panel on the left side until you find the correct one.

Quick start

  1. Click Quick Start to configure the eye tracker.

  2. A new window containing a marker will be displayed on the screen. Ensure that there is a green border around the marker indicating that it is being detected by the camera.

    Marker detected

     

    If the border is red, refer to the Marker detection section of the Troubleshooting Guide.

  3. While looking at the center of the marker, slowly rotate your head in the direction suggested by the arrow on the screen until the box is aligned with the marker.

    VOR Calibration

     

  4. Continue to look at the center of the marker, until the progress bar stops spinning and the window closes.

  5. (Optional) Ensure that the eye tracker is detecting good signal quality in Feature Preview window.

    The output of the eye tracking signals are displayed as a set of colored dots on the screen for each eye. The pupil signal (pink dot) also shows the ellipse representing the size and orientation of the pupil. The Fused signal (white dot) is the combination of all the other signals, and it must be available across the entire field-of-view (FOV).

    Good tracking

     

    If the Fused signal is not smooth or disappears across the FOV, then refer to the Tuning section of the Troubleshooting Guide. Otherwise, continue ahead to the next section to calibrate your eye tracker.

Calibration

  1. Before starting calibration, you must choose the calibration mode that is most suited for your system.

    Calibration is the process of mapping the eye tracking signal to an individual’s visual axis, which makes the eye tracking output meaningful for typical applications. The eye tracker performs best when the user’s gaze lies within the calibration range. Therefore, it is typically recommended to use as large a calibration range as necessary for the target application.

    There are currently two modes of calibration available: Grid and VOR.

    • In Grid mode, the user maintains their head in a fixed position while the target moves around the screen. This mode is suitable for large screens, which cover a large part of user’s FOV.
    • In VOR mode, the user turns their head, while continuously looking at a fixed target at the center of the screen. This mode is suitable for small screens, which only cover a small portion of user’s FOV. Calibration comparison
  2. Select the desired Calibration Mode in the configuration panel on the left side.

    Calibration mode

     

  3. Click the Calibrate button to launch the calibration window.

  4. A new window containing a marker will be displayed on the screen. Ensure that there is a green border around the marker indicating that it is being detected by the camera.

    Marker detected

     

    If the border is red, refer to the Marker detection section of the Troubleshooting Guide.

  5. Adjust your head into the correct position:

    • For Grid calibration mode, the head should remain in a fixed position facing the screen.

    • For VOR calibration mode, slowly rotate your head in the direction suggested by the arrow on the screen until the box is aligned with the marker.

      VOR Calibration

       

  6. Look at the center of the marker, and press one of enter, return, or space keys to register the calibration point.

  7. Repeat the above step for all nine positions.

  8. Once the calibration process is complete, the calibration window will automatically close. The calibration process can be aborted at any time by closing the window.

  9. Upon successful calibration, it’s possible to remember the calibration for later by clicking the Save Calibration button.

Validation

Once the calibration process is complete, a dot representing gaze will be overlaid on top of the live camera image.

Live view

 

Gaze overlay

A simple way to evaluate the accuracy of the eye tracker is using the gaze overlay on top of the live view image.

  1. Look at a particular point; freeze the live preview; and validate that the gaze marker is overlaid correctly on the point that you were looking at.

    • The live preview can be frozen by clicking the pause button, or by pressing and holding the pause shortcut key, p.
  2. Record the video and the gaze output for further review by clicking the record button, or by pressing the record shortcut key, r. When finished, the recording can be stopped by clicking the record button, or by pressing the record shortcut key, r, a second time.

Validation tool

The validation tool can be used for a more accurate measure of the eye tracker’s accuracy and range. Similar to the calibration process, the validation tool supports both Grid and VOR modes.

  1. In Track tab, launch the validation window by clicking the Validate button.

  2. Follow the same instructions as the calibration process.

    By default, the validation process uses 16 different positions instead of the 9 used by the calibration process. The number of the validation positions can be adjusted in the configuration panel.

  3. The result of the validation is available along with other recorded data.

Review

The recorded gaze, video, and validation data can be reviewed in the Replay tab.

  1. Click on Replay tab at the top right side of the window.

  2. Load the desired recording by double clicking on it in the list of recordings displayed on the left.

Replay tab

 

Labels and metadata can be added to the recorded data through profiles. Refer to Profiles in User Guide for more details.

Recorded Data

Data recorded from a session is accessible by right clicking on the session in replay and selecting “Open File Location”

Users who wish to implement their own data processing should be aware that the recorded data is divided into 4 files.

gaze_data.csv: This file contains the following fields:

Timestamp: The time elapsed since the tracker was turned on represented as a float32 value

Gaze_X / Gaze_Y / Gaze_Z : The X/Y/Z coordinates of the estimated gaze point (combined binocular gaze data) with respect to the midpoint between the two scanners. The values are in meters represented as float32 values. See Gaze Coordinate System for details.

Gaze_X_Right / Gaze_Y_Right / Gaze_Z_Right : The X/Y/Z coordinates of a unit vector representing the right eye gaze with its origin considered to be the center of the right eyeball defined in the Gaze Coordinate System.

Gaze_X_Left / Gaze_Y_Left / Gaze_Z_Left : The X/Y/Z coordinates of a unit vector representing the left eye gaze with its origin considered to be the center of the left eyeball defined in the Gaze Coordinate System.

Vergence: The angle between the left and right eye gaze vectors in radians represented as a float32 value. This value should be within the range of [-0.017, +0.35] radians (corresponds to [-1, +20] degrees)

Image_X / Image_Y: the X/Y coordinates of the projected gaze point in the camera image (represented as a float 32 value) normalized to range [0, 1] where (0, 0) corresponds to the top/left corner of the image and (1, 1) corresponds to the bottom/right corner. Note that the values could exceed the range [0, 1] in case the gaze falls outside the image.

Frame_Index: The index of the video frame during which the gaze point was recorded, this is used for synchronization with the video data represented as an integer value

meta_data.csv: This document contains all the metadata belonging to that recorded session including user profile information and camera calibration details

Validation_.csv: This csv file contains the validation samples and is generated for each validation and contains the following fields:

Timestamp: The time elapsed since the tracker was turned on represented as a float32 value

Gaze_Alpha / Gaze_Beta: The estimated left/rigth combined gaze angle (azimuth and elevation) represented as float32 and radians unit.

Reference_Alpha / Reference_Beta: The ground truth version of the gaze data described above.

Timestamp_Sampling_Onset: The timestamp of the begining of the sampling for each target.

Precision_SD: Precision of the gaze samples defined as the standard deviation of the gaze angle measured in degrees

Session.mp4: This is the video captured by the AdHawk world camera. This video is captured at a variable frame rate, so it cannot be played back on its own. The frame index fields in the gaze data recordings (either projected or raw) can be used to synchronize the video frame with the correct elapsed time within the recording session.

Gaze Coordinate System

Coordinate System

 

The coordinate system used for the gaze outputs is centered at the midpoint of the scanners with:

  • X is oriented in the positive direction to the right (user’s point of view)
  • Y is oriented in the positive direction going up
  • Z is oriented in the positive direction when behind the user

User Guide

Profiles

AdHawk Hub provides the ability to customize the eye tracking experience for different users through the Profile option.

Switch profiles

  1. Click on the current profile at the top left of the screen.
  2. Select from one of the available profiles on the system

Add a new profile

  1. Click on the current profile at the top left of the screen.
  2. Select Add at the bottom of the list
  3. Follow through the wizard to set up the new profile.
  4. Once the profile creation completes, it will be automatically be selected.

Update profile details

  1. Click on the current profile at the top left of the screen.
  2. Click on Profile Edit for the desired profile.
  3. Update the profile information as desired.
  4. Click Save to save and close the profile dialog.

Delete a profile

  1. Click on the current profile at the top left of the screen.
  2. Click on Profile Edit for the desired profile.
  3. Click Delete in the profile dialog.

Shortcuts

Key Action
C Start Calibration in the Track tab
V Start Validation in the Track tab
P Pause gaze and camera feed in the Track tab
R Start/Stop Recordings in the Track tab

Troubleshooting

The following sections describe the tools and the workflows available for troubleshooting eye tracking problems.

Hardware setup

Camera preview is not working or not showing the correct picture

The camera index allows the user to selects the camera used by the AdHawk Hub when there are multiple cameras connected to a system (e.g. Webcam). The index of the AdHawk front facing camera may be different on each PC. If the front facing camera is not displayed by default, then another index should be tried until the front facing camera from the AdHawk glasses is displayed.

Poor eye tracking performance

If the eye tracking output is unstable or behaving erratically, the AdHawk Hub provides a set of tools to troubleshoot and eliminate the problem.

We recommend that you follow these steps under guidance from the AdHawk Support team.

The troubleshooting tools can be enabled in the settings dialog.

  1. Go to File > Settings on Windows or AdHawk Hub > Preferences on Mac OS X
  2. Select the Advanced section
  3. Check Enable troubleshooting
  4. Click OK or Apply to save the changes. You should see the Tune tab appear at the top of the main window.

Eye tracking signals

The Tune tab visualizes the incoming eye tracking signal and demonstrates how well the system is detecting the relevant features.

The eye tracking signals should be centered when looking straight ahead and should be free of noise on the edges of the screen similar to the example shown below.

Signals Plot Example

 

Each detector is assigned a distinct color to identify its particular signals. PD1PD6 represent the signals from the glint detectors. Pupil represents the signal from the pupil detector.

The pulses show the raw signal being detected by our eye tracking system. These pulses are shown as patterns of semi-transparent lines (each line representing a pulse).

The features show the processed eye tracking signal used by the calibration process. The center of glint and pupil features are identified using dots. The pupil signal (pink dot) also shows the ellipse representing the size and orientation of the pupil.

Auto-tune

AdHawk’s eye tracking system provides several configuration parameters to optimize the eye tracking experience for different individuals. The Auto-Tune function automatically detects the optimal parameters for an individual by performing an initial scan of their eye.

When performing an Auto-tune, ensure that the user’s gaze is fixed directly ahead until the scan is complete.

After successful completion of the Auto-tune function, the eye tracking signals should appear centered on the screen.

If the eye tracking signals do not appear a few seconds after running Auto-tune, please ensure that glasses are seated correctly (See Appendix A) and try again. A successful Auto-tune centers the eye tracking signals for each eye in their respective plots as shown in the previous section.

Manual tuning

In some cases, the Auto-tune function may not correctly determine the optimal eye tracking parameters for an individual. The Tune tab provides the ability to manually adjust the eye tracking parameters, if necessary, under guidance from AdHawk.

Eye tracking signals not centered

If the eye tracking signals are not centered, the zoom and pan of eye tracking sensors can be manually adjusted.

X Mean will move the scan box on the horizontal axis Y Mean will move the scan box on the vertical axis Width Changes the width of the scan box Height Changes the height of the scan box

Please contact AdHawk support for additional assistance.

Eye tracking signals are noisy

If there are additional signals outside of the glint and pupil regions, the brightness of the eye tracking sensors may be adjusted to reduce the noise.

Please contact AdHawk support for additional assistnace.

Pupil outline is not visible

The purple pupil signal should include the outline and the center of the pupil. If the pupil outline and center do not appear on the screen, repeat Auto-tune.

Reporting an Issue

To report an issue please follow the following procedure

Application issues

  1. Select Help > Report an issue from the menu bar

  2. Follow the steps in the wizard.

    The wizard will pre-populate the last recording and session of data as part of the issue report. You may remove the recording or session item by right-clicking on the item and selecting Remove.

  3. Completing the wizard will generate a report

  4. Contact support@adhawkmicrosystems.com and provide the report as instructed

Eye tracking issues

  1. Set the “Log mode” to “Full” under File > Settings > Logging on Windows or AdHawk Hub > Preferences > Logging on Mac OS X

  2. Ensure a profile is selected with as many details as possible

    Selecting a profile to start logging

     

  3. Click record in the track tab to record an eye tracking session

  4. Perform the actions that reproduce your issue

  5. Select Help > Report an issue from the menu bar

  6. Follow the steps in the wizard

    The wizard will pre-populate the last recording and session of data as part of the issue report. You may remove the recording or session item by right-clicking on the item and selecting Remove.

  7. Completing the wizard will generate a report

  8. Contact support@adhawkmicrosystems.com and provide the report as instructed

  9. Set the “Log mode” back to “Basic” under File > Settings > Logging on Windows or AdHawk Hub > Preferences > Logging on Mac OS X

Appendix

Nosepieces

The AdHawk glasses kit ships with a set of replaceable nosepieces. It is important to choose the correct nose piece for each user in order to achieve optimal eye tracking performance.

Inserting and removing a nosepiece

A nosepiece may be inserted or removed from the glasses by gently pinching the nosepads towards each other and sliding the bridge of the nosepiece into or out of the channel on the glasses. The index finger of the opposite hand can be used to guide the nosepiece in place. The nosepiece will snap in place once inserted fully.

Inserting and Removing a Nosepiece

 

Selecting a nosepiece

There are four nosepieces included in the kit. The four nosepieces are identified by coloured bands on the nose pads as shown below:

Included NosepiecesNosepieces

The nosepieces have been designed to accomodate different face and nose shapes to maintain a consistent glasses position relative to the eyeball. They follow the trend shown below:

Nosepiece-nose general trend

 

In theory they either change the height at which the glasses sit on a user’s face or change the distance the glasses sit from the eye (relief), but a combination of these may happen concurrently. In general, as you increase the nosepiece number it will add height to the glasses and increase the relief. The table below describes the change in glasses fit between the four nosepieces.

Nosepiece # Purpose Colour
1 Reduce Height (-Y) Pink
2 Reduce Relief (-Z) Yellow
3 Baseline Black
4 Increase Height (+Y) Blue
Correct Fit Example: Side

 

In order to determine the correct height the pupil should be located mid-way between the top and bottom of the frame.

Correct Fit Example: Front

 

People of east-Asian descent tend to have shallower noses and a shorter relief from the nose bridge to the surface of the cornea. For Asian faces it is recommended to begin with nosepiece 4 (blue). A visual assessment of the side profile is recommended to see if the nosepiece needs to be swapped before a demo is attempted.

Connecting the glasses

The AdHawk MindLink glasses can be used with either the included Android device or a PC. The AdHawk MindLink glasses use a MicroUSB connection and can be connected to each device with the included cables. The glasses ship with a micro USB to USB-A cable for connecting the glasses to a computer and a micro USB to USB-C cable for connecting them to the Android device. A USB-C to USB-C cable and charger is also included to charge the phone and an adapter is provided to allow the Android device to be connected to the computer for software updates and data transfer.

Calibrating the glasses using the calibration fixture

For best eye tracking results, the glasses should be calibrated using the calibration fixture and the AdHawk Hub software at the start of each day of use. Place the glasses in the calibration fixture (as shown below) and close the lid so that it snaps securely. Open the AdHawk Hub software on the computer and click Tools > Device Calibration to begin the process.

Calibrating the glasses

 

Removing and inserting the lenses

Removing a lens

Remove one lens at a time. Place your two thumbs on the inner part of the lens pushing from the inside out while applying pressure on the tips of both your thumbs. The lens should pop out. Do not be afraid to apply pressure to the lenses, they are designed to be removed and will not break

Removing the lenses

 

Inserting a lens

The lenses must be inserted from the outside of the glasses. Hold the lens with your index and thumb finger to position the lens into the v-groove, starting from the side closest to the arm. Press along the bottom of the lens and glasses so it is fitted into the v-groove and do the same for the top side applying pressure. Apply pressure on the outer corners of the lens until it pops in. If it is difficult to properly seat the lens, it can be pressed down against a hard surface with a soft covering to force the lenses into the v-groove.

Inserting the lenses

 

Installing the ear hooks

The ear hooks included in the kit may be installed on the arms of the glasses to improve the fit of the glasses. Slide the ear hook opening over the end of the glasses’ arm, starting with the pointed upper edge of the arm. Rotate the ear hook downwards to fully close the end of the arm with the ear hook. Slide and wiggle the ear hook until moved to desired location on the glasses’ arm.

Installing the ear hooks

 

Using the cable clips

A clip is provided on each of the USB cables. Clipping the cable onto an article of clothing is recommended to prevent the USB cable from tugging on the glasses during use.

Plug the cable into the glasses, place the glasses on your face, slide the cable clip along the cable to your preferred location and pinch the clip to affix it to the clothing. There should be enough room/gap between the clip and the glasses to allow the user to move their head freely without any tugging.

Glossary

Term Meaning
EVK Evaluation Kit
IR Infrared
MEMS Micro-Electro-Mechanical Systems
VCSEL Vertical Cavity Surface Emitting Laser
VR Virtual Reality

Copyright 2021 AdHawk Microsystems Inc

STRICTLY CONFIDENTIAL