The purpose of this document is to describe the APIs used to interface with AdHawk’s Eye Tracking module.
Communication with the AdHawk Eye Tracking module can be accomplished via:
C SDK, which is built on top of the communication protocol and communicates with the device directly over USB or SPI.
C SDK via USB or SPI
Python SDK, which is built on top of the communication protocol, but proxies commands through the AdHawk Backend Service. The C SDK can also be set up to proxy commands in this fashion. The advantage of running in this mode is that multiple apps can be run at the same time.
Python or C SDK via Backend
Communication Protocol
The communication packets consist of a packet type followed by zero or more data bytes:
Byte
Type
Description
0
uint8
The packet type
1…
…
Payload
The length of payload is variable depending on the packet type
All values are stored in little endian (least significant byte first)
Control Interface
The control interface provides the ability to interact with AdHawk’s Eye Tracking module. The client/host initiates a command over the control interface by sending a request packet. The Control Packets section describes the list of available commands that the client/host can request over the control interface.
A request to the control interface may take up to 8 seconds to complete. All requests to the control interface will receive a response, either containing the requested information, or containing the result of the request. The clients should only issue one request at a time.
The responses that contain the result of the request consist of a 1-byte integer, where 0 indicates no error and successful execution of the requested command, and a non-zero value denotes the nature of error encountered while servicing the request. See the return codes section for details.
Stream Interface
AdHawk’s Eye Tracking module streams status and tracking data. Some streams like the tracker ready signal is always broadcasted. Other streams, such as the gaze vector stream, must be subscribed to.
Backend Service
The AdHawk Backend Service is the PC host component that communicates with AdHawk’s eye tracking hardware over USB, and allows multiple clients to interact with and receive data from the eye tracking module. In order to support multiple clients simultaneously, all communications between clients and the Backend Service is performed using UDP sockets. There are two sockets: control and data.
The control socket is constant, and is configured to 11032. The backend service is always listening on the control socket for client requests, and the responses to those requests are sent back to the source UDP port that was specified in the request packet.
The data socket is used to stream periodic eye tracking data to clients. Each client specifies its data port by sending a register endpoint packet to the control socket. The client data port may be the same as its control data port if desired.
Getting Started
Installation
Python SDK
pip install adhawk_py_sdk-<version>.tar.gz
C SDK
Refer to the README.md in the C SDK package
Setting up Communication
If communicating with the AdHawk Backend Service, a communication channel must be established over UDP. Start by registering as an endpoint by sending the register endpoint packet with the UDP port of the peer.
If using the Python SDK, this is handled by the FrontendApi and can be accomplished as follows:
import adhawkapiimport adhawkapi.frontenddef on_connect():print('Connected to AdHawk Backend Service')def on_disconnect():print('Disconnected from AdHawk Backend Service')api = adhawkapi.frontend.FrontendApi()api.start(connect_cb=on_connect, disconnect_cb=on_disconnect)...# Terminate communicationapi.shutdown()
The C SDK only requires the AdHawk Backend Service when running in UDP mode. The USB and SPI modes communicate directly with the device and do not require the AdHawk Backend Service. The API initialization call will take care of all setup required depending on the specified mode. See ah_com.h for details on how to connect in different modes
ah_api_init(params)
Auto-Tune
To tune the device, send the trigger autotune packet while the user is looking straight ahead. The response indicates whether the Auto-Tune procedure was successful.
C
ah_api_runAutotune();
Python
api.trigger_autotune()
Calibrate
Once the tracker ready signal is received or get tracker status returns success, the device is ready to be calibrated.
Calibration is the process of mapping the eye tracking signal to an individual’s visual axis, which makes the eye tracking output meaningful for typical applications. To initiate the calibration procedure send a calibration start packet.
The calibration procedure consists of the collection of a set of X, Y, Z gaze references. The gaze references are collected by instructing the user to look at a prescribed point and signaling the registration of the calibration data point through a register calibration point packet. When registering a calibration data point, the client must instruct AdHawk’s eye-tracking module of the user’s expected gaze during the collection of the calibration point. The expected gaze is specified in the payload of the Register calibration data point packet. Note that a response of ‘Left Eye Not Found’ or ‘Right Eye Not Found’ can be treated as a warning in binocular operation, but ‘Eyes Not Found’ is an error.
After collecting the data points, send the calibration complete packet to indicate that no further points will be collected and that a new calibration should be generated.
Once the calibration complete packet is received the eye-tracking module will compute the calibration coefficients based on the correspondence of sensor data and gaze references. The AdHawk’s eye-tracking module will respond with a zero (0) return code for successful calibration, and a non-zero value if calibration fails. See the return codes section for details.
If calibration is successful, the device is capable of streaming the gaze vector stream and the pupil position stream.
C
ah_api_calibrationStart();for(unsigned i =0; i < num_points; i++){float x, y, z;// As the eye moves and fixates on each point on the screen,// register the position of the point// get_reference_point(&x, &y, &z); ah_api_calibrationRegisterPoint(x, y, z);}ah_api_calibrationComplete();
Python
def grid_points(nrows, ncols, xrange=20, yrange=12.5, xoffset=0, yoffset=2.5):'''Generates a grid of points based on range and number of rows/cols''' zpos =-0.6# typically 60cm to screen# calculate x and y range for a xrange x yrange degree calibration window xmin, xmax = np.tan(np.deg2rad([xoffset -xrange, xoffset +xrange])) * np.abs(zpos) ymin, ymax = np.tan(np.deg2rad([yoffset - yrange, yoffset + yrange])) * np.abs(zpos) cal_points = []for ypos in np.linspace(ymin, ymax, nrows):for xpos in np.linspace(xmin, xmax, ncols): cal_points.append((xpos, ypos, zpos))print(f'grid_points(): generated {nrows}x{ncols} points'f' {xrange}x{yrange} deg box at {zpos}: {cal_points}')return cal_pointsnpoints =9nrows =int(np.sqrt(npts))reference_points = grid_points(nrows, nrows)api.start_calibration()for point in reference_points:# As the eye moves and fixates on each point on the screen,# register the position of the point api.register_calibration_point(*point)api.stop_calibration()
Enable data streams
Once the device is calibrated, to start receiving eye tracking data from the device, send a set stream control packet with the stream you want to enable and the rate at which you want to enable it. See Supported rates.
At this point, the device will start streaming packets to the registered endpoint. The format of the streamed packets are listed in the Data Streams section.
C
Once a device has been calibrated, the calibration coefficients can be saved using the blobs framework. The blobs framework allows saving and restoring run-time configuration objects of the eye-tracking module.
Send a save blob packet with the blob type set to 1 (gaze calibration data). If the blob was saved successfully, a unique 32-bit identifier is returned in the response packet. The calibration can later be loaded by sending a load blob packet with this ID in the payload.
C
A calibration start command packet will instruct AdHawk’s eye-tracking module to reset the calibration table and prepare for a new calibration procedure.
API
A calibration complete command will instruct AdHawk’s eye-tracking module that no further points will be collected and that a new calibration should be generated. Once the Calibration complete packet is received the eye-tracking module will compute the calibration coefficients based on the correspondence of sensor data and gaze references. The AdHawk’s eye-tracking module will respond with a zero (0) return code for successful calibration, and a non-zero value if calibration fails.
API
A calibration abort packet will instruct AdHawk’s eye-tracking module to discard all collected data for the new calibration, and restores the previous calibration.
API
Once a target gaze point is established and the user is instructed to direct their gaze at the target, a calibration data point may be registered. Registering a data point requires the client to instruct the eye-tracking module of the intended gaze target such that a correspondence can be established between the user’s gaze and the sensor data.
AdHawk’s eye-tracking module will respond with a zero (0) return code if the calibration point is successfully registered. Otherwise, a non-zero payload will be returned indicating that a correspondence between the user’s gaze and sensor data could not be established.
X, Y and Z is the position of the point relative to the midpoint of the scanners. See AdHawk Coordinate System for details.
API
When trigger scanner ranging is signaled, the eye-tracking module will perform an auto-tune function on all attached trackers. The eye-tracking module will respond with a zero (0) to indicate that scanner ranging successfully completed on both eyes. Otherwise, a non-zero payload will indicate which eye encountered a failure.
API
Updates an existing calibration based on the specified calibration point. The payload of the calibration data point indicates the position of the gaze target.
X, Y and Z is the position of the point relative to the midpoint of the scanners. See AdHawk Coordinate System for details.
API
These functions use the appropriate commands detailed below.
Blob size [0x92]
Retrieves a run-time configuration object from the eye tracking module.
The client/host should get the size of the configuration object using get blob size command, and allocate the required buffer size to store the configuration content. Then, the client/host can get the blob data in chunks of max 25 bytes using the get blob data command until the entire configuration object is retrieved.
Applies a run-time configuration object to the eye tracking module.
The client/host should set the size of the configuration object using the set blob size command. This will cause the eye tracking module to allocate the required buffer size to store the configuration content.Then the client/host can set the blob data in chunks of max 25 bytes using the set blob data command until the entire configuration object is written.
def chunks(data):'''Generator that breaks the data into chunks''' size =len(data) offset =0 chunk_len = adhawkapi.defaults.BLOB_CHUNK_LENwhile (offset + chunk_len) <= size:yield offset, data[offset:offset + chunk_len] offset = offset + chunk_lenyield offset, data[offset:]blob_type = adhawkapi.BlobType.CALIBRATIONfor offset, chunk in chunks(data): api.set_blob_data(blob_type, offset, chunk)
Load blob [0x94]
Applies a run-time configuration object to the eye tracking module. This is similar to writing a blob, except the data for the blob was previously saved to disk in the AdHawk Backend folder under cache as blob_.bin
# Loading a calibration blob that was previously saved under cache/blob_1241123630.binapi.load_blob(adhawkapi.BlobType.CALIBRATION, 1241123630)
Save blob [0x95]
Retrieves and saves a run-time configuration object from the eye tracking module. This is similar to reading a blob, except that the blob data is stored within the eye tracking module to disk in the AdHawk Backend folder under cache as blob_.bin, and the 32-bit unique identifier is returned to the clients for later use.
Get the center of the scan range. The returned values will be non-zero only if a trigger autotune command completed successfully or if the values were manually set through set autotune position
In the case of monocular operation, 0xFFFF will be returned for the X mean and Y mean of the inactive eye
API
Get the currently configured component offsets. Component offsets can be used if the scanners are not in their nominal position. The offset is the position of the scanner relative to its nominal position. See AdHawk Coordinate System for details.
API
Configure the component offsets. Component offsets can be used if the scanners are not in their nominal position. The offset is the position of the scanner relative to its nominal position. See AdHawk Coordinate System for details.
API
Begin a device calibration, which runs on all active trackers and restores tuning configurations to factory values. A response value of zero (0) indicates that the calibration has started successfully.
Retrieves the state of the current device calibration. If the calibration has not encountered an error, the returned value is an integer which indicates the progress of the current calibration, ranging from [0, 100], where 100 indicates the calibration has completed successfully.
Retrieves the state of the current firmware update. If the update has not encountered an error, the returned value is an integer which indicates the progress of the update, ranging from [0, 100], where 100 indicates the update has completed successfully.
Must be used to ping the AdHawk backend service to indicate that the frontend is still alive, and to verify that the backend service is still reachable. Should be sent every 2s. This is automatically handled within the Python and C SDKs.
API
The disconnect_cb passed into start is executed if the pings aren’t successful
Data Streams
Tracker ready [0x02]
The tracker ready packet type is an asynchronous status packet that is transmitted by AdHawk’s eye-tracking module when the system is ready to receive calibration commands. Once the tracker ready signal packet is received, the client/host system may go through a calibration process (send Calibration start, followed by Register calibration data point, and finally Calibration complete).
Byte
Type
Description
Stream
0
uint8
0x02
Gaze vector stream [0x03]
X, Y and Z are the estimated coordinates of the user’s gaze point relative to the midpoint of the scanners. Vergence is the angle between left and right eye gaze vectors. See AdHawk Coordinate System for details.
API
X, Y and Z are coordinates of the pupil relative to the point between the center of the two (left and right) scanners. See AdHawk Coordinate System for details.
API
class StreamRates(enum.IntEnum):'''Enum representing the set of supported rates''' OFF =0 RATE_30 =30 RATE_60 =60 RATE_125 =125 RATE_200 =200 RATE_250 =250 RATE_333 =333 RATE_500 =500
Extended Usage Notes
Python SDK
In the Python SDK, you can chose to make the API calls blocking or non-blocking. To perform a non-blocking operation, provide a callback function as a parameter to be executed when the operation is complete. This mode of operation is useful for GUI applications. If a callback function is not provided, the call blocks till a response is received and returns the response on success or raises an exception on failure.
# blocking operationtry: api.trigger_autotune()except adhawkapi.APIRequestError as err:print(err.ackcode)
