This template implements a computer interface that is controlled by the participant's gaze. The participant can select one of a grid of letters by fixating on it. The template contains code to support many rectangular selection regions, but can be simplified if gaze in a single region is all that needs to be detected.
The "Control" template is implemented using the module trial.c. The bitmap for the trial is a grid of letters, generated by grid_bitmap.c, which is not covered in this manual, as it is similar to other bitmap-drawing functions discussed previously.
These are the files used to build "Control". Those that were covered previously are marked with an asterisk.
main.c * | WinMain() function for windows non console compilation and main() for other compilations, setup and shutdown link and graphics, open EDF file. |
trials.c | Called to run the trial: just calls the trial, and so is not analyzed in this manual. |
trial.c | Creates and displays a grid of letters and does the recording. |
regions.c | Implements a gaze-controlled interface: sets up an array of gaze-activated regions and performs gaze-activated selection. |
grid_bitmap.c * | Creates a bitmap, containing a 5 by 5 grid of letters. |
This module implements a computer interface controlled by the participant's gaze. A grid of letters is displayed to the participant, each of which can be selected by fixating it for at least 700 milliseconds. This demonstration also performs "dynamic recentering", where drift correction is performed concurrently with the selection process.
The type of gaze control in this example is intended for multiple gaze-selection regions, and long dwell threshold times. There is a substantial amount of extra code added to the module that supports this. If your task is to detect when gaze falls within one or two well-separated areas of the display, then there are simpler methods than that discussed here. For example, a saccade to a target can be detected by waiting for 5 to 10 samples where gaze position is within 2 degrees of the target. This is not suitable for detecting if gaze leaves an area, however.
A common use for real-time link data is to monitor eye position, to ensure the participant's gaze stays within a prescribed location throughout a trial. The obvious method of checking that each gaze-position sample falls within the prescribed region may not work, because blinks may cause gaze position to change rapidly, giving a false indication. Monitoring average gaze position during fixations is more reliable, as this excludes data from blinks and saccades, but this data is not available until the end of the fixation.
The EyeLink tracker implements fixation update (FIXUPDATE
) events, which report the average gaze position during a period of a fixation. By configuring the tracker to produce a FIXUPDATE event about every 50 milliseconds, fixation position is monitored in an efficient and timely manner. By looking at events instead of samples, the amount of processing required is also reduced. Blinks also are automatically excluded, and the sum of the time of fixation events in a region represent the true time of continuous gaze.
By default, fixation updates are disabled. Commands must be sent to enable fixation updates before recording is started, and to disable them afterwards. This code produces fixation updates every 50 milliseconds, and enables only FIXUPDATE events to be sent by the link (other events may be enabled if desired as well):
// Configure EyeLink to send fixation updates every 50 mseceyecmd_printf("link_event_filter = LEFT,RIGHT,FIXUPDATE");eyecmd_printf("fixation_update_interval = 50");eyecmd_printf("fixation_update_accumulate = 50");init_regions(); // Initialize regions for this display//ensure the eye tracker has enough time to switch modes (to start recording).pump_delay(50);// Start data recording to EDF file, BEFORE DISPLAYING STIMULUS// You should always start recording 50-100 msec before required// otherwise you may lose a few msec of dataerror = start_recording(1,1,0,1); // send events only through link
Commands are added to the end_trial()
function to disable fixation updates:
// End recording: adds 100 msec of data to catch final eventsstatic void end_trial(void){clear_full_screen_window(target_background_color); // hide displayend_realtime_mode(); // NEW: ensure we release realtime lockpump_delay(100); // CHANGED: allow Windows to clean up// while we record additional 100 msec of data// Reset link data, disable fixation event dataeyecmd_printf("link_event_filter = LEFT,RIGHT,FIXATION,SACCADE,BLINK,BUTTON");eyecmd_printf("fixation_update_interval = 0");eyecmd_printf("fixation_update_accumulate = 0");}
The code to read FIXUPDATE events for the link is similar to that used in playback_trial.c, except that only events are processed:
// GET FIXUPDATE EVENTS, PROCESSi = eyelink_get_next_data(NULL); // Check for data from link{// get a copy of the FIXUPDATE eventeyelink_get_float_data(&evt);// only process if it's from the desired eye?if(evt.fe.eye == eye_used){// get average position and duration of the updateprocess_fixupdate((int)(evt.fe.gavx), (int)(evt.fe.gavy),// Process eventevt.fe.entime-evt.fe.sttime);}}
The function process_fixupdate()
is passed the average x and y gaze data and the total time accumulated during the fixation update, which may vary. Each event will be processed to determine which letter's region it falls within. The time of consecutive fixation updates in a region is accumulated into the region's dwell
field. If a fixation update does not fall into the currently accumulating region, it is assumed that gaze has shifted and the total time in all regions is reset to zero. To prevent noise or drift from inadvertently interrupting a good fixation, the first event in a new region is discarded.
Once the dwell
time in a region exceeds the threshold, the letter is selected by inverting its rdraw
region. A drift correction is performed, based on the difference between the average fixation position during selection, and the center of the selection region. This assumes that the visual center of the stimulus in the region is at the location set by the cx
and cy
fields of the region's data. Each drift correction may cause a jump in eye-position data, which can produce a false saccade in the eye-movement record.
// Process a fixation-update event:// Detect and handle a switch between regions// Otherwise, accumulate time and position in region// Trigger region when time exceeds thresholdvoid process_fixupdate(int x, int y, long dur){long avgx, avgy;int i = which_region(x, y); // which region is gaze inif(i == -1) // NOT IN ANY REGION:{ // allow one update outside of a region before resettingif(last_region == -1) // 2 in a row: reset all regions{reset_regions();}}else if(i == current_region) // STILL IN CURRENT REGION{rgn[i].dwell += dur; // accumulate time, positionrgn[i].avgxsum += dur * x;rgn[i].avgysum += dur * y;if(rgn[i].dwell>dwell_threshold && !rgn[i].triggered) // did this region trigger yet?{trigger_region(i); // TRIGGERED:avgx = rgn[i].avgxsum / rgn[i].dwell; // compute avg. gaze positionavgy = rgn[i].avgysum / rgn[i].dwell;// correct for drift (may cause false saccade in data)eyecmd_printf("drift_correction %ld %ld %ld %ld",(long)rgn[i].cx-avgx, (long)rgn[i].cy-avgy,(long)rgn[i].cx, (long)rgn[i].cy);// Log triggering to EDF fileeyemsg_printf("TRIGGER %d %ld %ld %ld %ld %ld",i, avgx, avgy, rgn[i].cx, rgn[i].cy, rgn[i].dwell);}}else if(i == last_region) // TWO UPDATES OUTSIDE OF CURRENT REGION: SWITCH TO NEW REGION{reset_regions(); // clear and initialize accumulatorsrgn[i].dwell = dur;rgn[i].avgxsum = dur * x;rgn[i].avgysum = dur * y;current_region = i; // now working with new region}last_region = i;}
Each selection region is defined by a REGION
structure. This defines the rectangular area where it senses gaze, a rectangular area to highlight when selected, and the expected gaze position during selection for use in drift correction. It also contains accumulators for dwell time and gaze position.
// CONTROL REGION DEFINITIONtypedef struct {int triggered; // is triggeredlong avgxsum; // average position accumulatorslong avgysum;long dwell; // total time in regionSDL_Rect rsense; // region for gazeSDL_Rect rdraw; // rectangle for invertint cx, cy; // center for drift correction} REGION;REGION rgn[NREGIONS];
The selection regions are created by init_regions()
, which creates regions to match the display.
void SDL_SetRect(SDL_Rect *rect, int x, int y, int w, int h){rect->x = x-w;rect->y = y-h;rect->w = w+w;rect->h = h+h;}int SDL_PointInRect(SDL_Rect *rect, int x, int y){if(x >= rect->x && x <=(rect->x +rect->w)&& y>= rect->y && y <= (rect->y + rect->h) )return 1;return 0;}// Initial setup of region datavoid init_regions(void){int i;int x,y;for(i=0;i<NREGIONS;i++) // For all regions:{// compute center of regionx = (i%5) * (SCRWIDTH / 5) + (SCRWIDTH / 10);y = (i/5) * (SCRHEIGHT / 5) + (SCRHEIGHT / 10);rgn[i].cx = x; // record center for drift correctionrgn[i].cy = y;SDL_SetRect(&(rgn[i].rdraw), x,y,SCRWIDTH/30,SCRHEIGHT/22);SDL_SetRect(&(rgn[i].rsense), x,y,SCRWIDTH/10,SCRHEIGHT/10);}}
When a fixation update event arrives, its gaze position is checked against all selection regions:
// Determine which region gaze is in// return 0-24 for a valid region, -1 if not in any regionint which_region(int x, int y){int i;for(i=0;i<NREGIONS;i++) // scan all regions for gaze position matchif(SDL_PointInRect(&(rgn[i].rsense), x,y)) return i;return -1; // not in any region}
Finally, a mechanism is needed to update the highlighted region on the display. In this example, the highlight stays on the selected region. It may be more ergonomic to simply flash the region momentarily once selected.
void trigger_region(int region){int i;for(i=0;i<NREGIONS;i++) // scan thru all regions{if(i==region) // is this the new region?{if(rgn[i].triggered==0) // highlight new regioninvert_rect(&(rgn[i].rdraw), 1, i);rgn[i].triggered = 1;}else{if(rgn[i].triggered) // unhighlight old regioninvert_rect(&(rgn[i].rdraw), 0, i);rgn[i].triggered = 0;}}}