SR Research Support Site
Control

This template implements a computer interface that is controlled by the participant's gaze. The participant can select one of a grid of letters by fixating on it. The template contains code to support many rectangular selection regions, but can be simplified if gaze in a single region is all that needs to be detected.

The "Control" template is implemented using the module trial.c. The bitmap for the trial is a grid of letters, generated by grid_bitmap.c, which is not covered in this manual, as it is similar to other bitmap-drawing functions discussed previously.

Source Files for "Control"

These are the files used to build "Control". Those that were covered previously are marked with an asterisk.

main.c *WinMain() function for windows non console compilation and main() for other compilations, setup and shutdown link and graphics, open EDF file.
trials.cCalled to run the trial: just calls the trial, and so is not analyzed in this manual.
trial.cCreates and displays a grid of letters and does the recording.
regions.cImplements a gaze-controlled interface: sets up an array of gaze-activated regions and performs gaze-activated selection.
grid_bitmap.c *Creates a bitmap, containing a 5 by 5 grid of letters.

Analysis of "trial.c"

This module implements a computer interface controlled by the participant's gaze. A grid of letters is displayed to the participant, each of which can be selected by fixating it for at least 700 milliseconds. This demonstration also performs "dynamic recentering", where drift correction is performed concurrently with the selection process.

The type of gaze control in this example is intended for multiple gaze-selection regions, and long dwell threshold times. There is a substantial amount of extra code added to the module that supports this. If your task is to detect when gaze falls within one or two well-separated areas of the display, then there are simpler methods than that discussed here. For example, a saccade to a target can be detected by waiting for 5 to 10 samples where gaze position is within 2 degrees of the target. This is not suitable for detecting if gaze leaves an area, however.

Fixation Update Events

A common use for real-time link data is to monitor eye position, to ensure the participant's gaze stays within a prescribed location throughout a trial. The obvious method of checking that each gaze-position sample falls within the prescribed region may not work, because blinks may cause gaze position to change rapidly, giving a false indication. Monitoring average gaze position during fixations is more reliable, as this excludes data from blinks and saccades, but this data is not available until the end of the fixation.

The EyeLink tracker implements fixation update (FIXUPDATE) events, which report the average gaze position during a period of a fixation. By configuring the tracker to produce a FIXUPDATE event about every 50 milliseconds, fixation position is monitored in an efficient and timely manner. By looking at events instead of samples, the amount of processing required is also reduced. Blinks also are automatically excluded, and the sum of the time of fixation events in a region represent the true time of continuous gaze.

Enabling Fixation Updates

By default, fixation updates are disabled. Commands must be sent to enable fixation updates before recording is started, and to disable them afterwards. This code produces fixation updates every 50 milliseconds, and enables only FIXUPDATE events to be sent by the link (other events may be enabled if desired as well):

// Configure EyeLink to send fixation updates every 50 msec
eyecmd_printf("link_event_filter = LEFT,RIGHT,FIXUPDATE");
eyecmd_printf("fixation_update_interval = 50");
eyecmd_printf("fixation_update_accumulate = 50");
init_regions(); // Initialize regions for this display
//ensure the eye tracker has enough time to switch modes (to start recording).
// Start data recording to EDF file, BEFORE DISPLAYING STIMULUS
// You should always start recording 50-100 msec before required
// otherwise you may lose a few msec of data
error = start_recording(1,1,0,1); // send events only through link

Commands are added to the end_trial() function to disable fixation updates:

// End recording: adds 100 msec of data to catch final events
static void end_trial(void)
{
clear_full_screen_window(target_background_color); // hide display
end_realtime_mode(); // NEW: ensure we release realtime lock
pump_delay(100); // CHANGED: allow Windows to clean up
// while we record additional 100 msec of data
// Reset link data, disable fixation event data
eyecmd_printf("link_event_filter = LEFT,RIGHT,FIXATION,SACCADE,BLINK,BUTTON");
eyecmd_printf("fixation_update_interval = 0");
eyecmd_printf("fixation_update_accumulate = 0");
}

Processing Fixation Updates

The code to read FIXUPDATE events for the link is similar to that used in playback_trial.c, except that only events are processed:

// GET FIXUPDATE EVENTS, PROCESS
i = eyelink_get_next_data(NULL); // Check for data from link
if(i == FIXUPDATE) // only process FIXUPDATE events
{
// get a copy of the FIXUPDATE event
// only process if it's from the desired eye?
if(evt.fe.eye == eye_used)
{
// get average position and duration of the update
process_fixupdate((int)(evt.fe.gavx), (int)(evt.fe.gavy),// Process event
evt.fe.entime-evt.fe.sttime);
}
}

The function process_fixupdate() is passed the average x and y gaze data and the total time accumulated during the fixation update, which may vary. Each event will be processed to determine which letter's region it falls within. The time of consecutive fixation updates in a region is accumulated into the region's dwell field. If a fixation update does not fall into the currently accumulating region, it is assumed that gaze has shifted and the total time in all regions is reset to zero. To prevent noise or drift from inadvertently interrupting a good fixation, the first event in a new region is discarded.

Once the dwell time in a region exceeds the threshold, the letter is selected by inverting its rdraw region. A drift correction is performed, based on the difference between the average fixation position during selection, and the center of the selection region. This assumes that the visual center of the stimulus in the region is at the location set by the cx and cy fields of the region's data. Each drift correction may cause a jump in eye-position data, which can produce a false saccade in the eye-movement record.

// Process a fixation-update event:
// Detect and handle a switch between regions
// Otherwise, accumulate time and position in region
// Trigger region when time exceeds threshold
void process_fixupdate(int x, int y, long dur)
{
long avgx, avgy;
int i = which_region(x, y); // which region is gaze in
if(i == -1) // NOT IN ANY REGION:
{ // allow one update outside of a region before resetting
if(last_region == -1) // 2 in a row: reset all regions
{
reset_regions();
}
}
else if(i == current_region) // STILL IN CURRENT REGION
{
rgn[i].dwell += dur; // accumulate time, position
rgn[i].avgxsum += dur * x;
rgn[i].avgysum += dur * y;
if(rgn[i].dwell>dwell_threshold && !rgn[i].triggered) // did this region trigger yet?
{
trigger_region(i); // TRIGGERED:
avgx = rgn[i].avgxsum / rgn[i].dwell; // compute avg. gaze position
avgy = rgn[i].avgysum / rgn[i].dwell;
// correct for drift (may cause false saccade in data)
eyecmd_printf("drift_correction %ld %ld %ld %ld",
(long)rgn[i].cx-avgx, (long)rgn[i].cy-avgy,
(long)rgn[i].cx, (long)rgn[i].cy);
// Log triggering to EDF file
eyemsg_printf("TRIGGER %d %ld %ld %ld %ld %ld",
i, avgx, avgy, rgn[i].cx, rgn[i].cy, rgn[i].dwell);
}
}
else if(i == last_region) // TWO UPDATES OUTSIDE OF CURRENT REGION: SWITCH TO NEW REGION
{
reset_regions(); // clear and initialize accumulators
rgn[i].dwell = dur;
rgn[i].avgxsum = dur * x;
rgn[i].avgysum = dur * y;
current_region = i; // now working with new region
}
last_region = i;
}

Multiple Selection Region Support

Each selection region is defined by a REGION structure. This defines the rectangular area where it senses gaze, a rectangular area to highlight when selected, and the expected gaze position during selection for use in drift correction. It also contains accumulators for dwell time and gaze position.

// CONTROL REGION DEFINITION
typedef struct {
int triggered; // is triggered
long avgxsum; // average position accumulators
long avgysum;
long dwell; // total time in region
SDL_Rect rsense; // region for gaze
SDL_Rect rdraw; // rectangle for invert
int cx, cy; // center for drift correction
} REGION;
REGION rgn[NREGIONS];

The selection regions are created by init_regions(), which creates regions to match the display.

void SDL_SetRect(SDL_Rect *rect, int x, int y, int w, int h)
{
rect->x = x-w;
rect->y = y-h;
rect->w = w+w;
rect->h = h+h;
}
int SDL_PointInRect(SDL_Rect *rect, int x, int y)
{
if(x >= rect->x && x <=(rect->x +rect->w)
&& y>= rect->y && y <= (rect->y + rect->h) )
return 1;
return 0;
}
// Initial setup of region data
void init_regions(void)
{
int i;
int x,y;
for(i=0;i<NREGIONS;i++) // For all regions:
{
// compute center of region
x = (i%5) * (SCRWIDTH / 5) + (SCRWIDTH / 10);
y = (i/5) * (SCRHEIGHT / 5) + (SCRHEIGHT / 10);
rgn[i].cx = x; // record center for drift correction
rgn[i].cy = y;
SDL_SetRect(&(rgn[i].rdraw), x,y,SCRWIDTH/30,SCRHEIGHT/22);
SDL_SetRect(&(rgn[i].rsense), x,y,SCRWIDTH/10,SCRHEIGHT/10);
}
}

When a fixation update event arrives, its gaze position is checked against all selection regions:

// Determine which region gaze is in
// return 0-24 for a valid region, -1 if not in any region
int which_region(int x, int y)
{
int i;
for(i=0;i<NREGIONS;i++) // scan all regions for gaze position match
if(SDL_PointInRect(&(rgn[i].rsense), x,y)) return i;
return -1; // not in any region
}

Finally, a mechanism is needed to update the highlighted region on the display. In this example, the highlight stays on the selected region. It may be more ergonomic to simply flash the region momentarily once selected.

void trigger_region(int region)
{
int i;
for(i=0;i<NREGIONS;i++) // scan thru all regions
{
if(i==region) // is this the new region?
{
if(rgn[i].triggered==0) // highlight new region
invert_rect(&(rgn[i].rdraw), 1, i);
rgn[i].triggered = 1;
}
else
{
if(rgn[i].triggered) // unhighlight old region
invert_rect(&(rgn[i].rdraw), 0, i);
rgn[i].triggered = 0;
}
}
}

Copyright ©2002-2023, SR Research Ltd.