SR Research Support Site
GCWindow

The most useful real-time experiment is a gaze-contingent display, where the part of the display the participant is looking at is changed, or where the entire display is modified depending on the location of gaze. These manipulations require high sampling rates and low delay, which the EyeLink tracker can deliver through the link.

You should run this experiment at as high a display refresh rate as possible (at least 120 Hz). Higher refresh rates mean lower delays between eye movements and the motion of the window.

This template demonstrates how to use the link's real-time gaze-position data to display a gaze-contingent window. This is an area centered on the point of gaze that shows a foreground image, while areas outside the window show the background image. SDL_util library can be used to draw the gaze cursors. Note that you will have to call initialize_dynamic_cursor to initialize.

Gaze-Contingent Window Issues

The gaze-contingent window in this template can be used to produce moving- window and masking paradigms. It is not intended for boundary paradigms, however.

Fast Updates

The window code in gazecursor.c operates by copying areas from a foreground bitmap to draw within the window, and from a background bitmap to draw outside the window. Once the window and background are initially drawn, the code needs only to redraw those parts of the window or background that changed due to window motion.

The first time the window is drawn, both the background and window are completely drawn. This takes much longer than the incremental updates during window motion, and is similar to the time it takes to copy a bitmap to the entire display. If the window is hidden, or drawn after being hidden, it will also take more time to draw. This is also true if the window movement between updates is larger than the window size.

At high display resolutions (where pixels are as small as 0.02 degrees), eye tracker noise and microsaccades can cause "jitter" of the window, which makes the edges of the window more visible. This constant drawing also makes Windows less responsive. The gaze contingent window code implements a "deadband" filter to remove the jitter while not adding any effective delay. This filter can be visualized as a washer on a tabletop being moved by a pencil through its hole - the washer only moves when the pencil reaches the sides of the hole, and small motions within the hole are ignored. The deadband filter is set to 0.1 degrees, which results in a negligible error in window position.

Windowing and Masking

The window can be used for two purposes: to show new information at the fovea, or to mask information from the fovea. When initializing the window, you must specify which operation the window is performing. The window code uses this to optimize its drawing to pass the minimum foveal information. This is done by erasing unwanted sections of the window first when showing new information, and by drawing new sections of the window (mask) first when masking foveal information. The effect is most important during large saccades.

Eye Tracker Data Delays

An important issue for gaze-contingent displays is the system delay from an eye movement to a display change. There are three factors affecting this delay: the eye tracker delay, the drawing delay, and the display delay.

The EyeLink tracker delay from an eye movement to data available by eyelink_newest_sample() is shown in the table below. This delay includes a time of half sample, which is the average delay from an eye movement to the time the camera takes an image. Note that this delay only affects the latency of data available from the link - sample timestamps are still accurate to the time the camera imaged the eye.

The EyeLink eye trackers have separate link and file filter settings, so the link filter can be disabled without affecting recorded data. The heuristic filter setting is controlled by the heuristic_filter command, sent to the tracker by the eyecmd_printf() function. (See the reference section of this manual for information on the EyeLink tracker configuration commands). The heuristic filter is automatically re-enabled at the end of each recording session, so needs to be explicitly changed for each trial.

ModeDelay$^1$Notes
1000 Hz, heuristic filter off< 2 ms high jitter
1000 Hz, level 1 (standard) heuristic filter< 3 ms .
1000 Hz, level 2 (extra) heuristic filter< 4 ms minimizes jitter
500 Hz, heuristic filter off3 ms high jitter
500 Hz, level 1 (standard) heuristic filter5 ms .
500 Hz, level 2 (extra) heuristic filter7 ms minimizes jitter
250 Hz, heuristic filter off6 ms high jitter
250 Hz, level 1 (standard) heuristic filter10 ms .
250 Hz, level 2 (extra) heuristic filter14 ms minimizes jitter

Remarks
1. Reports average end-to-end latency, measured from an actual physical event to availability of first data sample that registered the event on the Display PC via Ethernet or Analog output in C code.

Display Delays

The second component of the delay is the time to draw the new gaze-contingent window, and for the changed image data to appear on the monitor. The drawing delay is the time from new data being read until drawing is completed. This is usually less than 1 millisecond for typical saccades using a 10-degree gaze-contingent window, as only small sections of the window are erased and redrawn. This delay will be higher for large display changes, such as hiding or re-displaying a large window. In any case, the delay will be less that the full-screen bitmap copy time.

The final delay is caused by the time it takes to read out the image from the video card's memory to the monitor. The SDL version of the gcwindow sample experiment does retrace-locked display drawing. This means that, depending on the current retrace position, it can take up to one refresh period (ranging from 17 milliseconds for a 60 Hz display to 6 milliseconds for a 160 Hz display) before the stimulus starts to be drawn on the screen. Therefore, it is advantageous to use a monitor with a high refresh rate to reduce the delays in the display updating.

Source Files for "GCWindow"

These are the files used to build gcwindow. Those that were covered previously are marked with an asterisk.

main.c *\c WinMain() function for windows non console compilation and \c main() for other compilations, setup and shutdown link and graphics, open EDF file. This file is unchanged for all templates, and can be used with small changes in your experiments.
trials.cCalled to run a block of trials for the "gcwindow" template. Performs system setup at the start of each block, then runs the trials. Handles standard return codes from trials to allow trial skip, repeat, and experiment abort. This file creates multiple stimuli: text and pictures.
trial.cImplements a trial with a real-time gaze-contingent window, displayed using two bitmaps.

Analysis of "trials.c"

There are 5 trials to demonstrate different types of gaze-contingent window conditions, two using text and three using pictures. This requires some care with background colors. The code for each type of trail setup is similar to the modules trials.c in text example and trials.c in picture example.

Setup and Block Loop

The block loop for trials.c for the GCWindow template is similar to data_trial.c from the eyedata template, except that the brightness of the calibration background is darker. This allows it to approximately match both the text and picture trials. In an actual experiment, mixing stimuli with extreme differences in background brightness is not optimal.

// INITIAL CALIBRATION: matches following trials
SETCOLOR(target_foreground_color ,0,0,0); //color of calibration target
SETCOLOR(target_background_color,200,200,200); // background for drift correction
set_calibration_colors(&target_foreground_color, &target_background_color); // tell EXPTSPPT the colors

Setting Up Trials

Unlink the previous templates, each of the 5 trials has different setup. The setup code for each trials is executed within a switch() statement.

For trials 1 and 2, two bitmaps of text are created, one with characters and the other with "Xxx" words. A monospaced font is used so that the two displays overlap. For trial 1, the normal text is in the foreground, and the "Xxx" text is outside the window.

switch(num)
{ // #1: gaze-contingent text, normal in window, "xx" outside
case 1:
// This supplies the title at the bottom of the eyetracker display
eyecmd_printf("record_status_message 'GC TEXT (WINDOW)' ");
// Always send a TRIALID message before starting to record.
// It marks the start of the trial and should precede any other messages
eyemsg_printf("TRIALID 1");
// !V TRIAL_VAR message is recorded for EyeLink Data Viewer analysis
// It specifies a trial variable and value for the given trial
// This must be specified within the scope of an individual trial (i.e., after
// "TRIALID" and before "TRIAL_RESULT")
eyemsg_printf("!V TRIAL_VAR Type TEXT");
eyemsg_printf("!V TRIAL_VAR Central TEXT");
eyemsg_printf("!V TRIAL_VAR Periphery MASK");
if(create_text_bitmaps(1))
{
eyemsg_printf("ERROR: could not create bitmap");
return SKIP_TRIAL;
}
// IMGLOAD command is recorded for EyeLink Data Viewer analysis
// It displays a default image on the overlay mode of the trial viewer screen.
// Writes the image filename + path info
eyemsg_printf("!V IMGLOAD FILL text.png");
bitmap_save_and_backdrop(fgbm, 0, 0, 0, 0,"text.png", get_output_folder(), SV_NOREPLACE,
i = gc_window_trial(fgbm, bgbm, SCRWIDTH/4, SCRHEIGHT/3, 0, 60000L); // Gaze-contingent window, normal text
SDL_FreeSurface(fgbm); fgbm = NULL;
SDL_FreeSurface(bgbm); bgbm = NULL;
return i;

For trial 2, the bitmaps are reversed, and the window type is set to masking (fifth argument to gc_window_trial() ).

case 2: // #2: gaze-contingent text, "xx" in window, normal outside
eyecmd_printf("record_status_message 'GC TEXT (MASK)' ");
// Always send a TRIALID message before starting to record.
// It marks the start of the trial and should precede any other messages
eyemsg_printf("TRIALID 2");
// !V TRIAL_VAR message is recorded for EyeLink Data Viewer analysis
// It specifies a trial variable and value for the given trial
// This must be specified within the scope of an individual trial (i.e., after
// "TRIALID" and before "TRIAL_RESULT")
eyemsg_printf("!V TRIAL_VAR Type TEXT");
eyemsg_printf("!V TRIAL_VAR Central MASK");
eyemsg_printf("!V TRIAL_VAR Periphery TEXT");
if(create_text_bitmaps(2))
{
eyemsg_printf("ERROR: could not create bitmap");
return SKIP_TRIAL;
}
eyemsg_printf("!V IMGLOAD FILL text.png");
bitmap_save_and_backdrop(fgbm, 0, 0, 0, 0,
"text.png", get_output_folder(), SV_NOREPLACE,
i = gc_window_trial(bgbm, fgbm, SCRWIDTH/4, SCRHEIGHT/3, 1, 60000L); // Gaze-contingent window, masked text
SDL_FreeSurface(fgbm); fgbm = NULL;
SDL_FreeSurface(bgbm); bgbm = NULL;
return i;

For trials 3, 4, and 5, a mixture of pictures and blank bitmaps are used. For trial 3, the background bitmap is blank, and the foreground image is a picture (a peripheral mask).

case 3: // #3: Image, normal in window, blank outside
eyecmd_printf("record_status_message 'GC IMAGE (WINDOW)' ");
// Always send a TRIALID message before starting to record.
// It marks the start of the trial and should precede any other messages
eyemsg_printf("TRIALID 3");
// !V TRIAL_VAR message is recorded for EyeLink Data Viewer analysis
// It specifies a trial variable and value for the given trial
// This must be specified within the scope of an individual trial (i.e., after
// "TRIALID" and before "TRIAL_RESULT")
eyemsg_printf("!V TRIAL_VAR Type IMAGE");
eyemsg_printf("!V TRIAL_VAR Central IMAGE");
eyemsg_printf("!V TRIAL_VAR Periphery MASK");
if(create_image_bitmaps(0))
{
eyemsg_printf("ERROR: could not create bitmap");
return SKIP_TRIAL;
}
bitmap_to_backdrop(fgbm, 0, 0, 0, 0,
0, 0, (UINT16)(BX_MAXCONTRAST|((eyelink_get_tracker_version(NULL)>=2)?0:BX_GRAYSCALE)));
copy_resource_to_output_folder("images/sacrmeto.jpg");
eyemsg_printf("!V IMGLOAD FILL images/sacrmeto.jpg");
i = gc_window_trial(fgbm, bgbm, SCRWIDTH/4, SCRHEIGHT/3, 0, 60000L); // Gaze-contingent window, normal image
SDL_FreeSurface(fgbm); fgbm = NULL;
SDL_FreeSurface(bgbm); bgbm = NULL;
return i;

For trial 4, the foreground bitmap is a picture, and the background bitmap is the picture (a foveal mask).

case 4: // #4: Image, blank in window, normal outside
eyecmd_printf("record_status_message 'GC IMAGE (MASK)' ");
// Always send a TRIALID message before starting to record.
// It marks the start of the trial and should precede any other messages
eyemsg_printf("TRIALID 4");
// !V TRIAL_VAR message is recorded for EyeLink Data Viewer analysis
// It specifies a trial variable and value for the given trial
// This must be specified within the scope of an individual trial (i.e., after
// "TRIALID" and before "TRIAL_RESULT")
eyemsg_printf("!V TRIAL_VAR Type IMAGE");
eyemsg_printf("!V TRIAL_VAR Central MASK");
eyemsg_printf("!V TRIAL_VAR Periphery IMAGE");
if(create_image_bitmaps(1))
{
eyemsg_printf("ERROR: could not create bitmap");
return SKIP_TRIAL;
}
copy_resource_to_output_folder("images/sacrmeto.jpg");
eyemsg_printf("!V IMGLOAD FILL images/sacrmeto.jpg");
bitmap_to_backdrop(bgbm, 0, 0, 0, 0,
0, 0, (UINT16)(BX_MAXCONTRAST|((eyelink_get_tracker_version(NULL)>=2)?0:BX_GRAYSCALE)));
i = gc_window_trial(fgbm, bgbm, SCRWIDTH/4, SCRHEIGHT/3, 1, 60000L); // Gaze-contingent window, masked image
SDL_FreeSurface(fgbm); fgbm = NULL;
SDL_FreeSurface(bgbm); bgbm = NULL;
return i;

In trial 5, the foreground bitmap is a clear image and the background is a blurred image of the same picture (a simple variable-resolution display).

case 5: // #5: Image, blurred outside window
eyecmd_printf("record_status_message 'GC IMAGE (BLURRED)' ");
// Always send a TRIALID message before starting to record.
// It marks the start of the trial and should precede any other messages
eyemsg_printf("TRIALID 5");
// !V TRIAL_VAR message is recorded for EyeLink Data Viewer analysis
// It specifies a trial variable and value for the given trial
// This must be specified within the scope of an individual trial (i.e., after
// "TRIALID" and before "TRIAL_RESULT")
eyemsg_printf("!V TRIAL_VAR Type IMAGE");
eyemsg_printf("!V TRIAL_VAR Central IMAGE");
eyemsg_printf("!V TRIAL_VAR Periphery BLURRED");
if(create_image_bitmaps(2))
{
eyemsg_printf("ERROR: could not create bitmap");
return SKIP_TRIAL;
}
copy_resource_to_output_folder("images/sac_blur.jpg");
eyemsg_printf("!V IMGLOAD FILL images/sac_blur.jpg");
bitmap_to_backdrop(fgbm, 0, 0, 0, 0,
0, 0, (UINT16)(BX_MAXCONTRAST|((eyelink_get_tracker_version(NULL)>=2)?0:BX_GRAYSCALE)));
i = gc_window_trial(fgbm, bgbm, SCRWIDTH/4, SCRHEIGHT/3, 0, 60000L); // Gaze-contingent window, masked image
SDL_FreeSurface(fgbm); fgbm = NULL;
SDL_FreeSurface(bgbm); bgbm = NULL;
return i;

The function create_image_bitmaps() creates the proper set of bitmaps for each trial, and checks that the bitmaps loaded properly. It also generates the EyeLink graphics, and sets the background color for the drift correction to match the images.

switch(type)
{
case 0: // blank background
{
fgbm = image_file_bitmap("images/sacrmeto.jpg", 0, SCRWIDTH,SCRHEIGHT, 1);
bgbm = blank_bitmap(target_background_color, 0);
break;
}
case 1: // blank fovea
fgbm = blank_bitmap(target_background_color,1);
bgbm = image_file_bitmap("images/sacrmeto.jpg", 0, SCRWIDTH,SCRHEIGHT,1);
// bgbm = image_file_bitmap(imagepathsac, 0, SCRWIDTH,SCRHEIGHT,1);
break;
case 2: // normal and blurred bitmaps, stretched to fit display
fgbm = image_file_bitmap("images/sacrmeto.jpg", 0, SCRWIDTH,SCRHEIGHT,1);
bgbm = image_file_bitmap("images/sac_blur.jpg", 0, SCRWIDTH,SCRHEIGHT,1);
// fgbm = image_file_bitmap(imagepathsac, 0, SCRWIDTH,SCRHEIGHT,1);
// bgbm = image_file_bitmap(imagepathsac_blur, 0, SCRWIDTH,SCRHEIGHT,1);
break;
}

For trials 3 and 5, the window contains the maximum information, and the window masking type is set to 0 (erase before draw). In trial 4, the foreground bitmap is blank, and the window masking type is set to 1 (draw before erase). The masking type is set by the fifth argument to gc_window_trial().

Drawing the Window

The code for moving the gaze-contingent window is similar to that used to move the gaze cursor in the eyedata template. However, we don't copy any bitmap to the display. The function redraw_gc_window() will draw the background the first time it is called. We place code here to accurately mark the time of the display onset, in case reaction time needs to be determined. There are two ways to handle a blink. The preferred way is to freeze the gaze-contingent window by not calling redraw_gc_window() during the blink. Another possibility is to erase the window, but this takes longer to draw and causes distracting flickering, and is not recommended.

// NEW CODE FOR GAZE CONTINGENT WINDOW
if(eyelink_newest_float_sample(NULL)>0) // check for new sample update
{
eyelink_newest_float_sample(&evt); // get the sample
x = evt.fs.gx[eye_used]; // yes: get gaze position from sample
y = evt.fs.gy[eye_used];
if(x!=MISSING_DATA && y!=MISSING_DATA && evt.fs.pa[eye_used]>0) // make sure pupil is present
{
if(first_display) // mark display start AFTER first drawing of window
{
drawing_time = current_msec(); // time of retrace
trial_start = drawing_time; // record the display onset time
}
draw_gaze_cursor((int)x, (int)y); // move window if visible
if(first_display) // mark display start AFTER first drawing of window
{
first_display = 0;
drawing_time = current_msec() - drawing_time; // delay from retrace
eyemsg_printf("%d DISPLAY ON", drawing_time); // message for RT recording in analysis
//SDL_Flip(window);
SDL_BlitSurface(bgbm,NULL,window,NULL);
}
}
else
{
// Don't move window during blink
}
}

Other Gaze-contingent Paradigms

Saccade Contingent Displays

Although the EyeLink system makes start-of-saccade and end-of-saccade events available through the link, these will not usually be used for gaze-contingent displays, unless the display change is to occur at a significant delay after the saccade ends. The EyeLink on-line saccade detector uses a velocity-detection algorithm, which adds 4 to 6 samples (16 to 24 msec) delay to the production of the event, in addition to the regular (2 to 14 msec) sample delay from an eye movement. The timestamps in saccade events are modified to indicate the true time of the eye movement. As well, saccade size data is only available in the end-of-saccade event, which is available after the saccade ends.

For a gaze-contingent saccade detector, we recommend the use of a position- based algorithm computed from sample data. To detect the start of a saccade, compute the difference between the position of the eye in a sample and the average position of the last 6 samples. Do this separately for X and Y directions, then sum the absolute value of the differences and compare to a threshold of 0.3 to 0.6 degrees. This will detect saccade onset with a delay of 2 to 4 samples in addition to the sample data delay, and is most sensitive to large (> 4 degrees) saccades. This delay is largely due to the time it takes the eye to move a significant distance at the beginning of a saccade, so the detector will usually trip before the eye has moved more than 1 degree.

Boundary Paradigms

In a boundary paradigm, all or part of the display changes depending on where the locus of gaze is. For example, a line of text may change when the reader proceeds past a critical word in the sentence.

The best way to implement this is to modify the display when at least two samples occur within a region (in the reading case, the right side of the display). The display change would be made by copying a bitmap to the display. To keep the delay low, only part of the display should be copied from the bitmap, in this case the line of text. The area that can be changed to meet a given delay depends on your video card, CPU speed, and the display mode.


Copyright ©2002-2021, SR Research Ltd.