The dynamic sample experiment is moderately complex, but very powerful. It uses refresh-locked drawing to present dynamic, real-time displays, including sinusoidal smooth pursuit and a saccade performance task. The display timing achieved in both of these is extremely precise, and the source code was written so that the programmer does not need to modify any time-critical drawing code for most uses. In addition, this template demonstrates one-dimensional calibration which results in greater accuracy and faster setup, and shows how to seamlessly integrate drift correction into the task.
You should run this experiment at as high a display refresh rate as possible (at least 120 Hz). Higher refresh rates mean smoother motion during smooth pursuit, and better temporal resolution for displaying stimuli.
The code consists of several new modules. The code that is specific to experiments is largely isolated in one file, trials.c, which contains the usual do_trials()
function with the block-of-trials loop, and calls do_dynamic_trial()
to set up trial identifier and call the proper execution functions. These are do_sine_trial()
which executes a sinusoidal pursuit trial, and do_saccadic_trial()
which executes a saccadic trial using the gap- step-overlap paradigm.
Each of these functions then sets up a number of variables to define the trial, and calls run_dynamic_trial()
, supplying a background bitmap and a pointer to a trial-specific "callback" function. This function is called by run_dynamic_trial()
after each vertical retrace, to handle computing of target positions and visibility, update the target position and visibility, and send any required messages to the EDF file. These callback functions are fairly simple, because almost all the work is handled by target-drawing code in targets.c and the realtime trial loop in trial.c.
These are the files used to build dynamic. Those that were covered previously are marked with an asterisk.
main.c * | WinMain() function for windows non console compilation and main() for other compilations, setup and shutdown link and graphics, open EDF file. This file is unchanged for all templates, and can be used with small changes in your experiments. |
trials.c | Called to run the trials, and implements setup and callback functions for saccadic and smooth pursuit trials. Implements 1-D calibration. |
trial.c | Implements a real-time, refresh locked trial loop. This includes monitoring of delayed or missed retrace to ensure the quality of data from the experiment. |
targets.c targets.h | Creates a set of targets, which can have one of 3 shapes or can be hidden. Multiple targets may be visible at the same time. |
This module implements a complete target graphics system, which can draw, erase, and move multiple targets. This code will not be examined in detail, as it should not have to be changed except to create new target sizes and styles. A set of target "shapes" are created by create_shape()
and initialize_targets()
, with shape 0 always being invisible. The set of available shapes may be expanded by redefining NSHAPES and adding code to the switch() function in create_shape()
to draw the foreground of the new target image.
Target shapes consist of a small bitmap, which has the target image on a rectangular background. This entire bitmap, including the background, is copied to the display when drawing the target. Since the target background will be drawn it should match the background on which the targets will be displayed. The foreground color of the targets will usually be white or red.
Targets are erased by copying sections of a background bitmap to the display, which should be assigned to the variable target_background_bitmap
. This bitmap is also copied to the display by run_dynamic_trial()
before drift correction. Usually the background bitmap will be blanked to target_background_color
, but it may optionally contain cues or patterns.
These are the useful functions in targets.c, extracted from targets.h:
extern SDL_Surface target_background_bitmap; // bitmap used to erase targets// Create the target patterns// set all targets as not drawn, pattern 0// redefine as needed for your targetsint initialize_targets(SDL_Color fgcolor, SDL_Color bgcolor)// clean up targets after trialvoid free_targets(void)// draw target n, centered at x, y, using shape// will only draw if target is erasedvoid draw_target(int n, int x, int y, int shape)// erase target by copying back background bitmap// will only erase if target is drawnvoid erase_target(int n)// call after screen erased so targets know they're not visible// this will permit them to be redrawnvoid target_reset(void)// handles moving target, changing shape// target will be hidden it if shape = 0void move_target(int n, int x, int y, int shape)
The only modifications you should do to targets.c are to add or change the size or shape of targets - any other changes could cause the code to stop working properly. These changes are made to the function create_shape()
, and should be limited to the sections of code shown below.
The modifiable section of create_shape()
consists of two switch()
statements that set the size of the shapes and draw the foreground parts of the target shape bitmaps:
switch(n) // set size of target{case 0: // invisible targetcase 1: // filled circlecase 2: // "\\" linecase 3: // "/" linedefault:width = (SCRWIDTH/30.0)*0.5; // all targets are 0.5 by 0.5 degreesheight = (SCRHEIGHT/22.5)*0.5;break;}
In the example, all shapes have a size of 0.5 by 0.5 degrees, as computed for a 30-degree display width (distance between display and participant is twice the display width). You can change the values assigned to height
and width
to change the bitmaps sizes, and these should be referenced when drawing the shape graphics to auto-size them to the bitmaps.
The second section of code draws the graphics for the target - the bitmap has already been created and cleared to the background color.
SDL_FillRect(hbm,NULL,SDL_MapRGB(hbm->format,bgcolor.r, bgcolor.g, bgcolor.b));switch(n) // draw the target bitmap{case 0: // invisible targetdefault:break;case 1: // filled circle// draw filled ellipsefilledCircleRGBA(hbm, (Sint16) (width/2), (Sint16)(height/2),(Sint16)(min(width/2, height/2)-1),fgcolor.r,fgcolor.g,fgcolor.b,255);break;case 2: // "\" lineaalineRGBA(hbm,0,2,(Sint16)(width-2), (Sint16)height, fgcolor.r,fgcolor.g, fgcolor.b,255);aalineRGBA(hbm,0,1,(Sint16)(width-1), (Sint16)height, fgcolor.r,fgcolor.g, fgcolor.b,255);aalineRGBA(hbm,0,0,(Sint16)width,(Sint16)height, fgcolor.r,fgcolor.g,fgcolor.b,255);aalineRGBA(hbm,1,0,(Sint16)width,(Sint16)(height-1), fgcolor.r,fgcolor.g,fgcolor.b,255);aalineRGBA(hbm,2,0,(Sint16)width,(Sint16)(height-2), fgcolor.r,fgcolor.g, fgcolor.b,255);break;case 3: // "/" lineaalineRGBA(hbm,0,(Sint16)(height-2),(Sint16)(width-2), 0, fgcolor.r,fgcolor.g, fgcolor.b,255);aalineRGBA(hbm,0,(Sint16)(height-1),(Sint16)(width-1), 0, fgcolor.r,fgcolor.g,fgcolor.b,255);aalineRGBA(hbm,0,(Sint16)height,(Sint16)width,0 ,fgcolor.r,fgcolor.g,fgcolor.b,255);aalineRGBA(hbm,1,(Sint16)height,(Sint16)width,1,fgcolor.r,fgcolor.g,fgcolor.b,255);aalineRGBA(hbm,2,(Sint16)height,(Sint16)width,2,fgcolor.r,fgcolor.g,fgcolor.b,255);break;}
Changes should be limited to adding new cases, and increasing the number of shapes by redefining the constant NSHAPES
. Shape 0 will never be drawn, and should be left blank. The sample shapes draw a filled ellipse and two angles thick lines. Note that line width adapts to the display resolution to keep constant visibility at higher resolutions. Graphics should, if possible, be drawn in fgcolor
so that your experiments can easily set the color to match experimental requirements.
This module implements a retrace-locked drawing trial in the function run_dynamic_trial()
. The only reason to modify this code would be to remove the ability to terminate by a button press (which was included to allow skipping long pursuit trials), or to modify or remove drift correction (this is NOT recommended, as saccadic and pursuit tasks are typically run in non-CR modes which require drift correction before each trial).
The trial function requires three arguments:
// first argument is background bitmap for targets// third argument is pointer to drawing function:// int drawfn(UINT32 t, int *x, int *y)// where t = time from trial start in msec (-1 for initial target)// x, y are pointers to integer to hold a reference position// which is usually center or fixation target X,Y for drift correction// this function is called immediately after refresh,// and must erase and draw targets and write out any messages// this function returns 0 to continue, 1 to end trialint run_dynamic_trial(SDL_Surface* hbm, UINT32 time_limit,int (__cdecl * drawfn)(UINT32 t, UINT32 dt, int *x, int *y))
The first argument is a background bitmap, which will be copied to the display before drift correction, and over which the targets will be drawn (including the drift correction target). The second argument is the trial timeout value in milliseconds (this can be set to a very large number, and the drawing function can be used to determine timeout accurately). The third function is a pointer to a function that will handle retrace-locked drawing, sending messages, and ending the trial - this will be discussed later.
The code of this trial function has been modified in a number of ways. First, the drift correction loop now draws its own background and target, which causes the drift correction to be integrated with the initial fixation of the trial. The background is drawn by displaying the background bitmap - this should not contain stimuli, but might contain static graphics that are visible throughout the trial. After redrawing the whole display, reset_targets()
must be called so that targets will know that they are erased and may be redrawn later. Finally, the drawing function is called with a time of 0 to display the initial fixation target. The coordinates of this target are placed in the variables x
and y
, and used to inform the eye tracker of the drift correction target.
Note that this redrawing is done within a loop, so it will be repeated if setup was done instead of drift correction, as this will clear the display.
// DO PRE-TRIAL DRIFT CORRECTION// We repeat if ESC key pressed to do setup.// we predraw target for drift correction in this examplewhile(1){ // Check link often so we can exit if tracker stopped// (re) draw display and target at starting pointSDL_BlitSurface(hbm, NULL, window, NULL);Flip(window);SDL_BlitSurface(hbm, NULL, window, NULL);target_reset();drawfn(0, 0, &x, &y);// We drift correct at the current target location// 3rd argument is 0 because we already drew the displayerror = do_drift_correct((INT16)x, (INT16)y, 0, 1);// repeat if ESC was pressed to access Setup menuif(error!=27) break;}
Unlike the previous examples, special code was added to mark the time of the first retrace in the trial as trial start. The "!V TARGET_POS TARG1" message is inserted to the EDF file to record the current screen position of the target so that Data Viewer can plot the target trace in the Temporal Graph View.
if(trial_start == 0) // is this the first time we draw?{trial_start = t; // record the display onset timedrawfn(t, 0, &x, &y);// message for RT recording in analysiseyemsg_printf("%d DISPLAY ON", 0);eyemsg_printf("!V TARGET_POS TARG1 (%d, %d) 1 0", x, y);}else // not first: just do drawing{if(drawfn(t, t-trial_start, &x, &y)){end_trial(); // local function to stop recordingbutton = 0; // trial result message is 0 if timeoutbreak; // exit trial loop}}
Next, the drawing function is called. This is supplied with the time of retrace (used to determine message delays), the time from the trial start (used to determine time in the drawing sequence), and pointers to variables to hold the primary fixation target position (these are only used for drift correction). For the first call after the start of the trial, the time of trial start is set to the time of the first retrace, and messages are placed in the EDF file. As before, messages are only written after drawing is finished, and the delay of the message from the retrace is appended.
If the drawing function returned with 1, the trial will immediately end with a timeout. This allows trial end to be precisely synchronized with stimulus sequences.
This is the module where almost all of the experiment is defined and implemented, and where you will make the changes required to adapt the template for your own experiments. This module contains the block-of-trials loop in run_trials()
, and the trial setup and selector in do_trial()
, which are similar to those in other experiments.
One important difference is that this template uses horizontal-only calibration, which requires only 4 fixations which is ideal for use with neurologically impaired participants. The following code is added to run_trials()
before tracker setup, which sets the vertical position for calibration, and sets the calibration type. After this, vertical position of the gaze data will be locked to this vertical position, which should match the vertical position of your stimuli:
target_background_color.r=target_background_color.g=target_background_color.b=0;target_foreground_color.r=target_foreground_color.g=target_foreground_color.b=192;set_calibration_colors(&target_foreground_color, &target_background_color);// SET UP FOR HORIZONTAL-ONLY CALIBRATION
The only differences is that run_trials()
reports any delayed retrace drawing events after the block is finished (if your experiment has multiple blocks, this should be moved to the end of the experiment). The trial dispatch function do_trial()
also uses lookup tables for trial titles and TRIALID messages.
Each type of trial is implemented by a trial setup function and a drawing function. The setup function sets up variables used by the drawing function, creates targets and the background bitmap, and calls run_dynamic_trial()
. The drawing function is then called every vertical retrace by run_dynamic_trial()
, and performs drawing based on the time from the start of trial, places messages in the EDF file to record significant drawing events for analysis, and determines if the end of the trial has been reached.
The function do_saccadic_trial()
sets up for each trial in a gap-step-overlap paradigm. The first step is to create a blank background bitmap and initialize the target shapes (if these do not change, this could be done once for the entire experiment, instead of for each trial). Next, the interval between the saccadic goal target appearing and the fixation target disappearing is computed - if negative, the fixation target disappears before the goal target onset. The position of the fixation and goal targets are next calculated, adapting to display resolution to cause a saccade of the desired amplitude to the left or right. The last step of the setup is to draw reference graphics to the eye tracker display - in this case, a box at the position of each target.
//********** SACCADIC GAP/STEP/OVERLAP **************INT32 overlap_interval = 200; // gap <0, overlap >0UINT32 prestep_delay = 600; // time from trial start to moveUINT32 trial_duration = 2000; // total trial durationint fixation_x; // position of initial fixation targetint fixation_y;int goal_x; // position of final saccade targetint goal_y;int target_drawn=0;int fixation_visible; // used to detect changesint goal_visible;/*********** SACCADIC TRIAL SETUP AND RUN **********// run saccadic trial// gso = -1 for gap, 0 for step, 1 for overlap// dir = 0 for left, 1 for rightint do_saccadic_trial(int gso, int dir){int i;SDL_Color target_color = { 200,200,200};// blank background bitmapSDL_Surface* background;background = blank_bitmap(target_background_color);if(!background) return TRIAL_ERROR;// white targets (for visibility)initialize_targets(target_color, target_background_color);overlap_interval = 200 * gso; // gap <0, overlap >0fixation_x = SCRWIDTH/2; // position of initial fixation targetfixation_y = SCRHEIGHT/2;target_drawn = 0 ;// position of goal (10 deg. saccade)goal_x = fixation_x + (dir ? SCRWIDTH/3 : -SCRWIDTH/3);goal_y = fixation_y;set_offline_mode(); // Must be offline to draw to EyeLink screeneyecmd_printf("clear_screen 0"); // clear tracker display// add boxes at fixation goal targetseyecmd_printf("draw_filled_box %d %d %d %d 7", fixation_x-16, fixation_y-16,fixation_x+16, fixation_y+16);eyecmd_printf("draw_filled_box %d %d %d %d 7", goal_x-16, goal_y-16,goal_x+16, goal_y+16);// run sequence and triali = run_dynamic_trial(background, 200*1000L, saccadic_drawing);// clean up background bitmapSDL_FreeSurface(background);free_targets();return i;}
The saccadic trial is then executed by calling run_dynamic_trial()
, passing it the background bitmap and the drawing function for saccadic trials. Finally, we delete the targets and background bitmap, and return the result code from the trial.
The drawing function saccadic_drawing()
is called after each vertical retrace to create the display. For the saccadic trials, this involves deciding when the fixation and saccadic targets will become visible or be hidden, and reporting these events via messages in the EDF file.
The drawing code for saccadic trials is fairly straightforward, which was the goal of all the support code in this template. First, the visibility of the fixation target is determined by comparing the time after the start of the trial (dt) to the computed delay for fixation target offset. The fixation target is redrawn, and the process is repeated for the saccadic goal target. Note that move_target()
may be called even if the target has not changed state, as this function will not do any drawing unless the shape, visibility, or position of the target has changed.
fv = (dt < prestep_delay+overlap_interval) ? 1 : 0;// compute goal visibilitygv = (dt >= prestep_delay) ? 1 : 0;// draw or hide fixation targetmove_target(0, fixation_x, fixation_y, fv);// draw or hide goal targetmove_target(1, goal_x, goal_y, gv);SDL_Flip(window);// draw or hide fixation targetmove_target(0, fixation_x, fixation_y, fv);// draw or hide goal targetmove_target(1, goal_x, goal_y, gv);
After drawing, we can output messages without causing delays in drawing the stimuli. We output messages only if the state of the target has changed from the values stored from the previous call. The text of the messages simply represents the expected change in target appearance.
if(dt > 0) // no message for initial setup{if(fv != fixation_visible) // mark fixation offset{eyemsg_printf("!V FILLBOX 0 0 0 %d %d %d %d",fixation_x-(outer_diameter+1)/2,fixation_y-(outer_diameter+1)/2,fixation_x+(outer_diameter+1)/2,fixation_y+(outer_diameter+1)/2);}if(gv != goal_visible) // mark target onset{eyemsg_printf("!V FIXPOINT 255 255 255 255 255 255 %d %d %d 0",goal_x, goal_y, outer_diameter);}}
Finally, we send the fixation target position back to run_dynamic_trial()
for use in drift correction, and check to see if the trial is completed.
fixation_visible = fv; // record state for change detectiongoal_visible = gv;if(xp) *xp = fixation_x; //return fixation point locationif(yp) *yp = fixation_y;// check if trial timed outif(dt > trial_duration)return 1;elsereturn 0;
A second type of dynamic display trial implements sinusoidal smooth pursuit, where a target moves very smoothly and is tracked by the participant's gaze. High refresh rates and highly accurate positioning of the target are critical to produce the solid, subjectively smooth motion required. Drawing the target at the time of display refresh allows this to be achieved. This template implements sinusoidal pursuit by computing target position in the retrace drawing function. In addition it can change the target appearance at random intervals, which has been found to improve participant concentration and pursuit accuracy.
The code for sinusoidal pursuit is somewhat more complex than the previous example, mostly because of the code for target switching. The basics are very simple, and can be adapted for many pursuit patterns.
The function do_sine_trial()
sets up for each trial in the sinusoidal pursuit paradigm, and in general follows the steps used in the saccadic trial setup. The first step is to create a blank background bitmap and initialize the target shapes (if these do not change, this could be done once for the entire experiment, instead of for each trial).
Next, the variables that set the frequency, phase (where the sinusoidal cycle starts), the amplitude of the motion and the target switching interval are set. (Phase of the sinusoid is in degrees, where 0-degree is at the center and moving right, 90-degree is at the right extreme of motion, 180-degree is at the center and moving left, and 270-degree is at the left extreme. Pursuit is usually begun at the left (270-degrees) as the target accelerates smoothly from this position. The last step of the setup is to draw reference graphics to the eye tracker display - in this case, a box at the center and each end of the pursuit motion, and a line along the axis of target motion.
//******************* SINUSOIDAL PURSUIT DRAWING AND MOTION *****************#define PI 3.14159265358979323846#define FRACT2RAD(x) (2*PI*(x)) // fraction of cycles-> radians#define DEG2RAD(x) ((2*PI/360*(x)) // degrees -> radians#define FRACT2RAD(x) (2*PI*(x)) // fraction of cycles-> radians#define DEG2RAD(x) ((2*PI/360*(x)) // degrees -> radiansint sine_amplitude; // amplitude of sinusoid (pixels, center-to-left)int sine_plot_x; // center of sinusoidint sine_plot_y;float sine_frequency; // sine cycles per secondfloat sine_start_phase; // in degrees, 0=center moving rightint sine_cycle_count;UINT32 min_target_duration = 2000; // random target change intervalUINT32 max_target_duration = 4000;UINT32 next_target_time; // time of last target switchint current_target; // current target usedint prev_x=-1, prev_y=-1, outer_diameter;//********** SINUSOIDAL TRIAL SETUP AND RUN *********// setup, run sinusoidal pursuit trial// do_change controls target changesint do_sine_trial(int do_change){int i;SDL_Color target_color = { 255,0,0} ;// blank background bitmapSDL_Surface * background = NULL;background = blank_bitmap(target_background_color);// red targets (for minimal phosphor persistence)initialize_targets(target_color, target_background_color);sine_amplitude = SCRWIDTH/3; // about 10 degrees for 20 deg sweepsine_plot_x = SCRWIDTH/2; // center of displaysine_plot_y = SCRHEIGHT/2;sine_frequency = 0.4F; // 0,4 Hz, 2.5 sec/cyclesine_start_phase = 270; // start at leftsine_cycle_count = 10; // 25 secondsif(do_change) // do we do target flip?{current_target = 2; // yes: set up for flippingnext_target_time = 0;}else{current_target = 1; // round targetnext_target_time = 0xFFFFFFFF; // disable target flipping}set_offline_mode(); // Must be offline to draw to EyeLink screen// add boxes at left, right extremeeyecmd_printf("draw_filled_box %d %d %d %d 7", sine_plot_x-16, sine_plot_y-16,sine_plot_x+16, sine_plot_y+16);eyecmd_printf("draw_filled_box %d %d %d %d 7", sine_plot_x+sine_amplitude-16, sine_plot_y-16,sine_plot_x+sine_amplitude+16, sine_plot_y+16);eyecmd_printf("draw_filled_box %d %d %d %d 7", sine_plot_x-sine_amplitude-16, sine_plot_y-16,sine_plot_x-sine_amplitude+16, sine_plot_y+16);// add expected track lineeyecmd_printf("draw_line %d %d %d %d 15", sine_plot_x+sine_amplitude-16, sine_plot_y,sine_plot_x-sine_amplitude+16, sine_plot_y);prev_x = -1;prev_y = -1;i = run_dynamic_trial(background, 200*1000L, sinusoidal_drawing);SDL_FreeSurface(background);free_targets();return i;}
The pursuit trial is then executed by calling run_dynamic_trial()
, passing it the background bitmap and the drawing function for sinusoidal pursuit trials. Finally, we delete the targets and background bitmap, and return the result code from the trial.
The drawing function sinusoidal_drawing()
is called after each vertical retrace to create the display. For the sinusoidal pursuit trials, this involves computing the new target location, checking whether the target shape needs to be changed (and choosing a random duration for the next interval), redrawing the target, and reporting target position and appearance changes via messages in the EDF file. The drawing code is somewhat more complex than that for the saccadic trial, mostly due to the target shape changing code.
int __cdecl sinusoidal_drawing(UINT32 t, UINT32 dt, int *xp, int *yp){float phase; // phase in fractions of cycleint tchange = 0;int x, y;// phase of sinusoidphase =(float) (sine_start_phase/360 + (dt/1000.0)*sine_frequency);x = (int)(sine_plot_x + sine_amplitude*sin(FRACT2RAD(phase)));y = sine_plot_y;
Computing the target position is relatively straightforward. First, the phase of the target is computed, as a fraction of a cycle. The sine of this phase multiplied by 2*pi gives a number between 1 and -1, which is multiplied by the amplitude of the motion and added to the center position. The vertical position of the target is fixed for horizontal motion.
Next, we check to see if the target shape change interval has expired. If so, a new random interval is selected and a new shape is selected (this alternates between 2 and 3 which are left and right tilted lines). After this, the target is redrawn with the new position and appearance:
// compute target positionif(dt >= next_target_time){current_target = (current_target==2) ? 3 : 2;next_target_time = dt +(rand()%(max_target_duration-min_target_duration))+min_target_duration;tchange = 1;}move_target(0, x, y, current_target);Flip(window);
After drawing, it is safe to send messages to the EDF file for Data Viewer integration. In this example, a "!V TARGET_POS TARG1" message is sent when the target moves. This message has the X and Y position, target visibility (which is 1 by default). The last value of 0 means that the target position is interpolated across samples (e.g., a target moves smoothly).
// The following code is for Data Viewer Drawingif (x!= prev_x)eyemsg_printf(" !V TARGET_POS TARG1 (%d, %d) 1 0", x, y);if(tchange){}
Finally, the position of the target is reported for drift correction at the start of the trial, and the trial duration is checked. We have set the trial to end after a precise number of cycles of sinusoidal motion in this example.
if(xp) *xp = x; // return fixation point locationif(yp) *yp = y;prev_x = x ;prev_y = y;// check if proper number of cycles executedif(floor(phase-sine_start_phase/360) >= sine_cycle_count)return 1;elsereturn 0;
The specific example trials (saccadic tasks and sinusoidal pursuit) used in this example can be easily modified to produce tasks that use similar stimuli, or to use slightly different target sizes and colors. However, there are many different types of paradigms that can benefit from refresh-locked drawing, and almost all of these can be implemented by the background bitmap and drawing function method implemented here. Some examples are given below:
It is of course possible to move the drawing code into the trial function, rather than calling an external function. However, it is less easy to re-use such code, which was the goal of this example.