The broadcast project is designed to show some alternative ways of performing multiple-computer experiments. These include:
The concept behind this example is to have this application listen in on any experiment, and to generate calibration and gaze-position displays, which would be combined with a video record of the experiment computer's display. (In fact, this demonstration would need some enhancements to be used in this way, as the overlay it generates would probably not align precisely with the video of the experiment computer's display. This could be fixed by changing the display mapping functions to incorporate additional correction factors).
To run this experiment, you will need two computers connected by a network hub or switch to the eye tracker. Start the broadcast application first, then the eyedata or gcwindow application on the other computer (or any application that uses real-time sample data). Run through the eyedata application in the usual way, and note that the display of the broadcast computer follows along, displaying calibration targets and with a gaze cursor during recording. After the eyedata application finishes, broadcast will wait for another session to begin.
The broadcast application starts by requesting time and status updates from the tracker, and checks to see if a connection has been opened by any other application. It then opens a broadcast connection to the tracker, and watches the tracker mode. When in the proper modes, it reproduces calibration targets or plots a gaze cursor. Otherwise, it displays a black display. When the other application disconnects from the eye tracker, broadcast is disconnected as well, and begins to poll the eye tracker for the start of the next session.
The source code for broadcast is mostly new. Only those parts that illustrate new concepts will be discussed in detail.
These are the files used to build broadcast. Those that were covered previously are marked with an asterisk.
Broadcast.h * | Declarations to link together the template experiment files. Most of the declarations in this file can be used in your experiments. |
Broadcast_main.c | A modified version of main.c in other templates, which does not do tracker setup. Instead, it polls the tracker state until another application opens a connection to it, then opens a broadcast connection. It then determines display resolution by reading tracker settings, and monitors tracker modes to determine when to display a gaze cursor or calibration targets. |
broadcast_record.c | A modified version of trial.c , from the eyedata template. This version does not start recording but simply turns on link data reception. It uses link data to plot a gaze cursor, and displays gaze position and tracker time at the top of the display. It exits when the tracker leaves recording mode. |
This module is derived from main.c used in most other templates, and does most of the usual setup, except for configuring the tracker and opening a data file. It begins by initializing the DLL so we can use the link to communicate with the tracker. By calling open_eyelink_connection(-1)
, this is done without opening a connection to the eye tracker. We also set our network name to "broadcast", so that comm_simple will be able to find us:
if(trackerip)set_eyelink_address(trackerip);elseset_eyelink_address("100.1.1.1");// open DLL to allow unconnected communicationsif(open_eyelink_connection(-1))return -1; // abort if we can't open link
The usual display and calibration are done next. We also call set_remap_hooks()
, which set up a "hook" function to remap the location of calibration targets to match our display resolution. We will discuss this code later.
Next, the code calls wait_for_connection()
(described later) to determine if another application has connected to the eye tracker. Once this has occurred, a broadcast connection is opened to the eye tracker to allow reception of link data and monitoring of the tracker interactions with the application. Next, we read the display resolution (actually, the gaze position coordinate system) that the tracker has been configured for. We then call track_mode_loop()
to monitor the tracker and determine when to display calibration targets or to plot the gaze cursor. When the other application closes its connection to the eye tracker, our broadcast connection is closed as well, and track_mode_loop()
returns.
while(1) // Loop through one or more sessions{// wait for connection to listen to, or abortedif(wait_for_connection()) goto shutdown;pump_delay(1000); // give remote and tracker time for setup// now we can start to listen inif(eyelink_broadcast_open()){alert_printf("Cannot open broadcast connection to tracker");goto shutdown;}clear_full_screen_window(transparent_key_color);can_read_pixel_coords = 1; // first try to read coordstracker_pixel_left = SCREEN_LEFT; // set defaults in case failstracker_pixel_top = SCREEN_TOP;tracker_pixel_right = SCREEN_RIGHT;tracker_pixel_bottom = SCREEN_BOTTOM;if(eyelink_is_connected())if(read_tracker_pixel_coords()==-1){alert_printf("Cannot determine tracker pixel coords:assuming %dx%d", SCRWIDTH, SCRHEIGHT);can_read_pixel_coords = 0;}track_mode_loop(); // listen and process by tracker modegoto shutdown;}
The function wait_for_connection()
loops until the tracker is connected to another application. It waits 500 milliseconds between tests (otherwise the tracker would be overloaded with our request) using the Sleep(500)
function, which also gives other Windows applications some time.
//******** WAIT FOR A CONNECTION TO TRACKER *********int wait_for_connection(void){int i;int first_pass = 1; // draw display only after first failurewhile(1) // loop till a connection happens{ // check if tracker is connectedi = preview_tracker_connection();if(i == -1){alert_printf("Cannot find tracker");return -1;}else if(i > 0)return 0; // we have a connection!if(first_pass) //If not, draw title screen{SDL_Color bg = {192,192,192};SDL_Color fg = {0,0,0};first_pass = 0; //don't draw more than onceclear_full_screen_window(bg);get_new_font("Times Roman", SCRHEIGHT/32, 1); //select a fonti = 1;graphic_printf(window, fg, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"EyeLink Broadcast Listening Demonstration");graphic_printf(window, fg, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"Copyright 2021 SR Research Ltd.");i++;graphic_printf(window, fg, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"Waiting for another computer to connect to tracker...");graphic_printf(window, fg, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"Press ESC to exit from this screen");graphic_printf(window, fg, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"Press ALT-F4 to exit while connected");SDL_Flip(window);}i = getkey(); //check for exitSleep(500); //go to background, don't flood the tracker}}
The tracker connection status is read by preview_tracker_connection()
, which communicates with the tracker without requiring a connection to be opened. A status and time request is sent by calling eyelink_request_time()
. When no connection has been opened by our application, this sends the request to the address set by set_eyelink_address()
(or the default address of "100.1.1.1" if this function has not been used to change this).
Next, we wait for a response to be returned from the tracker, monitoring this with eyelink_read_time()
which returns 0 until a response is received. This should take less than 1 millisecond, but we wait for 500 milliseconds before giving up (this means that no tracker is running at the specified address, or that the link is not functioning).
We then look at the link status flag data, which is part of the ILINKDATA
structure kept by the eyelink_core DLL. A pointer to this structure is returned by eyelink_data_status()
. Several flags indicate the connection status (for a complete list, see the eye_data.h header file).
//**************************** PREVIEW TRACKER STATE ************************// checks link state of tracker// DLL must have been started with open_eyelink_connection(-1)// to allow unconnected time and message communication// RETURNS: -1 if no reply// 0 if tracker free// LINK_CONNECTED if connected to another computer// LINK_BROADCAST if already broadcastingint preview_tracker_connection(void){UINT32 t, tt;eyelink_request_time(); // force tracker to send status and timet = current_msec();{tt = eyelink_read_time(); // will be nonzero if replyif(tt != 0){ // extract connection stateelse return 0;}message_pump(); // keep Windows happy}return -1; // failed (timed out)}
In order to properly plot gaze position and display calibration targets at the proper location on our display, we need to know the gaze position coordinate system used by the tracker, which was set to match the display resolution of the application that connected to the tracker. In the template comm_listener, this was done by intercepting the DISPLAY_COORDS message. In this example, we will read the gaze coordinate settings directly from the eye tracker. (We will need to read this before calibration as well, in case the resolution was changed by the application).
To read the gaze position coordinate system setting, we need to read the tracker setting for "screen_pixel_coords". We request this value by calling eyelink_read_request("screen_pixel_coords")
, then wait for a response by calling eyelink_read_reply()
. This will copy the tracker's response (as a series of numbers in a text string) into a buffer we supply. We then extract the desired numbers from this string using sscanf()
.
Note the variable can_read_pixel_coords
, which indicates if variables can be read from the tracker. Older EyeLink trackers my not allow reading of settings through a broadcast connection, and this variable will be set to 0 if the first read fails, preventing wasted time and error messages later.
//****** MAP TRACKER TO LOCAL DISPLAY ***********int can_read_pixel_coords = 1; // does tracker support read?float tracker_pixel_left = 0; // tracker gaze coord systemfloat tracker_pixel_top = 0; // used to remap gaze datafloat tracker_pixel_right = 0; // to match our display resolutionfloat tracker_pixel_bottom = 0;// Read setting of "screen_pixel_coords" from tracker// This allows remapping of gaze data if our display// has a different resolution than the connected computer// The read may fail with older tracker softwareint read_tracker_pixel_coords(void){char buf[100] = "";UINT32 t;eyelink_read_request("screen_pixel_coords");t = current_msec();while(current_msec()-t < 500){{sscanf(buf, "%f,%f,%f,%f", &tracker_pixel_left,&tracker_pixel_top, &tracker_pixel_right, &tracker_pixel_bottom );return 0;}message_pump(); // keep Windows happy}return -1; // timed out}
Once we have the gaze-position coordinate system of the tracker, we can map this to our display. These functions apply this mapping to X or Y position data:
// remap X, Y gaze coordinates to local displayfloat track2local_x(float x){return SCREEN_LEFT + (x - tracker_pixel_left) * SCRWIDTH /(tracker_pixel_right - tracker_pixel_left + 1);}float track2local_y(float y){return SCREEN_TOP + (y - tracker_pixel_top) * SCRHEIGHT /(tracker_pixel_bottom - tracker_pixel_top + 1);}
Finally, we need to ensure that calibration targets are drawn at the proper position on our display. We do this by setting a "hook" to the eyelink_core DLL, causing it to call our function setup_remap_hooks()
before drawing each calibration target.
// callback for calibration target drawing// this moves target to match position on other displaysstatic HOOKFCNS hfcns;void CALLBACK remap_cal_target(INT16 x, INT16 y){x = track2local_x(x);y = track2local_y(y);if(hfcns.draw_cal_target_hook)hfcns.draw_cal_target_hook(x,y);}// setup "hook" function to be called before calibration targets drawnvoid setup_remap_hooks(void){HOOKFCNS *hooks = get_all_hook_functions();memcpy(&hfcns,hooks,sizeof(HOOKFCNS));hooks->draw_cal_target_hook = remap_cal_target;setup_graphic_hook_functions(hooks);}
While connected to the eye tracker, we can monitor what mode it is in by calling eyelink_tracker_mode()
, and use this information to determine what we should be displaying. The function track_mode_loop()
contains a mode- monitoring loop to do this. For each pass through the loop, it determines if the tracker mode has changed and executes the proper operations for the new mode. It also performs the usual checks for disconnection or program termination, and also sends any local keypresses to the tracker (this may not be desirable for some applications).
// Follow and process tracker modes// Displays calibration and drift correction targets// Also detects start of recording// Black backgrounds would be transparent as video overlayvoid track_mode_loop(void){int oldmode = -1; // to force initial mode setupwhile(eyelink_is_connected()){int mode = eyelink_tracker_mode();unsigned key = getkey();else if(key) // echo to trackereyelink_send_keybutton(key,0,KB_PRESS);if(mode == oldmode) continue;
The core of track_mode_loop()
is a switch statement that performs the proper operations for each tracker mode. For most modes, the display is cleared to black ( transparent_key_color
) to allow the video to be seen. During camera setup or calibration, a gray background is displayed, with white calibration targets (black targets would be transparent to the video).
To handle calibration, validation, and drift correction, we call the DLL function target_mode_display()
, which handles display of calibration targets, calibration sounds, key presses, and so on. During recording, we call record_target_display()
, discussed below. Note that we call read_tracker_pixel_coords()
when entering the camera setup and calibration modes, to update this information.
switch(mode){case EL_RECORD_MODE: // Record mode: show gaze cursorclear_full_screen_window(transparent_key_color);record_mode_display();clear_full_screen_window(transparent_key_color);break;case EL_IMAGE_MODE: // IMAGE NOT AVAILABLE IN BROADCASTbreak;case EL_SETUP_MENU_MODE: // setup menu: just blank displayclear_full_screen_window(target_background_color);// read gaze coords in case changedif(eyelink_is_connected() && can_read_pixel_coords)read_tracker_pixel_coords();break;case EL_CALIBRATE_MODE: // show calibration targetsif(eyelink_is_connected() && can_read_pixel_coords)read_tracker_pixel_coords();case EL_VALIDATE_MODE:case EL_DRIFT_CORR_MODE:break;case EL_OPTIONS_MENU_MODE: // no change in visibilitybreak;default: // any other mode: transparent key (black)clear_full_screen_window(transparent_key_color);break;}oldmode = mode;}}
This module processes link data during a recording block. It plots samples as a gaze cursor (using code from the trial.c file in the eyedata template). It also prints the time (as the tracker timestamps) and gaze position. It exits when eyelink_tracker_mode()
indicates that the tracker has exited recording mode. This method is less precise than the monitoring of the data stream used in the comm_listener template, and can lose samples and messages at the start and end of the recording block, but is acceptable for simply plotting a visible gaze cursor.
Instead of starting recording, old link data is discarded by calling eyelink_reset_data(1)
, and data reception is enabled by eyelink_data_switch(RECORD_LINK_SAMPLES | RECORD_LINK_EVENTS)
. This will probably discard the first few samples in the data stream as well.
//******** PERFORM AN EXPERIMENTAL TRIAL ******int record_mode_display(void){ALLF_DATA evt;unsigned key;int eye_used = -1; // which eye to show gaze forfloat x, y; // gaze positionfloat ox=-1, oy=-1; // old gaze position (to determine change)// create font for position displayget_new_font( "Arial", SCRWIDTH/50, 0);//enable link data reception without changing tracker modeinitialize_cursor(window, SCRWIDTH/50);
The code then loops until exit conditions are met: disconnection, application termination, or the tracker switching to a non-recording mode.
while(1) // loop while in record mode{key = getkey(); // Local keys/abort testbreak;else if(key) // OTHER: echo to tracker for controleyelink_send_keybutton(key,0,KB_PRESS);
Finally, samples are read from the link to plot the gaze cursor. The gaze cursor code is similar to that in the trial.c file in the eyedata template, except for its color and shape, and will not be discussed here. The gaze position data is first converted to the local display coordinates by track2local_x()
and track2local_y()
, which are implemented in broadcast_main.c.
The eye to be plotted is determined when the first sample is read from the link, using eyelink_eye_available()
. In addition, the gaze position data and sample time are printed at the top left of the display.
// CODE FOR PLOTTING GAZE CURSOR{eyelink_newest_float_sample(&evt); // get the sample dataif(eye_used == -1) // set which eye to track by first sample{eye_used = eyelink_eye_available();eye_used = LEFT_EYE;}else{x = evt.fs.gx[eye_used]; // get gaze position from sampley = evt.fs.gy[eye_used];evt.fs.pa[eye_used]>0) // plot if not in blink{ // plot in local coordsdraw_gaze_cursor(track2local_x(x), track2local_y(y));// report gaze position (tracker coords)if(ox!=x || oy!=y) // only draw if changed{SDL_Rect r = {SCRWIDTH*0.87, 0,window->w -SCRWIDTH*0.87, 50};SDL_FillRect(window,&r,SDL_MapRGB(window->format,0, 0, 0));graphic_printf(window, target_foreground_color,NONE, SCRWIDTH*0.87, 0, " %4.0f ", x);graphic_printf(window, target_foreground_color,NONE, SCRWIDTH*0.93, 0, " %4.0f ", y);}ox = x;oy = y;}else{erase_gaze_cursor(); // hide cursor during blink}// print tracker timestamp of sample{SDL_Rect r = {SCRWIDTH*0.75, 0,SCRWIDTH*0.87 -SCRWIDTH*0.75, 50};SDL_FillRect(window,&r,SDL_MapRGB(window->format,0, 0, 0));graphic_printf(window,target_foreground_color,NONE, SCRWIDTH*0.75, 0, " % 8d ", evt.fs.time);}}}}erase_gaze_cursor(); // erase gaze cursor if visiblereturn 0;
The broadcast example is designed mainly to illustrate some advanced concepts, including monitoring the tracker mode and connection status, duplicating the display of calibration targets, and reading tracker variables. These allow it to function with almost any application that sends real-time sample data over the link. Many of these methods could be added to the comm_listener template as well. However, to be useful for real-time analysis broadcast would need to handle data during recording in a similar way to comm_listener.