The comm_simple and comm_listener projects are used together to show the required elements for an experiment where one application (comm_simple) is controlling the experiment, while a second application (comm_listener) opens a broadcast connection to monitor and analyze the real-time data. The programs are synchronized at startup by exchanging messages through the eyelink_core DLL, After this, comm_listener relies on the standard messages sent by comm_simple for synchronization and to identify the trial being executed. This data is simply used to reproduce the stimulus display, and to plot a gaze position cursor.
To run this experiment, you will need two computers connected by a network hub (a low-speed hub should be used, not a high-speed or multispeed hub) to the eye tracker. Start the comm_listener application first, then the comm_simple application. Run through the comm_simple application in the usual way, and note that the display of the comm_listener computer follows along, with a gaze cursor during recording. After the comm_simple application finishes, comm_listener will wait for another session to begin.
The comm_simple application first opens a connection to the eye tracker, then checks for the presence of the comm_listener application, and sends a message to inform comm_listener that the experiment has begun. The comm_listener then opens a broadcast connection to the tracker, and watches for messages and for the start and end of recording blocks. When comm_simple disconnects from the eye tracker, comm_listener is disconnected as well.
The source code for comm_simple is derived from the simple template, with only a few changes to enable messages and samples in real-time link data so these are available to comm_listener. A few minor rearrangements of commands and messages have also been made to ensure that messages are received by comm_listener at the proper time. The source code for comm_listener is mostly new, with plotting of the gaze cursor derived from the trial.c file used in the eyedata template.
These are the files used to build comm_simple. Those that were covered previously are marked with an asterisk.
comm_simple.h | Declarations to link together the template experiment files. Most of the declarations in this file can be used in your experiments. |
simple_trials.c | (Same file as used in the simple template). Called to run a block of trials for the simple template. Performs system setup at the start of each block, then runs the trials. Handles standard return codes from trials to allow trial skip, repeat, and experiment abort. This file can be modified for your experiments, by replacing the trial instance code. |
comm_simple_main.c | A modified version of main.c used in other templates, which changes the tracker configuration to allow messages in the real-time link data, and enables link data even when not recording. It also exchanges synchronizing messages with comm_listener . |
comm_simple_trial.c | A modified version of trial.c from the simple template. The only changes are to enable samples and events over the link while recording, and to ensure that recording has ended before returning to the trial loop. |
This module is almost identical to the file main.c used in most previous templates, and only the differences will be discussed. These are in tracker setup to enable messages in the link data, and a new function that synchronizes startup with the comm_listener application.
After being connected to the eye tracker, comm_simple must inform comm_listener that it may open a broadcast connection to the eye tracker. First, it polls the link for a remote named "comm_listener", calling eyelink_poll_remotes and checking for responses. If no remote named "comm_listener" is found, the application is probably not started and the program aborts. If the remote is found, its address is recorded and a message is sent to it. In this case, it is the name of the application, but in a real experiment it might be used to transfer other information.
// finds listener application// sends it the experiment name and our display resolution// returns 0 if OK, -1 if errorint check_for_listener(void){INT16 i, n;char message[100];ELINKNODE node; // this will hold application name and addresseyelink_poll_remotes(); // poll network for any EyeLink applicationspump_delay(500); // give applications time to respondn = eyelink_poll_responses(); // how many responses?for(i=1;i<=n;i++) // responses 1 to n are from other applications{{ // Found COMM_LISTENER: now tell it we're readymemcpy(listener_address, node.addr, sizeof(ELINKADDR));eyelink_node_send(listener_address, "NAME comm_simple", 40);// wait for "OK" replyif(get_node_response(message, 1000) <= 0) return -1;if(_stricmp(message, "OK")) return -1; // wrong response?return 0; // all communication checks out.}}return -1; // no listener node found}
Next, comm_simple waits for a message confirming that the message was received. The process of sending data and waiting for an acknowledging message prevents data from being lost, and could be extended to transfer multiple messages by simply repeating the process. Because it might be necessary to wait for a response at several places in the program a "helper" function has been supplied that waits for reception of a message, and returns an error if no message is received within a set time limit. This function could also be extended to check that the message address matches that of the comm_listener application.
ELINKADDR listener_address; // Network address of listener application// node reception "helper" function// receives text message from another application// checks for time out (max wait of 'time' msec)int get_node_response(char *buf, UINT32 time){UINT32 t = current_msec();ELINKADDR msgaddr; // address of message sender// wait with timeoutwhile(current_time()-t < time){if(i > 0) return i;}return -1; // timeout failure}
Finally, code is added to app_main()
to call the check_for_listener()
function. We set our network name to "comm_simple" first - this would allow an alternative means for a listening application to find or identify our application.
if(check_for_listener()) // check for COMM_LISTENER application{alert_printf("Could not communicate with COMM_LISTENER application.");goto shutdown;}
The remaining changes are to the tracker setup code. This has been rearranged so that the "DISPLAY_COORDS" message is sent after the link data configuration of the tracker is completed, to ensure that the comm_listener application will see this message. Alternatively, we could have re-sent this message later, or included the display resolution in the message sent directly to comm_listener earlier.
To enable messages in link data, the MESSAGE type is added to the list of event types for the "link_event_filter"
command.
// Select which events are saved in the EDF file. Include everything just in caseeyecmd_printf("file_event_filter = LEFT,RIGHT,FIXATION,SACCADE,BLINK,MESSAGE,BUTTON,INPUT");// Select which events are available online for gaze - contingent experiments. Include everything just in caseeyecmd_printf("link_event_filter = LEFT,RIGHT,FIXATION,SACCADE,BLINK,MESSAGE,BUTTON,FIXUPDATE,INPUT");//Select which sample data is saved in EDF file or available online.Include everything just in case// Check tracker version and include 'HTARGET' to save head target sticker data for supported eye trackerseyecmd_printf("file_sample_data = LEFT,RIGHT,GAZE,HREF,PUPIL,AREA,GAZERES,BUTTON,STATUS%s,INPUT", (tracker_software_ver >= 4) ? ",HTARGET" : "");eyecmd_printf("link_sample_data = LEFT,RIGHT,GAZE,GAZERES,AREA,STATUS%s,INPUT", (tracker_software_ver >= 4) ? ",HTARGET" : "");// NEW: Allow EyeLink I (v2.1+) to echo messages back to listenereyecmd_printf("link_nonrecord_events = BUTTON, MESSAGE");// Program button #5 for use in drift correctioneyecmd_printf("button_function 5 'accept_target_fixation'");// Now configure tracker for display resolutionif(dispinfo.refresh>40)
This module is almost identical to the file trial.c used in the simple template, with only two differences. The first is that samples and events are enabled by start_recording(1,1,1,1)
, to make this data available to comm_listener. The second difference is to call the function eyelink_wait_for_mode_ready()
at the end of the trial. The reason for this is that we called stop_recording()
to end the trial, which simply sends a command message to the eye tracker and does not wait for the tracker to actually stop recording data. This means that the TRIALID message for the next trial might actually be sent before recording ends, and comm_listener would see it arrive while processing eye data, and therefore not properly process it.
// Call this at the end of the trial, to handle special conditionserror = check_record_exit();// ensure we are out of record mode before returning// otherwise, TRIALID message could be send before// comm_listener sees end of recording block datareturn error;
These are the files used to build comm_listener. Those that were covered previously are marked with an asterisk.
comm_listener.h | Declarations to link together the template experiment files. Most of the declarations in this file can be used in your experiments. |
comm._listener_main.c | A modified version of main.c used in other templates, which does not do calibration setup or tracker setup. Instead, it waits for a message from comm_simple, then opens a broadcast connection and turns on link data reception. |
comm_listener_loop.c | New code, which listens to link data and messages. It determines the display resolution of comm_simple from DISPLAY_COORD messages, and reproduces the trial stimulus from TRIALID messages. When a recording block start is found in the data stream, it transfers control to comm_listener_record.c. |
comm_listener_record.c | A modified version of data_trial.c, from the eyedata template. This version does not start recording or draw trial stimulus (this was done previously in comm_listener_loop.c). It uses link data to plot a gaze cursor, and displays gaze position and time in the trial at the top of the display. Messages are read to determine trial start time from the "DISPLAY ON" message. It exits when the end of the recording block is found in the data stream. |
This module is derived from main.c used in other templates, but it uses only some of the application initialization code from that file. The startup code for comm_listener does not need to configure calibration graphics, open a data file, or send configuration commands to the eye tracker - this is all done by comm_simple. Instead, it waits for a message from comm_simple, then opens a broadcast connection and turns on link data reception.
First, the DLL is initialized so we can send and receive messages with comm_simple. By calling open_eyelink_connection(-1)
, this is done without opening a connection to the eye tracker. We also set our network name to "comm_listener"
, so that comm_simple will be able to find us:
// open DLL to allow unconnected communicationsif(open_eyelink_connection(-1))return -1; // abort if we can't open link
After the usual display and application setup, we then wait for a message from comm_simple, by calling wait_for_connection()
(described below). Once we have been contacted by comm_simple, a broadcast connection is opened and link data reception is enabled. Finally, we tell comm_simple that we are ready to proceed by sending an "OK" message, and call listening_loop()
(defined in comm_listener_loop.c). When comm_simple closes its connection to the eye tracker, our broadcast connection is closed as well, and listening_loop()
returns.
while(1) // Loop through one or more sessions{// wait for connection to listen to, or abortedif(wait_for_connection()) goto shutdown;// now we can start to listen inif(eyelink_broadcast_open()){alert_printf("Cannot open broadcast connection to tracker");goto shutdown;}//enable link data reception by EyeLink DLL//NOTE: this function can discard some link datapump_delay(500); // tell COM_SIMPLE it's OK to proceedeyelink_node_send(connected_address, "OK", 10);clear_full_screen_window(target_background_color);get_new_font("Times Roman", SCRHEIGHT/32, 1); // select a fonti = 1;graphic_printf(window, target_foreground_color, NONE, SCRWIDTH/15,i++*SCRHEIGHT/26, "Listening in on link data and tracker mode...");SDL_Flip(window);listening_loop(); // listen and process data and messages// returns when COMM_SIMPLE closes connection to trackergoto shutdown;}
The function wait_for_connection()
displays a startup message, and waits for a message from comm_simple. The contents of this message are ignored in this example, but the ELINKADDR
of comm_simple is saved for sending our reply.
ELINKADDR connected_address; // address of comm_simple (from message)//******** WAIT FOR A CONNECTION MESSAGE **********// waits for a inter-application message// checks message, responds to complete connection// this is a very simple example of data exchangeint wait_for_connection(void){int i;int first_pass = 1; // draw display only after first failurechar message[100];while(1) // loop till a message received{i = eyelink_node_receive(connected_address, message);if(i > 0) // do we have a message?{ // is it the expected application?if(!_stricmp(message, "NAME comm_simple")){ // yes: send "OK" and proceedreturn 0;}}if(first_pass) // If not, draw title screen{SDL_Color colr = { 0,0,0};first_pass = 0; // don't draw more than onceclear_full_screen_window(target_background_color);get_new_font("Times Roman", SCRHEIGHT/32, 1); // select a fonti = 1;graphic_printf(window, colr, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"EyeLink Data Listener and Communication Demonstration");graphic_printf(window, colr, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"Copyright 2024 SR Research Ltd.");i++;graphic_printf(window, colr, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"Waiting for COMM_SIMPLE application to send startup message...");graphic_printf(window, colr, NONE, SCRWIDTH/15, i++*SCRHEIGHT/26,"Press ESC to quit");SDL_Flip(window);}i = getkey(); // check for exit}}
The core of this module is listening_loop()
, which processes all link data broadcast from the tracker between recording blocks. It processes all messages (which are copies of those placed in the tracker data file by comm_simple) to determine display resolution (from DISPLAY_COORD messages) and to reproduce the trial stimulus (from TRIALIAD messages). In an actual data- listener application, the TRIALID message might be used to determine how to process recording data.
When the start of a recording block is encountered in the data stream, the function eyelink_in_data_block(1, 1)
will return 1. We then call listener_record_display()
to handle this data. Note that we look in the link data stream for the start of recording, rather than monitoring the eye tracker mode with eyelink_current_mode()
, as this ensures that we get all data and messages between the start and end of recording. This would not be as critical if the code for reading eye data samples and events was included in the same loop as code to read data between trials, as messages would always be processed properly by keyword.
//********** LISTENING LOOP *************void listening_loop(void){int i;int j = 6;char trial_word[40]; // Trial stimulus word (from TRIALID message)char first_word[40]; // first word in message (determines processing)tracker_pixel_left = (float)SCREEN_LEFT; // set default display mappingtracker_pixel_top = (float)SCREEN_TOP;tracker_pixel_right = (float)SCREEN_RIGHT;tracker_pixel_bottom = (float)SCREEN_BOTTOM;// Now we loop through processing any link data and messages// The link will be closed when the COMM_SIMPLE application exits// This will also close our broadcast connection and exit this loopwhile(eyelink_is_connected()){ALLF_DATA data; // link data or messages// exit if ESC or ALT-F4 pressedi = eyelink_get_next_data(NULL); // check for new data itemif(i == 0) continue;{eyelink_get_float_data(&data);#ifdef PRINT_MESSAGES // optionally, show messages for debuggingget_new_font("Times Roman", SCRHEIGHT/55, 1); // select a fontgraphic_printf(window, target_foreground_color, NONE, SCRWIDTH/15,j++*SCRHEIGHT/55, "MESSAGE=%s", data.im.text);#endifif(!_stricmp(first_word, "DISPLAY_COORDS")){ // get COMM_SIMPLE computer display sizesscanf(data.im.text, "%*s %f %f %f %f",&tracker_pixel_left, &tracker_pixel_top,&tracker_pixel_right, &tracker_pixel_bottom);}else if(!_stricmp(first_word, "TRIALID")){// get TRIALID informationsscanf(data.im.text, "%*s %s", trial_word);// Draw stimulus (exactly as was done in COMM_SIMPLE)#ifndef PRINT_MESSAGESclear_full_screen_window(target_background_color);#endif// We scale font size for difference in display resolutionsget_new_font("Times Roman", (int) (SCRWIDTH/25.0 *SCRWIDTH/(tracker_pixel_right-tracker_pixel_left+1)), 1);graphic_printf(window, target_foreground_color, NONE,(int) (SCRWIDTH/2), (int)(SCRHEIGHT/2), "%s", trial_word);Flip(window); //graphic_printf(window, target_foreground_color, NONE,(int) (SCRWIDTH/2), (int)(SCRHEIGHT/2), "%s", trial_word);}}// link data block opened for recording?if(eyelink_in_data_block(1, 1)){listener_record_display(); // display gaze cursor on stimulus// clear display at end of trial#ifndef PRINT_MESSAGESclear_full_screen_window(target_background_color);#endif}}}
It is very important to know the display resolution of the computer running comm_simple, as this sets the coordinate system that gaze data is reported in. Without this, differences in display settings between computers could cause gaze data to be plotted on the wrong position. This display information is read from the DISPLAY_COORDS message sent during tracker configuration. In addition, EyeLink trackers automatically insert a "GAZE_COORDS" message just before each recording block. Two mapping functions are supplied to convert data in the coordinates of the comm_simple display to local display coordinates:
//****** MAP TRACKER TO LOCAL DISPLAY ***********float tracker_pixel_left = 0; // tracker gaze coord systemfloat tracker_pixel_top = 0; // used to remap gaze datafloat tracker_pixel_right = 0; // to match our display resolutionfloat tracker_pixel_bottom = 0;// remap X, Y gaze coordinates to local displayfloat track2local_x(float x){return SCREEN_LEFT +(x - tracker_pixel_left) * SCRWIDTH / (tracker_pixel_right - tracker_pixel_left + 1);}float track2local_y(float y){return SCREEN_TOP +(y - tracker_pixel_top) * SCRHEIGHT / (tracker_pixel_bottom - tracker_pixel_top + 1);}
This module processes link data during a recording block. It plots samples as a gaze cursor. It also prints the time and gaze position. It exits when eyelink_in_data_block(1, 1)
return 0, indicating that the end of the recording block has been encountered in the data stream.
The first thing listener_record_display()
does is to determine which eye's data to plot. This is available from eyelink_eye_available()
, since we know we are in a recording block.
//******** PLOT GAZE DATA DURING RECORDING ******int listener_record_display(void){ALLF_DATA evt;UINT32 trial_start_time = 0;unsigned key;int eye_used; // which eye to show gaze forfloat x, y; // gaze positionfloat ox=-1, oy=-1; // old gaze position (to determine change)int i,j=1;// create font for position displayget_new_font( "Arial", SCRWIDTH/50, 0);initialize_cursor(window, SCRWIDTH/50);eye_used = eyelink_eye_available();// use left eye if both available
Next, we loop and process events and samples until the application is terminated, the link is closed, or the recording block ends.
{key = getkey(); // Local keys/abort testbreak;if(!eyelink_in_data_block(1, 1))break; // stop if end of record datai = eyelink_get_next_data(NULL); // check for new data item{eyelink_get_float_data(&evt); // get message#ifdef PRINT_MESSAGESgraphic_printf(window, target_foreground_color, NONE, SCRWIDTH/100,j++*SCRHEIGHT/50, "MESSAGE=%s", evt.im.text);#endif}
It is very useful to be able to detect gaps in the link data, which might indicate link problems, lost data due to delays in processing, or too many messages arriving for the Windows networking kernel to handle. This can be done using the new LOST_DATA_EVENT event, which is inserted by the eyelink_core DLL in the data stream at the position of the gap:
#ifdef LOST_DATA_EVENT // only available in V2.1 or later DLLalert_printf("Some link data was lost");#endif
Samples are processed in much the same way as in the data_trial.c file in the eyedata template. Before plotting the gaze cursor, the gaze position data is first converted from the coordinates of the comm_simple display to our display coordinates. Gaze position data is printed in its original form. The time of the sample is also printed, if the time of the trial start has been determined from the SYNCTIME message.
// CODE FOR PLOTTING GAZE CURSOR// new sample?if(eyelink_newest_float_sample(NULL)>0){// get the sample dataeyelink_newest_float_sample(&evt);// get gaze position from samplex = evt.fs.gx[eye_used];y = evt.fs.gy[eye_used];// if double-buffer then print time from start of trial here// and after draw_gaze_cursor flipif (ISPAGEFLIP(window))listener_time_print(evt.fs.time-trial_start_time, 0.75, 0.87);// plot if not in blinkevt.fs.pa[eye_used]>0){// plot in local coords// only draw if changedif(ox!=x || oy!=y)if (ISPAGEFLIP(window))listener_position_print(x, 0.87, y, 0.93);draw_gaze_cursor((int)track2local_x(x), (int)track2local_y(y));// report gaze position (tracker coords)// only draw if changedif(ox!=x || oy!=y)// print x coord at the top and 0.87 of display// print y coord at the top and 0.93 of displaylistener_position_print(x, 0.87, y, 0.93);ox = x;oy = y;}else{// hide cursor during blinkerase_gaze_cursor();}// print time from start of triallistener_time_print(evt.fs.time-trial_start_time, 0.75, 0.87);}}// erase gaze cursor if visibleerase_gaze_cursor();return 0;}
These templates are designed as examples of how to write cooperating applications, where one computer listens in on an experiment in progress. The code here is designed to show the basic elements, such as startup synchronization, enabling and processing link data, mapping gaze coordinates for differences in display resolution, and exchanging messages between applications.
There are other ways to achieve these operations: for example, messages could be exchanged directly between applications to transfer display resolution or TRIALID data, and the connection state of the tracker could be monitored instead of exchanging messages at startup (this will be used in the broadcast template, discussed next).