Arduino Latency Measurement Toolkit

In this Instructable we will show you how to do simple input latency measurements with an Arduino.
This instructable is part of a final project of the lecture Designing Interactive Systems 2 from the Media Computing Group at the RWTH Aachen University. It was created with the support of the Fablab Aachen.

Goal of this project is to describe how to build a measurement system with simple and cheap components, that measures the latency between a mouse click (or a different input signal) and the actual change of an object on the computer screen. This system can be used in HCI research to compare different input methods or simply measure the latency of different systems.

To measure the latency of a system reliable we need to connect sensors from the outside to it that can not be influenced by the system itself.
To accomplish this, we use a pressure sensor to detect the input of the system. A pressure sensor can be placed for example on top of any mouse button without disassembling the mouse and can sense when the mouse button is pressed. Additionally this sensor can be placed on any other button (like a key from a keyboard), a touchpad or in some cases even on a touchscreen. This external sensor is independent of the mouse button itself.
The change on the screen will be detected with a lightsensor (diode) that is placed directly on the screen surface. This diode will measure the change of the light emitted by the screen at a specific point. In our case we use a simple website that changes the color from black to white when clicked. The measurement itself will take place on an Arduino microcontroller board that is connected to the light sensor and the pressure sensor. This way the measurement system is independent from the system that will be measured.

Our system has its limitations and is meant to be seen as a basis for future latency measurement systems.

Step 1: What You Need:

For the most basic setup of the measurement system you will need the following:

• Arduino Uno
• Pressure sensitive resistor (Interlink FSR402short)
• Light sensing diode (OSRAM BPW 34 DIL)
• Resistors: 150k, 220k, 10k
• 10k Trimmer
• Wires
• Multimeter

Step 2: Circuit

The basic circuit we build is shown in the pictures above. Our first prototype was build on a breadboard and tested quickly on a smartphone, later we also soldered everything together on a self-made Arduino shield and did some tests on different computers.

Pin D2 of the Arduino is used to detect the input on the pressure sensor. D2 is connected to 5 V with an external 10k pullup resistor. The pressure sensor is connected on one side to ground and on one side to D2.
The pullup resistor will determine the point at which pressure value the Arduino will start the measurement. For our tests we used 10k. More information on how to determine this value is given in the section "How to test with our system: setting up the pressure sensor"

To detect the change from the light sensor we use the internal comparator of the Arduino connected to pin D6 and D7. The light sensor will be connected on one site to ground and on one site to D6. On D7 we connect a voltage divider build with the 10k Trimmer and the 220k resistor. This voltage divider is used to set a voltage in the range of approximately 0 - 200mV. This will be the reference voltage for the comparator. More information on how to determine this value is given in the section "How to test with our system: setting up the light sensor"

With this setup we are able to set threshold values with the hardware and only need to read digital values with the Arduino. This makes it possible to measure very fast without using a slow analog to digital converter.

Step 3: Code

For a simple measurement we used this Arduino code:

unsigned long startTime;
unsigned long endTime;
unsigned long latency;

void setup() {
   Serial.begin(115200);
   Serial.println("Start");
   DDRD = DDRD & B11111011; // set pin 2 as input
}

void loop() {
   while(PIND & B00000100){} //wait for pin 2 (preassure sensor) to get low
   startTime = micros(); //4 uS precise, can overflow after ~70min of arduino runtime without reset!
   while(ACSR & B00100000){} //wait for comparator to trigger (lightsensor)
   endTime = micros();
   while((ACSR & B00100000) == 0){} //wait for release
   while((PIND & B00000100) == 0){}
   latency = endTime - startTime;
   Serial.println(latency);
}

In our test we connected the Arduino to a computer to log the latency values over the serial connection.

In the setup routine we begin the serial connection and set pin 2 as an input.

To begin the first measurement we have to wait for the pressure sensor to detect a button press. Because the normal digital read function is very slow, we cycle through an empty while loop and check directly the input register that contains D2. When the pressure value is high enough the program jumps out of the loop and saves the current runtime of the Arduino (given with micros()) as the start time. Then the program waits for the comparator output pin (in the ACSR register) to detect the change in the light sensor and saves the runtime as the end time. It then waits for both sensors to get back to the normal state, calculates the latency and sends it over the serial connection.

This code is meant to be a simple example on how to measure latency. For different measuring situations the code has to be changed accordingly.

Step 4: How to Test With Our System: Setting Up the Light Sensor

1. Attaching the light sensor to the screen:
   During our tests we found out that its not enough to simply press the light sensor against the screen. Light coming from the side,hitting the sensor, can influence the emitted voltage drastically. The best way to assure clean signals is to put the light sensor in a housing, shielding it from the surrounding light and putting its light sensitive area as close to the screen as possible. For the first prototype we made a housing out of laser cut cardboard and foam that could be pressed against the screen. Fixating this prototype with tape was still hard so we build a second prototype with a micro-suction tape that sticked to the screen surface. This solution granted excellent results.
2. Measuring and setting the voltage values at the voltage divider:
   The light sensor we used produced a voltage of approximately 0-10 mV when the screen was black and around 200 mV when the screen was white. These values depend on the physical screen and on the virtual, on-screen-object where you want to detect the change. First of all it is important to measure these values. To do so disconnect the light sensor from the circuit and only leave the 150k resistor directly attached to it. Then simply put the light sensor on the screen on the object where you want to measure the change when a mouse click happens. In our case this was a simple website (download below as zip). Now measure the output voltage of the sensor with the multimeter and note the voltage values for both states. In our case this was 10mV for black and 200mV for white. Now you have to set the threshold value for the voltage divider. Plug in the Arduino attached to the circuit and use the multimeter to measure the voltage between ground and pin D7. With a screwdriver you are now able to change the trimmer resistance until the desired threshold voltage is reached. In our case we set the voltage to 100mV to be sure to catch the change from 0 to 200mV. The comparator will trigger when the voltage of the sensor is higher then the voltage of the voltage divider.
3. Check the setup with simple code:
   To check if the set voltage threshold value is correct you can use a simple code that nonstop runs:

Serial.println(ACSR & B00100000);

If everything is correct you should see the change in the serial console when switching the color of the object on screen.

Step 5: How to Test With Our System: Setting Up the Pressure Sensor

For a quick testing it suffices to use a 10k pullup resistor on pin D2 and simply attache the pressure sensor on the mouse button. Depending on the mouse button the sensor might activate a little bit before the real click happens. To prevent this from happening the mouse button should be clicked as fast and strong as possible to keep this problem to a minimum.

It is possible to test when the Arduino detects the press similar to the light sensor with a simple code that nonstop runs:

Serial.println(PIND & B00000100);

If the delay is to present it is possible to change the point where the pressure sensor triggers. This can be done by changing the 10k resistor to a different value or exchanging it to a trimmer to set the value in the finished circuit. When using a trimmer it is useful to add another resistor in series (e.g. 1k) to avoid destroying the sensor when setting the trimmer to 0 Ohm by accident. In general the Arduino will trigger when the resistor value of the pressure sensor will match the resistor value of the pullup. In our case we measured the resistance on the pressure sensor to be 10k at the pressure we want the Arduino to trigger. Therefore we used a 10k pullup as well.

Now you are able to use the example code with the provided html-website to do the first latency tests. Be sure to change the code depending on if you want to measure a switch from black to white or the other way around.

Step 6: How to Test With Our System: Things to Consider

To measure how precise our system is we used an oscilloscope that was tracking on one channel directly the mouse button and on another channel a second light sensor connected to the screen. In general the Arduino measurement was off by constant 1-2ms. This offset probably is introduced by the pressure sensor that is detecting the press before the mouse button does.

When measuring with the oscilloscope we found more interesting phenomena:

When an LCD screen changes from white to black or vice versa it takes approximately 10ms to completely change the brightness. This can be directly observed when watching the voltage of the light sensor. In comparison a smartphone screen with OLEDs takes less then 1ms to change.
Additionally we found out that the frame rate at which the screen refreshes is important. We discovered both on the oscilloscope and the Arduino that the same measurement repeated many times had differing results by up to 20ms. We assume this could be influenced if the screen refreshes 60 or 50 times per second resulting in a new image every 20ms. When the command that changes the screen arrives directly before a refresh happens it can be measured with almost no delay. If it arrives shortly after a refresh it could take up to 20ms until the change gets present.