Appendix 4 - System latencies

Brain computer interface (BCI) systems must process neural signals with consistent timing in order to support adequate system performance. Thus, it is important to have the capability to determine whether a particular BCI configuration (hardware or software) provides adequate timing performance for a particular experiment. This appendix describes the latencies present when using OpenBCI-Stream for acquisition and distributed EEG signal processing 1.

84f2373731bf4106bb29bb3099a5b7a8

Latency Definitions

Real-time

The term real-time is used somewhat loosely, indicating only that the BCI system is able to process an entire block of data and update the output device before the next sample block is ready for processing. However, the degree to which the BCI system can be considered real-time is dependent on many factors, including the operating system, computer hardware specifications, and output device hardware. Furthermore, even if the mean overall processing and output time is less than the sample block size, large variability in this timing can significantly affect BCI performance, and must therefore be accounted for 1.

Sample block size and Block duration (\(T_2\) - \(T_{-1}\))

Ideally, the block duration should be identical to the sample block size; however, inconsistencies in operating system timing may interrupt and delay data acquisition, causing the time period between data blocks to be different than the actual block size, introducing a timing jitter.

The block duration is the primary indicator of the system’s ability to perform online signal processing in a BCI experiment. It is important to realize that the block duration is measured from the perspective of the software, and is not the same as the block size. That is, the block duration will never typically be less than the block size (i.e., the length of a block of data acquired from the ADC), but it can be longer than the block size if the system latency is longer than the block size.

ADC buffer (\(T_{-1}\) - \(T_{-2}\))

The ADC (analog to digital converter) buffer needs to be filled with an amount of raw data before digitize, this factor can be configured from WiFi Shield properties. Over Serial protocol, this latency can be related to RFduino protocol transmission.

Acquisition (\(T_0\) - \(T_{-1}\))

This is the delay between the time that the final sample in a sample block is digitized to when the sample block has been acquired by the software and is available to the software for processing. Depending on configuration, this latency may comprise physical signal delay in the amplifier, digitization, transmission from the ADC to the PC, and processing time inside a hardware driver.

Methods for measuring latencies

All measures performed will use timestamps and software methods, this is possible due to the Kakfa protocol that registers production timestamps and other ones that are packed with the data stream. For this specific debugging, the acquisition runs on a Raspberry Pi 4 and the data stream is collected remotely.

This measures can be done with this simple script that use an existing data stream:

[72]:
def get_latencies(samples=60):
    openbci_consumer = KafkaConsumer(bootstrap_servers=[f'{HOST}:9092'],
                                         value_deserializer=pickle.loads,
                                         auto_offset_reset='latest')
    openbci_consumer.subscribe(['eeg'])

    latencies = {}
    for record in openbci_consumer:

        reference = datetime.fromtimestamp(record.timestamp / 1000)
        timestamp_binary = datetime.fromtimestamp(record.value['context']['timestamp.binary'])
        timestamp_binary_consume = datetime.fromtimestamp(record.value['context']['timestamp.binary.consume'])
        timestamp_eeg = datetime.fromtimestamp(record.value['context']['timestamp.eeg'])

        latencies.setdefault('Block duration', []).append(reference.timestamp())
        latencies.setdefault('Binary produced', []).append((reference - timestamp_binary).total_seconds() * 1000)
        latencies.setdefault('Binary consumed', []).append((reference - timestamp_binary_consume).total_seconds() * 1000)
        latencies.setdefault('EEG produced', []).append((reference - timestamp_eeg).total_seconds() * 1000)
        latencies.setdefault('_EEG consumed', []).append((datetime.now() - reference).total_seconds()*1000)

        if len(latencies['Block duration']) >= samples:
            print('done')
            break
    latencies['Block duration'] = np.diff(latencies['Block duration'])*1000

    return latencies

For a configuration of 100 ms blocks, 8 channels and 1000 samples per second:

[219]:
latencies_100 = get_latencies(samples=60)
show_latencies(latencies_100, title='for 100 ms block duration at 1000 SPS and 8 channels')
../_images/notebooks_A4-system_latencies_7_0.png

This plot shows the cumulative latency from the first process (acquisition) to deserialization of the EEG.

[220]:
latencies_500 = get_latencies(samples=60)
show_latencies(latencies_500, title='for 500 ms block duration at 1000 SPS and 8 channels')
../_images/notebooks_A4-system_latencies_9_0.png
[221]:
latencies_1000 = get_latencies(samples=60)
show_latencies(latencies_1000, title='for 1000 ms block duration at 1000 SPS and 8 channels')
../_images/notebooks_A4-system_latencies_10_0.png

Additional latency for distributed systems

Since the data transmission is not instantaneous, access to the stream remotely adds a latency that can measure mainly in two features.

NTP offset

Before measuring the latency in two different systems is needed to get the systems synchronized, the implementation used for this test is synchronized with the Network Time Protocol server running on Raspberry Pi.

[157]:
client = ntplib.NTPClient()
ntp_offset = client.request(HOST).offset * 1000
print(f" NTP offset: {ntp_offset :.2f} ms")
 NTP offset: 4.21 ms

Distributed latency

This latency is related with Kafka, is the time from the EEG producer (on Raspberry) to EEG consumed in the remote system.

[158]:
print(f"Distributed: {np.min(latencies_100['_EEG consumed'])} ms")
Distributed: 2.444 ms

The final offset is:

[227]:
offset = np.min(latencies_100['_EEG consumed']) + ntp_offset
print(f"Total offset: {offset:.2f} ms")
Total offset: 6.66 ms

This offset affects all latencies except the block duration.

[222]:
show_latencies(latencies_100, offset=offset, title='for 100 ms block duration at 1000 SPS and 8 channels')
../_images/notebooks_A4-system_latencies_19_0.png

Results

Has been proved that OpenBCI-Stream can distribute EEG signal (with metadata) in approximately the 50% of the block duration, this performance allows the user through asynchronous data processing the development of real-time multipurpose BCI systems.

Real-time issues are paradigm-agnostic since some implementations require windows of 2.5 seconds 2 or more, even short time control signals like P300 need commonly too many trials 3, which in practice is an even bigger window.

Block duration

measured block duration

measured block duration jitter

Latency system

Latency system jitter

50 ms

50.37 ms

13.23 ms

35.03 ms

11.74 ms

100 ms

100.07 ms

5.72 ms

56.00 ms

4.01 ms

300 ms

300.25 ms

101.64 ms

134.82 ms

9.71 ms

500 ms

500.15 ms

15.20 ms

236.72 ms

11.11 ms

800 ms

800.08 ms

23.01 ms

380.45 ms

15.27 ms

1000 ms

1000.36 ms

20.04 ms

453.53 ms

13.03 ms

After a set of simulations (at 1000 samples per second and 8 channels) is possible to determinate that the system latency is aproximately 50% of block size for block size highter that 500 ms

db7a7a247e2f44428ca8ceaaf266e0c2


1(1,2)

J Adam Wilson, Jürgen Mellinger, Gerwin Schalk, and Justin Williams. A procedure for measuring latencies in brain–computer interfaces. IEEE transactions on biomedical engineering, 57(7):1785–1797, 2010.

2

Aldo Mora-Sánchez, Alfredo-Aram Pulini, Antoine Gaume, Gérard Dreyfus, and François-Benoît Vialatte. A brain–computer interface for the continuous, real-time monitoring of working memory load in real-world environments. Cognitive neurodynamics, pages 1–21, 2020.

3

Luis Fernando Nicolas-Alonso and Jaime Gomez-Gil. Brain Computer Interfaces, a Review. Sensors, 12(2):1211–1279, January 2012. URL: http://www.mdpi.com/1424-8220/12/2/1211 (visited on 2021-05-04), doi:10.3390/s120201211.