High-throughput data distribution for online computing in the CBM experiment

The CBM experiment, a fixed-target heavy-ion experiment at the upcoming GSI/FAIR SIS100 facility, aims to investigate QCD at high baryon densities. The CBM First-level Event Selector (FLES) serves as the central event selection system of the experiment. It functions as a high-performance computer cluster tasked with the online analysis of physics data, including full event reconstruction, at an incoming data rate of up to 1 TB/s. The CBM detector systems operate in a free-running and self-triggered manner, delivering time-stamped data streams. Without inherent event separation, timeslice building replaces global event building. The FLES online data distribution integrates data from around 5000 input links into self-contained, overlapping processing intervals and distributes these to the compute nodes. Using a combination of RDMA and zero-copy techniques, timeslices can be built efficiently over a high-throughput InfiniBand network and distributed to available online computing resources for full online event reconstruction and analysis in a heterogeneous HPC cluster system. A new IPC online interface to timeslice data utilizes Posix shared memory governed by a referencecounting item distributor. This design combines maximum performance and flexibility with minimum memory consumption. These developments have been successfully field-tested in production at the CBM predecessor experiment mCBM at the GSI/FAIR SIS18.