ReTrace: Topological evaluation of white matter tractography algorithms using Reeb graphs

Abstract We present ReTrace, a novel graph matching-based topological evaluation and validation method for tractography algorithms. ReTrace uses a Reeb graph whose nodes and edges capture the topology of white matter fiber bundles. We evaluate the performance of 96 algorithms from the ISMRM Tractography Challenge and the the standard algorithms implemented in DSI Studio for the population-averaged Human Connectome Project (HCP) dataset. The existing evaluation metrics such as the f-score, bundle overlap, and bundle overreach fail to account for fiber continuity resulting in high scores even for broken fibers, branching artifacts, and mis-tracked fiber crossing. In contrast, we show that ReTrace effectively penalizes the incorrect tracking of fibers within bundles while concurrently pinpointing positions with significant deviation from the ground truth. Based on our analysis of ISMRM challenge data, we find that no single algorithm consistently outperforms others across all known white matter fiber bundles, highlighting the limitations of the current tractography methods. We also observe that deterministic tractography algorithms perform better in tracking the fundamental properties of fiber bundles, specifically merging and splitting, compared to probabilistic tractography. We compare different algorithmic approaches for a given bundle to highlight the specific characteristics that contribute to successful tracking, thus providing topological insights into the development of advanced tractography algorithms.