The Distance-Weighted ROC for Performance Evaluation of Damage Localization

The transition of SHM and NDE from research prototypes to industrial applications remains hindered by the lack of appropriate reliability metrics that account for the spatial nature of damage identification. While the Receiver Operating Characteristic (ROC) has been widely adopted for performance assessment, its application to damage localization poses practical and fundamental challenges that emerge from the underlying binary classification paradigm. This paper introduces the Distance-Weighted ROC (DW-ROC) to address these challenges, while retaining the familiar interpretability as a compromise between sensitivity and specificity, which were crucial to the popularity of the ROC. Starting from concrete application cases and progressing in increasing levels of abstraction, this paper demonstrates the following limitations of the standard ROC. It is too inflexible in the off-grid case where damage is not located at predefined imaging points. It lacks nuance towards localization error, and is unable to rank methods according to their localization accuracy. Finally, it treats all false positives and negatives as equally problematic regardless of their spatial proximity to actual damage. These limitations lead to inappropriate oversimplifications and produce misleading performance comparisons, thereby hindering development and validation of SHM systems. The proposed DW-ROC method extends the ROC by introducing fuzzy membership functions that weight the scoring of true and false positives based on spatial proximity to actual damage. The approach smoothly penalizes localization errors while preserving the interpretability and conventions of standard ROC analysis. The DW-ROC requires only a single, physically meaningful parameter that sets the distance at which a predicted positive transitions from mostly true-positive to mostly false-positive, making it adaptable across different use-cases and methods. Numerical verification demonstrates that the DW-ROC retains the ability to rank damage maps according to background noise, and offers in addition the ability to rank according to localization accuracy. Moreover, it handles the off-grid case without requiring ad-hoc adjustments or arbitrary margins of localization error. Finally, it remains consistent with established conventions by providing an Area Under the Curve (AUC) metric where 0.5 corresponds to random guessing, and 1.0 to perfect performance. Validation was conducted after inspecting two representative structures with a vibration-based damage imaging technique, using accelerometer measurements in the modal frequency range, and a data-driven model of their baseline dynamics.The DW-ROC produced more representative performance evaluations than the ROC. It successfully distinguished between damage maps that confused the standard ROC, and matched expert assessment. This provided a powerful tool to compare methods and parameters, and enabled automated, synthetic overviews of performance over large datasets. This paper contributes to industrial acceptance of SHM systems by introducing a strategy for performance assessment that accounts for the peculiarities of the damage localization task. By acknowledging the spatial nature of localization performance while retaining familiar ROC interpretability, the DW-ROC enables more faithful modality comparison studies of damage imaging, across different structures, applications, and SHM methods. The insights and results presented in this paper constitute a practical step toward quality assessment strategies that support the transition of SHM and NDE from research demonstrations to routine industrial applications.