LEG-DINO: Local-Evidence-Governed DINO Fusion for Reliable Dense Prediction in Medical Image Segmentation

Dense visual prediction requires representations sensitive to local evidence and consistent with global semantic structure. Convolutional encoder-decoder networks preserve boundaries, texture transitions, and small foreground regions, but their local inductive bias can limit long-range semantic reasoning. In contrast, self-supervised models such as DINOv2 provide object-level context, but their spatially coarse token features may suppress weak boundaries or reliable foreground cues when injected directly into a prediction decoder. This paper addresses the reliability problem in local-global feature fusion by proposing LEG-DINO, a Local-Evidence-Governed DINO fusion framework for medical image segmentation. The idea treats the frozen DINOv2 branch as an external semantic prior whose contribution must be verified locally before it influences pixel-level prediction. LEG-DINO maintains a convolutional encoder-decoder as the primary evidence pathway and introduces DINO features only through scale-matched governed fusion blocks. At each scale, CNN features query DINO tokens through asymmetric cross-attention, local-global disagreement is estimated, and a spatial evidence gate regulates bounded residual DINO injection. This design allows semantic context to support ambiguous regions while preventing coarse foundation-model features from dominating local boundary evidence. We evaluated LEG-DINO on five test settings across retinal, dermoscopic, and polyp segmentation: FIVES, ISIC2016, ISIC2018, Kvasir, and external ISIC2018$\rightarrow$PH2 transfer. Against MSDUNet, MDPUNet, nnWUNet, CFCM-Net, and YoloSegNet, LEG-DINO achieves the strongest five-test macro Dice, IoU, and MCC scores of 0.9215, 0.8547, and 0.8981. Per-case, size-stratified, boundary, surface-distance, and controlled ablations show that reliability-governed DINO fusion improves the balance between semantic consistency and local evidence preservation.