Do SDN Configuration Changes Get Reviewed Differently? An Empirical Study at TELUS

Abstract Configuration files are crucial in Software-Defined Networking (SDN) as they define policies required for the dynamic and safe management of large-scale network traffic. Frequent changes to these files can indicate adaptability and responsiveness but may also suggest instability or frequent reconfiguration needs, due to the intricate dependencies between network components and the risk of misconfigurations that can affect network performance and security. This complexity introduced by these configuration changes poses significant challenges, complicating both the development and review processes. Managing and reviewing these changes effectively is essential to ensure that the network remains robust, secure, and optimized, making it crucial to understand and address the difficulties associated with configuration files in SDN context. This paper presents the results of a study conducted in collaboration with our industrial partner, a telecom company providing SDN-based solutions, to investigate the specific challenges related to the review of configuration files compared to traditional development and documentation files. We analyze 8,495 GitLab Merge Requests (MRs) from five configuration-centric projects. Using both quantitative and qualitative methods, we compare configuration MRs with development and documentation MRs, and examine differences between configuration bug-related and non-bug-related MRs. Our findings show that configuration-dominant MRs (more than 50% of the files are configuration) receive significantly less review participation and activity than development MRs, with less consistent quality and more questions raised. However, configuration-inclusive MRs (less than 50% configuration files) show better review quality and alignment with development standards, featuring more suggestions and evaluations. When compared to documentation MRs, both configuration-dominant and configuration-inclusive MRs exhibit higher review activity and engagement, take longer to review, and are more frequently self-managed. Configuration-dominant bug-related MRs demonstrate significantly higher review activity, engagement, and approval, emphasizing technical accuracy and achieving high-quality ratings. Configuration-inclusive bug-related MRs related to bugs show consistent review quality similar to non-bug-related MRs. The main contributions of the paper are: a comprehensive quantitative and qualitative comparison of reviews on configuration and traditional files, and an assessment of the impact of bugs in configuration files on the review process. These contributions give practitioners valuable insights to identify inefficiencies that reduce participation and effectiveness in configuration reviews in SDN context, while highlighting areas for improving review practices.