Qualitative Analysis of Security-Related Code Reviews in NPM Packages: An Empirical Study

Security issues are a major concern in software packages, and their impact can be detrimental if exploited. Prior work has shown that code review is a widely-used practice that project maintainers adopt to improve software quality. However, in-depth analyses on code review concerning security issues are limited. This paper explores the role of code review in finding and mitigating security issues. We investigate 10 active and popular npm packages to understand what types of security issues are raised during code review, and what kind of mitigation strategies are employed by package maintainers to address them. We analyze 171 pull-requests with raised security issues. We find that such issues are discussed at length by package maintainers. Moreover, we find that code review is effective at identifying certain types of issues, e.g., Race Condition and ReDOS, as dealing with such concerns requires in-depth knowledge of the project domain. When analysing how maintainers respond to the raised security issues, we find that most of the issues (55%) are addressed and mitigated. Still, some security concerns ended up not being fixed or are ignored by maintainers. Finally, we offer some implications to support the role of reviewing code in finding and fixing security concerns.