Static Analysis and Penetration Testing from the Perspective of Maintenance Teams

Static analysis and penetration testing are common techniques used to discover security bugs in implementation code. Penetration testing is often performed in black-box way by probing the attack surface of a running system and discovering its security holes. Static analysis techniques operate in a white-box way by analyzing the source code of a system and identifying security weaknesses. Because of their different nature, the two techniques report their findings in two different ways. This paper presents an exploratory study meant to determine whether a vulnerability report generated by a security tool based on static analysis is more or less useful than a report generated by a security tool based on penetration testing. The usefulness is judged from the perspective of the developers that have to devise a vulnerability-fixing patch. The initial results show an advantage when using penetration testing in one of the two cases we investigated.