An Empirical Validation of a Web Fault Taxonomy and its Usage for Web Testing

Web testing is assuming an increasingly important role in Web engineering, as a result of the quality demands put onto modern Web-based systems and of the complexity of the involved technologies. Most of the existing works in Web testing are focused on the definition of novel testing techniques, while only limited effort was devoted to understanding the specific nature of Web faults. However, the performance of a new Web testing technique is strictly dependent on the classes of Web faults it addresses. In this paper, we describe the process followed in the construction of a Web fault taxonomy. We used an iterative, mixed top-down and bottom-up approach. An initial taxonomy was defined by analyzing the high level characteristics of Web applications. Then the taxonomy was subjected to several iterations of empirical validation. During each iteration the taxonomy was refined by analyzing real faults and mapping them onto the appropriate categories. Metrics collected during this process were used to ensure that in the final taxonomy bugs distribute quite evenly among fault categories; fault categories are not-too-big, not-too-small and not ambiguous. Testers can use our taxonomy to define test cases that target specific classes of Web faults, while researchers can use it to build fault seeding tools, to inject artificial Web faults into benchmark applications. The final taxonomy is publicly available for consultation: since it is organized as a Wiki page, it is also open to external contributions. We conducted a case study in which test cases have been derived from the taxonomy for a sample Web application. The case study indicates that the proposed taxonomy is very effective in directing the testing effort toward those test scenarios that have higher chances of revealing Web specific faults.