Talking Tests: an Empirical Assessment of the Role of Fit Acceptance Tests in Clarifying Requirements

The starting point for software evolution is usually a change request, expressing the new or updated requirements on the delivered system. The requirements specified in a change request document are often incomplete and inconsistent with the initial requirement document, as well as the implementation. Programmers working on the evolution of the software are often in trouble interpreting an under-specified change request document, resulting in code that does not meet the users` expectations and contains faults that can only be detected later through expensive testing activities. In this paper, we investigate the role of acceptance tests to clarify the requirements used in software evolution iterations. In particular we focus on Fit tables, a way to express acceptance tests which simplifies their translation into executable test cases. We designed and ran an experiment to assess whether availability of Fit tables affects the level of understanding and the productivity in understanding the requirements. Results indicate that Fit tables significantly improve requirement understanding, but tend to involve additional effort.