A Novel Method for Comparing Test Suites for Developing Reliable Software Applications

Test suites play an important role in developing reliable software applications. Generally, the behaviour of software applications is verified by executing test suites to find defects. The reliability of software correlates with the quality of the test suite used to test it. Therefore, the quality of a test suite needs to be evaluated and enriched (if needed) especially for testing critical systems, such as plane-navigation system.This paper presents a novel method for comparing concrete and executable test suites using equivalence classes. This comparison identifies gaps in test suites with respect to each other. These gaps indicate potential weaknesses in the test suites. Further, this method provides a mechanism to enrich the test suites using these gaps. In this method, we devise equivalence classes, and associate each test case to an equivalence class. We, then, simulate the comparison of test suites by comparing sets of equivalence classes.The method compares test suites in a platform independent manner. The test suites, which are actually compared, are smaller than the original test suites because the redundant test cases are removed from the test suites, which makes it efficient. Further, it allows us to use any arbitrary criterion for devising equivalence classes which makes this method viable in various situations.We exercise our method over a couple of case studies to demonstrate its viability and effectiveness. The effectiveness of the method is validated using mutation and coverage analyses.