A Novel Method for Comparing Test Suites for Developing Reliable Software Applications

Masoud Mohammadian, Zafer Javed

Research output: A Conference proceeding or a Chapter in BookConference contribution

Abstract

Test suites play an important role in developing reliable software applications. Generally, the behaviour of software applications is verified by executing test suites to find defects. The reliability of software correlates with the quality of the test suite used to test it. Therefore, the quality of a test suite needs to be evaluated and enriched (if needed) especially for testing critical systems, such as plane-navigation system.This paper presents a novel method for comparing concrete and executable test suites using equivalence classes. This comparison identifies gaps in test suites with respect to each other. These gaps indicate potential weaknesses in the test suites. Further, this method provides a mechanism to enrich the test suites using these gaps. In this method, we devise equivalence classes, and associate each test case to an equivalence class. We, then, simulate the comparison of test suites by comparing sets of equivalence classes.The method compares test suites in a platform independent manner. The test suites, which are actually compared, are smaller than the original test suites because the redundant test cases are removed from the test suites, which makes it efficient. Further, it allows us to use any arbitrary criterion for devising equivalence classes which makes this method viable in various situations.We exercise our method over a couple of case studies to demonstrate its viability and effectiveness. The effectiveness of the method is validated using mutation and coverage analyses.

Original languageEnglish
Title of host publication2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT)
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISBN (Electronic)9781538662243
ISBN (Print)9781538662250
DOIs
Publication statusPublished - 31 Oct 2018
Event5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018 - Amman, Jordan
Duration: 31 Oct 20181 Nov 2018
https://www.philadelphia.edu.jo/it/isiict2018/index.php (Website)

Publication series

Name5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018

Conference

Conference5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018
CountryJordan
CityAmman
Period31/10/181/11/18
Internet address

Fingerprint

Equivalence classes
Application programs
Software
Equivalence class
Navigation systems
Concretes
Defects
Testing
Navigation System

Cite this

Mohammadian, M., & Javed, Z. (2018). A Novel Method for Comparing Test Suites for Developing Reliable Software Applications. In 2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT) [8613292] (5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ISIICT.2018.8613292
Mohammadian, Masoud ; Javed, Zafer. / A Novel Method for Comparing Test Suites for Developing Reliable Software Applications. 2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT). IEEE, Institute of Electrical and Electronics Engineers, 2018. (5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018).
@inproceedings{e4f0838900f747cda49be223460a42a5,
title = "A Novel Method for Comparing Test Suites for Developing Reliable Software Applications",
abstract = "Test suites play an important role in developing reliable software applications. Generally, the behaviour of software applications is verified by executing test suites to find defects. The reliability of software correlates with the quality of the test suite used to test it. Therefore, the quality of a test suite needs to be evaluated and enriched (if needed) especially for testing critical systems, such as plane-navigation system.This paper presents a novel method for comparing concrete and executable test suites using equivalence classes. This comparison identifies gaps in test suites with respect to each other. These gaps indicate potential weaknesses in the test suites. Further, this method provides a mechanism to enrich the test suites using these gaps. In this method, we devise equivalence classes, and associate each test case to an equivalence class. We, then, simulate the comparison of test suites by comparing sets of equivalence classes.The method compares test suites in a platform independent manner. The test suites, which are actually compared, are smaller than the original test suites because the redundant test cases are removed from the test suites, which makes it efficient. Further, it allows us to use any arbitrary criterion for devising equivalence classes which makes this method viable in various situations.We exercise our method over a couple of case studies to demonstrate its viability and effectiveness. The effectiveness of the method is validated using mutation and coverage analyses.",
keywords = "Extending Test Adequacy, Quality of Testing, Test Adequacy, Test Suite Comparison, Test Suite Enrichment",
author = "Masoud Mohammadian and Zafer Javed",
year = "2018",
month = "10",
day = "31",
doi = "10.1109/ISIICT.2018.8613292",
language = "English",
isbn = "9781538662250",
series = "5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
booktitle = "2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT)",
address = "United States",

}

Mohammadian, M & Javed, Z 2018, A Novel Method for Comparing Test Suites for Developing Reliable Software Applications. in 2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT)., 8613292, 5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018, IEEE, Institute of Electrical and Electronics Engineers, 5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018, Amman, Jordan, 31/10/18. https://doi.org/10.1109/ISIICT.2018.8613292

A Novel Method for Comparing Test Suites for Developing Reliable Software Applications. / Mohammadian, Masoud; Javed, Zafer.

2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT). IEEE, Institute of Electrical and Electronics Engineers, 2018. 8613292 (5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018).

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - A Novel Method for Comparing Test Suites for Developing Reliable Software Applications

AU - Mohammadian, Masoud

AU - Javed, Zafer

PY - 2018/10/31

Y1 - 2018/10/31

N2 - Test suites play an important role in developing reliable software applications. Generally, the behaviour of software applications is verified by executing test suites to find defects. The reliability of software correlates with the quality of the test suite used to test it. Therefore, the quality of a test suite needs to be evaluated and enriched (if needed) especially for testing critical systems, such as plane-navigation system.This paper presents a novel method for comparing concrete and executable test suites using equivalence classes. This comparison identifies gaps in test suites with respect to each other. These gaps indicate potential weaknesses in the test suites. Further, this method provides a mechanism to enrich the test suites using these gaps. In this method, we devise equivalence classes, and associate each test case to an equivalence class. We, then, simulate the comparison of test suites by comparing sets of equivalence classes.The method compares test suites in a platform independent manner. The test suites, which are actually compared, are smaller than the original test suites because the redundant test cases are removed from the test suites, which makes it efficient. Further, it allows us to use any arbitrary criterion for devising equivalence classes which makes this method viable in various situations.We exercise our method over a couple of case studies to demonstrate its viability and effectiveness. The effectiveness of the method is validated using mutation and coverage analyses.

AB - Test suites play an important role in developing reliable software applications. Generally, the behaviour of software applications is verified by executing test suites to find defects. The reliability of software correlates with the quality of the test suite used to test it. Therefore, the quality of a test suite needs to be evaluated and enriched (if needed) especially for testing critical systems, such as plane-navigation system.This paper presents a novel method for comparing concrete and executable test suites using equivalence classes. This comparison identifies gaps in test suites with respect to each other. These gaps indicate potential weaknesses in the test suites. Further, this method provides a mechanism to enrich the test suites using these gaps. In this method, we devise equivalence classes, and associate each test case to an equivalence class. We, then, simulate the comparison of test suites by comparing sets of equivalence classes.The method compares test suites in a platform independent manner. The test suites, which are actually compared, are smaller than the original test suites because the redundant test cases are removed from the test suites, which makes it efficient. Further, it allows us to use any arbitrary criterion for devising equivalence classes which makes this method viable in various situations.We exercise our method over a couple of case studies to demonstrate its viability and effectiveness. The effectiveness of the method is validated using mutation and coverage analyses.

KW - Extending Test Adequacy

KW - Quality of Testing

KW - Test Adequacy

KW - Test Suite Comparison

KW - Test Suite Enrichment

UR - http://www.scopus.com/inward/record.url?scp=85062190929&partnerID=8YFLogxK

U2 - 10.1109/ISIICT.2018.8613292

DO - 10.1109/ISIICT.2018.8613292

M3 - Conference contribution

SN - 9781538662250

T3 - 5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018

BT - 2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT)

PB - IEEE, Institute of Electrical and Electronics Engineers

ER -

Mohammadian M, Javed Z. A Novel Method for Comparing Test Suites for Developing Reliable Software Applications. In 2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT). IEEE, Institute of Electrical and Electronics Engineers. 2018. 8613292. (5th International Symposium on Innovation in Information and Communication Technology, ISIICT 2018). https://doi.org/10.1109/ISIICT.2018.8613292