| Peer-Reviewed

An Empirical Study on the Effectiveness of Automated Test Case Generation Techniques

Received: 18 November 2014     Accepted: 3 December 2014     Published: 23 December 2014
Views:       Downloads:
Abstract

The advent of automated test case generation has helped to reduce the laborious task of generating test cases manually and is prominent in the software testing field of research and as a result, several techniques have been developed to aid the generation of test cases automatically. However, some major currently used automated test case generation techniques have not been empirically evaluated to ascertain their performances as many assumptions on technique performances are based on theoretical deductions. In this paper, we perform experiment on two major automated test case generation techniques (Concolic test case generation technique and the Combinatorial test case generation technique) and evaluate based on selected metrics (number of test cases generated, complexities of the selected programs, the percentage of test coverage and performance score). The results from the experiment show that the Combinatorial technique performed better than the Concolic technique. Hence, the Combinatorial test case generation technique was found to be more effective than the Concolic test case generation technique based on the selected metrics.

Published in American Journal of Software Engineering and Applications (Volume 3, Issue 6)
DOI 10.11648/j.ajsea.20140306.15
Page(s) 95-101
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2014. Published by Science Publishing Group

Keywords

Automated Test Case Generation Technique, Combinatorial, Concolic, Empirical Study, Software Testing, Software Metrics

References
[1] S. Anand, E. Burke, T. Y. Chen, J. Clark, M. B. Cohen, W. Grieskamp, M. Harman, M. J. Harrold, and P. McMinn, “An orchestrated survey on automated Software test case generation,” Antonia Bertolino, J. Jenny Li and Hong Zhu, Editor/Orchestrators, Journal of Systems and Software 2013.
[2] J. E. Bentley, “Software testing fundamentals-concepts, roles, and terminology,” Corporate Data Management and Governance, Wachovia Bank, 201 S. College Street, NC-1025, Charlotte NC 28210, 2001.
[3] J. Czerwonka, “Pairwise testing in real World: practical extensions to test case generators,” Microsoft Corporation, One Microsoft way Redmond, WA 98052, 2006.
[4] M. d'Amorim, C. Pacheco, T. Xie, D. Marinov, and M. D. Ernst, “An empirical comparison of automated generation and classification techniques for object-oriented unit testing,” Department of Computer Science, University of Illinois, Urbana-Champaign, IL, U.S.A., 2006.
[5] G. Fraser, M. Staats, P. McMinn, A. Arcuri, and P. Padberg, “Does automated White-Box test generation really help Software Testers,?” Department of Computer Science, University of Sheffield, United Kingdom, 2013.
[6] S. Han and Y. Kwon, “An empirical evaluation of test data generation techniques.” Journal of Computing Science and Engineering, vol. 2, No. 3, September, 2008.
[7] K. Kahkonen, R. Kindermann, K. Heljanko and I. Niemela, “Experimental comparison of Concolic and Random Testing for Java Card Applets,” Department of Information and Computer Science Aalto University, P.O. Box 15400, FI-00076 AALTO, Finland, 2010.
[8] B. Korel, “Automated Software test data generation,” IEEE Transactions on Software Engineering, Vol. 16, No. 8, 1990.
[9] D. R. Kuhn, R. N. Kacker, and Y. Lei, “Practical Combinatorial Testing,”. National Institute of Standards and Technology (NIST), U.S. Government Printing Office, Washington, U.S.A., 2010.
[10] K. Lakhotia, P. McMinn, and M. Harman, “Automated test data generation for coverage: haven’t we solved this problem yet?,” King’s College, CREST centre, London,WC2R 2LS, U.K., 2009.
[11] L. Luo, “Software Testing Techniques,” Institute for Software Research International, Carnegie Mellon University, Pittsburgh, PA15232, U.S.A., 2001.
[12] T. J. McCabe, “A complexity measure,” IEEE Transactions on Software Engineering, Vol. Se-2, No., 4, 1976.
[13] J. Pan, “Software Testing, Dependable Embedded Systems,” Electrical and Computer Engineering Department, Carnegie Mellon University, 1999.
[14] X. Qu, and B. Robinson, “A case study of Concolic Testing tools and their limitations,” ABB Corporate Research 940 main campus drive, Raleigh, NC, U.S.A., 2010.
[15] M. Roper, J. Miller, A. Brooks, and M. Wood, “Towards the experimental evaluation of Software testing techniques,” EuroSTAR ’94, pp 44/1-44/10October 10-13, 1994, Brussels.
[16] K. Sen, “Concolic testing and constraint satisfaction,” Proceedings, 14th International Conference on Theory and Applications of Satisfiability Testing (SAT’11), 2011.
[17] S. Wang, and J. Offutt, “Comparison of unit-level automated test generation tools,” Software Engineering, George Mason University, Fairfax, VA 22030, USA, 2008.
Cite This Article
  • APA Style

    Bolanle F. Oladejo, Dimple T. Ogunbiyi. (2014). An Empirical Study on the Effectiveness of Automated Test Case Generation Techniques. American Journal of Software Engineering and Applications, 3(6), 95-101. https://doi.org/10.11648/j.ajsea.20140306.15

    Copy | Download

    ACS Style

    Bolanle F. Oladejo; Dimple T. Ogunbiyi. An Empirical Study on the Effectiveness of Automated Test Case Generation Techniques. Am. J. Softw. Eng. Appl. 2014, 3(6), 95-101. doi: 10.11648/j.ajsea.20140306.15

    Copy | Download

    AMA Style

    Bolanle F. Oladejo, Dimple T. Ogunbiyi. An Empirical Study on the Effectiveness of Automated Test Case Generation Techniques. Am J Softw Eng Appl. 2014;3(6):95-101. doi: 10.11648/j.ajsea.20140306.15

    Copy | Download

  • @article{10.11648/j.ajsea.20140306.15,
      author = {Bolanle F. Oladejo and Dimple T. Ogunbiyi},
      title = {An Empirical Study on the Effectiveness of Automated Test Case Generation Techniques},
      journal = {American Journal of Software Engineering and Applications},
      volume = {3},
      number = {6},
      pages = {95-101},
      doi = {10.11648/j.ajsea.20140306.15},
      url = {https://doi.org/10.11648/j.ajsea.20140306.15},
      eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajsea.20140306.15},
      abstract = {The advent of automated test case generation has helped to reduce the laborious task of generating test cases manually and is prominent in the software testing field of research and as a result, several techniques have been developed to aid the generation of test cases automatically. However, some major currently used automated test case generation techniques have not been empirically evaluated to ascertain their performances as many assumptions on technique performances are based on theoretical deductions. In this paper, we perform experiment on two major automated test case generation techniques (Concolic test case generation technique and the Combinatorial test case generation technique) and evaluate based on selected metrics (number of test cases generated, complexities of the selected programs, the percentage of test coverage and performance score). The results from the experiment show that the Combinatorial technique performed better than the Concolic technique. Hence, the Combinatorial test case generation technique was found to be more effective than the Concolic test case generation technique based on the selected metrics.},
     year = {2014}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - An Empirical Study on the Effectiveness of Automated Test Case Generation Techniques
    AU  - Bolanle F. Oladejo
    AU  - Dimple T. Ogunbiyi
    Y1  - 2014/12/23
    PY  - 2014
    N1  - https://doi.org/10.11648/j.ajsea.20140306.15
    DO  - 10.11648/j.ajsea.20140306.15
    T2  - American Journal of Software Engineering and Applications
    JF  - American Journal of Software Engineering and Applications
    JO  - American Journal of Software Engineering and Applications
    SP  - 95
    EP  - 101
    PB  - Science Publishing Group
    SN  - 2327-249X
    UR  - https://doi.org/10.11648/j.ajsea.20140306.15
    AB  - The advent of automated test case generation has helped to reduce the laborious task of generating test cases manually and is prominent in the software testing field of research and as a result, several techniques have been developed to aid the generation of test cases automatically. However, some major currently used automated test case generation techniques have not been empirically evaluated to ascertain their performances as many assumptions on technique performances are based on theoretical deductions. In this paper, we perform experiment on two major automated test case generation techniques (Concolic test case generation technique and the Combinatorial test case generation technique) and evaluate based on selected metrics (number of test cases generated, complexities of the selected programs, the percentage of test coverage and performance score). The results from the experiment show that the Combinatorial technique performed better than the Concolic technique. Hence, the Combinatorial test case generation technique was found to be more effective than the Concolic test case generation technique based on the selected metrics.
    VL  - 3
    IS  - 6
    ER  - 

    Copy | Download

Author Information
  • Department of Computer Science, University of Ibadan, Ibadan, Nigeria

  • Department of Computer Science, University of Ibadan, Ibadan, Nigeria

  • Sections