Clemson University Technical Report 93-101

Performing Strong Mutation Analysis Using

Mutant Schemata

Roland H. Untch
Department of Computer Science
Clemson University
Clemson, SC 29634-1906
[email protected]

A. Jefferson Offutt
ISSE Department
George Mason University
Fairfax, VA 22030
[email protected]

Mary Jean Harrold
Department of Computer Science
Clemson University
Clemson, SC 29634-1906
[email protected]

Abstract

Mutation analysis is a powerful technique for assessing and improving the quality of test data used in unit testing software. Unfortunately, current automated mutation analysis systems suffer from severe performance problems. This paper presents a new technique for performing mutation analysis that uses program schemata to encode all mutants for a program into one metaprogram, which is subsequently compiled and run at speeds substantially higher than achieved by previous interpretive systems. Preliminary performance improvements of over 300% are reported. This technique has the additional advantages of being easier to implement than interpretive systems, easy porting across a wide range of hardware and software platforms, and using the same compiler and run-time support system that is used during development and/or deployment.

Keywords: Fault-based testing, mutation analysis, program schemata, software testing.

1 Introduction

Programs1 are tested by executing them against test inputs and examining the resulting outputs for errors. The intent of this testing process is to increase our confidence in the correctness of the tested code. However, when testing is poorly conducted, in an ad-hoc manner, our confidence may be misplaced. Poorly selected test data that does not adequately exercise a program must be deemed low quality". Systematic testing techniques establish test data adequacy criteria that seek to measure the quality of the test data used to exercise a given program. One powerful testing technique that uses an adequacy criterion is mutation testing [12, 14]. In mutation testing, the test set is analyzed to determine a quality measure called the mutation adequacy score ; this process is called mutation analysis.

1 e use the wor pro ra to enote the so tware un er test, wh ch a e a co ete ro ra or so e s a er un t, such as a roce ure.