An experimental analysis of fault detection capabilities of covering array

Combinatorial Interaction Testing (CIT) aims at constructing an effective test suite, such as a Covering Array (CA), that can detect faults that are caused by the interaction of parameters. In this paper, we report on some empirical studies conducted to examine the fault detection capabilities of five popular CA constructors: ACTS, Jenny, PICT, CASA, and TCA. The experimental results indicate that Jenny has the best performance, because it achieves better fault detection than the other four constructors in many cases. Our results also indicate that CAs generated using ACTS, PICT, or CASA should be prioritized before testing.