Test Case Analytics: Mining test case traces to improve risk-driven testing


Written by: T.B. Noor and H. Hemmati. Software Analytics Workshop (SWAN), 2015.


In risk-driven testing, test cases are generated
and/or prioritized based on different risk measures. For example,
the most basic risk measure would analyze the history of the
software and assigns higher risk to the test cases that used to
detect bugs in the past. However, in practice, a test case may not
be exactly the same as a previously failed test, but quite similar. In
this study, we define a new risk measure that assigns a risk factor
to a test case, if it is similar to a failing test case from history.
The similarity is defined based on the execution traces of the test
cases, where we define each test case as a sequence of method
calls. We have evaluated our new risk measure by comparing it
to an old-fashion risk measure (where the risk measure would
be increased only if the very same test case, not a similar one,
failed in the past). The results of the study on two open source
projects show that our new risk measure is by far more effective
in identifying failing test cases compared to the old-fashion risk