gradle-pitest-plugin

Gradle plugin for PIT Mutation Testing


Project maintained by szpak Hosted on GitHub Pages — Theme by mattgraham

Gradle plugin for PIT Mutation Testing

The plugin provides an ability to perform a mutation testing and calculate a mutation coverage of a Gradle-based projects with PIT.

Quick start

Add gradle-pitest-plugin and pitest itself to the buildscript dependencies in your build.gradle file:

buildscript {
    repositories {
        mavenCentral()
        mavenLocal()
        //Needed only for SNAPSHOT versions
        //maven { url "http://oss.sonatype.org/content/repositories/snapshots/" }
    }
    dependencies {
        classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:1.0.0'
    }
}

Apply plugin:

apply plugin: "pitest"

Call Gradle with pitest task:

gradle pitest

After the measurements a report created by PIT will be placed in ${PROJECT_DIR}/build/reports/pitest directory.

Plugin configuration

The Pitest plugin has to be configured. All the command line options are supported. To make life easier taskClasspath, mutableCodePaths, sourceDirs, reportDir and pitestVersion are automatically set by a plugin. In addition sourceDirs, reportDir and pitestVersion can be overridden by an user.

In the past there was one mandatory parameter - targetClasses - which points to the classes which should be mutated. Starting from 0.32.0 it is only required if a group for the project is not set. Otherwise value "${project.group}.*" is set by default (which can be overridden using pitest.targetClasses parameter).

In case of using not default PIT version the pitestVersion parameter should be used to override it.

The configuration in Gradle is the real Groovy code which makes all assignments very intuitive. All values expected by PIT should be passed as a corresponding types. There is only one important difference. For the parameters where PIT expects a coma separated list of strings in a Gradle configuration a list of strings should be used (see outputFormats in the following example).

pitest {
    targetClasses = ['our.base.package.*']  //by default "${project.group}.*"
    pitestVersion = "1.0.0" //not needed when a default PIT version should be used
    threads = 4
    outputFormats = ['XML', 'HTML']
}

Check PIT documentation for a list of all available command line parameters. The expected parameter format in a plugin configuration can be taken from PitestPluginExtension.

There are a few parameters specific for Gradle plugin:

For example:

pitest {
    ...
    enableDefaultIncrementalAnalysis = true
    testSourceSets = [sourceSets.test, sourceSets.integrationTest]
    mainSourceSets = [sourceSets.main, sourceSets.additionalMain]
    jvmArgs = ['-Xmx1024m']
}

Multi-module projects support

gradle-pitest-plugin can be used in multi-module projects. The plugin has to be applied in all subprojects which should be processed with PIT. A sample snippet from build.gradle located for the root project:

subprojects {
    ...
    apply plugin: 'pitest'

    pitest {
        threads = 4

        if (project.name in ['module-without-any-test']) {
            failWhenNoMutations = false
        }
    }
}

Currently PIT does not provide an aggregated report for multi-module project. A report for each module has to be browsed separately. Alternatively a PIT plugin for Sonar can be used to get aggregated results.

Versions

Every gradle-pitest-plugin version by default uses a predefined PIT version. Usually this a the latest released version of PIT available at the time of releasing a plugin version. It can be overridden by using pitestVersion parameter in a pitest configuration closure.

Note. There could be some issues when using non default PIT versions.

gradle-pitest-plugin 1.0.x uses PIT 1.0.x, 0.32.x uses PIT 0.32, 0.30.x uses PIT 0.30, etc.

Note. PIT 0.27 is not supported due to issue 47. Note. Due to internal refactoring in PIT versions >=0.32 require gradle-pitest-plugin >=0.32.x and PIT versions <=0.31 gradle-pitest-plugin <=0.30.x.

gradle-pitest-plugin 1.0.0 requires Gradle 1.6+ and was tested with Gradle 1.6 to 1.12 under OpenJDK 8, OpenJDK 7 and Sun 1.6.

See changelog file for more detailed list of changes in the plugin itself.

FAQ

  1. Why have I got java.lang.VerifyError: Expecting a stackmap frame... when using Java 7?

    It should be fixed in PIT 0.29. As a workaround in older versions add jvmArgs = '-XX:-UseSplitVerifier' to a pitest configuration block

    pitest {
        ...
        //jvmArgs = '-XX:-UseSplitVerifier'     //<0.33.0
        jvmArgs = ['-XX:-UseSplitVerifier']     //>=0.33.0
    }
    
  2. Why have I got GroovyCastException: Cannot cast object '-Xmx1024', '-Xms512m' with class 'java.lang.String' to class 'java.util.List' after upgrade to version 0.33.0?

    To keep consistency with the new mainProcessJvmArgs configuration parameter and make an input format more predicable jvmArgs parameter type was changed from String to List<String> in gradle-pitest-plugin 0.33.0. The migration is trivial, but unfortunately I am not aware of the way to keep both parameter types active at the same time.

    pitest {
        ...
        //jvmArgs = '-Xmx1024 -Xms512m'     //old format
        jvmArgs = ['-Xmx1024', '-Xms512m']  //new format
    
    }
    
  3. Why my Spring Boot application doesn't work correctly with gradle-pitest-plugin 0.33.0 applied?

There is an issue with the way how spring-boot-gradle-plugin handles JavaExec tasks (including pitest task which in the version 0.33.0 became JavaExec task to resolve classpath issue with configured non default PIT version - see issue #7).

Luckily there is a workaround which allows to run PIT 0.33 (with Java 8 support) with gradle-pitest-plugin 0.32.0:

buildscript {
    (...)
    dependencies {
        classpath("info.solidsoft.gradle.pitest:gradle-pitest-plugin:0.32.0") {
          exclude group: "org.pitest"
        }
        classpath "org.pitest:pitest-command-line:0.33"
    }
}

pitest {
    pitestVersion = "0.33"
}

Known issues

Development

gradle-pitest-plugin cloned from the repository can be built using Gradle command:

./gradlew build

The easiest way to make a JAR with local changes visible in another project is to install it into the local Maven repository

./gradlew install

Support

gradle-pitest-plugin was written by Marcin ZajÄ…czkowski. The author can be contacted directly via email: mszpak ATT wp DOTT pl. There is also Marcin's blog available: Solid Soft - working code is not enough.

The plugin surely has some bugs and missing features. They can be reported using an issue tracker. However it is often a better idea to send a questions to the PIT mailing list first.

The plugin is licensed under the terms of the Apache License, Version 2.0.