Back to Chapters

Chapter 15: Continuous Integration

Continuous Integration (CI) is a development practice where developers integrate code into a shared repository frequently, preferably several times a day. Each integration can then be verified by an automated build and automated tests. In this chapter, we'll explore how to integrate TestNG with popular CI systems to automate your testing process.

Benefits of CI with TestNG

Integrating TestNG with a CI system provides several benefits:

  1. Automated Testing: Tests run automatically on every code change, ensuring immediate feedback.
  2. Consistent Test Environment: Tests run in a controlled, reproducible environment.
  3. Early Bug Detection: Issues are caught early in the development cycle.
  4. Improved Code Quality: Regular testing helps maintain and improve code quality.
  5. Faster Release Cycles: Automation speeds up the testing and release process.
  6. Historical Test Results: CI systems store test results, allowing you to track test performance over time.

Setting Up TestNG with Maven

Before integrating with a CI system, let's set up TestNG with Maven, which is commonly used in CI environments.

Maven POM Configuration

Here's a basic pom.xml configuration for a Java 21 project with TestNG:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>testng-ci-example</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <testng.version>7.8.0</testng.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>${testng.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.11.0</version>
                <configuration>
                    <source>21</source>
                    <target>21</target>
                    <compilerArgs>--enable-preview</compilerArgs>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.1.2</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                    <argLine>--enable-preview</argLine>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

TestNG XML Configuration

Create a testng.xml file in the src/test/resources directory:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="CI Test Suite">
    <test name="Unit Tests">
        <packages>
            <package name="com.example.unit.*"/>
        </packages>
    </test>
    <test name="Integration Tests">
        <packages>
            <package name="com.example.integration.*"/>
        </packages>
    </test>
</suite>

Running Tests with Maven

To run the tests locally with Maven:

mvn clean test

This command will compile the code and run the TestNG tests defined in the testng.xml file.

Integrating with GitHub Actions

GitHub Actions is a CI/CD platform that allows you to automate your build, test, and deployment pipeline directly from your GitHub repository.

Creating a GitHub Actions Workflow

Create a file named .github/workflows/maven.yml in your repository:

name: Java CI with Maven

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Set up JDK 21
      uses: actions/setup-java@v3
      with:
        java-version: '21'
        distribution: 'temurin'
        cache: maven

    - name: Build with Maven
      run: mvn -B clean test

    - name: Publish Test Report
      uses: mikepenz/action-junit-report@v3
      if: always()
      with:
        report_paths: '**/target/surefire-reports/TEST-*.xml'

This workflow will:

  1. Trigger on pushes to the main branch or pull requests targeting the main branch
  2. Set up a Java 21 environment
  3. Build the project and run tests with Maven
  4. Publish the test results as a report

Viewing Test Results

GitHub Actions will display the test results in the workflow run summary. You can also use the JUnit report action to generate a more detailed report.

Integrating with Jenkins

Jenkins is a popular open-source automation server that can be used to implement CI/CD pipelines.

Setting Up a Jenkins Pipeline

Create a Jenkinsfile in the root of your repository:

pipeline {
    agent any

    tools {
        jdk 'jdk21'
        maven 'maven3'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Build') {
            steps {
                sh 'mvn clean compile'
            }
        }

        stage('Test') {
            steps {
                sh 'mvn test'
            }
            post {
                always {
                    junit '**/target/surefire-reports/TEST-*.xml'
                }
            }
        }
    }
}

This pipeline will:

  1. Check out the code from your repository
  2. Compile the code
  3. Run the tests
  4. Publish the test results

Configuring Jenkins

  1. Install the necessary plugins:
    • JDK Tool Plugin
    • Maven Integration Plugin
    • Pipeline Plugin
    • JUnit Plugin
  2. Configure JDK and Maven in Jenkins:
    • Go to Manage Jenkins > Global Tool Configuration
    • Add JDK installation with name 'jdk21'
    • Add Maven installation with name 'maven3'
  3. Create a new Pipeline job:
    • Select "Pipeline" as the job type
    • Configure the pipeline to use your repository and Jenkinsfile

Viewing Test Results in Jenkins

Jenkins will display the test results in the job's build page. You can see:

  • Overall test status
  • Number of tests passed, failed, and skipped
  • Test trend over time
  • Detailed test reports

Integrating with GitLab CI/CD

GitLab CI/CD is a built-in CI/CD solution for GitLab repositories.

Creating a GitLab CI/CD Pipeline

Create a file named .gitlab-ci.yml in the root of your repository:

image: maven:3.9-eclipse-temurin-21

variables:
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"

cache:
  paths:
    - .m2/repository

stages:
  - build
  - test

build:
  stage: build
  script:
    - mvn compile

test:
  stage: test
  script:
    - mvn test
  artifacts:
    reports:
      junit:
        - target/surefire-reports/TEST-*.xml
    paths:
      - target/surefire-reports
    expire_in: 1 week

This pipeline will:

  1. Use a Maven image with Java 21
  2. Cache Maven dependencies to speed up builds
  3. Compile the code in the build stage
  4. Run tests in the test stage
  5. Publish test results as JUnit reports

Viewing Test Results in GitLab

GitLab will display the test results in the pipeline's job page. You can see:

  • Overall test status
  • Number of tests passed, failed, and skipped
  • Detailed test reports

Advanced CI Configurations

Let's explore some advanced CI configurations for TestNG.

Parallel Test Execution

To run tests in parallel in a CI environment, update your testng.xml:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Test Suite" parallel="methods" thread-count="4">
    <test name="Parallel Tests">
        <packages>
            <package name="com.example.*"/>
        </packages>
    </test>
</suite>

And update your Maven Surefire configuration:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.1.2</version>
    <configuration>
        <suiteXmlFiles>
            <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
        </suiteXmlFiles>
        <argLine>--enable-preview</argLine>
        <properties>
            <property>
                <n>surefire.testng.verbose</n>
                <value>2</value>
            </property>
        </properties>
    </configuration>
</plugin>

Test Groups and Profiles

You can define different test groups and Maven profiles to run specific tests in different CI stages:

Update your testng.xml:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="CI Test Suite">
    <test name="Unit Tests">
        <groups>
            <run>
                <include name="unit"/>
            </run>
        </groups>
        <packages>
            <package name="com.example.*"/>
        </packages>
    </test>
</suite>

Create another file named integration-testng.xml:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="CI Test Suite">
    <test name="Integration Tests">
        <groups>
            <run>
                <include name="integration"/>
            </run>
        </groups>
        <packages>
            <package name="com.example.*"/>
        </packages>
    </test>
</suite>

Update your pom.xml with profiles:

<profiles>
    <profile>
        <id>unit-tests</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <suiteXmlFiles>
                            <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
                        </suiteXmlFiles>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
    <profile>
        <id>integration-tests</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <suiteXmlFiles>
                            <suiteXmlFile>src/test/resources/integration-testng.xml</suiteXmlFile>
                        </suiteXmlFiles>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Then in your CI configuration, you can run different profiles:

# GitHub Actions example
- name: Run Unit Tests
  run: mvn -B test -P unit-tests

- name: Run Integration Tests
  run: mvn -B test -P integration-tests

Test Coverage Reports

You can add test coverage reporting to your CI pipeline using JaCoCo:

Update your pom.xml:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.10</version>
    <executions>
        <execution>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>report</id>
            <phase>test</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Update your CI configuration to publish the coverage report:

# GitHub Actions example
- name: Upload Coverage Report
  uses: actions/upload-artifact@v3
  with:
    name: coverage-report
    path: target/site/jacoco/