Test Reporter

This Github Action displays test results from popular testing frameworks directly in GitHub.

✔️ Parses test results in XML or JSON format and creates nice report as GitHub Check Run or GitHub Actions job summaries

✔️ Annotates code where it failed based on message and stack trace captured during test execution

✔️ Provides final conclusion and counts of passed, failed and skipped tests as output parameters

How it looks: ||||| |:--:|:--:|:--:|:--:|

Supported languages / frameworks: - .NET / dotnet test ( xUnit / NUnit / MSTest ) - Dart / test - Flutter / test - Go / go test - Java / JUnit - JavaScript / JEST / Mocha - Python / pytest / unittest - PHP / PHPUnit / Nette Tester - Ruby / RSpec - Swift / xUnit

For more information see Supported formats section.

Do you miss support for your favorite language or framework? Please create Issue or contribute with PR.

Example

Following setup does not work in workflows triggered by pull request from forked repository. If that's fine for you, using this action is as simple as:

on:
  pull_request:
  push:
permissions:
  contents: read
  actions: read
  checks: write
jobs:
  build-test:
    name: Build & Test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4     # checkout the repo
      - run: npm ci                   # install packages
      - run: npm test                 # run tests (configured to use jest-junit reporter)

      - name: Test Report
        uses: dorny/test-reporter@v3
        if: ${{ !cancelled() }}       # run this step even if previous step failed
        with:
          name: JEST Tests            # Name of the check run which will be created
          path: reports/jest-*.xml    # Path to test results
          reporter: jest-junit        # Format of test results

Recommended setup for public repositories

Workflows triggered by pull requests from forked repositories are executed with read-only token and therefore can't create check runs. To workaround this security restriction, it's required to use two separate workflows: 1. CI runs in the context of the PR head branch with the read-only token. It executes the tests and uploads test results as a build artifact 2. Test Report runs in the context of the repository main branch with read/write token. It will download test results and create reports

The second workflow will only run after it has been merged into your default branch (typically main or master), it won't run in a PR unless after the workflow file is part of that branch.

PR head branch: .github/workflows/ci.yml

name: 'CI'
on:
  pull_request:
jobs:
  build-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4         # checkout the repo
      - run: npm ci                       # install packages
      - run: npm test                     # run tests (configured to use jest-junit reporter)
      - uses: actions/upload-artifact@v4  # upload test results
        if: ${{ !cancelled() }}           # run this step even if previous step failed
        with:
          name: test-results
          path: jest-junit.xml

default branch: .github/workflows/test-report.yml

name: 'Test Report'
on:
  workflow_run:
    workflows: ['CI']                     # runs after CI workflow
    types:
      - completed
permissions:
  contents: read
  actions: read
  checks: write
jobs:
  report:
    runs-on: ubuntu-latest
    steps:
    - uses: dorny/test-reporter@v3
      with:
        artifact: test-results            # artifact name
        name: JEST Tests                  # Name of the check run which will be created
        path: '*.xml'                     # Path to test results (inside artifact .zip)
        reporter: jest-junit              # Format of test results

Usage

- uses: dorny/test-reporter@v3
  with:

    # Name or regex of artifact containing test results
    # Regular expression must be enclosed in '/'.
    # Values from captured groups will replace occurrences of $N in report name.
    # Example:
    #   artifact: /test-results-(.*)/
    #   name: 'Test report $1'
    #   -> Artifact 'test-result-ubuntu' would create report 'Test report ubuntu'
    artifact: ''

    # Name of the Check Run which will be created
    name: ''

    # Comma-separated list of paths to test results
    # Supports wildcards via [fast-glob](https://github.com/mrmlnc/fast-glob)
    # All matched result files must be of the same format
    path: ''

    # The fast-glob library that is internally used interprets backslashes as escape characters.
    # If enabled, all backslashes in provided path will be replaced by forward slashes and act as directory separators.
    # It might be useful when path input variable is composed dynamically from existing directory paths on Windows.
    path-replace-backslashes: 'false'

    # Format of test results. Supported options:
    #   dart-json
    #   dotnet-nunit
    #   dotnet-trx
    #   flutter-json
    #   golang-json
    #   java-junit
    #   jest-junit
    #   mocha-json
    #   phpunit-junit
    #   python-xunit
    #   rspec-json
    #   swift-xunit
    reporter: ''

    # Allows you to generate only the summary.
    # If enabled, the report will contain a table listing each test results file and the number of passed, failed, and skipped tests.
    # Detailed listing of test suites and test cases will be skipped.
    only-summary: 'false'

    # Allows you to generate reports for Actions Summary
    # https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/
    use-actions-summary: 'true'

    # Optionally specify a title (Heading level 1) for the report. Leading and trailing whitespace are ignored.
    # This is useful for separating your test report from other sections in the build summary.
    # If omitted or set to whitespace/empty, no title will be printed.
    report-title: ''

    # Customize the title of badges shown for each Actions Summary.
    # Useful when distinguish summaries for tests ran in multiple Actions steps.
    badge-title: 'tests'

    # Limits which test suites are listed:
    #   all
    #   failed
    #   none
    list-suites: 'all'

    # Limits which test cases are listed:
    #   all
    #   failed
    #   none
    list-tests: 'all'

    # Limits number of created annotations with error message and stack trace captured during test execution.
    # Must be less or equal to 50.
    max-annotations: '10'

    # Set action as failed if test report contains any failed test
    fail-on-error: 'true'

    # Set this action as failed if no test results were found
    fail-on-empty: 'true'

    # Controls whether test report details are collapsed or expanded.
    # Supported options:
    #   auto: Collapse only if all tests pass (default behavior)
    #   always: Always collapse the report details
    #   never: Always expand the report details
    collapsed: 'auto'

    # Relative path under $GITHUB_WORKSPACE where the repository was checked out.
    working-directory: ''

    # Personal access token used to interact with Github API
    # Default: ${{ github.token }}
    token: ''

Output parameters

Name	Description
conclusion	success or failure
passed	Count of passed tests
failed	Count of failed tests
skipped	Count of skipped tests
time	Test execution time [ms]
url	Check run URL
url_html	Check run URL HTML
slug_prefix	Random anchor links slug prefix generated for the summary headers

Supported formats

dart-json

Test run must be configured to use JSON reporter. You can configure it in dart_test.yaml:

file_reporters:
  json: reports/test-results.json

Or with CLI arguments:

dart test --file-reporter="json:test-results.json"

For more information see: - test package - test configuration

dotnet-trx

Test execution must be configured to produce Visual Studio Test Results files (TRX). To get test results in TRX format you can execute your tests with CLI arguments:

dotnet test --logger "trx;LogFileName=test-results.trx"

Or you can configure TRX test output in *.csproj or Directory.Build.props:

<PropertyGroup>
  <VSTestLogger>trx%3bLogFileName=$(MSBuildProjectName).trx</VSTestLogger>
  <VSTestResultsDirectory>$(MSBuildThisFileDirectory)/TestResults/$(TargetFramework)</VSTestResultsDirectory>
</PropertyGroup>

Supported testing frameworks: - xUnit - NUnit - MSTest

For more information see dotnet test

dotnet-nunit

Test execution must be configured to generate NUnit3 XML test results. Install the NUnit3TestAdapter package (required; it registers the nunit logger for dotnet test), then run tests with:

dotnet test --logger "nunit;LogFileName=test-results.xml"

Supported testing frameworks: - NUnit

For more information see dotnet test

flutter-json

Test run must be configured to use JSON reporter. You can configure it in dart_test.yaml:

file_reporters:
  json: reports/test-results.json

Or with (undocumented) CLI argument:

flutter test --machine > test-results.json

According to documentation dart_test.yaml should be at the root of the package, next to the package's pubspec. On current stable and beta channels it doesn't work, and you have to put dart_test.yaml inside your test folder. On dev channel, it's already fixed.

For more information see: - test package - test configuration - flutter-cli - unit testing introduction

golang-json

You must use the -json flag and output the results to a file (ex: go test -json > testresults.json)

java-junit (Experimental)

Support for JUnit XML is experimental - should work but it was not extensively tested. To have code annotations working properly, it's required your directory structure matches the package name. This is due to the fact Java stack traces don't contain a full path to the source file. Some heuristic was necessary to figure out the mapping between the line in the stack trace and an actual source file.

phpunit-junit

PHPUnit can generate JUnit XML via CLI: phpunit --log-junit reports/phpunit-junit.xml

tester-junit

Nette Tester can generate JUnit XML via CLI:

tester -s -o junit tests/ > reports/tester-junit.xml

Note: Nette Tester's JUnit output doesn't include test suite names. The parser will use the report file name as the suite name and automatically group tests by directory structure.

jest-junit

JEST testing framework support requires the usage of jest-junit reporter. It will create test results in Junit XML format which can be then processed by this action. You can use the following example configuration in package.json: ```json "scripts": { "test": "jest --ci --reporters=default --reporters=jest-junit" }, "devDependencies": { "jest": "^26.5.3", "jest-junit": "^12.0.0" }, "jest-junit": { "outputDirectory": "reports", "outputName": "jest-junit