CI/CD: Run Test Packs from GitHub Actions

Overview

The Shift-Left API GitHub Action runs one of your test run packs from a GitHub Actions workflow. It signs in to your Shift-Left API server, starts the pack, waits for that run to finish, applies a quality gate, and writes JUnit XML and a JSON summary into the workspace. If the gate fails, the step fails, and so does the check on the pull request.

The action lives in the public repository Total-Shift-Left/Shift-Left-API-Integrations. You reference it by tag, Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1. It is not listed on the GitHub Marketplace, and you do not need it to be: GitHub resolves the action straight from the tag. It runs on the node20 runtime from a prebuilt bundle, so there is no install step.

The GitHub Action, the Azure DevOps task and the Jenkins plugin apply the same gate and give the same decision for the same run. Any other CI system can use the REST API.

Before you begin

1. A test run pack. Build a pack for the tests you want CI to run and pick its environment in the pack. Copy the pack ID. See Test Run Pack Wizard.
2. The Public API switched on, with the API user's role in Allowed Roles, under Settings → Integrations → Public API. See Public API.
3. An API user. The action signs in with an email and password. Use a dedicated user for CI, so a person changing their password never breaks the pipeline.
4. A server GitHub can reach. The pack runs on your Shift-Left API server, not on the GitHub runner. For a self-hosted installation inside your network, run the job on a self-hosted GitHub runner that can reach the server.

Step 1 — Store the connection details

In the repository (or organisation), add these under Settings → Secrets and variables → Actions:

Never put the password in the workflow file.

Step 2 — Add the workflow

Create .github/workflows/api-tests.yml:

# .github/workflows/api-tests.yml
name: API Tests
on:
  pull_request:
    branches: [main]

jobs:
  api-tests:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      checks: write   # lets dorny/test-reporter create the check
    steps:
      - uses: actions/checkout@v4
      - name: Run API test pack
        id: shiftleft
        uses: Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1
        with:
          server-url: ${{ secrets.SHIFTLEFT_URL }}
          api-email: ${{ secrets.SHIFTLEFT_EMAIL }}
          api-password: ${{ secrets.SHIFTLEFT_PASSWORD }}
          pack-id: ${{ vars.SHIFTLEFT_TEST_PACK_ID }}
          pass-threshold-percent: '100'   # gate: fail below this pass rate
          fail-on-error-tests: 'true'
      - name: Publish test results
        uses: dorny/test-reporter@v1
        if: always()
        with:
          name: API Test Results
          path: shiftleft-test-pack-results.xml   # JUnit XML written by the action
          reporter: java-junit

The second step is optional. Any JUnit reporter works; dorny/test-reporter turns the XML into a check with per-test results on the pull request.

Step 3 — Read the result

Each run writes two files into the workspace and sets step outputs you can use in later steps (steps.shiftleft.outputs.decision with the step id above, for example):

• shiftleft-test-pack-results.xml: JUnit XML, one test case per test in the pack.
• shiftleft-test-pack-summary.json: the run's summary and the gate decision.

Inputs

Good to know

• It grades the run it started. The action notes the pack's latest run before it triggers, and only grades once a new run appears. A script that polls straight away can read the previous run's result instead.
• One run per pack at a time. If the pack is already running, the server refuses the second start (HTTP 409). Give each pipeline its own pack if they may overlap.
• The environment comes from the pack. There is no per-run environment or base-URL override. To test dev on feature branches and staging on release branches, keep one pack per environment and choose the pack ID by branch in the workflow.
• Results show as a check, not a comment. The step passes or fails on the pull request, and the JUnit file feeds your test reporter. The action does not post a pull-request comment.
• It is audited. CI sign-ins and CI-started runs appear in the audit log.

Troubleshooting

Related articles

• CI/CD integrations on the product site
• GitHub Actions integration overview
• Any CI via the REST API
• Public API
• Manage and Run Execution Packs
• Plans and editions