GitHub Actions integration

API testing in GitHub Actions

Run a Total Shift Left test pack from any workflow with a single step. Gate the build on its pass rate, publish JSON and JUnit results, and block merges on API regressions — no scripting required.

Available now — reference the action as Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1. Source on GitHub.

What this integration gives you

One step runs a Total Shift Left test pack as part of any GitHub Actions workflow: it authenticates, triggers the run, waits for that run to finish, applies your quality gate, and writes results into the workspace. Use it to gate pull-request merges on API quality, run scheduled regression suites against staging, or chain API tests into a wider release pipeline.

  • Quality gates on PRs

    The step exits non-zero when the pass rate misses your threshold — blocks the merge via branch protection.

  • JSON + JUnit artifacts

    A JSON summary and JUnit XML land in the workspace on every run, ready for upload-artifact or any test reporter.

  • Step outputs

    decision, success_rate, execution_id and more are exposed as step outputs for downstream jobs and PR comments.

  • Self-hosted runner support

    Works on GitHub-hosted or self-hosted runners — the latter when your Shift-Left deployment sits behind a firewall.

Workflow example

Drop this into .github/workflows/api-tests.yml. It reads credentials from GitHub Secrets, runs on every pull request and main-branch push, and fails the job when the pass rate drops below 95%.

name: API Tests

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  api-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1
        id: api-tests
        with:
          server-url: ${{ secrets.SHIFTLEFT_URL }}
          api-email: ${{ secrets.SHIFTLEFT_EMAIL }}
          api-password: ${{ secrets.SHIFTLEFT_PASSWORD }}
          pack-id: ${{ vars.SHIFTLEFT_TEST_PACK_ID }}
          pass-threshold-percent: '95'

      - name: Publish results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: shiftleft-results
          path: shiftleft-test-pack-results.xml

Several packs in parallel

The action is an ordinary step, so a matrix strategy fans test packs out across parallel jobs.

strategy:
  matrix:
    pack: [pack_checkout, pack_accounts]
steps:
  - uses: Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1
    with:
      pack-id: ${{ matrix.pack }}
      server-url: ${{ secrets.SHIFTLEFT_URL }}
      api-email: ${{ secrets.SHIFTLEFT_EMAIL }}
      api-password: ${{ secrets.SHIFTLEFT_PASSWORD }}

Using the outputs

Give the step an id and later steps can read the gate decision and pass rate — useful for PR comments, dashboards, or deciding whether a deploy job runs.

- uses: Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1
  id: api-tests
  continue-on-error: true
  with:
    server-url: ${{ secrets.SHIFTLEFT_URL }}
    api-email: ${{ secrets.SHIFTLEFT_EMAIL }}
    api-password: ${{ secrets.SHIFTLEFT_PASSWORD }}
    pack-id: ${{ vars.SHIFTLEFT_TEST_PACK_ID }}

- name: Comment the result on the PR
  run: |
    echo "Decision:   ${{ steps.api-tests.outputs.decision }}"
    echo "Pass rate:  ${{ steps.api-tests.outputs.success_rate }}%"
    echo "Execution:  ${{ steps.api-tests.outputs.execution_id }}"

Inputs

Four inputs are required; the rest have defaults that suit most pipelines. These are the same values every Total Shift Left CI integration takes — the action, the Azure DevOps task, the CircleCI orb, the Bitbucket pipe and the GitLab component all call one shared runner, so they behave identically.

InputDefaultDescription
server-urlRequiredBase URL of your deployment, no trailing slash.
api-emailRequiredAPI user. Its role must be allowed on the public API.
api-passwordRequiredPassword for that user. Always pass it from a secret.
pack-idRequiredID of the test run pack to execute.
tenant-id""Multi-tenant installs only — sent as the X-Tenant-ID header.
wait-for-completiontrueSet false to trigger the run and continue without waiting.
poll-interval-seconds10How often to poll for status while waiting.
timeout-minutes60Give up waiting after this many minutes and fail the step.
pass-threshold-percent100Minimum pass rate for the gate. 0 disables the threshold check.
fail-on-error-teststrueFail the gate when any test finishes in ERROR, whatever the pass rate.
gate-failure-resultfailedUse succeeded-with-issues to warn on a failed gate instead of failing the step.
write-json-summarytrueWrite a JSON summary of the run.
json-summary-pathshiftleft-test-pack-summary.jsonPath for the JSON summary, relative to the working directory.
write-test-results-xmltrueWrite JUnit XML for your CI test reporter.
test-results-xml-pathshiftleft-test-pack-results.xmlPath for the JUnit XML, relative to the working directory.
working-directoryautoWhere artifacts are written. Detected from the CI's own workspace variable when unset.

Outputs and gate decisions

Every value below is set as a step output, so steps.<id>.outputs.<name> works in any later step.

OutputDescription
execution_idID of the execution that was graded. Empty if the run never started.
trigger_execution_idReceipt id from the trigger call. Not a lookup key — use execution_id.
decisionGate decision code (see below).
success_ratePass rate of the graded execution.
passed"true" when the quality gate passed.
task_completionsucceeded, succeededWithIssues or failed.
json_summary_pathAbsolute path to the JSON summary that was written.
test_results_xml_pathAbsolute path to the JUnit XML that was written.

Gate decision codes

DecisionMeaning
PASSEDThe run finished and cleared every gate you configured.
GATE_FAIL_THRESHOLDThe run finished below your pass-rate threshold.
GATE_FAIL_ERROR_TESTSAt least one test finished in ERROR and fail-on-error-tests was on.
FAILEDThe execution itself failed.
COMPLETED_WITH_ISSUESThe gate failed but gate-failure-result was set to warn instead.
OKThe run completed with no gate configured to judge it.
TIMEOUTThe run had not finished within timeout-minutes.
TRIGGER_ONLYwait-for-completion was false, so nothing was graded.

Why not a curl script?

GET /status reports a pack’s most recent execution, and POST /run returns before the scheduler has claimed the pack — so a script that polls immediately can read the previous run’s result and pass a build whose new run has not started. Every Total Shift Left integration records the execution id before triggering and treats the run as complete only once that id changes.

If you would still rather call the API directly — an air-gapped runner with no access to the action, or a workflow that needs custom logic — every endpoint the action uses is public. See the REST API pattern on the integrations page, and remember to guard against the stale-result case above.

GitHub Actions integration — FAQ

Contact us at

info@totalshiftleft.com

to learn more

  • Is there a first-party Total Shift Left GitHub Action?
    Yes. Reference it as Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1. It runs on the node20 runner with a pre-built bundle, so there is no install or build step in your workflow — the action authenticates, triggers the test pack, waits for that run to finish, applies your quality gate, and writes JSON plus JUnit XML into the workspace.
  • Where do I store the Total Shift Left credentials in GitHub?
    Store the server URL, email and password as encrypted GitHub Secrets at the repository or organization level, and pass them into the server-url, api-email and api-password inputs. The pack id is not a secret, so a repository or organization variable is the natural home for it. Never commit credentials to workflow YAML. For self-hosted Shift-Left deployments, the URL secret should point at your internal hostname.
  • Does the action work with self-hosted Shift-Left deployments?
    Yes. The action calls the public /api/v1 endpoints, so any reachable deployment works — multi-tenant SaaS or self-hosted on your own infrastructure. For a deployment behind a firewall, run the job on a self-hosted GitHub runner inside the same network. Multi-tenant installs set the tenant-id input, which is sent as the X-Tenant-ID header.
  • How do I fail the build on a quality gate?
    That is the default. Set pass-threshold-percent to your minimum pass rate and the step exits non-zero when the run misses it, which fails the job — combine with branch protection to block the merge. fail-on-error-tests additionally fails the gate when any test finishes in ERROR, whatever the pass rate. If you would rather warn than block, set gate-failure-result to succeeded-with-issues.
  • Can I get the results into the GitHub UI?
    The action writes JUnit XML to shiftleft-test-pack-results.xml and a JSON summary to shiftleft-test-pack-summary.json by default. Upload either with actions/upload-artifact, or feed the XML to any test-reporter action to get annotations on the pull request. Both paths are configurable inputs, and both are returned as step outputs.
  • Can I run multiple test packs in parallel?
    Yes. Use a matrix strategy with one entry per pack id — the action is a plain step, so it parallelises exactly like any other. Give each matrix entry a distinct artifact name if you are uploading results.
  • Why not just call the REST API with curl?
    You still can, and the pattern is documented below. The difference is that `GET /status` reports a test pack’s most recent execution, and `POST /run` returns before the scheduler has claimed the pack — so a naive script can read the previous run’s result and pass a build whose new run has not started. The action records the execution id before triggering and only grades the run once that id changes.

Add Total Shift Left to your next pull request

Forever-free Citizen Developer or 15-day Enterprise trial. Add one step to your workflow and gate the merge on API quality.