Run API Tests in GitHub Actions: A Hands-On Workflow Lab

This is lesson 7.2 of API Testing in CI/CD, Regression and Performance. Lesson 7.1 explained the step contract: an exit code, a results file and a gate. This lab builds that step in GitHub Actions, end to end.

How do you run API tests in GitHub Actions?

To run API tests in GitHub Actions, you add a workflow file that, on every push and pull request, starts your API tests, waits for them, and fails the job when they fail. GitHub reads one thing from each step: did it exit with zero? The rest of this lab is about making that answer trustworthy: the right trigger, the secrets, a runner that can reach your systems, a gate, and a results file that names the failing test.

The parts of an API test workflow

A workflow file lives in .github/workflows/ and has four parts that matter for API tests:

Two more settings earn their place. concurrency stops two runs of the same workflow from testing the same environment at once. timeout-minutes stops a hung job from holding a runner for six hours, GitHub's default limit.

The runner question: who can reach what

This is where most API test workflows break the first time. Three systems have to talk:

1. The runner that executes the job.
2. The test runner that sends the requests: a CLI on the GitHub runner, or a test server the job calls.
3. The API under test.

A GitHub-hosted runner is a fresh virtual machine on the internet. It can reach public addresses only. If your API is on a private network, or your test server runs on your own computer, the hosted runner simply times out. The fix is a self-hosted runner: GitHub's runner application installed on a machine inside that network, registered under the repository's Settings → Actions → Runners. Then runs-on: self-hosted sends the job there.

Secrets, variables and what never goes in YAML

Store credentials under Settings → Secrets and variables → Actions. Secrets are encrypted and masked in logs; variables are plain text and fit IDs such as which test pack to run. Refer to them as ${{ secrets.NAME }} and ${{ vars.NAME }}. Pull requests from forks don't receive secrets, so a workflow that needs them won't authenticate on a fork's pull request: that's GitHub protecting you, not a bug.

The lab: a pass-rate gate on a Test Run Pack

In Shift-Left API, the tests stay in the product and a Test Run Pack is what a pipeline runs. Packs and CI need the Professional or Enterprise edition, or the 15-day trial, so do this lab on a trial.

1. Make the pack. Import the sandbox spec from https://demo.totalshiftleft.ai/openapi/rest.json and add a few read tests: the product list, the Wireless Mouse, a user, an order. In Execution, put them in a Test Run Pack. Its environment is the sandbox. Copy the pack's ID.
2. Check the API user. As an administrator, open Settings, then Integrations, then Public API, and make sure the role of the user the workflow signs in as is allowed.
3. Add three secrets and a variable to your repository: SHIFTLEFT_URL (your Shift-Left API server's address, no trailing slash), SHIFTLEFT_EMAIL and SHIFTLEFT_PASSWORD as secrets, and SHIFTLEFT_PACK_ID as a variable.
4. Pick the runner. If your Shift-Left API server is reachable from the internet, ubuntu-latest works. If it runs on your own machine or inside your network, register a self-hosted runner there and use self-hosted.
5. Commit the workflow below as .github/workflows/api-tests.yml.

name: API tests

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]
  workflow_dispatch:

concurrency:
  group: api-tests-${{ github.ref }}
  cancel-in-progress: false

jobs:
  api-tests:
    runs-on: ubuntu-latest   # or self-hosted, if your server is on a private network
    timeout-minutes: 30
    steps:
      - name: Run the Store API pack
        id: api-tests
        uses: Total-Shift-Left/Shift-Left-API-Integrations/github-actions@v1
        with:
          server-url: ${{ secrets.SHIFTLEFT_URL }}
          api-email: ${{ secrets.SHIFTLEFT_EMAIL }}
          api-password: ${{ secrets.SHIFTLEFT_PASSWORD }}
          pack-id: ${{ vars.SHIFTLEFT_PACK_ID }}
          pass-threshold-percent: '90'

      - name: Keep the JUnit results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: shiftleft-results
          path: shiftleft-test-pack-results.xml

      - name: Print the gate decision
        if: always()
        run: echo "Decision ${{ steps.api-tests.outputs.decision }}, pass rate ${{ steps.api-tests.outputs.success_rate }}%"

The action is referenced by its tag, @v1, from the public Shift-Left-API-Integrations repository; it isn't listed on GitHub Marketplace, and it doesn't need to be for uses: to work. The if: always() lines matter: without them, a failed gate would skip the steps that keep the results, which is exactly when you need them.

What the step does, line by line in the log

1. It signs in to the public API under /api/v1 with the API user's email and password.
2. It notes the pack's latest run, then starts the pack. If that pack is already running, the start is refused, so a nightly schedule and a push never run it twice at once.
3. It polls every 10 seconds, for up to 60 minutes by default, until a new run finishes. It never grades the previous run by mistake.
4. It applies the gate and logs a line such as [ShiftLeft] Decision: PASSED (completion=succeeded).
5. It writes shiftleft-test-pack-results.xml (JUnit) and shiftleft-test-pack-summary.json into the workspace, and sets the outputs decision, success_rate, passed and execution_id.

The tests themselves run on the Shift-Left API server, not on the GitHub runner. The runner only has to reach the server; the server has to reach the API under test, which the public sandbox makes easy.

Break it on purpose

A gate you've never seen fail is a gate you don't trust. In Shift-Left API, edit one test in the pack so it expects something the sandbox doesn't return: change the Wireless Mouse's expected price from 24.99 to 29.99. Push any commit, or run the workflow by hand.

With four tests and one failing, the pass rate is 75 percent, below the 90 you set. The step fails with Quality gate: GATE_FAIL_THRESHOLD, the job turns red, and the decision step still prints the numbers. Download the shiftleft-results artifact: the JUnit file names the failing test. The same run is in Reporting in Shift-Left API, with the failed check on it. Put the price back and the next run passes.

Three inputs change how the gate behaves:

Make the check required

A red job only stops a merge if GitHub is told to wait for it. In the repository's Settings, add a branch protection rule or ruleset for main, require status checks to pass, and pick the api-tests check. From now on a pull request that breaks the API tests can't be merged until they pass again.

Two habits keep this healthy. One pack per environment: there's no per-run environment override, so a workflow that tests dev on pull requests and staging on main picks a different pack ID for each, for example with a matrix or an if: on the branch. And keep the pull request pack short; save the long regression pack for push to main or a nightly schedule.

Try it yourself

On a trial, put a few sandbox read tests in a Test Run Pack, add the workflow above with your server address, API user and pack ID, and choose a runner that can reach your Shift-Left API server. Push, and look for [ShiftLeft] Decision: PASSED in the step's log and shiftleft-test-pack-results.xml in the run's artifacts. Then break one test, push again, and watch the job fail.

Recap

• To run API tests in GitHub Actions you need a workflow trigger, secrets, a runner that can reach your test server, and a step whose exit code is the gate.
• The Shift-Left API action, by tag @v1, runs a Test Run Pack on your server, waits for that run, and writes JUnit and a JSON summary.
• A required status check turns a failed gate into a blocked merge, which is the point of running API tests in GitHub Actions at all.

For the wider workflow toolkit (matrices, reusable workflows, deployment protection rules), read API testing in GitHub Actions: complete workflow guide, and the GitHub Actions integration page lists every input and output. Next, in lesson 7.3, you do the same with Jenkins.