GitLab CI Integration for API Testing

.gitlab-ci.yml example

Drop this into your .gitlab-ci.yml. Stores credentials as protected, masked CI/CD variables, runs on every merge request and main-branch push, and fails the job when the pass rate drops below 95%.

stages:
  - test

api-tests:
  stage: test
  image: alpine:latest
  variables:
    SHIFTLEFT_URL: $SHIFTLEFT_URL
    SHIFTLEFT_EMAIL: $SHIFTLEFT_EMAIL
    SHIFTLEFT_PASSWORD: $SHIFTLEFT_PASSWORD
    SHIFTLEFT_TEST_PACK_ID: $SHIFTLEFT_TEST_PACK_ID
  before_script:
    - apk add --no-cache curl jq
  script:
    - |
      set -euo pipefail
      PACK="$SHIFTLEFT_TEST_PACK_ID"
      TOKEN=$(curl -sS -f -X POST "$SHIFTLEFT_URL/api/v1/login" \
        -H "Content-Type: application/json" \
        -d "{\"email\":\"$SHIFTLEFT_EMAIL\",\"password\":\"$SHIFTLEFT_PASSWORD\"}" \
        | jq -r .token)
      AUTH="Authorization: Bearer $TOKEN"
      STATUS_URL="$SHIFTLEFT_URL/api/v1/test-packs/$PACK/status"

      # Note the pack's latest run first, so the previous run is never graded
      BEFORE=$(curl -sS -f "$STATUS_URL" -H "$AUTH" | jq -r '.executionId // empty')
      # Start the pack (the server answers 409 if it is already running)
      curl -sS -f -X POST "$SHIFTLEFT_URL/api/v1/test-packs/$PACK/run" -H "$AUTH" > /dev/null

      # Wait up to 60 minutes for a NEW run to finish
      DONE=0
      for i in $(seq 1 360); do
        RESULT=$(curl -sS -f "$STATUS_URL" -H "$AUTH")
        RUN_ID=$(echo "$RESULT" | jq -r '.executionId // empty')
        STATE=$(echo "$RESULT" | jq -r '.status // empty' | tr '[:lower:]' '[:upper:]')
        if [ -n "$RUN_ID" ] && [ "$RUN_ID" != "$BEFORE" ]; then
          case "$STATE" in RUNNING|PENDING|QUEUED|"") ;; *) DONE=1; break ;; esac
        fi
        sleep 10
      done
      [ "$DONE" = 1 ] || { echo "Timed out waiting for a new run"; exit 1; }

      echo "$RESULT" | jq .
      # Reads executionId, status and successRate; if your installation's Swagger page
      # names them differently, adjust the jq filters. Fails closed below 95%.
      echo "$RESULT" | jq -e '(.successRate // 0) >= 95' > /dev/null
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"

The component source is public and wraps @totalshiftleft/ci — the single implementation behind the GitHub Action, the Azure DevOps task, the CircleCI orb and the Bitbucket pipe, so it grades a run identically to all of them. It is not in the CI/CD Catalog yet, so the snippet below is what the include will look like, not something you can run today.

stages:
  - test

include:
  - component: gitlab.com/totalshiftleft/shiftleft-ci/run-test-pack@1.0.0
    inputs:
      server-url: https://tenant.totalshiftleft.ai
      pack-id: pack_123
      pass-threshold-percent: 95

The component reads credentials by variable name — api-email-variable and api-password-variable default to SHIFTLEFT_API_EMAIL and SHIFTLEFT_API_PASSWORD — so no secret ever lands in .gitlab-ci.yml. It publishes both artifacts with when: always and feeds the JUnit XML to artifacts:reports:junit, so results show up in the merge request widget and the pipeline's Tests tab.

Gate decision codes

Printed to the job log as [ShiftLeft] Output decision=<code> and repeated in the JSON summary artifact. The job exits 0 when the gate passes and 1 when it fails, so it gates the pipeline.