Azure DevOps Integration for API Testing
The Shift-Left API Azure DevOps task is a first-party integration for Azure Pipelines. Install it from the Visual Studio Marketplace or as a VSIX; one YAML task then triggers a test pack, applies a pass-rate and fail-on-error quality gate, writes JUnit XML and a JSON summary, and sets output variables for later steps.
Install from Marketplace or download the VSIX
Use the one-click install from the Visual Studio Marketplace, or grab the latest VSIX package from our GitHub releases for self-hosted Azure DevOps Server.
Install the Extension
One-click install into your Azure DevOps organization. Published as TotalShiftLeft.shift-left-api-automation-integration.
Get the VSIX
The full task source is public at Total-Shift-Left/Shift-Left-API-Integrations under azure-devops/. Packaged builds are attached to release tags rather than committed to the tree, so they cannot drift from the source beside them. Useful for air-gapped or self-hosted Azure DevOps Server installs, where the Marketplace is not reachable.
Overview
The Shift-Left API Automation Integration extension for Azure DevOps adds a pipeline task that connects to your Shift-Left API instance using the public API. It handles login, test pack triggering, status polling, and detailed results — all within your existing Azure Pipeline workflow.
Prerequisites
- Node.js 20+ on the build agent (task handler Node20_1).
- Network access from the agent to your Shift-Left API server URL.
- A user with permission to call the Shift-Left CI/CD public API.
Key features
- Trigger test run packs from any YAML or Classic pipeline.
- Configurable quality gate with pass-rate thresholds and error-test policies.
- Automatic JUnit XML publishing for native Azure DevOps test result tabs.
- JSON summary artifact for custom dashboards or downstream steps.
- Output variables (execution ID, trigger ID, gate decision) for pipeline logic.
- Multi-tenant support via optional X-Tenant-ID header.
Pipeline YAML example
Reference the task as totalshiftleft.shiftleft-api-integration-task@1 in your pipeline YAML.
variables:
- group: shiftleft-secrets # contains ShiftLeftEmail, ShiftLeftPassword
steps:
- task: totalshiftleft.shiftleft-api-integration-task@1
displayName: Run ShiftLeft pack
inputs:
serverUrl: 'https://your-shiftleft-host'
tenantId: '' # optional
apiEmail: '$(ShiftLeftEmail)'
apiPassword: '$(ShiftLeftPassword)'
packId: 'your_pack_id'
waitForCompletion: true
pollIntervalSeconds: '10'
timeoutMinutes: '60'
passThresholdPercent: '100'
failOnErrorTests: true
gateFailureResult: 'failed'
writeJsonSummary: true
jsonSummaryPath: 'shiftleft-test-pack-summary.json'
writeTestResultsXml: true
testResultsXmlPath: 'shiftleft-test-pack-results.xml'
workingDirectory: '$(Build.SourcesDirectory)'
- task: PublishTestResults@2
displayName: Publish ShiftLeft test results
condition: succeededOrFailed()
inputs:
testResultsFormat: JUnit
testResultsFiles: '**/shiftleft-test-pack-results.xml'
searchFolder: '$(Build.SourcesDirectory)'
mergeTestResults: true
failTaskOnFailedTests: false
- task: PublishPipelineArtifact@1
displayName: Upload JSON summary
condition: always()
inputs:
targetPath: '$(Build.SourcesDirectory)/shiftleft-test-pack-summary.json'
artifact: 'shiftleft-summary'Configuration reference
| Input | Required | Description |
|---|---|---|
| serverUrl | Required | Base URL of your Shift-Left API instance. |
| tenantId | Optional | Optional tenant ID for multi-tenant environments. Sends X-Tenant-ID header. |
| apiEmail | Required | Login email for Shift-Left API. Use a secret variable. |
| apiPassword | Required | Login password. Must be a secret pipeline variable. |
| packId | Required | ID of the test run pack to execute. |
| waitForCompletion | Optional | Poll until the run finishes or times out. Default: true. |
| pollIntervalSeconds | Optional | Seconds between status polls. Default: 10. |
| timeoutMinutes | Optional | Overall timeout in minutes. Default: 60. |
| passThresholdPercent | Optional | Minimum pass rate (0–100) for a successful gate. Default: 100. |
| failOnErrorTests | Optional | Fail the gate when any test has ERROR status. Default: true. |
| gateFailureResult | Optional | Task result on gate failure: failed (default) or succeeded-with-issues, which finishes the task as Succeeded with issues. |
| writeJsonSummary | Optional | Write a JSON summary file to the workspace. Default: true. |
| jsonSummaryPath | Optional | Workspace-relative path for the JSON summary file. |
| writeTestResultsXml | Optional | Write JUnit XML results to the workspace. Default: true. |
| testResultsXmlPath | Optional | Workspace-relative path for the JUnit XML file. |
Output variables
Give the task a name: (for example ShiftLeft) and read these variables in downstream steps.
| Variable | Description |
|---|---|
| shiftLeftExecutionId | Execution ID from the status/results response. |
| shiftLeftTriggerExecutionId | ID returned from the initial trigger response. |
| shiftLeftDecision | Gate decision code: PASSED, GATE_FAIL_THRESHOLD, TIMEOUT, etc. |
Quality gate mapping
How the task sets the Azure Pipelines step outcome based on your gate configuration.
| Task result | Meaning |
|---|---|
| Succeeded | Gate passed — run completed as expected. |
| Failed | Gate or run failed. Requires gateFailureResult set to failed. |
| Succeeded with issues | Gate failed but configured as non-blocking (gateFailureResult: succeeded-with-issues). |
Step-by-step setup, every input and troubleshooting: Run test packs from Azure DevOps Pipelines (help centre). For the tool-neutral how-to on running API tests in Azure Pipelines, read API test automation with CI/CD, step by step; to run security or load tests from the same packs, see API security testing and API load testing (paid add-ons).
A ready-made Azure Pipelines file from a test pack
On a test run pack, Run in your pipeline writes a ready-made Azure Pipelines file for you. It has no field for a password or token, so nothing you type is pasted into the file unchecked. Studio generates the same kind of file for GitHub Actions, GitLab CI, Jenkins.
- +Where the runner comes from. Keep the runner in your repository, or give an https address and your own SHA-256 checksum. If the checksum does not match or the download fails, the build stops before the runner starts. The runner needs Node 18 or newer and has no npm dependency.
- +Options the file can carry.
--verdict: Fail the build on the release verdict. A verdict of "no evidence yet" fails the build by default, and a partial verdict fails unless you allow it.--tickets: Run only the tests behind the tickets you name; a ticket with no tests fails the build unless you allow that.--sarif, --fail-on-severity, --policy: Options for security runs: SARIF output, a severity gate and a committed policy file. These need the API security testing add-on. - +Pull request comments. The verdict comment appears on Azure Pipelines pull requests.
- +Plans. Professional, Trial and Enterprise. The Citizen Developer edition cannot generate pipeline files.
See the release verdict for what --verdict judges.
Azure DevOps FAQs
How do I reference the task in YAML?
Use totalshiftleft.shiftleft-api-integration-task@1 (publisher ID + task name + major version).
Where should I store credentials?
Create a variable group in Pipelines → Library with ShiftLeftEmail (plain) and ShiftLeftPassword (secret). Link the group to your pipeline.
What Node.js version does the agent need?
The task handler requires Node.js 20 or later on the build agent (Node20_1 handler).
Can I use this with YAML and Classic pipelines?
Yes. The YAML task reference works in both YAML pipelines and Classic release definitions that support YAML tasks.
Ready to add API testing to your Azure Pipeline?
Start your free 15-day trial. Install the extension and run your first test pack today.