API Versioning Tools: Testing Two Versions of One Contract

This is lesson 5.8 of OpenAPI, Contract and Schema Testing. Lesson 5.6 ended with "when the contract has to break, version it"; this lesson shows what that looks like and how to test it. The demo uses a trial edition of Shift-Left API and two versions of the store spec.

API versioning tools answer one question before a release: which changes between two versions of a contract break the clients you already have? This lesson works through one such change on the sandbox store, shows the strategies for shipping it, and then tests both versions side by side.

An API versioning example: one field becomes required

Here is an API versioning example you can try on the sandbox. Today, creating a user needs only name and email; role is optional and defaults to user. Version 1.1 of the store spec (download it) makes role required. Every client that leaves it out would start getting a 400. That is a breaking change, and versioning is how you ship it without breaking those clients: the old contract stays available, unchanged, while the new one is introduced beside it.

Breaking or compatible?

Ask one question of each change: would a request that worked yesterday fail today, or would a response a client parsed yesterday confuse it today?

The "usually" and "often" rows are where teams disagree. Write your rule down, and apply it the same way every time.

API versioning strategies

Take the required role and ship it four ways:

All four can carry the change. They differ in who notices it and what your tests must send.

API versioning header vs URL

The choice most teams actually face is header vs URL:

• URL versions are visible in logs, links and bug reports, easy to call from a browser or curl, and cached separately with no extra work. The cost: /v2/users reads like a different resource, and teams tend to version everything at once.
• Header versions keep URLs stable and make small, frequent or date-based versions practical. The cost: the version is invisible unless you log headers, caches must vary on the header, and a client that forgets the header gets the default, which must be chosen carefully.

For testing, the difference is concrete: with URL versions each version is a different path in the spec; with header versions it is the same path, and every test must send the header explicitly, including the negative case of sending no header at all.

What to test when you version

1. Diff each new spec against the last release. A tool such as oasdiff, or your test platform's change detection, classifies every change before code runs.
2. Keep a suite per supported version. The v1 tests don't change when v2 arrives. They are the proof that old clients still work.
3. Test the default. For header or media-type versions: what does a request with no version get?
4. Test the edges between versions. A resource created through v2 and read through v1 should come back in v1's shape.
5. Test deprecation. A deprecated version should announce it, for example with a Deprecation header and a Sunset header (RFC 8594) giving the retirement date; and after that date it should answer the way you documented, often 410 Gone.

API versioning tools, by job

No single tool covers versioning. Each kind answers a different question:

Run the diff on every pull request that touches the spec, and the per-version suites on every build.

API versioning best practices

• Don't version for additive changes; version only when something breaks.
• Support a small, stated number of versions, with a published retirement date for each.
• Treat the version as part of the contract: put it in the spec, and in every test.
• Never change an old version's behaviour while it is supported. Bug fixes, yes; contract changes, no.
• Watch the traffic: retire a version when its usage says you can, not when the calendar says so.

For a longer, CI-focused walk-through, read API versioning testing: backward compatibility checks in CI.

In Shift-Left API: two spec versions side by side

Shift-Left API versions the contract inside the project. When you approve a re-imported spec, each new or changed endpoint becomes a new version and the earlier one is kept, deprecated, with its tests. No existing test is rewritten. Approving changes needs Professional, Enterprise or the trial. Lesson 9.3 walks through the review screen; here you test both versions.

1. Create a project from version 1.0 of the store spec, and generate tests for Create a new user in Users.
2. Import version 1.1 into the same project. Review API Changes marks Create a new user as Breaking: role became required. Click Approve & Import.
3. The changed endpoint arrives in a new feature, Users v2. The original Users keeps its version 1 endpoints and their tests.
4. In Users v2, open Create a new user and click Generate tests. Among them is a missing required field test for role, expecting a 400.
5. Run it. It fails: the live sandbox doesn't require a role yet (send a user without one and it answers 201, with role set to user), so it can't return the 400 that version 1.1 documents. The spec moved ahead of the implementation, and the test says so.
6. In Execution, build a Test Run Pack with the version 1 tests from Users and the new tests from Users v2, and click Run Now. One pack now answers both questions: do old clients still work, and does the new contract hold?

These versions are versions of the spec in the project. Both point at the same sandbox path, because the sandbox has one implementation; with a real URL- or header-versioned API, each version's tests would carry its own path or header. The API change detection page lists the breaking-change rules.

Try it yourself

With a trial, create a project from store-api-v1.0.json, import store-api-v1.1.json, and click Approve & Import. Generate tests for Create a new user in Users v2 and run the test for a missing role. Does it pass? What would have to change, the API or the spec, for it to pass?

Recap

• API versioning tools split the job: a spec diff finds breaking changes, a linter enforces your rules, and a suite per version proves old clients still work.
• This API versioning example, a field that becomes required, breaks every old client unless the old contract stays available beside the new one.
• API versioning strategies (URL path, query parameter, header, media type) all work; API versioning header vs URL is a trade between visibility and stable URLs.
• API versioning best practices: version only breaking changes, keep a test suite per supported version, and test the default and the deprecation path.

Next: lesson 5.9, what an API test coverage tool should measure.