WebSocket-RPC Testing

Overview

Some services are not called with independent HTTP requests. You open a WebSocket connection, sign in once during the handshake, and then send a series of calls over that same connection. The server remembers what happened earlier on the connection, and a refused call is just a message on a perfectly healthy socket.

Shift-Left Studio tests these WebSocket-RPC gateways directly. You connect to a gateway, see the actions it offers, choose which ones to test, and Studio creates an endpoint and generated tests for each. For stateful scenarios — sign in, do something, check the result on the same connection — you build a workflow with dedicated WS Connect, WS Call and WS Close blocks.

Three things make WebSocket-RPC different from HTTP, and this article explains each: there is no HTTP status code at all, a refused call is not the same as a broken connection, and the connection itself carries the credential.

Key concepts

Before you begin

• Your licence must be Trial or Enterprise. On other editions the WebSocket options do not appear.
• You need the gateway address (starting with ws:// or wss://) and, if the gateway requires one, an access key.
• WebSocket-RPC tests run on the Engine (the server, including the Engine built into the Desktop app). They do not run on the Agent (local runner).
• A WebSocket-RPC endpoint does not need a project base URL. It carries its own full gateway address.

Step 1 — Connect to a gateway

1. In the project tree, open the import dialog (Import API to Project, or Import API to Feature).
2. Under Import Method, click WebSocket Gateway.
3. Enter the Gateway address, for example ws://127.0.0.1:18050/ws.
4. Under Access key (if your gateway needs one):
   • Choose a saved key from the list, or No key (the gateway is open), or
   • Click Add a new key, enter a name in Name this key, e.g. Administrator, paste the key into Paste the key, and click Save key.
5. Click Test connection.

When the connection works, a Connected card shows:

If the card says the gateway type does not confirm keys when connecting, the first real call will tell you whether the key works.

Note: Access keys are saved per project, so two projects can each have an Administrator key pointing at different gateways. Saving a key under an existing name replaces the credential without breaking any endpoint that uses it.

Step 2 — Choose which actions to test

Many gateways announce every action they support when you connect. Studio reads that list and classifies each action in the same step, so testing the connection and choosing what to test happen together.

1. Read the summary under Choose what to test, for example Your gateway listed 544 actions across 90 areas. The 120 safe ones are already selected.
2. Review each action's badge:
   • Safe to run — a recognised read action (such as list, get or status). Selected by default.
   • Changes data — the action changes the gateway when a test runs. Not selected unless you tick it.
   • Changes data? — the gateway did not say what the action does, so Studio assumes the careful answer and treats it as changing data.
3. With more than 12 actions, use Search actions…, Select all N, Select safe only and Clear to manage the selection.
4. Click Create tests for N actions.

Important: A generated happy-path test really runs its action. The classification is deliberately cautious: only a recognised read verb counts as safe, and actions such as run, send, test, check or validate are treated as changing data because each of them executes something. An action like files.updateList is changing; sessions.listActive is safe. Tick a Changes data action only if you are testing against a gateway you can afford to change.

If the gateway cannot list what it supports, Studio says so; you can add methods one at a time after connecting. You can also import a JSON descriptor file: set API Type: to WebSocket-RPC, choose Upload File, and upload a file such as { "wsUrl": "ws://host/ws", "codec": "jsonrpc2", "methods": ["agents.list"] }.

Step 3 — Review an endpoint

Each action becomes a WebSocket-RPC endpoint. Open one to see or edit:

There is deliberately no place to type a literal token. A handshake token authorises the whole connection and is often long-lived and powerful; a literal would travel with every export and backup.

Step 4 — Generate and understand the tests

Generation creates, for each method:

Methods that change data get no happy-path test, but their negative tests are still generated, because the gateway refuses those calls before acting.

Why there is no status code assertion

A WebSocket call has no HTTP status. A status check such as "status equals 200" would pass on a call the gateway flatly refused, so generated WebSocket-RPC tests never include one. Use WebSocket: Call Succeeded instead. For negative tests that expect a refusal for authorization reasons, the handshake assertion is also left out, because a correct gateway may refuse either at the handshake or at the call.

Assertion reference

All assertions are in the WebSocket-RPC category. Each one fails with a readable message if it is used on a response that is not from a WebSocket gateway.

The run result shows a verdict in words — Accepted, Refused or No reply — instead of a status code.

Step 5 — Build a stateful workflow

For scenarios that depend on connection state, use workflow blocks from the WebSocket Session group in the flow editor palette.

1. Open or create a workflow.
2. Drag WS Connect onto the canvas and connect it after Start. Set WebSocket URL, Frame codec and Auth reference. Optionally set Handshake overrides (JSON), Handshake timeout (ms) and Idle timeout (ms). This block publishes the granted role, scopes and server information as variables you can assert on.
3. Add one or more WS Call blocks. Set RPC method and Params (JSON) (just the params), and the same WebSocket URL and Auth reference as the Connect block.
4. Add WS Close at the end with the same WebSocket URL and Auth reference, then connect it to Stop.
5. Save and run.

Important: A WS Call block does not open a connection by itself. It must match an earlier WS Connect block — the same URL and auth reference together name the session. If Call blocks connected on their own, each step would get its own fresh connection and a stateful test would silently become several unrelated ones.

Testing permissions. Run the same flow with two auth references — for example an administrator and a viewer. Assert WebSocket: Role Equals after each Connect, and WebSocket: Call Succeeded set to false on the call the viewer should be denied. Two auth references against one URL always produce two separate connections, so the two roles never share a session.

Refused versus dead

A refusal is what negative tests are designed to provoke, so it does not stop the run. WS Close passes on an already-dead connection so an earlier failure is not reported twice. All connections are closed when the run ends, however it ends.

Step 6 — Protect gateway state during a pack run (optional)

Because happy-path tests really run their actions, a pack of WebSocket-RPC tests can change live gateway state. A test run pack can be wrapped in a snapshot: Studio calls a backup action before the first test and a restore action after the last.

• If the backup fails, the pack stops and every test is recorded as not run, with the gateway's reason.
• Restore runs on every exit — pass, fail, cancel or error.
• If the restore fails, it appears as an error row on the run, not just in a log.

This setting is not yet available in the pack editor; it is set through the API (see "Automating this"). The default backup and restore action names for OmniTroop gateways come from the vendor's documentation and have not been verified against a live gateway, so confirm them for your gateway.

Troubleshooting

Best practices

• Keep Changes data actions unticked unless you test against a disposable gateway.
• Save one access key per role and pick it by name; rotate by saving a new credential under the same name.
• Use workflows, not single tests, for anything that depends on earlier steps.
• End every connection with WS Close.
• Run the Workflow Doctor (Check workflow) on WebSocket workflows before running them.

Related articles

• Workflows
• Test run packs
• Test run
• JSON-RPC 2.0 and MCP Server Testing
• Workflow Doctor: Diagnose and Fix a Workflow

Automating this

The pack snapshot is the wsRpcGuard field on PUT /api/test-run-packs/:id:

Saved access keys are listed with GET /api/wsrpc/keys (names only) and saved with POST /api/wsrpc/keys.

For administrators (self-hosted installations)