# WebSocket-RPC Testing

> Some services are not called with independent HTTP requests. You open a WebSocket connection, sign in once during the handshake, and then send a series of calls over that same connection.

Source: https://totalshiftleft.ai/help-center/product-documentation/websocket-rpc-testing

> **Applies to:** Trial and Enterprise editions · Web app and Desktop app · Runs on the Engine only (not the Agent)

## Overview

Some services are not called with independent HTTP requests. You open a WebSocket connection, sign in once during the handshake, and then send a series of calls over that same connection. The server remembers what happened earlier on the connection, and a refused call is just a message on a perfectly healthy socket.

Shift-Left Studio tests these WebSocket-RPC gateways directly. You connect to a gateway, see the actions it offers, choose which ones to test, and Studio creates an endpoint and generated tests for each. For stateful scenarios — sign in, do something, check the result on the same connection — you build a workflow with dedicated **WS Connect**, **WS Call** and **WS Close** blocks.

Three things make WebSocket-RPC different from HTTP, and this article explains each: there is no HTTP status code at all, a refused call is not the same as a broken connection, and the connection itself carries the credential.

## Key concepts

| Term | Meaning |
|---|---|
| **Gateway** | The WebSocket server you are testing, reached at a `ws://` or `wss://` address. |
| **Method (action)** | The name of an operation you call on the gateway, such as `agents.list`. Each method becomes one endpoint. |
| **Frame codec** | The message format the gateway speaks. Two are supported: **JSON-RPC 2.0 over WebSocket** and **OmniTroop**. |
| **Handshake** | The first exchange on a new connection, which signs you in. It decides your trust level (role) for the whole connection. |
| **Auth token reference** | A pointer to the credential — for example `key:Administrator` — resolved at run time. You never type the token itself into an endpoint. |
| **Access key** | A credential you save once under a name (such as *Administrator*) and then pick by name. Stored encrypted; it is never shown again. |
| **Session** | One open connection. Steps in a workflow share it, so later steps see what earlier ones did. |
| **Refused** vs **dead** | A refused call is a logical failure on a working connection. A dead connection is an error. They are reported differently. |

## Before you begin

- Your licence must be **Trial** or **Enterprise**. On other editions the WebSocket options do not appear.
- You need the gateway address (starting with `ws://` or `wss://`) and, if the gateway requires one, an access key.
- WebSocket-RPC tests run on the **Engine** (the server, including the Engine built into the Desktop app). They do **not** run on the **Agent** (local runner).
- A WebSocket-RPC endpoint does **not** need a project base URL. It carries its own full gateway address.

## Step 1 — Connect to a gateway

1. In the project tree, open the import dialog (**Import API to Project**, or **Import API to Feature**).
2. Under **Import Method**, click **WebSocket Gateway**.
3. Enter the **Gateway address**, for example `ws://127.0.0.1:18050/ws`.
4. Under **Access key (if your gateway needs one)**:
   - Choose a saved key from the list, or **No key (the gateway is open)**, or
   - Click **Add a new key**, enter a name in **Name this key, e.g. Administrator**, paste the key into **Paste the key**, and click **Save key**.
5. Click **Test connection**.

When the connection works, a **Connected** card shows:

| Line | Meaning |
|---|---|
| **Recognised as** | The gateway type (for example *OmniTroop gateway* or *JSON-RPC gateway*) and version, if it reports one. |
| **Signed in as** | The role the gateway granted this key, for example *Admin — full access*. |
| **Response** | How fast the handshake was: *Fast*, *Normal* or *Slow*. |

If the card says the gateway type does not confirm keys when connecting, the first real call will tell you whether the key works.

> **Note:** Access keys are saved per project, so two projects can each have an *Administrator* key pointing at different gateways. Saving a key under an existing name replaces the credential without breaking any endpoint that uses it.

## Step 2 — Choose which actions to test

Many gateways announce every action they support when you connect. Studio reads that list and classifies each action in the same step, so testing the connection and choosing what to test happen together.

1. Read the summary under **Choose what to test**, for example *Your gateway listed 544 actions across 90 areas. The 120 safe ones are already selected.*
2. Review each action's badge:
   - **Safe to run** — a recognised read action (such as list, get or status). Selected by default.
   - **Changes data** — the action changes the gateway when a test runs. Not selected unless you tick it.
   - **Changes data?** — the gateway did not say what the action does, so Studio assumes the careful answer and treats it as changing data.
3. With more than 12 actions, use **Search actions…**, **Select all N**, **Select safe only** and **Clear** to manage the selection.
4. Click **Create tests for N actions**.

> **Important:** A generated happy-path test really runs its action. The classification is deliberately cautious: only a recognised read verb counts as safe, and actions such as run, send, test, check or validate are treated as changing data because each of them executes something. An action like `files.updateList` is changing; `sessions.listActive` is safe. Tick a **Changes data** action only if you are testing against a gateway you can afford to change.

If the gateway cannot list what it supports, Studio says so; you can add methods one at a time after connecting. You can also import a JSON descriptor file: set **API Type:** to **WebSocket-RPC**, choose **Upload File**, and upload a file such as `{ "wsUrl": "ws://host/ws", "codec": "jsonrpc2", "methods": ["agents.list"] }`.

## Step 3 — Review an endpoint

Each action becomes a WebSocket-RPC endpoint. Open one to see or edit:

| Field | Purpose |
|---|---|
| **RPC method** | The action name, for example `agents.list`. |
| **WebSocket URL** | The full gateway address. Must start with `ws://` or `wss://`. |
| **Frame codec** | **JSON-RPC 2.0 over WebSocket** or **OmniTroop**. |
| **Auth token reference** | `key:NAME` for a saved access key, `env:NAME` for an environment variable, or `secretref://provider/path#field` for a secret manager. |
| **Default params** | The params sent with the call. |
| **Call timeout (ms)**, **Idle timeout (ms)**, **Max session (ms)** | Limits for one call, for a quiet connection, and for a whole session. |

There is deliberately no place to type a literal token. A handshake token authorises the whole connection and is often long-lived and powerful; a literal would travel with every export and backup.

## Step 4 — Generate and understand the tests

Generation creates, for each method:

| Test | Assertions |
|---|---|
| **Happy path** (not created for methods marked as changing data) | **WebSocket: Handshake Succeeded**, **WebSocket: Call Succeeded** (true), and a response-time check. |
| **Unknown method rejected** | **WebSocket: Handshake Succeeded**, **WebSocket: Call Succeeded** (false), grounded in the gateway's own error codes. |
| **Missing required field** (where the gateway names required fields) | **WebSocket: Call Succeeded** (false). |

Methods that change data get no happy-path test, but their negative tests are still generated, because the gateway refuses those calls before acting.

### Why there is no status code assertion

A WebSocket call has no HTTP status. A status check such as "status equals 200" would pass on a call the gateway flatly refused, so generated WebSocket-RPC tests never include one. Use **WebSocket: Call Succeeded** instead. For negative tests that expect a refusal for authorization reasons, the handshake assertion is also left out, because a correct gateway may refuse either at the handshake or at the call.

## Assertion reference

All assertions are in the **WebSocket-RPC** category. Each one fails with a readable message if it is used on a response that is not from a WebSocket gateway.

| Assertion | What it checks | Example |
|---|---|---|
| **WebSocket: Handshake Succeeded** | The connection opened and the gateway accepted the credentials. | Every positive test. |
| **WebSocket: Role Equals** | The trust level the gateway granted this connection. | `admin`, `viewer` |
| **WebSocket: Scope Granted** | The connection was granted a scope. A `*` grant satisfies any scope. | `agents:read` |
| **WebSocket: Call Succeeded** | The reply reported success. Set it to false to require a refusal. | true for positives, false for negatives |
| **WebSocket: Error Code Equals** | The error code in a refusal. Accepts text or numbers. | `METHOD_NOT_FOUND`, `-32601` |
| **WebSocket: Event Received** | The gateway pushed an event with this name during the call. Keepalive messages are ignored. | `agent.updated` |
| **WebSocket: Event Field Equals** | Some pushed event has a field at this path equal to a value. | `payload.status` = `ready` |

The run result shows a verdict in words — **Accepted**, **Refused** or **No reply** — instead of a status code.

## Step 5 — Build a stateful workflow

For scenarios that depend on connection state, use workflow blocks from the **WebSocket Session** group in the flow editor palette.

1. Open or create a workflow.
2. Drag **WS Connect** onto the canvas and connect it after Start. Set **WebSocket URL**, **Frame codec** and **Auth reference**. Optionally set **Handshake overrides (JSON)**, **Handshake timeout (ms)** and **Idle timeout (ms)**. This block publishes the granted role, scopes and server information as variables you can assert on.
3. Add one or more **WS Call** blocks. Set **RPC method** and **Params (JSON)** (just the params), and the same **WebSocket URL** and **Auth reference** as the Connect block.
4. Add **WS Close** at the end with the same **WebSocket URL** and **Auth reference**, then connect it to Stop.
5. Save and run.

> **Important:** A **WS Call** block does not open a connection by itself. It must match an earlier **WS Connect** block — the same URL and auth reference together name the session. If Call blocks connected on their own, each step would get its own fresh connection and a stateful test would silently become several unrelated ones.

**Testing permissions.** Run the same flow with two auth references — for example an administrator and a viewer. Assert **WebSocket: Role Equals** after each Connect, and **WebSocket: Call Succeeded** set to false on the call the viewer should be denied. Two auth references against one URL always produce two separate connections, so the two roles never share a session.

### Refused versus dead

| What happened | Step status | Connection |
|---|---|---|
| The reply refused the call | **FAILED** | Stays open; later steps can continue. |
| The handshake was rejected, the connection dropped, the server closed it, or it went idle too long | **ERROR** | Gone. A later **WS Connect** opens a new one. |
| **WS Close** with nothing left to close | **PASSED** | — |

A refusal is what negative tests are designed to provoke, so it does not stop the run. **WS Close** passes on an already-dead connection so an earlier failure is not reported twice. All connections are closed when the run ends, however it ends.

## Step 6 — Protect gateway state during a pack run (optional)

Because happy-path tests really run their actions, a pack of WebSocket-RPC tests can change live gateway state. A test run pack can be wrapped in a snapshot: Studio calls a backup action before the first test and a restore action after the last.

- If the backup fails, the pack stops and every test is recorded as not run, with the gateway's reason.
- Restore runs on every exit — pass, fail, cancel or error.
- If the restore fails, it appears as an error row on the run, not just in a log.

This setting is not yet available in the pack editor; it is set through the API (see "Automating this"). The default backup and restore action names for OmniTroop gateways come from the vendor's documentation and have not been verified against a live gateway, so confirm them for your gateway.

## Troubleshooting

| Symptom | Why it happens | What to do |
|---|---|---|
| **WebSocket Gateway** is missing from the import dialog | Your edition does not include it, or it is turned off on your installation. | Check your licence (Trial or Enterprise), or ask your administrator. |
| *The URL must start with ws:// or wss://* | The address uses `http://` or has no scheme. | Enter the full WebSocket address. |
| *WebSocket-RPC tests are not supported by the local runner* | The test was run on the **Agent**. | Switch to **Engine** and run again. |
| Every call fails with an unauthorized error | The auth reference names a key that does not exist, or the key is wrong. | Check the key name; a missing key fails by name rather than connecting anonymously. |
| A **WS Call** step errors before sending anything | No earlier **WS Connect** with the same URL and auth reference, so there is no session to call on. | Add or match a **WS Connect** block. |
| A status-code assertion passes on a refused call | WebSocket calls have no status. | Replace it with **WebSocket: Call Succeeded**. |
| The "unknown method rejected" test fails because the call succeeded | The test is calling a real method. | Regenerate the test so it targets a method the gateway cannot know. |

## Best practices

- Keep **Changes data** actions unticked unless you test against a disposable gateway.
- Save one access key per role and pick it by name; rotate by saving a new credential under the same name.
- Use workflows, not single tests, for anything that depends on earlier steps.
- End every connection with **WS Close**.
- Run the Workflow Doctor (**Check workflow**) on WebSocket workflows before running them.

## Related articles

- [Workflows](/help-center/product-documentation/understanding-workflows)
- [Test run packs](/help-center/product-documentation/test-run-pack)
- [Test run](/help-center/product-documentation/test-run)
- [JSON-RPC 2.0 and MCP Server Testing](/help-center/product-documentation/json-rpc-and-mcp-testing)
- [Workflow Doctor: Diagnose and Fix a Workflow](/help-center/product-documentation/workflow-doctor)

## Automating this

The pack snapshot is the `wsRpcGuard` field on `PUT /api/test-run-packs/:id`:

| Field | Default | Meaning |
|---|---|---|
| `enabled` | `false` | Turn the snapshot on. |
| `backupMethod` / `restoreMethod` | Gateway defaults | The actions called before the first test and after the last. |
| `backupParams` / `restoreParams` | `{}` / unset | Params for each. When `restoreParams` is unset, the backup's reply is sent back. Values may use `{{backup.<path>}}`. |
| `restoreWhen` | `always` | `onSuccess` leaves a failed run's state in place for inspection. |
| `requireBackup` | `true` | A failed backup stops the pack. |

Saved access keys are listed with `GET /api/wsrpc/keys` (names only) and saved with `POST /api/wsrpc/keys`.

## For administrators (self-hosted installations)

| Setting | Default | What switching it does |
|---|---|---|
| `WSRPC_TRANSPORT_ENABLED` | `true` | Set to `false` to remove WebSocket-RPC testing entirely. The import option, endpoints and workflow blocks become unavailable. |