Fill Tests From a Data Set

Overview

Use this when a project already has tests and you have test data you want those tests to use. The data can be a CSV, Excel, JSON, YAML or XML file, or a table you type in. Shift-Left Studio sends the values into every existing test in the project, one test at a time. You see a preview first and can undo the change afterwards.

Explore the capability at Fill tests from a data set. To run one test once per row of data instead, see Data-driven testing.

Where to find it

On the Data Sets screen, or under Project Settings → Data Sets:

• Fill tests… in the "Fill tests from data" panel opens the dialog on the first data set.
• Fill tests… on a data set's row opens the dialog on that data set.
• Recent fills lists earlier fills. Each one has an Undo button.

You can also ask the Project Assistant, for example "fill our tests with the Users data set". It shows the same preview as an offer. Nothing is written until you accept it.

If your data is still a file, import it under Import a dataset first. The dialog works from data sets.

Two ways to use the data

What gets filled

• Columns are matched to fields by name. USER_ID, userId and user-id are treated as the same name. A dotted column such as address.city fills that nested body field.
• A column fills only a field the request already sends. It never adds a field. Columns that no request sends are listed as unused.
• The columns can fill:
  • path, query and form parameters
  • JSON body fields
  • GraphQL variables
  • JSON-RPC and WebSocket-RPC params
  • SOAP parameters and SOAP XML elements
  • ordinary headers
• Values are converted to the field's current type. "42" goes into a number field as 42. A cell that can't be converted is not sent, and the preview says why.
• Rows for each endpoint. If the sheet has a column that names the endpoint or operation (for example Operation = Add), each endpoint gets only its own rows.

What is left alone, and why

• A value a test chose on purpose. On one endpoint, a boundary test sends age: 151 and a wrong-type test sends age: "abc", while the passing tests share age: 30. Only the shared value is replaced. A value that only one test sends is what that test checks, so it is kept. To overwrite those values too, untick Keep values a test chose on purpose.
• A test that expects the API to refuse the request. Its fields are filled only where a passing test sends the same value. The bad value stays.
• Linked values such as {{env.token}} or {{userId}}. These come from an environment or an earlier step, and they stay linked.
• Credentials and transport headers: Authorization, Cookie, API-key-style headers, Content-Type, Accept and SOAPAction. Credentials belong in an authentication profile.
• INVALID_<column> columns. They are never written into a test that must pass. They drive negative data-driven tests.
• In fill mode, a test that already runs once per row of a data set is skipped. Its values would be replaced on every run anyway.

Checks that move with the data

• Repeated values. If a check repeats a value the request sent, it is updated to the new value. For example, if the check is "response name equals John" and the name is filled with Alice, the check becomes "equals Alice".
• EXPECTED_STATUS column. It decides which rows a test can take. A test that must pass only takes rows expecting a success status. In "run once per row" mode, each row checks its own status.
• EXPECTED_<path> columns. They set the value each row expects in the response.

Undo

Undo is in Recent fills. You can also ask the assistant, for example "undo the last fill" or "undo the fill from Users". It shows which tests it would restore before you accept.

Undo restores every test the fill changed. The one exception is a test someone edited after the fill: it keeps its current values, and Undo names it. A value the fill wrote counts as your edit, so regenerating tests keeps it.

Switching it off

An administrator of a self-hosted installation can switch the feature off by setting TEST_DATA_PROPAGATION_ENABLED=false.

Related

• Data-driven testing
• Data sources
• Project Assistant actions