What Is Synthetic Test Data Generation, and When Is It Enough?

This is lesson 6.5 of Test Data, Mocks and End-to-End API Testing. Lesson 6.4 listed synthetic data as one strategy; this lesson shows how it's made and where it stops being enough.

What is synthetic test data generation?

What is synthetic test data generation? It's creating artificial records that follow the shape, types and rules of real data but describe no real person, account or order. A synthetic customer has a plausible name, a valid-format email and a believable age, and none of it was copied from production. Because nothing is real, synthetic data can go into any environment, any log and any bug report without a privacy review.

The synthetic test data generation meaning, in four approaches

For API functional testing, the first two usually cover what you need. Statistical and AI approaches earn their place when realism itself is under test, such as a fraud rule or a search ranking.

A worked example: synthetic users for the store

The sandbox store's POST /api/v1/users needs a name and an email, and accepts an optional role and age from 0 to 150. A synthetic batch for a data-driven run might be:

Notice three deliberate choices:

1. A reserved domain. example.com is set aside for documentation and testing, so a test that sends a real email never reaches a real person.
2. A number in the email. Small name lists collide quickly; a number, a sequence or a UUID keeps values unique where the API requires it.
3. Boundary ages on purpose. Random values rarely land on 0 or 150. Synthetic data for testing mixes random rows with chosen edge rows.

Seeds: synthetic, but repeatable

A random generator with no seed produces a new batch on every run. When a test fails on row 3, you can't send row 3 again. A seed fixes the sequence: the same seed gives the same rows, a different seed gives a different set. Seed the batch you run in CI, and change the seed on purpose when you want fresh coverage.

Two things usually stay unseeded: UUIDs and dates relative to "now". If a test compares them, generate them once and store them.

Referential integrity: the hard part

Single records are easy. Related records are not. An order needs a user_id and a product_id that exist. Generating three columns of random UUIDs gives orders that point at nothing. In the sandbox that's allowed (an order for an unknown user is accepted, and its expanded user comes back null), but in most systems it's a 400 or a foreign-key error. Generate parents first, capture their IDs, then generate children that reference them. That's a workflow, not a spreadsheet (see lesson 6.6).

Synthetic test data generation using generative AI

A language model can turn "twenty customers from Nordic countries, some with long double-barrelled surnames, two with no phone number" into rows in seconds. Use it like this:

1. Give the model the schema and the rules, not production samples.
2. Validate every row against the schema; reject, don't repair.
3. Check uniqueness and the edge cases you need; add the ones it missed by hand.
4. Freeze the accepted rows as a file, versioned with the tests.

Where the prompt goes matters for regulated teams: a hosted model receives whatever you send it. If even the rules are sensitive, run the model inside your own network.

Synthetic test data vs production data

Synthetic data is enough for functional, contract and most negative testing. It's not enough when the test depends on the real distribution of values: a search ranking, a fraud threshold, a migration of twenty years of records. For those, a masked subset of production, handled under your data-protection rules, is the usual answer.

Common mistakes

• Random without a seed in CI. Failures you can't replay.
• Realistic-looking but invalid. A generated postcode in the wrong format tests your validator, not your feature.
• Real domains in emails. Use a reserved domain such as example.com.
• Forgetting the edges. Random data clusters in the middle of every range.

Synthetic users in Shift-Left API

Shift-Left API includes a built-in Random generator as a data source for data-driven tests. It's rule-based: values come from built-in lists and patterns, with no external library and no AI model, so it works on every edition. Fake card numbers and IBANs have the right shape and aren't valid numbers, and generated emails use example.com.

1. Import https://demo.totalshiftleft.ai/openapi/rest.json and make a test for POST /api/v1/users that checks for a 201.
2. Open the test's Data tab and switch Data-driven testing on.
3. Under Data source, choose Random generator. Set How many rows to 5. Seed (for repeatable data) is optional: left empty, the generator uses a fixed default, so the rows are the same on every run; type a number to get a different, equally repeatable set.
4. Under Fields to generate, click + Add field twice: name with fullName, and email with email. The picker has 26 generators, among them int, float, date, pastDate, futureDate, company, city, phone, postalCode, uuid, sequence and pattern (# for a digit, ? for a letter).
5. Click Auto-map columns to bind name and email to the body fields of the same name, then Preview iterations to see the rows.
6. Run the test. Each row is its own result, numbered 1 to 5 of 5, with a Data Row panel showing what was sent.

If the generator doesn't have what you need, import your own synthetic sheet as a data set instead, as in lesson 9.7. The data-driven testing feature page lists every source.

Try it yourself

On a Create a user test, switch on Data-driven testing, pick Random generator with 5 rows of name (fullName) and email (email), auto-map, preview and run. Then run it again without changing the seed and compare the names.

Recap

• What is synthetic test data generation: making artificial records with real data's shape and rules and none of its people, so they can go anywhere.
• Synthetic test data generation tools range from seeded random generators to statistical models; for API functional tests, seeded and schema-driven generation usually covers it.
• Synthetic test data generation using generative AI adds realism; validate, freeze and version what it produces.

For the full comparison with masked production data, read synthetic test data vs production data.

Next: lesson 6.6, how to do end to end API testing.