AI Settings

Overview

AI Settings is the system-wide control panel for AI-assisted test generation: whether the AI engine is on, which provider and model are used, the model parameters, and the encrypted API keys. These settings apply to the whole system—to control what gets generated for a single project, see Test Generation Settings.

Where to find everything: Administration Settings. Explore the capability at multi-provider AI.

Before you begin

• You must be an Administrator. Non-admins see Access Denied ("You need Administrator privileges to access AI settings.").
• Have the provider API key ready for the model you plan to use. Keys are encrypted at rest and visible only to administrators.
• Nav path: Settings → Integrations → AI Settings. The page header reads AI Settings with Reset to Defaults and Save Settings at the top-right.

Step 1 — Turn the AI engine on

In the Use AI Engine card, toggle the switch on. The help text reads "When enabled, Selected AI Model will be used for test generation. When disabled, default test generation engine will be used." With it off, generation falls back to the deterministic engine everywhere.

Step 2 — Configure Test Generation Settings

The Test Generation Settings card holds system-wide generation defaults:

Step 3 — Pick a model provider

In AI Model & API Configuration, use the Model Provider dropdown. There are ~20 provider families: OpenAI, Anthropic, Google Gemini, Meta Llama, xAI Grok, Mistral AI, Perplexity, Azure OpenAI, API Free, Cohere, Together AI, Groq, DeepSeek, AI21 Labs, Replicate, Hugging Face, and Self-hosted (Ollama / LM Studio / vLLM).

Choosing a provider selects its first model by default. A Security Notice on this card reminds you that keys are encrypted and admin-only.

Self-hosted (Enterprise only)

Self-hosted (Ollama / LM Studio / vLLM) is an Enterprise capability. On non-Enterprise plans the option shows "— Enterprise only" and selecting it raises "Self-hosted LLM is available on the Enterprise plan only." with an Upgrade link. When available, selecting it reveals:

• Base URL (OpenAI-compatible /v1) — with Quick fill buttons for Ollama (http://localhost:11434/v1), LM Studio (http://localhost:1234/v1), and vLLM (http://localhost:8000/v1). From Docker, use http://host.docker.internal:<port>/v1.
• An Advanced — embedding model section (collapsed) with an optional Embedding model field (default nomic-embed-text) used only for offline RAG / semantic search when no cloud embedding key is set.

Step 4 — Choose the specific model

Under Specific Model:

1. Pick a model from the dropdown. Groups are Current, ✓ Available for your key (or ✓ Installed on your server for self-hosted), ★ Custom (added), and — Catalog / — Presets.
2. Click Refresh available models to query the provider for the models your key/server can actually use (tooltip: "Query this provider for the models its key can use"). Tick Show all to include snapshots, previews, and legacy models.
3. To use a model not in the catalog, choose ➕ Add a custom model…, type the model id, and press Enter. Custom models appear as removable chips (click × to remove).

Step 5 — Set model parameters

In the three parameter fields:

Step 6 — Manage API keys and test the connection

Under API Keys Management:

1. Enter the key in the {Provider} API Key (Currently Selected) field—the label and placeholder match the selected model's provider (e.g. sk-..., sk-ant-..., AIza...).
2. Click Test Connection (label shows Testing…). This "Sends a tiny ping to {provider} to verify the key and model. Nothing is saved." Success shows ✓ Connection successful (with the reply); failure shows ✗ Connection failed plus a Code. A successful test offers Save These Settings.
3. Expand Edit All API Keys (Hide to collapse) to manage every provider's key in one place, including a Self-hosted API key (optional) and Self-hosted Base URL.
4. Azure OpenAI models additionally require Azure OpenAI Endpoint (e.g. https://your-resource.openai.azure.com/) and Azure OpenAI Deployment (the deployment name).

The Selected Model Information box summarizes the model, provider, required key, and Key Status (Verified, Key set — not verified yet, or Not configured). Changing the model, a key, the base URL, or Azure endpoint/deployment clears a prior verification.

Step 7 — Save (or reset)

• Click Save Settings (top-right; shows Saving…). On success: "AI settings saved successfully."
• Reset to Defaults opens a confirmation ("Are you sure you want to reset all AI settings to defaults?…"); confirm with Reset. This cannot be undone.

Troubleshooting

• Access Denied — you're not an Administrator; ask an admin to make the change.
• ✗ Connection failed — check the key, model, and (for Azure/self-hosted) endpoint/base URL; the Code line names the failure. Re-run Test Connection.
• Key Status: "Not configured" / "Key set — not verified yet" — enter the key and run Test Connection before saving so generation can use it.
• A key field is blank with a warning after an update — the stored ciphertext couldn't be decrypted (typically after an encryption-key change); re-enter the flagged provider keys and save.

Best practices

• Keep one stable model/configuration per environment so results are comparable.
• Never commit API keys to source control or share them in tickets; rotate on your standard schedule.
• After changes, run Test Connection, then spot-check AI-generated tests on a sample endpoint.

Related articles

• AI-generated tests
• Test Generation Settings
• User Policies