AI Settings

The AI Settings module allows administrators to configure system-wide AI test generation, API models, and authentication keys. These settings define how the system leverages AI to automatically generate realistic, schema-compliant, and executable test cases across different scenarios. 

Key Sections  #

1. Test Generation Settings : 

• Test Types: Define the categories of tests the AI can generate (e.g., status, format, data, performance, security). 

• Custom Instructions: Provide advanced guidance for AI behavior. 

• • Force AI to return output in valid JSON arrays only. 

• • Require 4–6 test cases per request, covering happy path, negative tests, authentication, edge cases, and performance checks. 

• • Strict schema compliance ensures generated cases are executable without modifications. 

• Test Case Structure: 
  Each AI-generated test includes: 

• • Metadata: Name, description, HTTP method, endpoint, expected status code. 

• • Parameters & Headers: Concrete values (no placeholders). 

• • Request Body: Schema-compliant data (realistic values such as emails, IDs, ISO timestamps). 

• • Assertions: Status code, content type, response time, and critical field validation. 

• • Expected Response: Complete, realistic JSON response. 

• • Authentication: Bearer tokens or required auth configuration. 

2. Generation Options : 

• • Enable automatic test generation: AI suggests and creates test cases whenever a new schema or endpoint is defined. 

• • Enable smart test suggestions: AI proposes optimized edge cases and variations to improve coverage. 

3. AI Model & API Configuration : 

Admins can configure which AI model is used for test generation, smart suggestions, and schema compliance checks. The system supports multiple providers, enabling flexibility across OpenAI, Anthropic, Google Gemini, Meta, xAI, Mistral, Perplexity, and Azure OpenAI. 

• Model Selection : 
  
  
  • OpenAI Models (Highest Capability) 
    
    
    • GPT-4o – Latest OpenAI model with improved performance 

• • • GPT-4 – Most capable model for complex test generation 

• • • GPT-4o Mini – Fast and efficient for most tasks 

• • • GPT-3.5 Turbo 16K – Extended context window 

• • • GPT-3.5 Turbo – Fast and efficient for most tasks 
      
      
  • Anthropic Models (High Capability) 
    
    
    • Claude 3 – Advanced reasoning and analysis 

• • • Claude 3 Sonnet – Balanced performance 

• • • Claude 3 Haiku – Fast and efficient 
      
      
  • Google Gemini Models (High Capability) 
    
    
    • Gemini 1.5 Pro – Most capable model with advanced reasoning 

• • • Gemini 1.5 Flash – Fast and efficient 

• • • Gemini 1.0 Pro – Reliable and well-tested 
      
      
  • Meta Llama Models (High Capability)
     
    • Llama 3.1 405B – Most capable 

• • • Llama 3.1 70B – High performance 

• • • Llama 3.1 8B – Fast and efficient 
      
      
  • xAI Grok Models (High Capability)
     
    • Grok-2 – Enhanced reasoning capabilities 

• • • Grok-1 – Real-time information access 
      
      
  • Mistral AI Models (Medium-High Capability)
     
    • Nemo – Latest advanced model 

• • • Mixtral 8x22B – High capability 

• • • Mixtral 8x7B – Balanced performance 

• • • Mistral 7B – Fast and efficient 
      
      
  • Perplexity Models (Medium Capability)
     
    • Llama 3.1 70B – High quality 

• • • Mixtral 8x7B – Balanced performance 

• • • Llama 3.1 8B – Fast and efficient 
      
      
  • Azure OpenAI Models (Enterprise Capability) 
    
    
    • Azure GPT-4o – Enterprise OpenAI model 

• • • Azure GPT-4 – Enterprise OpenAI model 

• • • Azure GPT-4o Mini – Enterprise OpenAI model 

• • • Azure GPT-3.5 Turbo – Enterprise OpenAI model 
      
      
  • API Free Models (Basic Capability)
     
    • API Free Default – Free tier model 

• • • API Free Advanced – Enhanced free model 

• Configuration Parameters: 

• • Max Tokens: 4000 → Limits response size. 

• • Temperature: 0.7 → Balances creativity with accuracy. 

• • Timeout: 90s → Ensures requests do not hang indefinitely. 

4. API Keys Management

• API Key Security: Keys are encrypted and stored securely; only administrators can modify them. 

• Current Provider: OpenAI (Model: GPT-4). 

• API Key Status: Configured and validated. 

• Multi-Key Support: Add multiple keys and switch between providers or models seamlessly. 

Workflow Example 

1. Administrator configures AI Settings with test categories and custom rules. 

1. User selects an API schema and requests AI-generated test cases. 

1. AI returns JSON-only executable test cases with realistic values. 

1. Tests are automatically added to the project for execution. 

Benefits 

• Consistency: Every generated test follows a standardized structure. 

• Coverage: Includes positive, negative, edge, performance, and auth scenarios. 

• Efficiency: Eliminates manual test writing, accelerating test cycle creation. 

• Security: Keys and sensitive configurations are protected with encryption. 

⚡ With AI Settings, teams can transform raw API schemas into a comprehensive, executable test suite in minutes, ensuring higher quality and faster delivery.