TestTeller Architecture

Overview

TestTeller features a modular architecture built around a dual-feedback RAG (Retrieval-Augmented Generation) system that enables continuous learning and improvement. The system consists of interconnected agents that share knowledge through a centralized vector store.

System Architecture Diagram

graph LR
    A[Generator Agent] --> B[Quality Assessment]
    B --> C[High-Quality Storage]
    C --> D[Vector Store]
    D --> E[Automator Agent]
    E --> F[Context Discovery]
    F --> G[Code Generation]
    G --> H[Validation Results]
    H --> D
    D --> A

Core Components

1. Generator Agent

Purpose: Intelligent test case generation with self-improving capabilities

Key Features:

Architecture:

2. Automator Agent

Purpose: RAG-enhanced test automation code generation

Key Features:

Architecture:

3. Core Engine

Purpose: Unified infrastructure for LLM integration and vector storage

Components:

Dual-Feedback Learning System

Feedback Loop Architecture

Generation Feedback

  1. Quality Assessment: AI-powered scoring (0.0-1.0) based on:
    • Completeness of test cases
    • Structure and organization
    • Detail level and actionability
  2. Intelligent Storage:
    • Only high-quality tests (>0.7 score) stored
    • Smart deduplication prevents redundant storage
    • Metadata enrichment for better retrieval
  3. Learning Integration:
    • High-quality content becomes training examples
    • Patterns learned for future generations
    • Context enrichment improves retrieval accuracy

Automation Validation Feedback

  1. Code Validation Results:
    • Syntax and framework compliance checking
    • Execution success/failure tracking
    • Performance metrics collection
  2. Metadata Enrichment:
    • Successful automation patterns stored
    • Framework-specific optimizations learned
    • Error patterns used for improvement
  3. Cross-Agent Learning:
    • Automation results inform test generation
    • Successful patterns enhance future code generation
    • Knowledge shared between Generator and Automator agents

Architectural Benefits

Self-Improving Capability

Cross-Agent Intelligence

Quality Control

Technical Implementation

Vector Store Architecture

Technology: ChromaDB with advanced querying capabilities

Features:

Storage Strategy:

Collection Structure:
├── Original Documents (requirements, specs, code)
├── Generated Test Cases (high-quality only, >0.7 score)  
├── Automation Results (successful patterns, framework configs)
└── Metadata (quality scores, timestamps, source information)

Document Processing Pipeline

Universal Parser

  1. Format Detection: Automatic detection of document types
  2. Content Extraction: Format-specific content extraction with structure preservation
  3. Intelligent Chunking: Context-aware splitting that maintains semantic meaning
  4. Metadata Enrichment: Document type, source, quality indicators

Enhanced RAG Ingestion

  1. Document Intelligence: Automatic categorization (requirements, test cases, API docs, specifications)
  2. Semantic Chunking: Preserves document structure and relationships
  3. Batch Processing: Concurrent processing with configurable performance settings
  4. Quality Assessment: Content quality evaluation during ingestion

LLM Integration Architecture

Multi-Provider Support

LLM Interface Layer
├── Google Gemini Integration
├── OpenAI Integration  
├── Anthropic Claude Integration
└── Ollama (Llama) Integration

Provider Abstraction:

Model Selection Strategy

Context Discovery Engine (Automator Agent)

9 Specialized RAG Queries

  1. API Discovery Query: “Find API endpoints, request/response schemas, authentication patterns”
  2. UI Pattern Query: “Discover UI component patterns, selectors, interaction methods”
  3. Authentication Query: “Identify authentication flows, token management, session handling”
  4. Data Model Query: “Extract data models, validation rules, schema definitions”
  5. Test Pattern Query: “Find existing test implementations and patterns”
  6. Similar Implementation Query: “Locate similar test cases and automation patterns”
  7. Configuration Query: “Discover environment configs, setup requirements”
  8. Error Handling Query: “Identify error patterns, exception handling, edge cases”
  9. Integration Query: “Find integration patterns, service communication, dependencies”

Query Optimization

Configuration & Environment Management

Environment Configuration

Configuration Layers:
├── Provider Configuration (API keys, models, endpoints)
├── Document Processing (chunk sizes, overlap, file types)
├── Vector Store Settings (host, port, collection management)
├── Quality Thresholds (scoring, storage, retention)
└── Output Configuration (formats, destinations, templates)

Modular Configuration System

Scalability & Performance

Performance Optimizations

Scalability Features

Security & Privacy

Data Security

Privacy Considerations