We built an event-driven AI engine on AWS that routes user questions to either SQL database queries, RAG document search, or a hybrid execution path combining both.
Core Architecture & Stack
- Amazon Bedrock Multi-Model Setup: Uses Claude Haiku 4.5 to classify user intent (SQL, RAG, or Hybrid). For Text-to-SQL generation, query formatting, and answer synthesis, the engine uses Claude Sonnet 4.6. Embeddings are generated using Titan Embed Text v2.
- Asynchronous API Gateway & SQS: A WebSocket API receives client questions and immediately places job IDs onto an Amazon SQS queue, returning HTTP 202. An orchestrator AWS Lambda processes the job and streams tokens back over the WebSocket.
- Automated ETL & Textract Pipeline: Uploaded documents in S3 trigger a Lambda pipeline. Standard text PDFs are parsed natively , while scanned files run through Amazon Textract. Invoice prefixes trigger Textract AnalyzeExpense to extract vendor metadata and logo text.
- Semantic Layer: A YAML-based schema definition in S3 maps database entities, joins, business vocabulary, and sample SQL pairs, steering the LLM without requiring structural changes to the client's database.
- Observability & Telemetry: Engineered structured JSON logging and agent-trace telemetry, enabling the client's engineering team to securely monitor LLM reasoning steps, generated SQL queries, and execution latency directly through AWS CloudWatch.
Key Findings & Optimizations
During the development, several technical optimizations significantly impacted the final architecture:
- Intent Routing Efficiency: Offloading query classification to Claude Haiku reduced latency and token usage , reserving Claude Sonnet for SQL construction and answer generation. An environment toggle allows running the entire pipeline on Haiku , cutting model costs by 61% with a minor drop in official evaluation score (from 70% to 63%).
- Targeted Textract Strategy: Running AnalyzeExpense ($0.01/page) only on designated invoice/work-order prefixes —and falling back to standard OCR ($0.0015/page) or native text extraction elsewhere —prevented unnecessary document processing costs.
- Predictable Cost Modeling: We established a transparent cost framework based on user activity profiles (Light, Medium, Heavy). This allowed the client to accurately project per-query costs across different model configurations (Sonnet vs. Haiku) and decouple variable LLM expenses from fixed infrastructure costs.
