System

The mechanics behind every answer.

This is the architecture reference: seven governed stages, budget rules, and routing logic. For what it actually produces, see Console and Evaluation.

What this architecture has producedMeasured

Full evaluation →

Faithfulness, avg

0.000

Relevancy, avg

0.000

Filings indexed

Chunks indexed

Execution graph

Measured

Model

LLaMA 3.1 8B

Input

Original query

Output

1-3 step research plan

Behavior

Skipped on YELLOW/RED tier

Reflector decision

is_grounded=True OR loops≥maxEmit response

needs_more_info=TrueLoop: rewrite

is_grounded=FalseLoop: re-reason

error in stateFallback

Budget governance

Measured

GREEN0 – 60k

Full pipeline. All nodes active.

Max loops: 6
Retrieval depth: 8
Model: llama-3.3-70b-versatile

YELLOW60k – 85k

Planner skipped. Reduced loops and retrieval.

Max loops: 3
Retrieval depth: 5
Model: llama-3.3-70b-versatile

Bypasses: Planner

RED85k+

Minimum viable path. Budget preservation mode.

Max loops: 2
Retrieval depth: 4
Model: llama-3.1-8b-instant

Bypasses: Planner, Heavy Model

Model routing table

Measured

Task	Applies at	Model	Rationale	Est. cost
Planner	ALL	llama-3.1-8b-instant	Simple structured task	~800 tokens
Query Rewriter	ALL	llama-3.1-8b-instant	JSON structured output	~1,200 tokens
Reasoner (GREEN)	GREEN/YELLOW	llama-3.3-70b-versatile	Complex synthesis + citations	~3,500 tokens
Reasoner (RED)	RED	llama-3.1-8b-instant	Budget-constrained synthesis	~2,000 tokens
Reflector	ALL	llama-3.1-8b-instant	JSON hallucination check	~1,000 tokens
Responder (fallback)	ERROR	llama-3.3-70b-versatile	Graceful degradation	~2,500 tokens

StaleReflects src/optimization/config.py's configured model IDs as of the last precompute run. Groq has since retired the Llama 3.x IDs shown here (confirmed against the live /v1/models endpoint, see deployment_roadmap.md §9 Phase 2) -- the live serverless chat route now calls openai/gpt-oss-120b and openai/gpt-oss-20b instead; this table has not been repointed at those.

Response cache

Measured

Similarity Threshold: 0.92 cosine sim
Max Cache Size: 500 entries (LRU)
TTL: 3,600 seconds (1hr)
Embedding Model: all-MiniLM-L6-v2
Thread Safety: RLock protected
Eviction Policy: Least-recently-used
Shared Model: Yes: avoids reload across components

Context window

Measured

Top-K Chunks Maximum chunks injected per query: 8
Max Chunk Chars Individual chunk character hard cap: 1500
Max Total Tokens Total context budget across all chunks: 6,000
Relevance Floor Minimum cosine sim to include a chunk: 0.15
Truncation Method Splits at '.', '!', '?' before hard cut: Sentence boundary
Token Estimate Fast approximation (configurable): 4 chars ≈ 1 token