ZephAI — Model Gateway

Active Configuration

Loading...

Switch Provider

Ollama (Local)
Runs on your server. Free, private, slower on CPU.
Active
Claude API
Anthropic cloud API. Fast, high quality, pay-per-use.
Active
Azure OpenAI
GPT-4 via Azure. Enterprise SLA, your subscription.
Active
AWS Bedrock
Claude & Llama via AWS. IAM auth, pay-per-token.
Active
OpenRouter
200+ models via one API. Gemini, GPT-4o, Llama, Mistral & more.
Active
OpenAI (Direct)
GPT-4o, o1, o3 via direct OpenAI API. Enterprise contracts & ZDR.
Active
Google Gemini (Direct)
Gemini Pro / Flash via Google AI Studio. Direct contract, 1M context.
Active

Performance

🚩 These toggles are backed by the Feature Flags system. Changes made here or there are immediately in sync.
Cross-Encoder Reranking
Improves result quality by rescoring candidates. Adds ~20s on CPU — disable when using fast cloud providers (Bedrock, Claude).
Contextual Retrieval §2.4
Prepend AI-generated document context to each chunk before embedding. Improves recall for queries matching document theme. Applies to new uploads only — adds 1 LLM call per ingested document.
Multi-Hop Retrieval §3.3
Decompose complex queries into 2–4 sub-questions, search each independently, and merge results via RRF. Best for multi-faceted analytical questions. Adds 1 extra LLM call per query.
HyDE (Hypothetical Document Embeddings) §3.3
Generate a hypothetical ideal answer and embed it instead of the raw sub-question. Improves recall for abstract queries where question vocabulary differs from document vocabulary. Requires Multi-Hop enabled. Adds 1 LLM call per sub-question.
Graph RAG §3.1
Augment vector retrieval with Knowledge Graph traversal — extracts entities from the query, follows KG edges, and merges linked chunks via RRF. Build the KG first via Knowledge Graph → Build. Default OFF.
Multi-Modal Ingestion OCR + Vision
Enable Azure Document Intelligence OCR for scanned PDFs + Azure AI Vision / LLM captioning for images (PNG, JPG, TIFF, BMP, WEBP). Requires AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT and AZURE_DOCUMENT_INTELLIGENCE_KEY in .env. Default OFF.
Chunking Strategy
Controls how new documents are split at ingest time. Applies to the next upload — existing indexed documents are not re-chunked.

Semantic Cache

Enable Semantic Cache
Return cached answers for semantically similar queries — 30–50% fewer LLM calls, sub-100ms for hits.
0.80 (broad)0.99 (strict)
Cache Stats
—
Hits
—
Misses
—
Hit Rate
—
Entries

Runtime Config

Override env-var parameters at runtime without restarting containers. Changes take effect immediately and persist until reset. Redis-backed

Usage Stats

No requests recorded yet.

Query Intent Breakdown

Loading…