case · 008
Hybrid retrieval RAG pipeline.
A search engine that reads a folder of documents and answers natural-language questions in plain English, with citations pointing to the exact passage each fact came from. Combines ChromaDB dense retrieval with BM25 sparse retrieval via Reciprocal Rank Fusion (RRF), then re-ranks with a cross-encoder, then generates the answer with inline citations and a confidence score. Ships with a golden eval dataset and an evaluation harness that catches chunking/embedding regressions.
fig. 0.1
Cover image · retrieval pipeline diagram
Traditional keyword search finds files that *might* contain the answer; the user then opens them and reads through. Our pipeline finds the specific passage, reads it, and gives back a direct answer with a citation the user can open to verify.
Real uses from the project's own README: ingest a contract and ask "who owns the IP after termination", ingest a codebase and ask "how does the transcription pipeline work", ingest a folder of research papers and ask "what did we decide about chunking", or ingest a resume + job listing and ask "which of my projects match a Senior AI Engineer role".
retrieval modes (BM25-only or dense + BM25 + re-rank)
eval metrics (faithfulness, citation accuracy, retrieval, calibration)
unit tests covering the API contract
of generated answers ship with source citations
The pipeline runs in two modes. The zero-config BM25-only mode works without an API key, retrieving by keyword match with full BM25 scoring and producing citations against matched chunks. The full hybrid mode adds ChromaDB dense vectors (configurable embedding model via OpenRouter), fuses dense + sparse by RRF with tunable weights (default 0.7 dense, 0.3 sparse), and re-ranks with a cross-encoder before LLM generation.
Citation verification is built in — every generated answer surfaces source chunks, which means a downstream eval can grade faithfulness, citation accuracy, retrieval relevance, and confidence calibration. The harness ships in-tree with a golden Q&A dataset, so chunking or embedding regressions are caught in CI rather than at the user. FastAPI service with `/v1/ask`, `/v1/ingest`, `/v1/stats`, Dockerfile, and 12 unit tests cover the API contract end-to-end.
/ stack
- n/a — off-chain information retrieval
- Python
- ChromaDB (dense)
- BM25 (sparse)
- Cross-encoder re-ranker
- OpenRouter (LLM)
- FastAPI server
- Docker + docker-compose
- pytest
- golden dataset
next case study
AI-augmented MT5 scalping bot..
An autonomous MT5 bot that runs technical-indicator analysis (RSI, MACD, Bollinger Bands, ATR) and cloud-LLM reasoning in parallel, then executes trades with hard daily-loss and drawdown limits behind a WebSocket operator dashboard. Every decision logged with its AI reasoning and indicator state.
/ start a project