Who buys it
Engineering leadership paying for AI coding tools, and whoever owns the rule that source code cannot leave the network.
Business case
Retrieval layer for AI coding agents
Who uses it
Developers running AI coding agents against repositories too large for the agent to hold in context.
The problem
What this costs you today.
An agent without retrieval reads whole files. Each cold read costs roughly 740 tokens of injected context on average, most of it irrelevant to the question being asked, and the cost is paid on every tool call by every developer.
The second cost is worse and harder to measure: the agent confidently edits the wrong thing. Deprecated modules and data models that share the right vocabulary are exactly what naive similarity search surfaces first.
The mechanism
What changes.
The repository is indexed once with AST-aware chunking, so chunks follow function and class boundaries rather than character counts. Hash-based incremental ingestion means only changed files are re-embedded.
A hook intercepts the agent's tool calls and injects the top eight chunks before the query runs. Hybrid retrieval combines vector search and BM25 with reciprocal-rank fusion, then a late-interaction reranker compresses 30 candidates to 8.
Everything runs locally through an ONNX runtime, which removes the PyTorch dependency entirely — about 2 GB of RAM saved and the largest latency source off the critical path. No source code leaves the network.
The hook fails open on a two-second budget. If retrieval is slow or unavailable, the agent's normal file read proceeds and nothing breaks.
Measured
What was actually observed.
These are observed figures from the tool running against real repositories, not projections. The saving is per retrieval, and it is a 29% reduction in injected context — not the order-of-magnitude claim this kind of tool is usually sold on.
| Metric | Observed | Basis |
|---|---|---|
| Average injected tokens per hook call | ~524 | Observed in running use |
| Average cold file read (baseline) | ~740 | Same workload without retrieval |
| Tokens saved per retrieval | ~216 (~29%) | Difference between the two above |
| Query pipeline latency, end to end | ~150 ms | ~30 ms embed, ~50 ms vector search, ~20 ms keyword, ~50 ms rerank |
| Memory saved by ONNX over PyTorch | ~2 GB | PyTorch dependency removed entirely |
| Corpus indexed in current use | 16 projects, 380+ chunks | Across 5 typed collections |
| Hook failure budget | 2 s, fail-open | Agent's normal read proceeds on timeout |
Where the money is
Your numbers, not ours.
Defaults below are the measured per-retrieval figures. Only volume and price are yours to set — change those and leave the measured values unless you have your own benchmark.
Every figure above is arithmetic on the values you entered. Nothing is a measured result from a prior deployment, and no customer savings are claimed anywhere on this site.
The honest part
When not to buy this.
- Repositories small enough that an agent can hold the relevant parts in context. Retrieval adds operational load and buys nothing.
- Teams of one or two. The indexing infrastructure is not worth maintaining at that scale.
- Anyone expecting the token bill alone to justify it. At a 29% per-retrieval saving the direct spend case is real but small; buy this for precision and for keeping code in-network.
- Organisations comfortable sending source to a hosted indexing service, who will find managed options cheaper to run.
Adoption
What it actually takes.
Prerequisites
A host for the vector store and the local embedding service. No GPU required, and no PyTorch, since both models run through ONNX.
Setup effort
Initial index is minutes for a mid-size repository; incremental runs re-embed only changed files. Hook configuration is per developer machine.
Ongoing cost
Index scope needs an owner. Anything indexed can be surfaced to an agent later, so adding a directory is a data-handling decision, not a config detail.
Engagement
How this would be delivered.
- Build and hand over, run in your own infrastructure.
- Supported deployment with index-scope policy defined for your repositories.
Caveat: The measured figures come from this tool running against real repositories, including this one. Your codebase, languages, and query patterns will move them. A one-week trial on one repository is the only way to know your own numbers.
Next step
Run the demo, then tell us your numbers.
The fastest way to test any of this is against your own volumes rather than the placeholders above.