Ask
Ask the work.
Retrieval over everything on this site: case studies, every upstream fix, the timeline. It finds and cites passages. It does not write answers, and it says so when nothing fits.
Loading the index…
Evaluation
26 questions · gold passage known
How often the right passage comes back.
Each question was written in different words from the passage that answers it, because an eval in the corpus’s own vocabulary only proves keyword search can find words it was given. Keyword results are computed at build time. Meaning and Both run here, in your browser, once the model is loaded. Twenty-six questions is a small set: a difference of one or two questions between methods is noise, not a finding.
| Question | Keywords | Meaning | Both |
|---|---|---|---|
| Recall@1 | 54% | — | — |
| Recall@3 | 69% | — | — |
| Recall@5 | 81% | — | — |
| MRR | 0.65 | — | — |
| Off-topic declined | 10/12 | — | — |
| Answerable withheld | 0/26 | — | — |
| What stops one applicant getting two hard credit enquiries? | 5 | · | · |
| What happens to applications while the credit bureau is down? | 1 | · | · |
| Why RabbitMQ rather than Kafka? | 2 | · | · |
| How is work not lost if the message broker is unavailable when a customer applies? | 8 | · | · |
| What are the weak points of the onboarding design? | miss | · | · |
| How are card numbers protected in the database? | 2 | · | · |
| What would wake someone up on call? | miss | · | · |
| What happens if the phone resends the same request? | 1 | · | · |
| Why do emoji and Hindi text get mangled when documents are chunked for embeddings? | 1 | · | · |
| How do you know the Unicode segmentation itself is right? | 1 | · | · |
| Which LangChain component failed the audit? | 1 | · | · |
| How does the code search engine avoid parsing the same file again? | 4 | · | · |
| Tell me about a concurrency bug you fixed | 6 | · | · |
| How can an agent find every caller of a function? | 1 | · | · |
| How do you stop an AI coding assistant declaring victory too early? | 1 | · | · |
| How do you keep an agent's prompt small? | 5 | · | · |
| Will the installer overwrite settings I already have? | 1 | · | · |
| Experience replacing SOAP web services | 1 | · | · |
| How much of the old banking code ended up under test? | 1 | · | · |
| Has he contributed to Prometheus? | 1 | · | · |
| Files cut short during upload but reported as successful | 1 | · | · |
| Which change made HTML partitioning faster? | 1 | · | · |
| Sentences disappearing after tables when scraping articles | 2 | · | · |
| Open-source work in Java | 2 | · | · |
| Where did he go to university? | miss | · | · |
| Is he looking for a job right now? | 1 | · | · |
rank of the first relevant passage in the top 10 · miss = not in the top 10