Suppose you hold an archive covering a subject from 1941 to 1944. You have one document from 1941, one from 1942, one from 1944, and nothing at all from 1943. Now ask your search system: "what am I missing?"
Full-text search cannot answer. Neither can a vector database, an embedding model, or a RAG pipeline built on either. Not because they are badly built — because the thing you are asking about is not a document, and every one of those systems is a function from a query to a ranked list of documents. The 1943 material is not ranked low. It does not exist to be ranked.
Absence is a property of a set
The reason this is hard is worth being precise about. A gap is not visible in any individual record. It only appears once you know the shape the set should have had — that these documents form a series, that the series is indexed by year, and that the years run continuously.
That makes it an aggregate question, not a retrieval question. It is the sort of thing databases have been excellent at since the 1970s and that nobody thought to point at a document corpus, because we spent twenty years assuming the interesting question about documents was always "which ones match."
Four absences worth reporting
In practice the useful gaps fall into four categories, and they generalise across wildly different corpora:
- A hole in an otherwise covered span. You have 1942 and 1944. Where is 1943?
- A claim resting on one source. Nothing corroborates this. In a literature review it is the single most important property of a finding, and relevance ranking actively hides it.
- An entity connected to nothing. Named once and never placed — either a dead end or the loose thread you have been looking for.
- Material never read. Held but never extracted, and therefore invisible to all three questions above. This is the most common finding in any real archive and the one nobody has a report for.
The false-positive trap
The naive implementation of gap detection is worse than useless, and it is worth explaining why, because it is the failure mode that makes people distrust the whole idea.
If you report a gap wherever two data points have space between them, then a single mis-parsed date will manufacture decades of imaginary leads. Indexing this very project produced exactly that: a stray 1800 extracted from a 2026 document, generating two centuries of confident nonsense.
So the rule is three data points, not two. Two years establish no pattern; three establish a series that a gap can interrupt. A tool that raises false leads in a research corpus is not a neutral inconvenience — it spends somebody's weekend in the wrong archive.
Report, never explain
There is a strong temptation to say why a gap exists. Resist it completely. The absence of 1943 material might mean the records burned, the unit was disbanded, the archive was never digitised, or nobody has looked yet. Software cannot distinguish these and should not pretend to.
Conservators have had this discipline for centuries. A lacuna in a manuscript is recorded, never filled — because a documented absence is a lead and an invented certainty is a lie that outlives the person who invented it. A tool that guesses at meaning in a research corpus is not a tool. It is a source of errors with a confident interface.
Doing it offline
Every result above is set-based SQL over data you already hold. No model, no API key, no network call. That is not only a privacy property, though it is that too. It means the results are reproducible — the same corpus produces the same gaps tomorrow, and they do not shift because a vendor shipped a new checkpoint.
For a research group that matters more than it might sound. An institution that can only reason about its own archive while somebody's subscription is current does not really own that archive.