Your RAG Pipeline Can't Answer "Where Did That Come From?"

Every RAG system eventually gets the question. A customer, an auditor, a compliance officer, or just an engineer debugging a bad answer at 2am: where did that come from?

You can point to the vector store. You can show the retrieved chunks. But can you trace a specific chunk back to the exact bytes in the source document that produced it? Can you prove it hasn’t been silently corrupted? Can you do this for every chunk, deterministically, without an LLM in the loop?

Most teams can’t. And until someone asks, it doesn’t feel like a problem.

The provenance gap

Here’s what a typical RAG ingestion pipeline looks like:

document → chunking → embedding → vector store → retrieval → LLM → answer

Every step after chunking has tooling. Embedding models are benchmarked. Vector stores have indexes and metadata. Retrieval has rerankers. The LLM has evals.

But chunking — the step that actually decides what text becomes a retrievable unit — usually happens in a split_text() call that throws away all connection to the source. The chunk exists. The document exists. The link between them is gone.

This is fine until it isn’t:

• An AI agent surfaces a data point from a financial filing. The auditor asks which paragraph. You grep the source doc and hope.
• A healthcare chatbot gives a dosage recommendation. HIPAA wants an audit trail. Your trail starts at the embedding, not the document.
• A regulatory submission links each claim to a source. Your chunker split the source mid-sentence. The link points to a fragment that doesn’t mean what the claim says.

Why “just add metadata” doesn’t work

The common fix is to attach metadata to chunks — page number, section header, maybe a character offset. This is what LangChain’s add_start_index parameter is supposed to do. It doesn’t work reliably.

LangChain’s implementation splits the text first, then uses text.find(chunk) to locate each chunk in the source afterward. This breaks on duplicate substrings (every chunk gets start_index: 0), on token-based splitters (character offsets don’t match token boundaries), and on overlapping chunks. There are multiple open issues documenting this, some closed as “not planned.”

The architecture is wrong. You can’t split first and search later. Offsets have to be tracked during the split, or duplicates and encoding mismatches will always produce garbage.

Three deeper problems with the metadata approach:

Offsets drift. Character offsets from a PDF parser depend on the parser version, encoding handling, and whitespace normalization. Parse the same PDF six months later with an updated library and your offsets point somewhere else. Byte ranges against the raw file don’t drift.

No integrity check. Metadata says “this chunk came from page 12.” But did the content actually come from page 12? There’s no hash to verify. If a preprocessing step silently mutated the text — normalized Unicode, stripped whitespace, truncated — the metadata still says page 12.

No quality signal. Metadata describes where a chunk came from, not whether it should exist. A three-word fragment from a table boundary gets the same metadata as a well-formed paragraph. The downstream pipeline has to discover the problem itself, usually by producing a bad answer.

What chunk-level provenance looks like

Every chunk should carry a receipt: here’s exactly where I came from, and here’s proof I haven’t changed.

{
  "content": "Patients receiving the 200mg dose showed...",
  "source": {
    "file": "trial-results.txt",
    "byteStart": 14832,
    "byteEnd": 15091,
    "contentHash": "sha256:a3f2e8..."
  },
  "assessment": {
    "verdict": "pass",
    "confidence": {
      "boundaryScore": 0.95,
      "completenessScore": 0.92,
      "hashVerified": true
    }
  }
}

The byte range is exact — you can slice the source file at those offsets and get the chunk content back, byte for byte. The SHA-256 hash proves it. No fuzzy matching, no “most likely from page 7.”

The assessment is a quality gate before anything downstream touches the chunk. A chunk that got split mid-word? Flagged. A chunk that’s just a header with no content? Flagged. A chunk where the hash doesn’t verify against the byte range? Rejected. This happens at ingestion time, not after the LLM hallucinates from a bad chunk.

Try it

npx @watthem/stela README.md

No signup, no API key, no config file. It reads the file, chunks it, and prints every chunk with its provenance and assessment verdict. Every chunk’s byte range is verified against the source before it leaves the pipeline; if the bytes don’t match, it throws rather than emitting false provenance.

npx @watthem/stela --strategy heading --json report.txt

stela works on UTF-8 text. Extract from PDF or DOCX first — byte ranges then refer to a stable text artifact rather than a parser’s shifting interpretation.

It also works as a library:

import { run } from "@watthem/stela";

const result = run(documentText, {
  strategy: "paragraph",
  file: "source.txt",
});

for (const chunk of result.chunks) {
  console.log(chunk.text);
  console.log(chunk.source.contentHash);
  console.log(chunk.source.byteStart, chunk.source.byteEnd);
  console.log(chunk.assessment.verdict);
}

For exact disk-file provenance, pass readFileSync(path) as a Buffer instead of a decoded string.

When this matters

You don’t need chunk provenance for a chatbot that answers questions about your docs. You need it when the answer has consequences:

Regulated industries. FDA submissions, SOX audits, HIPAA-covered records, insurance underwriting. When an AI-surfaced claim traces back to the wrong paragraph, the cost isn’t a bad user experience — it’s a compliance finding.

Multi-document pipelines. When chunks from dozens of source files end up in the same vector store, provenance is the only way to debug “where did the model get that?”

AI agents making decisions. An agent that processes denial letters or prior authorization docs needs to trace every decision back to the source paragraph. Not for the user — for the engineer who debugs it when the agent gets it wrong.

The EU AI Act and recent state-level AI legislation are moving toward requiring documentation of data provenance in high-risk AI systems. The regulation is coming, but the engineering reason is already here: you can’t fix what you can’t trace.

What stela is not

It’s not a vector store. It’s not a retrieval engine. It’s not an embedding model. It’s the preprocessing layer that runs before all of those, and it gives every chunk a receipt that survives the rest of the pipeline.

If your RAG stack works fine without provenance, it’ll keep working fine. But the first time someone asks “where did that come from?” and you can answer with an exact byte range and a cryptographic hash instead of “somewhere in that PDF,” you’ll understand why this layer exists.

---

stela is on npm. Node.js 22+. Zero runtime dependencies. Deterministic. Free to use.