Preparing Source Documents for Retrieval: Cleaning, Chunking, Overlap, and Metadata

Hello. In the previous lesson, you separated a RAG prototype into small, reviewable functions for embedding, source formatting, and grounded generation—and kept secrets outside the notebook. That scaffold is only as reliable as the evidence it receives. A perfectly written grounded prompt cannot repair a document that was parsed badly, split at the wrong boundary, or stripped of its provenance.

This lesson focuses on the preparation layer between raw documents and embeddings. By the end, you should be able to specify and inspect a document-preparation policy covering cleaning, chunking, overlap, and metadata—the decisions that determine what a retriever can later find and what an answer can credibly cite.

Plan for about 40 minutes: roughly 8 minutes of video, 12–15 minutes of reading, and the remaining time applying the ideas to a practical RAG corpus.

---

The retrieval unit is a design decision

A source document is rarely the right unit to retrieve. A 40-page policy, product manual, or incident runbook may contain one relevant paragraph and many pages of distraction. Supplying the full document to an LLM wastes context capacity, increases cost, and makes it harder for the model to identify the controlling evidence.

Instead, RAG normally retrieves chunks: small, self-contained units of text that are later embedded and ranked against a question. The important word is self-contained. A chunk should retain enough context that, if it is retrieved alone, a reader can tell:

• what document it came from;
• what topic or section it addresses;
• what claim, rule, procedure, or explanation it contains;
• where to find the original source.

The document-preparation pipeline has several connected decisions. The following image provides a useful high-level checklist.

For an engineering leader, the key point is that chunking is not merely a library setting. It is a product and quality decision. If the assistant is intended to answer narrow policy questions, its chunks should make narrow rules easy to retrieve. If it supports open-ended technical research, chunks may need somewhat more surrounding explanation. The choice should be explicit, testable, and recorded.

---

Preserve meaning before you optimize retrieval

Preparation begins with parsing, then cleaning. These are related but different operations.

• Parsing extracts usable content from a source format such as PDF, HTML, Markdown, a Word document, a scanned image, or a transcript.
• Cleaning removes artifacts that do not carry useful meaning for the intended RAG use case and normalizes the remaining text into a consistent representation.

A PDF might visually contain headings, columns, tables, page numbers, footnotes, and headers. A simplistic parser can turn that into a misleading stream of text: a footer may appear in the middle of a sentence; a table’s headers may become disconnected from its values; a two-column page may be read across rather than down. Likewise, HTML includes navigation menus, cookie notices, and repeated page chrome that users never mean to ask the assistant about.

The objective is not to make text look cosmetically tidy. It is to preserve the source’s meaning while removing retrieval noise.

A conservative cleaning policy

Start by retaining an immutable or access-controlled copy of the raw source, then create a separate canonical text representation for chunking. The canonical version should make the transformations inspectable and repeatable.

Common useful cleaning actions include:

Source issue	Usually appropriate action	Important caution
Repeated headers, footers, page numbers	Remove when clearly templated noise	A page number can be valuable provenance; retain it as metadata
HTML navigation, sidebars, cookie banners	Exclude from extracted main content	Do not accidentally remove substantive notices or release notes
Inconsistent whitespace and line breaks	Normalize spacing and paragraph breaks	Preserve meaningful list, code, and table boundaries
OCR artifacts	Correct obvious extraction errors when verified	Do not silently “correct” technical terms or numbers
Repeated copies of the same document	Deduplicate, selecting an authoritative version	Keep version history and source provenance
Scanned tables or diagrams	Preserve labels and relationships where possible	Flat text may lose meaning; validate with a human sample

Avoid destructive “cleanup” rules such as removing all punctuation, lowercasing everything, deleting all numbers, or stripping text that looks repetitive. In a technical or operational corpus, punctuation, capitalization, version strings, error codes, and numeric thresholds can be the entire answer.

For example, consider the following policy text:

External Customer Notices
For a SEV-1 incident affecting regulated markets, Legal must approve any external customer notice before publication.
The executive liaison coordinates leadership and customer communications.

The heading identifies the scope. “SEV-1,” “regulated markets,” “Legal,” and “before publication” are conditions and constraints. Removing any of them can turn a specific rule into an unsafe generalization.

A useful inspection question is: Would this cleaned output let a reviewer reconstruct the same operational meaning as the source? If the answer is uncertain, preserve more structure and investigate the parser.

---

Chunk at boundaries that a user would recognize

Chunking divides canonical text into units suitable for embedding and later retrieval. Two competing goals must be balanced:

1. Specificity: a focused chunk is easier to match to a focused question.
2. Context: a chunk needs enough surrounding material to make its meaning and constraints clear.

Very small chunks can retrieve a keyword or isolated sentence but omit the exception, actor, scope, or precondition that makes it correct. Very large chunks preserve context but dilute the semantic signal with unrelated text. They also consume more of the model’s context window when several retrieved chunks are supplied to generation.

There is no universal best chunk size. A reasonable initial hypothesis for prose policies and technical documentation is a few hundred tokens per chunk, perhaps 300 to 600 tokens, with a moderate overlap. This is a starting configuration to test, not a production standard. The unit should be tokens when possible, because model limits and costs are token-based; character counts are only a rough implementation proxy.

More important than a numeric size is the splitting hierarchy. Prefer meaningful document boundaries before imposing a maximum size:

1. Split at document and major-section boundaries.
2. Keep a section heading with the content it governs.
3. Split large sections at paragraphs, list groups, speaker turns, or topic changes.
4. Split an oversized paragraph at sentence boundaries only if necessary.
5. Avoid breaking a sentence, code block, table row group, or procedural sequence unless no better alternative exists.

This gives you a useful default strategy: structure first, size second.

Match the strategy to the source type

Source type	Meaningful boundaries to preserve	Weak default to avoid
Markdown or product documentation	Heading hierarchy, paragraphs, callouts, code blocks	Splitting at arbitrary characters
HTML help centre	Main article, heading hierarchy, sections, lists	Including navigation and related-article widgets
PDF policies or manuals	Title, section heading, subsection, page reference, paragraphs	Flattening columns or separating table headers from values
Incident runbooks	Service name, alert condition, procedure step group, escalation section	Splitting a numbered procedure midway
Meeting or call transcripts	Speaker turn, agenda topic, question-and-answer pair	Separating a statement from the speaker who made it
Source code	File, class, function, method, adjacent comments	Breaking identifiers or function bodies purely by length

A transcript illustrates why content-aware chunking matters. The text “We will not approve the launch until Legal reviews the notice” is not enough if the question is, “Who blocked the launch?” The speaker name belongs with the statement. A speaker-turn boundary is therefore more meaningful than a generic paragraph or character boundary.

Similarly, a runbook may contain a heading, a condition, and numbered actions. If the condition is in one chunk while the actions are in another, retrieval may provide an instruction without the restriction that makes it safe.

---

Overlap protects boundaries, but creates redundancy

Even careful structure-aware chunking sometimes creates a harmful boundary. One paragraph may state a rule, while the following paragraph gives a crucial exception. A speaker name may fall immediately before the statement likely to be retrieved. Overlap repeats a small amount of text from the end of one chunk at the beginning of the next chunk so that a boundary does not entirely sever the connection.

Consider this simplified policy section:

Chunk choice	Retrieved text for a question about notice approval	Consequence
No overlap	“Approval must be obtained before publication.”	The actor who approves is absent
With overlap	“Legal approval is required for regulated markets. Approval must be obtained before publication.”	The rule remains interpretable if this chunk is retrieved alone

Overlap is most valuable when using basic content-independent splits, because those methods know little about the document’s meaning. It is less necessary when chunks already align cleanly with coherent sections or speaker turns, though a small overlap can still be useful.

Treat overlap as a guardrail, not a cure for poor chunk boundaries:

• Start with a modest overlap, often around 10 to 20 percent of the target chunk size.
• Use overlap that preserves a complete preceding sentence or a small paragraph fragment, rather than an arbitrary number of characters when your tooling allows it.
• Add a section title or short structural prefix to each chunk where appropriate. This is often more useful than duplicating a large amount of prose.
• Inspect whether the overlap keeps a condition, exception, or actor connected to the relevant statement.
• Watch for excessive duplication. Larger overlaps increase embedding cost, index size, and the chance that search returns near-identical chunks rather than distinct evidence.

For instance, an initial policy configuration might state:

PIPELINE_CONFIG = {
    "chunking_strategy": "section_then_paragraph",
    "target_chunk_tokens": 450,
    "overlap_tokens": 60,
    "preserve_section_heading": True,
    "pipeline_version": "2025-01",
}

This is not a magic configuration. It is a testable hypothesis. The configuration should be versioned so that, when retrieval quality changes, the team can identify whether a parser change, a source update, or a chunking change caused it.

---

Metadata makes chunks usable, citable, and governable

A chunk’s text tells the retrieval system what it is about. Metadata tells the application where it came from, how it may be used, and how it relates to other content.

Metadata should not be an indiscriminate copy of every available field. Choose fields by asking three questions:

1. Provenance: Can we trace an answer back to the authoritative location?
2. Relevance: Could this field narrow a future search to the right corpus?
3. Governance: Does this field express version, sensitivity, authority, or permitted access?

A practical chunk record might look like this:

prepared_chunk = {
    "id": "incident-comms:v3:external-notices:002",
    "text": """
Incident Communications Standard
External Customer Notices

For a SEV-1 incident affecting regulated markets, Legal must approve
any external customer notice before publication.
""".strip(),
    "metadata": {
        "document_id": "incident-comms",
        "document_title": "Incident Communications Standard",
        "source_uri": "knowledge-base/incident-communications",
        "section_path": "Communications > External Customer Notices",
        "chunk_number": 2,
        "document_version": "3.0",
        "published_at": "2025-01-15",
        "content_status": "approved",
        "audience": "incident-response",
        "jurisdiction": "regulated-markets",
        "data_classification": "internal",
        "pipeline_version": "2025-01",
    },
}

Several details are deliberate:

• id identifies the specific chunk, not merely the document.
• document_id identifies the conceptual source across updates.
• document_version distinguishes the version that supplied the text.
• section_path and source_uri provide a readable route for later citations and human verification.
• chunk_number preserves ordering, which is useful when reviewing neighboring chunks.
• content_status helps distinguish approved material from drafts.
• jurisdiction, audience, and classification fields support scoped retrieval and safer operation when those distinctions matter.

The title and section heading appear in both useful human-readable metadata and in the chunk text. This is intentional: a heading can add semantic context to the embedding, while metadata supports traceability and controlled filtering. Fields such as ingestion timestamp, parser version, and checksum are usually operational metadata; they are valuable for debugging but do not need to be placed in the text sent for embedding.

Access-related metadata deserves special care. Labelling a chunk “confidential” does not enforce access control. In a real system, authorization must be checked before retrieved text is delivered to a user or model. Still, retaining classification and entitlement-related attributes with the chunk is necessary so that future retrieval layers have the information needed to enforce that policy.

---

Create an inspectable preparation contract

At this point, do not judge the pipeline by whether it produces a large number of chunks. Judge it by whether a reviewer can inspect a sample and understand the choices.

For each representative document type, inspect several prepared chunks against the original source. Use this lightweight review checklist:

• Fidelity: Did parsing preserve headings, speaker names, conditions, exceptions, lists, tables, and technical identifiers that matter?
• Noise removal: Did repeated page furniture, navigation, or malformed extraction disappear without removing content that users may need?
• Chunk coherence: Could one chunk plausibly answer a narrow question on its own?
• Boundary quality: Are procedures, conditions, and their exceptions kept together? Where not, does overlap preserve the connection?
• Provenance: Can a reader locate the original document, version, page or section, and chunk sequence?
• Authority and recency: Does metadata distinguish approved current guidance from drafts or superseded versions?
• Governance: Are source scope, classification, and intended audience recorded where relevant?
• Repeatability: Is the parser and chunking configuration versioned so the same source can be processed consistently?

This is where your prior program-management experience is directly useful: turn a vague implementation choice into an explicit contract with acceptance criteria, sample-based review, known risks, and an accountable owner. “We use default chunking” is not an engineering decision. “We preserve procedure boundaries, use section-first chunks with a documented overlap, retain source and version metadata, and inspect representative outputs before indexing” is one.

You do not yet need a vector database to carry out this review. A list of dictionaries in the notebook is sufficient:

prepared_chunks = [prepared_chunk]

for chunk in prepared_chunks:
    print(chunk["id"])
    print(chunk["metadata"]["section_path"])
    print(chunk["text"])
    print()

The immediate objective is to make the corpus legible before it becomes vectors. In the next lesson, these prepared chunks will receive embeddings and be placed into a simple vector index. If the chunk records are coherent and traceable now, the retrieval behavior will be much easier to interpret later.

---

Key takeaways

Effective RAG preparation preserves meaning while making evidence retrievable:

• Parsing extracts content; cleaning removes noise and normalizes it without destroying substantive meaning.
• Begin with source structure—headings, paragraphs, speaker turns, procedures, functions—before applying a size limit.
• Smaller chunks improve specificity but may omit context; larger chunks preserve context but can dilute relevance.
• Overlap protects important relationships across boundaries, but it also increases redundancy and cost.
• A chunk should ideally stand alone with a meaningful heading or contextual prefix.
• Metadata provides provenance, relevance signals, versioning, and governance information; it is essential for trustworthy citations and controlled retrieval.
• Chunking settings should be versioned and tested against representative documents, not treated as defaults that need no review.

Next, you will create embeddings for these prepared chunks and store them in a simple vector index—the point at which the preparation choices made here begin to show up as ranked retrieval results.