Working with long documents and files

Students paste lecture notes, job seekers paste job descriptions, analysts paste reports. Long inputs fail in predictable ways: the model skims, mixes up sections, or answers from general knowledge instead of your document. This tutorial shows how to prevent each failure.

You will practise on any long document you have: a PDF chapter, a company policy, a research paper, or a long chat export. You will also write a small Python script that splits a text file into chunks, for the times when a document is too big for one prompt.

Prerequisites: you know the parts of a prompt from *Anatomy of a good prompt*. Python 3.8+ is only needed for the last section.

A model can only read a limited amount of text at once. That limit is the context window, measured in tokens (small pieces of words; roughly, 100 tokens is about 75 English words, though it varies by language). Your prompt, any uploaded files, the earlier conversation and the model's reply all share this one window.

Two things follow. First, a very long chat slowly pushes early messages out of the window or makes the model pay less attention to them. Second, even when a document fits, the model does not read it like a person with a highlighter. It tends to work best when the question is specific and the relevant part is easy to find.

Window sizes differ by model and change often, so check the tool's help page rather than trusting a number from a blog post. Upload limits (pages, file size) are separate from the window and also differ by app and plan.

Put your instructions before the document, mark where the document starts and ends, and repeat the key question after it if the document is very long. Tags such as <document> work well because they cannot be confused with the text inside.

You will answer questions about the document between the <document> tags.

Rules:
- Use only the document. Do not use outside knowledge.
- After each answer, quote the exact sentence(s) that support it, in quotation marks.
- If the document does not contain the answer, reply: "The document does not say."

<document>
[paste the text here]
</document>

Question: [your question]

What to expect: Check that every quote really appears in your document (use Ctrl+F). If a quote is missing or reworded, the answer is not grounded and you should not trust it.

"Summarise this report" gives an average summary. Narrow tasks get better results because the model knows what to look for. Compare these on a 40-page company policy:

For a long document, do not ask for one big summary. Ask for a structure first, then go deeper only where you need to. This is a small prompt chain (see *Prompt chaining: breaking big tasks into steps*).

Read the document below and give me an outline: each section or chapter heading (or, if there are none, each major topic), followed by one line on what it covers. Do not summarise the details yet.

<document>
[paste or attach the document]
</document>

What to expect: A good outline lists every section in order. Use it to see whether the model has understood the whole document before you spend time on details.

Using only the document, summarise sections [list of sections] for a reader who [who you are, e.g. is preparing for an exam / must decide whether to sign this].

For each section give:
- 3 to 5 bullet points of the main ideas
- Any numbers, dates or conditions, exactly as written
- Anything that looks unclear or contradictory

If a section is not in the document, say so.

What to expect: The last line stops the model from inventing a section that doesn't exist. Numbers and dates should match the document word for word.

You can give the model two or three documents and ask it to compare them. Label each clearly so it can refer to them. A table is usually the best output for comparisons because gaps become visible.

Compare the two documents below on these points: [point 1], [point 2], [point 3].

Reply as a table with columns: Point | Document A | Document B | Difference. Use short quotes where possible. If a document says nothing about a point, write "not stated".

<document name="A">
[text of first document]
</document>

<document name="B">
[text of second document]
</document>

What to expect: Spot-check two or three rows against the originals. Comparisons are where models most often mix up which document said what.

If a file is too large for one prompt, split it into chunks, process each chunk with the same instructions, then combine the results. This is often called map-reduce: *map* each chunk to a short result, then *reduce* all the short results into one answer.

Two details matter. Split at natural boundaries (paragraphs, not the middle of a sentence), and let chunks overlap slightly so an idea cut at the edge appears whole in one of them. The script below splits a plain-text file by paragraphs and writes numbered chunk files you can paste one by one.

import sys
from pathlib import Path


def split_into_chunks(text, max_chars=8000, overlap_paragraphs=1):
    """Split text on blank lines into chunks of at most max_chars characters.

    The last `overlap_paragraphs` paragraphs of each chunk are repeated at the
    start of the next chunk, so ideas cut at a boundary are not lost.
    """
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks = []
    current = []
    size = 0
    for para in paragraphs:
        if current and size + len(para) > max_chars:
            chunks.append("\n\n".join(current))
            current = current[-overlap_paragraphs:] if overlap_paragraphs else []
            size = sum(len(p) for p in current)
        current.append(para)
        size += len(para)
    if current:
        chunks.append("\n\n".join(current))
    return chunks


if __name__ == "__main__":
    if len(sys.argv) != 2:
        sys.exit("Usage: python split_text.py yourfile.txt")
    source = Path(sys.argv[1])
    chunks = split_into_chunks(source.read_text(encoding="utf-8"))
    for number, chunk in enumerate(chunks, start=1):
        out = source.with_name(f"{source.stem}_part{number:02d}.txt")
        out.write_text(chunk, encoding="utf-8")
        print(f"{out.name}: {len(chunk)} characters")
    print(f"Done: {len(chunks)} chunks.")

Run it with python split_text.py notes.txt. A single paragraph longer than max_chars stays whole in its own chunk. Lower max_chars if your tool rejects the chunks.

This is part [n] of [total] of a longer document about [topic]. Do not try to summarise the whole document; work only on this part.

Extract:
- The main points (max 5 bullets)
- Any definitions, formulas, dates or numbers, exactly as written
- Open questions the next part might answer

<part>
[paste chunk n]
</part>

What to expect: Save each reply. Keep the output format identical for every chunk so the results are easy to combine.

Below are notes extracted from [total] consecutive parts of one document about [topic]. Combine them into a single study summary.

Rules:
- Merge duplicate points; keep the order of the original document.
- Keep numbers, definitions and formulas unchanged.
- If two parts seem to contradict each other, list the contradiction at the end instead of choosing one.

<notes>
[paste all the chunk replies]
</notes>

What to expect: Because the reduce step sees only your extracted notes, anything the map step dropped is gone. That is why the map step asks for exact numbers and definitions.