Skip to main content

Working with PDFs

A PDF that looks readable to you may contain no usable text for a document-search system. This is one of the most common reasons local AI appears to miss obvious information.

Before changing the model or prompt, check the PDF.

Text PDF or scanned PDF?​

A text PDF contains actual characters. You can usually select a sentence with the mouse, copy it and paste it into a plain-text editor. Search inside the PDF also works.

A scanned PDF may be only a set of page images. It looks like a document because you can see the words, but there may be no text layer for document search to index.

A scanned PDF becomes searchable after OCR, optical character recognition, creates a text representation of the page.

A 30-second test​

Open the PDF in your normal reader and try three things:

  1. Search for a distinctive word that is clearly visible on the page.
  2. Select a full sentence and copy it.
  3. Paste that sentence into Notepad.

If you get nothing, an image, random characters or badly scrambled text, the local AI will probably have the same problem unless its document extractor performs OCR.

OCR is not the same as understanding​

OCR tries to recognise characters. The language model then works with the text that OCR produced.

Historical Māori names are particularly vulnerable to OCR errors because the source may include old type, handwriting, damaged paper, unusual spacing, diacritics, faint photocopies or spellings that the OCR engine has seen rarely.

A model may then make the error worse by silently converting the damaged text into a more familiar-looking name.

For ingoa wāhi, keep an image or page reference to the original source and record the OCR form separately from any corrected form.

Common PDF problems​

Complex layouts​

Two-column reports, footnotes, sidebars and boxed text can be extracted in the wrong reading order. The words may all be present while the sentences are mixed together.

Tables​

Tables can become a stream of disconnected values. Before relying on a model summary, compare the extracted result with the visual table.

Maps inside PDFs​

A document-search pipeline that extracts text does not automatically understand a map image. A place label visible on a historical map may never enter the text index.

Treat embedded maps as images that need their own review or OCR process. For historical spatial work, see Historical Māori maps and the Retrolens workflow.

Old scans​

Skew, fold marks, bleed-through and low contrast can make OCR unreliable. A better scan often improves results more than a larger language model.

Missing macrons​

A source may genuinely pre-date widespread digital macron use, or the macron may have been lost during OCR. Do not silently insert one and then describe the changed form as the source spelling.

Use separate fields for place_as_written and possible_normalised_name.

A practical local OCR route​

For a Windows beginner, the safest workflow is to create a searchable working copy before adding an important scanned PDF to the AI workspace.

If you already use a trusted desktop PDF application with OCR, run OCR locally and save a new working copy while keeping the untouched original. Verify several pages manually, including pages with Māori names.

For a fully open local route, OCRmyPDF can add a searchable text layer using local OCR tools. It is capable but is more technical to install on Windows than the main AnythingLLM workflow, so it is not required for a first wānanga setup.

Open WebUI can also be configured with document extraction engines that support scanned PDFs and images, but that is an advanced configuration and the privacy properties depend on the extractor you select. Do not assume an “OCR” setting is local without checking the engine.

Keep the original​

Use a simple source structure:

source-documents\
  originals\
    report-volume-1.pdf
  working-ocr\
    report-volume-1-ocr.pdf
  notes\
    report-volume-1-ocr-check.txt

The original is evidence. The OCR copy is a derived working file.

In the notes, record the OCR tool, date and any pages where names or tables need manual checking.

Check the extracted text before indexing hundreds of pages​

Test several kinds of page:

  • ordinary paragraphs
  • a page with several Māori names
  • a table
  • footnotes
  • a page with poor scan quality
  • a map or figure page

If the extraction is poor, fix the document first. Indexing the same broken text into a different model will not repair the evidence.

When a visual model helps​

Some local models can accept images as well as text. This can help with an individual page, screenshot or map legend, but it is not a substitute for careful georeferencing or source interpretation. A multimodal model can misread labels and can still invent plausible details.

For large document collections, searchable text plus human review remains the more auditable path.

Record extraction uncertainty​

For historical material, include a field such as:

text_quality = good | usable | poor | image-only

and, where OCR was used:

ocr_review = unchecked | sampled | checked

This lets a later reviewer distinguish a clean born-digital report from a name recovered from a difficult scan.

Next, use the extracted evidence to find candidate ingoa wāhi.

Sources​

Last verified: 22 August 2026

Last updated on