What can go wrong?
Local AI removes some risks and introduces others. Most failures are not dramatic. They are ordinary mistakes such as retrieving the wrong page, misreading a scan, using an unexpected cloud provider or turning an uncertain name into a precise-looking point.
The useful response is to make the workflow inspectable.
A fluent answer with no evidence
A language model can produce a convincing answer even when the relevant passage was not retrieved.
Reduce this by requiring the document title and page or section, then opening the source. If the cited text does not support the answer, treat the answer as wrong rather than trying to rescue it with more prompting.
A plausible but nonexistent ingoa wāhi
A model can create a name that looks linguistically plausible, merge two nearby names or “repair” damaged OCR into a name that was never in the source.
Preserve place_as_written, keep the source context, and reject any candidate that cannot be found on the cited page.
False coordinates
Do not ask an LLM to supply coordinates from memory. A plausible latitude and longitude can still be the wrong feature, a town centroid, a different place with the same name or an invented point.
Use a named geographic source such as the New Zealand Gazetteer or an approved local dataset. Record the source identifier with the geometry.
OCR has changed the evidence
A scanned report may turn Pāpāwai into a damaged form or lose macrons completely. The language model may then make a confident guess about what the OCR “must have meant”.
Check the page image. Keep the original PDF, the OCR working copy and any corrected form as separate evidence states. See Working with PDFs.
Māori names are silently normalised
A model may replace a historical spelling with a current official form. This can make a table look tidy while destroying useful historical evidence.
Keep the source spelling and checked modern match in different fields. A modern name can be added later without overwriting the source.
A name is separated from its context
A place mentioned inside evidence about land, whakapapa, customary use or a historical event can lose meaning when extracted as a standalone row.
Keep a source-context field and page reference. For sensitive or culturally significant kōrero, the answer may be that the name should not become a public point at all.
The model interprets tikanga or whakapapa as if it has authority
Local models are statistical language systems. They can summarise text and identify patterns, but they do not acquire mana, mandate or whakapapa through document access.
If the task requires an interpretation held by whānau, hapū or iwi, the model can help locate the relevant evidence but should not be presented as the authority that resolves it.
A desktop application is still using the cloud
AnythingLLM, LM Studio and Open WebUI can all be configured in ways that involve external services. Possible pathways include hosted language models, external embedding APIs, web search, plugins, connectors or remote model servers.
Check the selected providers and repeat a known document question with the network disconnected. If the workflow only works online, investigate what is leaving the machine.
Telemetry and update checks
“Local inference” does not necessarily mean an application makes no network connections. Software may check for updates or send product telemetry separately from model inference.
For material requiring stronger controls, inspect the application's current privacy and telemetry documentation, organisational firewall logs or endpoint controls rather than assuming that a local model implies zero outbound traffic.
AnythingLLM's current desktop documentation makes the storage location inspectable, while LM Studio documents offline model and document operation. Open WebUI provides an offline mode but notes that its air-gap tutorial is community contributed. These are useful technical facts, not substitutes for your own network test.
Third-party plugins change the boundary
A local application may allow plugins or tools that call external APIs. A weather tool, web search provider or remote database can create a network path even when the language model itself is local.
Start with external tools disabled. Add them only when the kaupapa requires them and their data flows are understood.
Old models give weak results
Local AI changes quickly. A model that was a good recommendation a year ago may now be unnecessarily slow or weak.
This guide puts Last verified dates on technology pages. Recheck the model card and application documentation before a major deployment or wānanga.
Not enough RAM
Symptoms include very slow model loading, Windows becoming unresponsive, the application closing, or generation taking so long that people assume it is broken.
Use a smaller quantised model. A 5 GB model file does not mean only 5 GB of system RAM is required because the operating system, context, runtime buffers and other applications also need memory.
CPU inference is slow
A normal laptop can run a useful small model without a dedicated GPU, but it may produce text slowly. That is not a privacy failure. It is a hardware trade-off.
For document research, retrieval quality and source checking often matter more than raw tokens per second. A smaller model that responds promptly can be better for a wānanga than a larger model that makes the whole exercise wait.
The embedding model is wrong for the collection
The language model can be good while retrieval is poor. This often happens when the embedding model is weak for the language or terminology in the documents.
For mixed English and te reo Māori collections, this guide recommends testing a multilingual embedding model such as Qwen3-Embedding-0.6B. After changing embedding models, re-index the documents. Old vectors created by another embedder should not be mixed into the new index.
Retrieval missed the page
Try a direct keyword or name search. If the text is definitely in the extracted document but the semantic search misses it, narrow the document set, adjust the question or test a stronger multilingual embedder.
If the text is absent from the extracted document, fix the PDF or OCR. The LLM cannot retrieve text that never reached the index.
The map looks more certain than the evidence
A point symbol can make a tentative match appear settled. Keep review_status, confidence, geometry_source and review notes in the GIS data. Where the location is genuinely approximate, represent that uncertainty appropriately rather than inventing precision.
See Why GIS is different and Sensitive locations.
A simple failure test
Before relying on a workspace, deliberately ask three questions:
- one answer you know is in the document
- one name that is easy to confuse with an iwi, hapū or person
- one question that the documents do not answer
A useful system should retrieve the first, handle the second carefully, and say “not found” for the third. If it confidently answers all three, tighten the prompts and evidence checks before doing important mahi.
Last verified: 22 August 2026