How AI actually learns: training, inference, RAG and what happens to your data
Artificial intelligence is often explained with one sentence that causes more confusion than it resolves: “AI learns from your data.” Sometimes that is true. Sometimes it is not. Sometimes your information is used only for the few seconds needed to answer a question. Sometimes a service stores it. Sometimes a provider may later use selected conversations to improve future models. Sometimes an organisation can contractually prevent that. Sometimes the entire process happens on a local computer and no document content leaves the machine at all.
The problem is that several different technical processes are being collapsed into one idea. If Māori organisations, GIS teams, researchers and data-governance practitioners are going to make sensible decisions about AI, those processes need to be separated.
This article explains the main stages in plain English, with mapping examples throughout. It is not an argument for or against AI. It is an argument for knowing which part of the system is actually doing what.
Start with the model
A modern language model is not usually a giant digital library that looks up a stored paragraph whenever somebody asks a question. During training, text is broken into small units called tokens. The model repeatedly tries to predict what comes next. Each error causes tiny adjustments to the numerical values inside the model. Those values are usually called parameters or weights.
After an enormous number of repetitions, the model becomes very good at predicting language. It has absorbed statistical relationships between words, concepts, code, places, names, facts and patterns. The original documents influenced those numerical weights, but the trained model is not normally carrying a tidy folder of source files inside itself.
This distinction matters for Māori material. If a public Te Ara page was present in a large web-training corpus, it may have contributed in a very small way to the final model weights. That is different from an AI research tool indexing an iwi report and retaining a searchable copy of the report. Both involve AI, but the data relationship is completely different.
OpenAI describes its foundation-model training in these broad terms, explaining that models are developed from information that is publicly available on the internet, licensed or provided by partners, and supplied or generated by users, trainers and researchers. It also explains that training changes the model’s parameters rather than creating a simple searchable copy of every source. See How ChatGPT and our foundation models are developed.
Pre-training
Pre-training is the huge first stage. A foundation model is exposed to an immense collection of text and, increasingly, images, audio, video and code. The purpose is not to teach the model one job. It is to give the model broad statistical knowledge of language and patterns so that later it can perform many jobs.
The scale is difficult to imagine. GPT-3’s published methodology described hundreds of billions of tokens drawn from filtered Common Crawl, WebText, books and Wikipedia. Meta says Llama 3 used more than 15 trillion tokens. DeepSeek-V3 disclosed 14.8 trillion. Qwen3 disclosed roughly 36 trillion tokens across 119 languages and dialects. A token is not the same as a word, but these numbers are still useful for understanding the scale.
This is one reason small languages can be both present and underrepresented at the same time. A corpus containing hundreds of thousands of Māori-language pages or passages can still be tiny compared with tens of trillions of tokens dominated by English, Chinese and other major languages.
For GIS, pre-training is what lets a general model already know that QGIS has expressions, that a GeoPackage is a spatial data format, that NZTM2000 is a New Zealand coordinate reference system, or that an awa is a river. That knowledge comes from the model’s broad training. It is not the same as the model having access to your organisation’s current GIS project.
Post-training
A raw foundation model is not automatically a useful assistant. Post-training teaches it how to follow instructions, answer questions, reason through problems, avoid some unsafe behaviour and produce outputs in forms people find useful.
Developers use human-written examples, rankings, reinforcement learning, synthetic examples and other methods. This is one reason two models with similarly large pre-training datasets can feel very different to use. Their post-training can emphasise different behaviour.
For mapping, post-training may be what makes a model better at following a request such as “write a QGIS expression that labels only features where review_status equals checked” rather than merely continuing a paragraph about QGIS.
Fine-tuning
Fine-tuning is additional training carried out on a smaller, more focused dataset. It changes model weights. An organisation might fine-tune a model for a particular style, technical domain or task.
This is different from simply attaching documents to a chat. That distinction is important because people regularly describe any use of organisational documents as “training the AI”. In many modern systems that is not what is happening.
A GIS team could fine-tune a model on thousands of examples of feature classifications, for example. That is genuine additional training. By contrast, giving an already-trained model ten planning reports to search at runtime is normally an inference or retrieval problem, not fine-tuning.
Inference
Inference is what happens when you actually use the trained model.
You type a question. The system places that question into the model’s current context. The model uses its existing parameters to calculate a response token by token. The model is being used, not necessarily retrained.
This is the key point:
Entering information into an AI service does not automatically mean that the model’s weights are being changed by that information at that moment.
That does not mean the information disappears. The service may retain the prompt and response. The provider may have logging, abuse-monitoring or product-improvement processes. Depending on the product and settings, some interactions may later be selected for future training. Those are separate stages.
A good governance discussion should therefore ask three different questions. What happens during the current inference? What does the service retain afterwards? Can retained material later be used for model improvement?
Context
The model’s context is the information available to it for the current interaction. That can include your prompt, previous messages, system instructions, retrieved document passages, tool outputs and sometimes images or files.
A model may therefore know something for two very different reasons. It may have learned a broad pattern during pre-training, or the relevant information may simply have been placed into its current context.
Suppose an iwi environmental team asks a model about an erosion report written last week. A general model cannot have learned that report during historical pre-training if the report did not exist then. But if the report is attached to the current chat, the model can work with it because the report content has been added to the current context.
That distinction is important for source checking. A model answer may sound equally confident whether the claim came from a supplied source or from broad model memory. Good GIS research should preserve the source path so the practitioner can tell the difference.
RAG
Retrieval-augmented generation, usually called RAG, is one of the most useful concepts for Māori GIS document work.
Instead of trying to place every document into the model’s context at once, the system first searches a collection of documents. It retrieves the passages that appear most relevant to the question. Those passages are then supplied to the language model, which prepares an answer from them.
A useful mental model is:
find the evidence first, then ask the model to work with that evidence.
RAG does not normally retrain the foundation model on the documents. The model weights can remain unchanged. The document collection is held separately and searched when needed.
That is why local RAG can be particularly useful for Māori organisations. The documents, parsed text, embeddings, search index and model can all be kept on infrastructure controlled by the organisation. The model can help find relevant passages without the collection becoming part of a new external training run.
Google and Microsoft describe RAG in essentially these terms: external information is retrieved and supplied to a model at inference time rather than requiring the foundation model to be retrained for every knowledge source. See Google Cloud’s retrieval-augmented generation overview and Microsoft’s RAG concepts.
Embeddings
RAG commonly relies on embeddings. An embedding model converts a piece of text into a long list of numbers representing aspects of its meaning. Similar passages tend to end up closer together in this mathematical space than unrelated passages.
This allows semantic search. A user can ask about river access even if the source paragraph uses terms such as customary use, crossing point or riparian entry rather than the exact words in the question.
For GIS practitioners there is an unfortunate vocabulary collision. An AI vector is not GIS vector geometry. It is not a point, line or polygon. Likewise, a vector database used for semantic search is not the same thing as a geodatabase or spatial database.
The distinction matters because an AI system can create a vector index from documents while the actual GIS geometry remains in a GeoPackage, PostGIS database or geodatabase. Those are different data stores serving different purposes.
Product retention
Now we reach the part that is most often confused with training.
A company can process your prompt to generate a response and still retain a copy afterwards. Retention can exist for security, abuse monitoring, account history, compliance, support, audit, product improvement or other purposes.
Retention does not automatically mean training either. An enterprise service may retain prompts for audit while contractually excluding them from foundation-model training. Conversely, a consumer product may allow some conversations to be used for future model improvement unless the user changes a setting.
The words “does not train on your data” therefore answer only one question.
They do not automatically answer where the data are processed, how long they are retained, who can administer the environment, what logs exist, what subprocessors are involved, or whether the organisation can delete every derived copy.
Future model improvement
Some consumer AI products allow conversations to contribute to future model improvement. If that happens, the interaction may later enter a training or evaluation pipeline. The provider may filter, de-identify, sample or process it before use.
This is a later step. It is not the same thing as the model changing itself in real time while the user types.
That distinction is useful because it makes practical controls possible. A user may disable model improvement. An enterprise contract may exclude customer content from training by default. A local AI system may prevent the provider from receiving the information at all.
Different products make different promises, so product-specific terms matter more than general slogans about “AI”.
Local AI
Local AI changes the architecture again.
A model can be downloaded and run on a desktop computer, workstation or organisational server. If the model, document parser, embedding model and search index all operate locally, the working documents do not need to be sent to the model developer.
That does not erase the history of the model’s original training. A downloaded model may still have been pre-trained on enormous international datasets. It may contain biases, weak representation of te reo Māori, uncertain provenance or knowledge derived from public Māori material.
But local operation can change what happens to new organisational information from that point onward.
This is a useful distinction for Māori data sovereignty. The historic training question and the current data-flow question should both be considered, but they should not be treated as the same thing.
Agents and tools
Newer AI systems may do more than generate text. An agent can use tools, call APIs, execute code, search the web, query databases or interact with software.
That changes the data boundary again. A local model may be running on your machine while a web-search tool sends a query to an external provider. A local GIS assistant may call an online geocoder. A plugin may transmit part of a prompt to another service.
The phrase “local model” therefore does not prove that the entire workflow is local.
For important Māori or organisational data, draw the system. Show the model, document store, embeddings, spatial database, web tools, map services, APIs and user devices. Then draw an arrow wherever information travels.
That diagram often explains more than pages of generic AI policy.
Why this matters for Māori GIS
GIS turns information into spatial objects. That adds an extra consequence to AI errors and AI-derived information.
A vague historical description can become a point. A model-generated classification can become a polygon. A set of public reports can be combined to infer a likely location. A retrieved place name can be matched to the wrong feature and then appear on a map with false precision.
AI is therefore most useful when it helps people reach evidence rather than replacing the evidence.
A strong workflow might use AI to search hundreds of reports for candidate ingoa wāhi, preserve the wording and page reference, have a person check the source, verify the geography using an identified source, then create a GIS record with provenance and review status.
The AI reduces reading and data-preparation effort. GIS manages the geography. People retain authority over interpretation and use.
The seven questions to keep separate
When somebody says “the AI is using our data”, ask what they actually mean. Is the material part of historic foundation-model pre-training? Is it being used for current inference? Is it in the current context? Has the service retained it? Could retained material later be used for model improvement? Has it been indexed into a RAG system? Is an external tool or agent sending part of it somewhere else?
Those are seven different technical pathways.
Māori data sovereignty becomes easier to apply, not weaker, when those pathways are visible. The goal is not to simplify the kaupapa down to technology. It is to make sure governance is responding to the system that actually exists.
Sources and further reading
OpenAI: How ChatGPT and our foundation models are developed
OpenAI: GPT-3 model card and training information
Meta: Introducing Meta Llama 3
Qwen: Qwen3 technical overview
DeepSeek: DeepSeek-V3 technical information
Google Cloud: Retrieval-augmented generation
Continue with What Māori material is already in AI training data?, which looks at the evidence from Common Crawl, mC4, Te Ara, historical newspapers, public reports and other Māori-related material on the open web.