3 posts tagged with "ai-training"

Artificial intelligence is often explained with one sentence that causes more confusion than it resolves: “AI learns from your data.” Sometimes that is true. Sometimes it is not. Sometimes your information is used only for the few seconds needed to answer a question. Sometimes a service stores it. Sometimes a provider may later use selected conversations to improve future models. Sometimes an organisation can contractually prevent that. Sometimes the entire process happens on a local computer and no document content leaves the machine at all.

The problem is that several different technical processes are being collapsed into one idea. If Māori organisations, GIS teams, researchers and data-governance practitioners are going to make sensible decisions about AI, those processes need to be separated.

This article explains the main stages in plain English, with mapping examples throughout. It is not an argument for or against AI. It is an argument for knowing which part of the system is actually doing what.

When people ask whether ChatGPT, Claude, Gemini, Llama, Qwen or DeepSeek has been trained on Māori material, the honest answer is more complicated than yes or no. The developers of current frontier models do not publish complete lists of every webpage, PDF, book, image or other item used during training. That means we cannot point to a current model and prove that it learned from a particular Waitangi Tribunal report or iwi website unless the developer has disclosed it.

But that does not mean we know nothing. We can inspect some of the large public web corpora that have fed the modern AI ecosystem. When we do that, Māori material is plainly there.

The most useful evidence comes from mC4, the multilingual version of Google’s C4 corpus built from Common Crawl. The publicly accessible Māori-language subset contains roughly 101,000 training records labelled mi. Those records include genuine Māori-language and Māori-related pages from Te Ara, Dictionary of New Zealand Biography material, Māori media, arts organisations and at least one iwi website. They also include classification errors and rubbish, so the number must not be misrepresented as 101,000 clean Māori documents. What it does prove is that Māori online material has already been swept into web-scale corpora used by the AI research community.

The public discussion around artificial intelligence often treats American and Chinese models as if they come from completely different technological worlds. Politically and legally, they do operate under different jurisdictions. Their companies have different ownership structures, regulatory obligations, cloud environments and terms of service. Those differences matter.

But when we look at how the models themselves are trained, the broad pattern is strikingly similar. OpenAI, Anthropic, Google, Meta, Alibaba and DeepSeek all rely on some mixture of large public datasets, licensed or partner material, human-created examples, synthetic data and extensive post-training. The details vary, but the core industrial process is recognisable across borders.

For Māori organisations, this means “American model” versus “Chinese model” is not a sufficient risk framework. The better questions are what data sources were used, how transparent the developer is, what product is being used now, where current prompts are processed, what jurisdiction applies, and whether the model can be run locally without sending new Māori information back to the developer.