AI Glossary

A neural network trained on large amounts of text to predict the next token. LLMs power chat assistants, drafting, extraction, classification and reasoning tasks. Examples include Claude, GPT, Gemini and open-weight models such as Llama and Mistral.

The unit of text a model reads and writes — roughly three-quarters of an English word. Model pricing, speed and context limits are all measured in tokens.

The maximum number of tokens a model can consider in one request, including instructions, retrieved documents, conversation history and its own answer. Bigger windows help, but sending only what is relevant is usually cheaper and more accurate.

Designing the instructions, examples and structure given to a model so it produces reliable output. In production, prompts are versioned and tested like code.

An architecture that retrieves relevant passages from your own documents and gives them to the model as context, so answers are grounded in your data and can cite sources — without retraining the model.

Numeric vectors that represent the meaning of text (or images). Texts with similar meaning have nearby vectors, which is what makes semantic search possible.

A store optimised for searching embeddings by similarity. Options range from Postgres with pgvector to dedicated engines; the right choice depends on scale, filtering needs and what you already run.

Splitting documents into passages before indexing them for retrieval. Chunk size and boundaries (by heading, paragraph or semantic break) strongly affect RAG answer quality.

Combining keyword search (good for exact names, codes and IDs) with semantic vector search, then using a re-ranking model to order the best candidates. Usually the biggest single improvement to retrieval quality.

Further training a model on your examples to change its style, format or narrow-task behaviour. It is not the best way to give a model new facts — RAG is usually cheaper and easier to keep current for that.

A system in which a model plans and takes multi-step actions — calling tools, reading results and deciding what to do next — towards a goal. Production agents need clear permissions, limits and human checkpoints.

Letting a model request structured calls to your APIs — look up an order, create a ticket, query a database — which your code executes and returns. This is how LLMs act on real systems safely.

An open standard for connecting AI applications to tools and data sources through a common interface, so one integration can serve many models and assistants.

Constraining a model to return data in a defined schema (for example JSON with required fields) so its output can flow straight into other systems without fragile parsing.

When a model produces fluent but false or unsupported content. It is reduced — never eliminated — by grounding answers in retrieved sources, requiring citations, validating output and routing low-confidence cases to people.

Checks around a model that enforce policy: input filtering, output validation, PII redaction, topic restrictions and limits on which actions an agent can take.

An attack where text in a user message, document or web page tries to override a model's instructions — for example to leak data or trigger a tool. Defences include least-privilege tool access, treating retrieved content as untrusted data, and human approval for sensitive actions.

Repeatable tests that score a model or prompt against a labelled set of real cases. Evals are how you know a change made the system better rather than just different, and they belong in CI like unit tests.

A design where people review, approve or correct AI output at defined points — typically for low-confidence or high-impact decisions. Their corrections also become evaluation data.

The practice of running LLM systems in production: tracing each request, tracking cost and latency, versioning prompts, monitoring quality and catching regressions.

Sending each request to the most suitable model — a small, fast, cheap model for simple tasks and a larger one for hard reasoning — to balance quality, latency and cost.

Reusing the processed form of a repeated prompt prefix (such as long instructions or a reference document) across requests, which cuts cost and latency for high-volume workloads.

A model whose weights are published so it can be self-hosted, for example Llama, Mistral or Qwen. Useful when data must never leave your infrastructure, at the cost of running and scaling it yourself.

An agreement with a model provider that prompts and outputs are not stored after the request is processed, beyond what abuse monitoring legally requires. A common requirement for regulated data.