Hunter Brennick AI Systems & Advisory ↗
AI Systems Orchestration
Part II · Patterns And Practice/Chapter 05
05

Context, Tools, And Protocols

Where usefulness lives or dies: knowledge sources, context engineering, memory, tool shape, MCP, A2A, Skills, and AGENTS.md.

12 min read3 figures
The gist
  • Knowledge lives in sources. Context is what the system selects for this run. Memory is retained state.
  • Design the context contract first: authority, provenance, freshness, access, budget, evidence.
  • Tool output shape matters as much as prompt wording.
  • Protocols belong at boundaries: MCP for tools, A2A for delegation, Skills for expertise, AGENTS.md for repos.

Module 5: Knowledge, Context, Memory, And Tool Design

This is where most agent systems either become useful or fall apart.

Knowledge Is Not The Context Window

These terms are related. They describe different parts of the system.

Term What it means What it is not
Knowledge source A durable corpus or system that may contain relevant facts, rules, examples, or procedures. The text currently loaded into the model.
Model knowledge Patterns encoded in model weights during training. A current, attributable source of truth.
Retrieval The process that finds and selects candidate material for a task. Proof that the selected material is authoritative or complete.
Context The instructions, evidence, tool results, and working state available during a run. Everything the organization or system knows.
Tokens and context window The representation and capacity limit for what the model can process at once. A knowledge architecture or a measure of truth.
Memory Information retained across runs to improve continuity or future retrieval. An authority that should override current sources.
Artifact An output or evidence record produced by a run. Automatically validated knowledge.

A useful flow is:

FROM SOURCES TO EVIDENCE Knowledge sourcesdurable corpora with owners Retained memorycandidate input · not authority Retrieve and rankselect · ground · keep current Selected contextfor this run only Context windowtoken budget · capacity Model and toolsact on the selection Artifact or actionevidence the run happened Evaluationverify before it counts THE CONTRACT Authority · which source wins when sources disagree. Provenance · where a claim came from. Freshness · how stale material is caught. Access · what this agent may retrieve or act on. Selection and budget · what deserves scarce window space. Evidence · what must survive the run.
FIG 05.1From knowledge sources to evidence: the selection pipeline that feeds a run.

The model may also draw on its training-time knowledge. Current authoritative sources should win when the two disagree. Memory should accelerate discovery and preserve lessons. It should not bypass source verification.

Design The Context Contract

Optimizing embeddings or vector databases or token counts comes second. The contract comes first. Authority decides which source wins when sources disagree. Provenance is whether the system can show where a claim came from. Freshness is how the system detects stale material. Access defines what an agent may retrieve. It also defines what the agent may reveal or act on. Selection decides what belongs in the current context and what should stay out. Budget decides what deserves scarce context window space. Evidence is what must survive the run. Citations. Artifacts. Checks.

Company knowledge is not a special AI primitive. It shows up as several kinds of domain knowledge sources:

  • policies
  • product documentation
  • repositories
  • customer records
  • operational systems

Each one has its own owner. Each has its own permissions. Each has its own freshness requirements. An agent system should point to and retrieve from those sources rather than silently becoming a second source of truth.

Learn the pieces that make this work in practice. Source authority. Provenance. Freshness. Knowledge sources versus context versus memory. Progressive disclosure. Retrieval quality. Tool naming. Input schemas. Output schemas. Batch APIs. Response filtering. Tool result summarization. Context compaction. Memory as retained data. Not magic.

Build a context contract for one recurring task. Which sources are authoritative? How are they retrieved? What must be fresh? What can memory contribute? What evidence must the run preserve? What is the context budget?

Then run a tool fitness test on any MCP or tool in use. Is the name obvious? Are parameters hard to misuse? Can the agent request small results? Does the tool expose batch operations? Does it return structured data? Does it fail with useful messages? Can it be evaluated?

A well-built customer-support MCP tool is a good reference point for practical tool design:

  • read-only access
  • field filtering
  • batch operations
  • rate-limit handling
  • scoped OAuth

Knowledge lives in sources. Context is what the system selects for this run. Tokens are how that context fits. Memory is retained state. A connector is an access path. It is not the knowledge itself. More context is not automatically better context. Personas and instructions can shape judgment and procedure. They should not become hidden fact stores. A great model with bad tools behaves like a smart person using a broken keyboard. Tool output shape matters as much as prompt wording.

SAME MODEL · TWO TOOLS A TOOL THAT FIGHTS THE MODEL do_stuff( ) parameters that are easy to misuse returns ten thousand rows at once no batch operations fails with a stack trace or silence the model guesses · retries · burns the window and looks less capable than it is A TOOL THAT HELPS search_tickets(query) a name that says what it does a schema that is hard to misuse small filtered results · batch operations failures that explain what to try instead the model discovers it · calls it correctly and recovers when things fail tool shape is a multiplier a great model with bad tools behaves like a smart person using a broken keyboard
FIG 05.2Same model, two tools: shape decides whether the model looks brilliant or broken.

Module 6: Protocols And Interoperability

This is the layer that turns isolated agents into ecosystems.

Four protocols matter most here.

MCP covers
  • tools
  • resources
  • prompts
  • roots
  • sampling
  • elicitation
A2A covers
  • Agent Cards
  • tasks
  • messages
  • artifacts
  • streaming
  • push notifications

Skills are portable procedural knowledge with progressive disclosure. AGENTS.md holds repo-local operating instructions for coding agents.

A clean shorthand:

MCP: what can this agent app use?
A2A: what other agents can this system delegate to?
Skills: what packaged expertise can this agent load?
AGENTS.md: how should agents behave inside this repo?
THE INTEROPERABILITY QUAD MCP What can this agent app use? tools · resources · prompts · roots · sampling · elicitation A2A What other agents can this system delegate to? agent cards · tasks · messages · artifacts · streaming Skills What packaged expertise can this agent load? portable procedural knowledge · progressive disclosure AGENTS.md How should agents behave inside this repo? repo local operating instructions for coding agents protocols belong at boundaries · discovery without trust is dangerous
FIG 05.3The interoperability quad: what each protocol answers for an agent system.

Build a protocol map for your own machine covering:

  • local skills
  • MCP servers
  • workflow artifacts
  • generated client surfaces
  • repo AGENTS.md files

Mark which parts are portable. Mark which are local-only. Mark which depend on private credentials.

Interoperability is not just convenience. It is how you avoid rebuilding the same integration for every model and client. Protocols need security boundaries too. Discovery without trust is dangerous.

Next chapterChapter 06 · Runtime Orchestration What happens when the workflow actually runs: graphs, checkpoints, handoffs, parallel agents, human interrupts, and framework choices.