Context Development Lifecycle

Introduction

The context development lifecycle is the missing discipline between writing a clever prompt and running a dependable AI agent in production. A Chroma study of 18 large language models found that reliability drops as input length grows, even on simple tasks, so more context is not automatically better context. Teams that scatter instructions across rule files, wiki pages, and chat threads discover this the hard way when an agent quietly follows stale guidance. A context development lifecycle treats those instructions, retrieved documents, memories, and tool descriptions as engineered artifacts that are generated, evaluated, distributed, and observed. This guide walks through each stage, the runtime techniques that keep a context window healthy, and the security and governance questions that follow. By the end you will have a working plan for building, testing, and maintaining context the same way you already build and test code.

Quick Answers on the Context Development Lifecycle

What is the context development lifecycle?

The context development lifecycle is a four-stage loop of generating, evaluating, distributing, and observing the instructions, knowledge, and tools given to AI agents, so context receives the same testing and versioning as code.

How is it different from context engineering?

Context engineering curates the tokens for each model call at runtime, while the lifecycle governs how context assets are authored, tested, versioned, shipped, and monitored over many months.

Why does context need a lifecycle?

Models degrade as inputs grow and guidance goes stale, so untested context quietly lowers accuracy. A lifecycle process catches regressions before release and turns production failures into new tests.

Key Takeaways

• The context development lifecycle applies software discipline to prompts, retrieval, memory, tools, and agent skills through four stages: generate, evaluate, distribute, and observe.
• Context is a finite resource, so quality, relevance, and freshness matter more than raw token volume or ever larger context windows.
• Evaluation is the stage most teams skip; a small suite of scenario tests with pass thresholds catches most context regressions before they reach users.
• Shared context packages are a supply chain, which means versioning, review, provenance, and prompt injection scanning belong in the process from day one.

What Is the Context Development Lifecycle?

The context development lifecycle is the repeatable process of generating, evaluating, distributing, and observing the prompts, retrieved knowledge, memory, and tool descriptions an AI model receives, so context is versioned, tested, and maintained like software.

Where the Context Development Lifecycle Came From and How the Term Is Used Today

Building on that definition, it helps to trace where the phrase comes from, because two communities currently use it in slightly different ways. Patrick Debois, a DevOps veteran, published a short post on the context development lifecycle on February 19, 2026. He later turned it into a talk called Context Is the New Code at AI Engineer Europe 2026. His argument is that the bottleneck in AI-assisted software work has moved from writing code to communicating intent. He names four stages, which are generate, evaluate, distribute, and observe, and he asks a pointed question: if you would not ship untested code, why ship untested context? The audience he has in mind is teams whose coding agents read rule files, Markdown notes, and tribal knowledge that nobody versions or tests.

Shifting to the data and analytics side, the same phrase appears in vendor writing aimed at enterprise agents that must answer business questions correctly. Atlan describes a six-stage context development lifecycle that runs from discovery and extraction through structuring, validation, activation and serving, monitoring, and finally deprecation. Its engineering core is a loop of build, test, review, approve, deploy, and learn, in which AI drafts and tests the context and humans decide what becomes canonical. The unit of work here is not a rules file but a governed business definition, such as what counts as revenue or which table holds the trusted customer list. Readers should note that this is a vendor framing, so its performance claims describe one product rather than an independent benchmark.

Stepping back, the two usages are the same idea applied to different artifacts. Both treat context as an asset with an owner, a version, and a test suite. Both also borrow from DevOps the habit of closing the loop with production feedback. The term is still young and has no formal standard, which means the stage names vary between authors while the underlying practice stays consistent. In this guide the term means the general practice, covering coding-agent instructions, enterprise knowledge, retrieval pipelines, memory, and tool definitions. Where an author’s stage names differ, the sections below point out the mapping so you can translate between them.

From Prompt Engineering to Context Engineering to a Full Lifecycle

Beyond the history of the phrase, the practical question is how a lifecycle differs from the two disciplines that came before it. Prompt engineering is the craft of writing and organizing instructions, especially system prompts, so a model produces good outcomes. Context engineering widened that scope to everything the model sees at inference time, including tools, retrieved data, and message history. Anthropic defines it as the set of strategies for curating and maintaining the optimal set of tokens during inference. Andrej Karpathy called it the delicate art and science of filling the context window with the right information for the next step. Shopify chief executive Tobi Lutke named it a core skill in mid 2025.

Stepping up one level, a lifecycle is not another technique for filling the window; it is the process that produces and maintains the things you put in it. Our guide to context engineering for LLM agents covers the runtime craft, such as what to select, compress, or isolate on each model call. The context development lifecycle sits around that craft and answers a different set of questions. Who wrote this instruction, how do we know it still works, which version is live, and what happened when it failed? Prompt engineering produces a good prompt, context engineering assembles a good window, and the lifecycle keeps both honest over time. Without it, a team can have excellent runtime assembly feeding on instructions that nobody has reviewed in six months.

Why Context Became the Main Engineering Constraint

Stepping back from definitions, the reason a context development lifecycle is needed at all is that context behaves like a scarce resource rather than a free input. Anthropic explains that models spend an attention budget as they read, because transformer attention relates every token to every other token and stretches thinner as the window grows. Its guidance is to find the smallest set of high-signal tokens that maximize the likelihood of the outcome you want. Context is therefore described as a finite resource with diminishing marginal returns. Every new token you add draws down that budget by some amount, whether or not the token helps.

Turning to the evidence, controlled experiments show the cost of ignoring this. Chroma evaluated 18 models and reported that performance becomes less reliable as input length grows, even on simple tasks, and that models do not use their context uniformly. On a conversational memory benchmark, every model family scored significantly higher on a focused prompt of about 300 tokens than on a full prompt of about 113,000 tokens. Our own explainer on context rot in large language models walks through why this degradation is gradual rather than a sudden cliff. A single distractor document also lowered accuracy against the needle-only baseline, and four distractors lowered it further.

Given those findings, bigger windows alone cannot rescue a team whose context is disorganized. Debois compares dumping everything into a large window to handing a new junior developer the whole codebase with no onboarding. The problem is rarely volume; it is quality, relevance, and freshness. Cost reinforces the point, because agents read far more than they write. The Manus team reports roughly 100 input tokens for every output token in its agent loops, so the content of the prompt, not the answer, dominates both latency and spend.

The Anatomy of Context: Prompts, Retrieval, Memory, Tools, and Skills

Turning to the raw materials, a lifecycle only works if you know exactly which assets it manages. In practice, five families cover almost everything an agent reads. The first is instructions, meaning system prompts, role definitions, and output formats, which Anthropic says should sit at the right altitude between brittle hardcoded logic and vague guidance. Under a managed lifecycle, each family becomes a versioned artifact with an owner, a test suite, and a release path. Treating them as separate asset types matters because they fail in different ways and are tested differently.

Retrieved knowledge is the second family, covering documents, database rows, chunks from a vector index, and anything fetched at run time. Its failures are usually silent: a stale index, a chunk that lost its surrounding meaning, or a retriever that surfaces a plausible but wrong passage. Because retrieval quality is measurable, it is the easiest family to put under automated evaluation. Teams often discover that the index, not the model, explains most of their wrong answers. Chunking choices, embedding refreshes, and reranking all belong in the lifecycle as configuration that is reviewed and tested.

Memory is the third family and includes scratchpads that last for one task and long-term stores that persist across sessions. Anthropic describes structured note-taking, where an agent writes a NOTES file or to-do list outside the window and reads it back later. The storage patterns vary from plain files to vector stores and knowledge graphs. Memory deserves lifecycle treatment because it is written by the agent itself, which means it can accumulate errors, outdated facts, or sensitive data without any human review. A memory store that nobody audits is context that nobody owns.

The fourth and fifth families are tool definitions and packaged skills. A tool description is context in the strictest sense, since the model reads the name, parameters, and documentation before deciding whether to call it. The Model Context Protocol standardizes how those tool descriptions and resources reach a model, and the mechanics of function calling in LLMs explain how the model chooses among them. Skills and rule files such as CLAUDE.md, agent.md, or Cursor rules bundle instructions, examples, and scripts that an agent loads when relevant. All five families share one property: a small edit can change behavior everywhere, which is exactly why they need change control.

Stage One: Generating Context Worth Keeping

Moving on to the first stage, generation is about turning knowledge that lives in people’s heads into structured material an agent can act on. Debois splits this material into technical context such as coding standards and patterns, project context such as timelines and priorities, and business context such as customer expectations and compliance duties. Most teams already hold all three kinds, but scattered across chat threads, ticket comments, and the memory of the senior engineer who happens to be on call. The generation stage gathers that material, decides what is stable enough to write down, and gives each piece an owner. The goal is not to write more context but to write the smallest amount that reliably changes agent behavior.

Sourcing is easier when you pull from systems of record instead of asking people to write essays. A semantic knowledge graph for LLM agents is one way to capture business relationships so they can be retrieved rather than retyped. Commit history, pull request threads, and issue trackers carry the reasons behind a design. Candede’s article on the context development life cycle adds that library documentation can be fetched live to avoid version hallucinations. Atlan reports that most customers using its workflow find AI-drafted business definitions beat their own first drafts, although that is a vendor claim about the drafting step only. Whatever the source, a drafted instruction is a hypothesis until the evaluation stage tests it.

Quality at this stage follows Anthropic’s advice to start minimal and let observed failures drive additions. Write the instruction at the right altitude, specific enough to guide behavior but flexible enough to let the model apply judgment. Prefer a few diverse, canonical examples to a long list of edge-case rules, because examples work like pictures that are worth a thousand words. Record the intent behind every instruction, since a rule without its rationale becomes impossible to retire safely later. A short front-matter block listing the owner, purpose, last review date, and target agents turns a loose paragraph into an asset the rest of the context development lifecycle can manage.

Stage Two: Evaluating Context Before It Ships

Next, the evaluation stage is where the context development lifecycle earns its keep, because it is the step most teams skip. The core idea is to apply test-driven habits to specifications: define scenarios, run the agent with the candidate context, and check whether the output matches intent. Candede’s write-up of the lifecycle lists a ladder of checks that gets progressively more expensive. Context linting verifies structure and required fields, such as character limits or mandatory metadata. A context review asks a separate model whether the instructions are clear, complete, and free of contradictions before anything runs.

Rule-based evaluations come next, and here an LLM judge scores the agent’s output against company standards. Anthropic found this approach practical in its multi-agent research work. There, a single LLM judge prompt scored outputs from 0.0 to 1.0 and gave a pass or fail grade, which proved the most consistent option. Our walkthrough of evaluating LLM outputs with an LLM judge explains how to write rubrics and avoid common judge biases. End-to-end evaluations go further by giving the judge execution tools, such as a sandbox in which generated code actually runs. Each rung catches problems the previous one cannot see, so mature teams use all of them.

Because model output is non-deterministic, a single pass or fail means little. Candede suggests an error budget, where a test passes if the agent meets the threshold in a set number of repeated runs, with four of five given as the example. This turns a flaky signal into a stable gate that a release pipeline can enforce. It also forces the team to state how reliable the context must be. That is a product decision rather than an engineering default. A context change should merge only when it passes its scenarios at the agreed budget and does not break the scenarios that passed before.

Starting small is perfectly acceptable for a first release of your suite. Anthropic reports that about 20 real-world queries were enough to reveal large effect sizes early in its research system. A team therefore does not need a thousand-case benchmark on day one. Pull the first scenarios from real failures, real tickets, and real user requests rather than inventing tidy examples. Hold some scenarios back as a test set so that you do not tune the context to the exact cases you measure. Human review still matters for edge cases that automated judges miss, and a lightweight sampling routine keeps people close to the outputs.

Building a Context Evaluation Harness That Teams Actually Run

Building on those evaluation ideas, the harness is the small piece of infrastructure that makes the whole lifecycle routine rather than aspirational. A good harness is deliberately simple, consisting of scenario files, a runner that loads a candidate context bundle, a judge, and a pass or fail report. Each scenario holds an input, the context bundle under test, and a rubric describing a good outcome. Anthropic’s tool-writing guidance adds that each evaluation task should be paired with a verifiable outcome and that verifiers should stay lenient enough to accept valid phrasings. Run each task in its own simple loop so that one scenario cannot contaminate another through leftover state.

Track more than accuracy, because cost and behavior reveal problems that scores hide. Anthropic recommends recording runtime, total tool calls, token consumption, and tool errors for every run. Frequent redundant calls can signal that pagination or token limits need tuning, while frequent invalid-parameter errors usually point to unclear descriptions. Our guide to measuring AI agent performance lists the metrics worth adding to a dashboard. Wire the harness into the same pull request pipeline that runs unit tests, so a failing context change blocks a merge exactly like a failing code change.

Stage Three: Packaging, Versioning, and Distributing Context

From there, the distribution stage of the lifecycle asks how approved context reaches every agent and developer who needs it. Debois frames context as versioned, published packages with integrity guarantees and update mechanisms, much like software dependencies. In practice that means storing context in source control, tagging releases, and sharing them through a registry so teams stop copying rule files between repositories. A consumer then pins a version, reads a changelog, and upgrades on purpose rather than discovering silent edits. Treating context as a dependency is what turns a shared folder of notes into a managed product.

Dependencies bring their familiar problems along with them, and context packages are no exception to that rule. Candede calls the resulting conflict problem Dependency Hell 2.0, because two context packages can give an agent contradictory instructions and nothing in the toolchain reports it. Conflict detection therefore becomes part of the release gate, using the same scenario tests that evaluate each package alone. Semantic versioning helps, provided the team agrees what counts as a breaking change, such as removing a rule or changing an output format. Anthropic’s production notes describe rainbow deployments that run old and new agent versions side by side. That is a sensible rollout pattern when a context update could disrupt agents in the middle of a task.

Third-party context is also a supply chain that needs the same scrutiny as any other dependency. Imported skills and rule files can leak credentials, expose private details, or carry hidden instructions aimed at hijacking an agent. That is why Candede suggests an AI software bill of materials recording who built a skill, how, and with which models. Our guide to MCP server security risks shows how tool descriptions themselves can be poisoned. Scanning, signing, and review therefore belong before a package enters the registry, not after an incident. A package that fails its scan or its scenarios never reaches the agents that would have trusted it.

Stage Four: Observing Context in Production

On top of distribution, the final stage closes the loop by learning from what agents actually do. Debois describes it as finding the gaps where context was insufficient, detecting drift between assumptions and reality, and iterating. Agent logs are the raw material, and they reveal missing context through stalls, retries, and repeated clarifying questions. Rejected pull requests and review comments are equally valuable, since each one is a human telling you the context produced the wrong thing. Candede adds the idea of turning every production failure into a new test case so the same mistake cannot recur unnoticed.

Freshness deserves its own monitoring because context rots even when nobody touches it. Atlan lists staleness, drift, and contradiction as the conditions to check for, and it cites a view that context goes stale in three places: vector indexes, caches, and prompt templates. A policy changes on Monday, the wiki updates on Tuesday, and the embedding index still serves the old text on Friday. Every context asset needs an expiry or review date, because a fact that was true at authoring time carries no guarantee after that. Scheduled checks that compare indexed content to its source of truth are cheap and catch most staleness.

Tracing and good telemetry make the whole observation stage practical for busy teams. LangChain recommends tracing token usage with LangSmith and testing whether each context change really helps, which is the observation half of the loop. Add runtime protections as well, because observation alone will not stop a bad instruction from acting. Candede describes context filters that work like a web application firewall for prompts, and our piece on deterministic guardrails for AI agents shows how rule-based checks sit beside model judgment. The findings flow back to stage one, where new gaps become new drafts, and the context development lifecycle begins its next turn.

Runtime Context Management: Compaction, Memory, and Isolation

Looking beyond the build pipeline, agents also manage context while they run, and the context development lifecycle must account for those runtime behaviors. Anthropic describes four main techniques for handling long tasks that outgrow a single window. Compaction summarizes a conversation near the limit and restarts with the summary, with clearing old tool results as the lightest form. Structured note-taking has the agent write progress to storage outside the window and read it back later. Sub-agents handle focused work in clean windows, returning condensed summaries of often 1,000 to 2,000 tokens. Just-in-time retrieval keeps lightweight references such as file paths or links and loads content only when needed.

The Manus team adds practical lessons that bear on lifecycle design. It calls the key-value cache hit rate the single most important metric for a production agent, so prompts should keep a stable prefix and avoid per-second timestamps at the start. It treats the file system as unlimited memory, compressing in restorable ways by dropping a page’s content while keeping its URL. It also has agents rewrite a to-do file during a task of about 50 tool calls, which keeps goals near the end of the window. Our overview of AI agent memory architecture covers storage options, and the Manus engineering write-up details the cache lessons. Runtime policies such as when to compact and what to keep are themselves context assets, so they belong under version control and evaluation.

Retrieval and Tool Context Across the Lifecycle

Given those runtime techniques, retrieval and tool definitions deserve their own lifecycle treatment because they change more often than prompts. A retrieval pipeline is configuration plus data, which includes the chunking strategy, the embedding model, the index contents, and the reranker. Any of those can change answer quality, so each should have a version and an evaluation set of questions with known good passages. The choice between retrieval and training is a design decision that interacts with this, and our comparison of retrieval versus fine-tuning trade-offs helps frame it. Re-running the retrieval evaluation after every index rebuild catches the silent regressions that otherwise reach users.

Graph-based retrieval adds relationships that flat vector search misses, which matters when answers depend on how entities connect. Our explainer on GraphRAG versus traditional RAG describes when the extra indexing effort pays off. Whichever method you choose, record the retrieval configuration alongside the context bundle so that a result can be reproduced later. Reproducibility turns an unexplained wrong answer into a debuggable event with a known index version and query. It also lets you compare two retrieval strategies fairly on the same scenario set before switching.

Tool context follows the same logic, because every tool description is read by the model before it decides what to call. Anthropic’s guidance says more tools do not always lead to better outcomes, and that a few tools aimed at high-impact workflows beat thin wrappers around every endpoint. Descriptions should be written like onboarding notes for a new teammate, with unambiguous parameter names and strict schemas. Because small description edits can move results substantially, tool text should pass through the same review and scenario testing as any prompt. Treat a tool description as production code that a model executes in its head.

Roles, Ownership, and Review Workflows for Context

For teams adopting the context development lifecycle, ownership is usually harder than tooling. Someone must be accountable for each context asset, and that person is rarely the platform engineer who runs the pipeline. The domain expert who knows what a correct answer looks like should own the content, while a context engineer owns the structure, the tests, and the release process. A reviewer from outside the author’s team checks for conflicts and unclear wording, and a security owner reviews anything imported from outside the organization. Code ownership files in the repository can encode these responsibilities so the right people are asked automatically.

A good review workflow for context closely mirrors an ordinary pull request. The author proposes a change, the harness runs the scenarios, and the reviewers see the diff in instructions next to the diff in results. Atlan’s loop keeps humans in charge of approval, with AI drafting and testing and people deciding what becomes canonical. Commenters on Debois’s post made a related observation: agents are finally pushing developers to write documentation, because poor documents now produce visibly poor output. Another linked the idea to Peter Naur’s 1985 paper on programming as theory building, which treats software work as building shared understanding. A lifecycle works when shared understanding is written down, tested, and owned, rather than carried in one person’s head.

Measuring Context Quality, Cost, and Drift

Rounding out the four stages, measurement gives the whole context development lifecycle a scoreboard that leaders can read. Quality metrics come first: scenario pass rate at the agreed error budget, retrieval recall on a labeled question set, and the share of production sessions flagged by users or reviewers. Freshness metrics follow, such as days since an asset’s last review and the age gap between an index and its source. Cost metrics matter just as much, because Anthropic reports that agents use about four times the tokens of chat and multi-agent systems about fifteen times. In its research evaluation, token usage alone explained roughly 80 percent of the variance in browsing benchmark performance, so spend and quality are entangled.

Caching changes the cost picture and therefore belongs on the dashboard. Anthropic says prompt caching can cut costs by up to 90 percent and latency by more than two times for reused prompts. Our primer on prompt caching to cut costs explains the mechanics and the prefix rules that make caches hit. A lifecycle that edits the top of a system prompt every week can quietly destroy its own cache hit rate. Track cost per successful task rather than cost per call, since a cheaper context that fails more often is not actually cheaper.

Drift needs a baseline to be visible, so record scenario results for every released version. When a model upgrade or an index rebuild lands, rerun the suite and compare against the last known good run. Differences that the team cannot explain are exactly the signals worth investigating before users find them. Budgeting guides such as our look at FinOps for agentic AI show how to turn these numbers into forecasts and alerts. Over time the same dataset also shows which context assets are carrying their weight and which can be retired to save tokens.

Security Risks in Shared and Retrieved Context

Despite the convenience of shared context in any lifecycle, every channel that carries text to a model is also a channel for an attacker. The OWASP project lists the main exposure in its Top 10 for LLM applications. Prompt injection is LLM01, data and model poisoning is LLM04, system prompt leakage is LLM07, and vector and embedding weaknesses are LLM08. Each risk maps neatly onto a stage of the lifecycle. Prompt injection arrives through retrieved documents and tool outputs during runtime, while poisoning enters through training data, memory stores, and imported packages. Leakage and embedding weaknesses are failures of how context is stored and exposed.

Shared rule files and skills deserve particular suspicion because agents follow them with elevated trust. A malicious line in an imported instruction file can tell an agent to exfiltrate secrets, install a dependency, or ignore a safety rule. The user may never even read the file that was imported. Tool descriptions are an equally attractive target, since the model reads them as authoritative guidance. Our guide to prompt injection attacks and defenses shows concrete attack patterns and the layered mitigations that work. The distribution stage is the natural checkpoint, because a registry can scan every package before any agent loads it.

Memory adds a persistence problem on top of the immediate injection risk. If an attacker plants a false fact once, a long-term memory store can replay it in every future session, long after the malicious page has gone. Retrieval indexes have a similar weakness, since anyone who can write to a source document can influence what the agent later retrieves. Treat every piece of retrieved or remembered text as untrusted input, no matter how trusted the system that stored it. Provenance tags that record where each chunk came from make it possible to quarantine a bad source and to explain a bad answer.

Defense in depth follows directly from these facts about trust and persistence. Limit what an agent can do with least-privilege tools so a successful injection has little to steal, and keep secrets out of context altogether. Scan packages at publication and again at install time, and sign releases so tampering is detectable. Add runtime filters and deterministic checks on high-risk actions, and require human approval for anything irreversible. Finally, rehearse failure by including adversarial scenarios, such as documents that contain hostile instructions, in the evaluation suite so regressions in defenses are caught like any other bug.

Where the Context Development Lifecycle Falls Short

Among the limits worth admitting, the first is that evaluation of context is inherently noisy. No suite of scenarios can prove that a context asset is correct for every future request. Model output varies between runs, judges have their own biases, and a suite of twenty scenarios cannot represent every user request. Error budgets and held-out sets reduce the problem but never remove it. A lifecycle can therefore give a team more confidence than the evidence supports if the scenarios are narrow. The honest framing is that the lifecycle lowers risk and speeds learning; it does not guarantee correctness.

Maturity of tooling is the second limit, because the ecosystem is young. Commenters on Debois’s post raised a chicken-and-egg problem about whether tooling or methodology should come first, and standards for context packages remain informal. Model upgrades can also invalidate carefully tuned instructions overnight, which means every release of a new model reopens the testing work. Smaller teams may find a full registry and review board heavier than the problem justifies. Anthropic’s long-standing advice to do the simplest thing that works applies here, so adopt only the stages that address a failure you have actually seen.

Ethics, Privacy, and Accountability in Context Design

Looking at the human side of the context development lifecycle, context is where an organization’s values, data, and assumptions are written into agent behavior. An instruction that tells an agent how to prioritize customers or screen applicants encodes a policy, and it deserves the same scrutiny as a written policy. Reviewers should ask who is affected by each rule, whether it treats groups fairly, and whether a person can appeal an outcome. Documenting the intent behind an instruction, as the generation stage recommends, makes these questions answerable later. If a context asset shapes decisions about people, it should have a named owner who can explain it to an auditor.

Privacy risks concentrate mostly in memory stores and in production logs. Long-term memory can store personal details that users never expected to persist. LangChain notes a case where a stored location surfaced in an unrelated image request, leaving users feeling the window no longer belonged to them. Apply data minimization, set retention periods, and give people ways to view and delete what is remembered. The observation stage creates its own tension, since mining logs for gaps means handling real conversations. Redact sensitive fields before analysis, restrict access, and delete context assets through the deprecation step when the business reason for them ends.

Common Mistakes Teams Make With Context

In practice, the most frequent mistake in any lifecycle is treating more context as safer. Teams add a rule after every incident and never remove one, so the instruction file grows until it contradicts itself. Chroma’s results suggest that irrelevant material actively hurts, since distractors lowered accuracy even when the correct passage was present. A second mistake is writing context with no owner and no review date, which guarantees rot. Pruning should be scheduled work, with every rule justified by a scenario that fails without it.

Testing mistakes come next on the list of common errors. Many teams run only happy-path scenarios, so the suite stays green while real users hit the edge cases. Others tune context against the same examples they use to score it, which inflates results and hides overfitting. Judges also drift when the judging prompt itself is never tested, so calibrate the judge against a small set of human-graded outputs. A passing test suite is only as trustworthy as the scenarios and the judge behind it.

Design mistakes round out the list of problems worth avoiding. Prompts that start with a changing timestamp destroy cache hits, and tool lists that change mid-session force expensive rework. The Manus team also warns about few-shot ruts, where repetitive examples cause an agent to copy a pattern even when it no longer fits, and it suggests adding structured variation. Copying another team’s rule file without adapting it imports assumptions that do not match your codebase. Finally, many teams ignore the model change, leaving instructions written for last year’s model in place after an upgrade.

A Maturity Path for Adopting the Context Development Lifecycle

With that catalogue of pitfalls in mind, adopting the context development lifecycle works best as a staged climb rather than a big-bang program. At the first level, context is ad hoc, living in personal notes and chat history. At the second level it is versioned in source control with owners and review dates, which is the cheapest step and removes the worst chaos. At the third level it is tested, with scenarios and a judge gating changes. At the fourth it is distributed through a registry with scanning, and at the fifth it is observed in production with failures feeding new tests.

Scale decides how fast a team needs to climb that ladder. Atlan’s framing suggests that one to three agents is bootstrapping, that lifecycle management becomes necessary somewhere between three and thirty agents, and that beyond thirty, sprawl and drift take over. Those thresholds come from a vendor and are rough, yet they match the common experience that informal habits stop working once many agents share the same knowledge. Start with versioning and a ten-scenario test suite, then add distribution and observation when the number of consumers makes manual coordination painful. Review the maturity level each quarter and pick the single next step with the best payoff.

The Future of the Context Development Lifecycle

Looking ahead, several trends suggest the context development lifecycle will become a standard part of AI engineering, though the details remain speculative. Registries for skills and rule files are already appearing, and talks such as Debois’s Context Is the New Code describe evaluation, distribution, dependency management, and feedback as the next frontier. Expect security scoring and software bills of materials for context packages to become routine, borrowing directly from software supply chain practice. Model providers are also adding built-in memory and file-based tools, which will push more context management into platforms. Teams that already version and test their context will adopt those features faster and with fewer surprises.

Automation will likely take over a larger share of the loop. AI systems already draft context, and the same systems can propose fixes when a production failure appears, leaving humans to approve. Enterprise vendors describe serving approved context to agents through protocols such as MCP, which would make a governed context layer a shared service rather than a per-project folder. Evaluation will likely shift from occasional benchmark runs to continuous testing on live traffic with automatic rollback. Standards for portable context are plausible, since the same instructions currently must be rewritten for each agent product.

The deeper change ahead is cultural rather than technical in nature. If context is the main constraint on useful agents, then documentation, domain knowledge, and clear intent become first-class engineering outputs rather than chores. Teams will hire and measure people on how well they capture and maintain what the organization knows. Time once spent writing code is expected to shift toward writing rigorous evaluations, as Candede argues. The organizations that treat context as a maintained product, not a one-time prompt, will see agents that improve with each release instead of decaying. Our explainer on the Model Context Protocol is a good place to follow that shift.

How to Implement Context Lifecycle Management in Your Team

Putting the stages into practice does not require new platforms, because a repository, a test runner, and a pipeline you already own are enough for a first version. The six steps below build a minimal context development lifecycle around a single coding agent or support agent. Each step produces one artifact that you can commit, review, and improve later. Expect the first pass to take a few days of focused work rather than a quarter. The snippets are deliberately small so you can adapt them to your own stack and model provider.

Step 1 - Inventory the context your agents read

Begin by listing every file, page, and store that an agent reads before it acts, because you cannot manage assets you have not found. A typical team finds between 10 and 40 items on the first sweep, spread across rule files, wiki pages, tool definitions, and saved memories. Classify each item into one of the five families from earlier: instructions, retrieved knowledge, memory, tool descriptions, or skills. Note who touches it, how often it changes, and which agents consume it. The command below finds the common agent instruction files in a repository so you have a starting list.

find . -type f \( -name "CLAUDE.md" -o -name "AGENTS.md" -o -name "*.mdc" \) -not -path "./node_modules/*"

Step 2 - Add a manifest with an owner, version, and review date

Next, give each asset a small manifest so that ownership and freshness are visible to tools and to people. Put the metadata in front matter at the top of the file, or in a sidecar for assets that cannot hold it. Limit the manifest to five fields or fewer so authors keep it current. The review date is the most important field, because it lets a scheduled job flag any asset older than 90 days. Version numbers follow semantic versioning, where removing a rule or changing an output format counts as a breaking change. Pro tip: start with only owner, version, and review date, because teams that demand ten fields on day one stop filling them in.

---
name: support-refund-policy
owner: billing-team
version: 1.3.0
review_by: 2026-12-01
agents: [support-bot]
intent: Keep refund answers consistent with the published policy.
---

Step 3 - Write the first ten scenarios from real failures

Then write scenarios, which are the tests of the whole lifecycle, starting from 10 genuine failures or tricky requests rather than invented ones. Each scenario holds an input, the context bundle it runs against, and a plain-language rubric describing what a good answer must contain or avoid. Mix easy cases with edge cases and at least two adversarial inputs, such as a retrieved document that contains a hostile instruction. Keep rubrics short and checkable so a judge model can score them consistently. Store the files next to the context they test so a change to one prompts review of the other.

- id: refund-after-30-days
  input: "I bought this 45 days ago. Can I still get a refund?"
  context: support-refund-policy
  rubric:
    must: ["states the 30 day window", "offers store credit if policy allows"]
    must_not: ["promises a cash refund", "invents an exception"]
  runs: 5
  pass_threshold: 4

Step 4 - Build a minimal evaluation runner

After that, write a runner that loads each scenario, runs the agent several times, asks a judge to grade each output against the rubric, and applies the error budget. A first version is about 40 lines and uses whichever model API your team already pays for. Keep the judge prompt in its own versioned file and spot-check it against 20 human-graded outputs before trusting it. Record tokens and latency for every run so cost shows up beside quality. Run scenarios in isolation, with no shared state, so one failure cannot hide or cause another.

Treat the runner as a product with its own tests. A judge that always answers pass will make the whole suite meaningless, so include a few scenarios that are known to fail and confirm that the runner reports them. Keep the output machine-readable, because the pipeline in the next step needs a clear exit code. Print the failing scenario identifiers so a reviewer can jump straight to the cause. Resist the urge to add clever scoring until the basic pass and fail signal is trusted by everyone.

import json, sys, yaml

def run_suite(path, run_agent, judge):
    failures = []
    for sc in yaml.safe_load(open(path)):
        passes = 0
        for _ in range(sc["runs"]):
            answer = run_agent(sc["input"], sc["context"])
            if judge(answer, sc["rubric"]):
                passes += 1
        ok = passes >= sc["pass_threshold"]
        print(json.dumps({"id": sc["id"], "passes": passes, "ok": ok}))
        if not ok:
            failures.append(sc["id"])
    return failures

if __name__ == "__main__":
    bad = run_suite(sys.argv[1], run_agent=lambda q, c: "", judge=lambda a, r: False)
    sys.exit(1 if bad else 0)

Step 5 - Gate every context change in your pipeline

Now connect the runner to the pipeline that already reviews code, so that every change to a context file triggers the suite automatically. Configure the job to run only when context paths change, which keeps costs down and makes the signal clear for reviewers. Require the job to pass before merge, and show the diff in scenario results next to the diff in text. Add a second scheduled job that fails when any manifest is past its review date. A third job can rerun the suite weekly against the live model to catch silent drift. Set the error budget deliberately, since 4 passes out of 5 runs is a common starting point.

name: context-ci
on:
  pull_request:
    paths: ["context/**", "evals/**"]
jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python evals/run_suite.py evals/scenarios.yaml

Step 6 - Release versions and feed production failures back

Finally, tag each approved change as a release, publish the bundle where agents can pull it, and let consumers pin a version. Log every agent session with the context version it used, so a bad outcome can be traced to a specific release. Review a sample of traces each week and convert every real failure into a new scenario within 24 hours, which is how the suite grows from 10 cases to hundreds. Retire assets through a deprecation note when the business reason ends, rather than silently deleting them. This closes the loop, and the lifecycle then improves with each cycle of observed use.

git tag -a context-v1.3.0 -m "Refund policy: add store credit rule"
git push origin context-v1.3.0

Key Insights

• Chroma's test of 18 models found that performance becomes less reliable as input length grows even on simple tasks, so a long window never substitutes for curated context.
• Anthropic's contextual retrieval cut failed top-20 retrievals from 5.7 percent to 1.9 percent once reranking was added to contextual embeddings, a 67 percent drop proving retrieval needs tests.
• The Manus team averages about 100 input tokens per output token and pays $0.30 per million for cached input against $3 uncached, so prompt prefix stability directly controls agent cost.
• Token usage alone explained about 80 percent of the performance variance on BrowseComp in Anthropic's multi-agent research system, which also consumed roughly 15 times the tokens of ordinary chat.
• A concise response format used 72 tokens against 206 for the detailed one in Anthropic's guidance on writing tools for agents, so response design is context design.
• Atlan cites MIT research finding that 95 percent of enterprise AI pilots deliver no measurable return, a figure it uses to argue that governed business context is missing.
• About 20 real-world queries were enough for Anthropic to see large effect sizes early, so a small scenario suite is a credible starting point for testing context changes.

The evidence points in one direction: context quality, not context volume, decides whether an agent is useful. Controlled studies show accuracy falling as windows fill, while production teams report that cost is dominated by what the model reads rather than what it writes. Retrieval, tool descriptions, and prompt structure each move results by double-digit percentages, which means they behave like code and deserve code-grade discipline. The same sources show that small evaluation suites reveal large effects, so the barrier to testing context is lower than most teams assume. A context development lifecycle packages these lessons into a loop of generation, evaluation, distribution, and observation that any team can adopt incrementally.

Dimension	Ad hoc context	Versioned and owned	Tested and gated	Full lifecycle
Ownership	Whoever wrote it last	Named owner per asset	Owner plus independent reviewer	Owner, reviewer, and security owner
Versioning	Edited in place, no history	Semantic versions in source control	Versions tied to test results	Pinned releases in a registry with changelogs
Testing	Manual spot checks	Occasional manual review	Scenario suite with an error budget	Scenarios, adversarial cases, and held-out sets in CI
Distribution	Copied between repositories	Shared repository folder	Shared repository with release tags	Registry with scanning, signing, and update channels
Change review	None	Pull request review of text	Pull request with scenario diff	Pull request, scenario diff, and approval gates
Observability	Anecdotes from users	Basic logs	Logs tagged with context version	Traces, failure mining, and automatic new test cases
Security posture	Trusts everything it imports	Informal reading of imports	Scans for secrets and injection	Provenance, bills of materials, and runtime filters
Typical failure	Silent rot and conflicts	Stale but tidy rules	Overfitting to the test set	Process overhead for small teams

Context Lifecycle in Practice: Three Examples With Numbers

Anthropic's Contextual Retrieval for Better Search Context

Anthropic described contextual retrieval, a method that prepends a short, model-generated note about where each chunk sits in its source document before the chunk is embedded and indexed. The company ran the approach across several knowledge domains and measured how often the top 20 retrieved chunks failed to include a relevant one. Contextual embeddings alone reduced failures by 35 percent, from 5.7 percent to 3.7 percent, and adding contextual BM25 raised the reduction to 49 percent. A reranking step on top cut failures by 67 percent, down to 1.9 percent. The one-time cost of writing the chunk notes was about $1.02 per million document tokens under the stated assumptions. Retrieval quality is a context asset that can be measured, so it should be versioned and tested like any other. The method still requires reprocessing documents whenever they change, and the gains come from Anthropic's own datasets, so teams should confirm them on their own corpus.

Manus and the Economics of a Stable Prompt Prefix

The team behind the Manus agent published its production lessons, built around the key-value cache hit rate, which it calls the most important metric for a production agent. It measured an average of about 100 input tokens for every output token, so reading the prompt costs far more than writing the answer. With Claude Sonnet, cached input tokens cost $0.30 per million against $3 for uncached tokens, a tenfold difference, or a 90 percent saving on every cached token. To protect the cache, the team keeps the prompt prefix stable, makes context append-only, avoids timestamps at the start, and masks tool choices instead of removing tools mid-session. Their engineering write-up also describes using the file system as restorable memory. The trade-off is reduced freedom to edit prompts, because any change to the prefix still invalidates the cache for that session.

Anthropic's Evaluation-Driven Tool Descriptions

Anthropic's engineers built evaluation tasks from realistic data, ran them in simple agent loops, and used the transcripts to refine tool definitions. They added a response format option so agents can request concise or detailed output. In a Slack example the concise response used 72 tokens against 206 for the detailed one, about 65 percent fewer. Claude Code caps tool responses at 25,000 tokens by default, and truncated results include guidance on how to narrow the next request. Anthropic's tool-writing guide reports that small changes to tool descriptions had large effects, and that refining them helped Claude Sonnet 3.5 reach state-of-the-art results on SWE-bench Verified. A limitation is that strong evaluation tasks may need dozens of tool calls, which makes them slow to build and verify. Overfitting to the training tasks remains a risk, so held-out test sets are still required.

Lessons From Case Studies of Context Lifecycle Management

Case Study: Anthropic's Multi-Agent Research System

The challenge was that open-ended research questions need broad, parallel exploration, and a single agent working in one window quickly runs out of room and focus. Anthropic built a system in which a lead agent plans the work and spawns subagents, each with its own clean context and a narrow task. Every subagent explores extensively but returns only a condensed summary to the lead. The team started with about 20 real-world test queries rather than a large benchmark. It scored outputs with an LLM judge that gave a grade from 0.0 to 1.0 plus a pass or fail verdict. Human testers then looked for the edge cases that the automated judge could not see.

The measured impact was large and clearly visible in the team's own tests. In Anthropic's account of the research system, a lead Claude Opus 4 agent with Sonnet 4 subagents beat a single Opus 4 agent by 90.2 percent. Parallel work also cut research time by up to 90 percent on complex queries. The cost is a serious limitation, because multi-agent runs use roughly 15 times the tokens of ordinary chat, so the design suits only valuable tasks. Errors compound in long-running agents, which required checkpointing, retry logic, and rainbow deployments for safe updates. The lesson for a lifecycle is that isolation and evaluation worked together, and that neither would have been enough alone.

Case Study: Atlan's Governed Context Lifecycle for Enterprise Agents

The problem Atlan describes is that most enterprise AI pilots stall because agents lack trustworthy business definitions to work from. It cites MIT research finding that 95 percent of pilots deliver no measurable return. It also repeats a view that 83 percent of organizations experiment with AI while only 17 percent are able to drive value. Atlan developed a six-stage context lifecycle covering discovery, structuring, validation, serving, monitoring, and deprecation. Its engineering loop runs through build, test, review, approve, deploy, and learn. Candidate context is tested against historical traces and reviewed by a human owner before release, then served to agents through protocols such as MCP.

According to Atlan's description of the lifecycle, added context improved text-to-SQL accuracy by 38 percent in its AI Labs benchmark. The company also says most customers find AI-drafted definitions beat their own first drafts. These are vendor claims about a vendor product, and the page does not publish the underlying data, so readers should treat them as directional rather than proven. The framework suggests that lifecycle management becomes necessary once an organization runs more than about three agents, which remains a rough heuristic. The six-stage structure is still a useful template for teams that need governance, auditability, and a clear retirement path for outdated business facts.

Case Study: Chroma's Context Rot Experiments

The core problem was that many teams assumed larger context windows would remove the need to curate inputs, yet little controlled evidence existed either way. Chroma built experiments that varied input length, the similarity between question and answer, the number of distractors, and the structure of the surrounding text. It ran them across 18 models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3. The findings were consistent, since performance became less reliable as inputs grew, even on simple tasks. On a conversational memory benchmark, focused prompts of about 300 tokens beat full prompts of about 113,000 tokens for every model family. One distractor lowered accuracy against the baseline, and four distractors lowered it further.

The study also carries several limitations that Chroma itself states openly. Shuffled haystacks produced better results than coherent ones, an outcome the authors call counterintuitive and could not fully explain. The test of topic similarity used only two topics, so no general conclusion followed. Tasks were also synthetic, which means real workloads may degrade differently, and one older model was excluded after content filtering blocked 60.29 percent of its tasks. Even so, the Chroma context rot report gave teams a practical reason to measure and prune context instead of simply adding more.

Common Questions About Context Lifecycle Management