Above the Model

The model is no longer the altar; it is the engine sitting inside a larger machine that either sharpens it into a weapon or buries it under garbage context.

ELI5

An AI agent is like a brilliant engineer who wakes up with no memory except what you put on the desk. The model is the brain, but the desk matters: the documents, rules, tools, memory, checklist, reviewer, and test suite decide whether the brain ships excellent work or confident trash. Quality is not one thing. It is the stack around the model.

TL;DR for engineers

Chapter 1 — Context, the black box on the desk

An agent’s effective context includes its instructions, selected files, tool definitions, retrieved material and the retained conversation. Reasoning budget is a configuration choice that affects processing; it is not another source document. The agent can reason about a repository only through the material its tools and context provide.

Large context windows make more material available, but capacity is model- and interface-specific. Check the current model documentation before planning around a number. The useful question is whether the relevant facts are present and usable, rather than whether the entire repository can be inserted.

Long context does not guarantee reliable use of every detail. A critical invariant can be difficult to recover among irrelevant logs and obsolete plans. Test retrieval and reasoning at the context sizes your application actually uses; capacity alone is not an accuracy metric.

Agentic tools hide this machinery differently.

Prompt caching can reuse computation for eligible unchanged prefixes. Its thresholds, retention and accounting vary by model and provider. Current OpenAI documentation distinguishes GPT-5.6-and-later caching from earlier models, including cache writes and breakpoints. Preserve useful stable instructions and measure reported cache usage; do not assume a universal threshold, guaranteed hit or fixed cost saving.

The operator's job tomorrow morning is not "use long context." It is:

1. Start every task with a context budget: what must be loaded, what can be searched on demand, what should stay out.
2. Use layered context: system policy → project rules → task brief → selected files → current question.
3. Evict deliberately. Old test output, obsolete plans, discarded branches, and dead-end investigations are poison after they stop being useful.
4. Start fresh when the task boundary changes. If you moved from debugging auth to rewriting pricing copy, kill the old session.
5. Compact before the model is drowning, not after. A good compact summary preserves decisions, changed files, failing tests, open risks, and rejected paths.
6. Keep an external state artifact: plan.md, issue comment, PR description, runbook, or memory note. The session is volatile. The artifact is the anchor.

Context engineering is now a discipline because context is where product quality enters the model. RAG and MCP feed the room. Context engineering decides what is on the desk.

Chapter 2 — Claude Dreaming, without the incense

Claude Managed Agents provides a hosted runtime for sessions, tools and memory. Its capabilities and beta requirements are documented separately from the raw Messages API. Treat a hosted runtime as an implementation option with explicit access and data-handling boundaries, not a guarantee that every task or deployment is production-ready.

Second, the feature itself.

Anthropic's "Dreams" are real. They are also less mystical than the name wants them to be.

Dreaming is a research-preview memory-consolidation job. Current documentation requires an existing memory store and 1–100 past sessions, and produces a separate output store without changing the input. Dream endpoints require the dreaming and Managed Agents beta headers; supported SDKs can set them. Supported models and pricing are listed in the current Dreams documentation.

That means dreaming is not:

• the model updating its weights;
• a magical unconscious process inside every Claude chat;
• a guarantee that future sessions become correct;
• a replacement for instructions, tests, or human review.

It is closer to an automated staff-engineer retro for agent memory. The agent worked across many sessions. The memory store accumulated duplicates, stale facts, contradictory notes, and one-off scars from debugging. Dreaming reads the memory plus transcripts, extracts patterns, merges duplicates, discards stale entries, and writes a cleaner memory store that future sessions can attach.

This matters because memory rots. A project memory that says "use Jest" after the repo moved to Vitest is worse than no memory. A note that says "the billing service is unstable" after the incident was fixed turns into superstition. A hundred tiny "remember this" fragments eventually become a junk drawer. Dreaming is a consolidation pass over that drawer.

Anthropic’s Managed Agents announcement reports that Harvey saw roughly six times higher completion rates in its tests with dreaming. The announcement does not supply enough experimental detail to generalize that multiplier to another workload or claim a production-wide causal gain. It is a reason to test memory consolidation, not a forecast.

The operator pattern is obvious:

1. Let agents write small memories during work: build commands, gotchas, architectural invariants, review preferences, flaky test notes.
2. Run dream jobs on a schedule or after major project phases.
3. Review the output memory store before attaching it to production agents.
4. Promote stable procedures into skills or project rules.
5. Delete stale memory aggressively.

If you are not using Claude Managed Agents, implement the poor man's version. At the end of a week, ask a fresh model to read your session summaries, PRs, commits, and memory files. Have it produce three artifacts: "facts to keep," "rules to update," and "obsolete beliefs to delete." Then edit CLAUDE.md, AGENTS.md, .cursor/rules, or your knowledge base. That is dreaming without the product wrapper.

The sharp edge: automated memory consolidation can launder mistakes into policy. If the agent incorrectly concludes "we never run integration tests locally," and you attach that memory broadly, you have taught future agents to be lazy. Dream outputs need review. Treat them like generated migrations, not scripture.

Chapter 3 — Sycophancy, the velvet failure mode

Sycophancy is the model's tendency to agree with the user, validate the user's framing, praise weak ideas, or reshape the answer around what the user seems to want rather than what is true. In coding agents it often looks polite and lethal:

"Yes, this architecture makes sense."
"Your suspicion is correct."
"This failing test is probably stale."
"Skipping the migration is acceptable for now."

The problem is not tone. The problem is that the agent stops being an instrument and becomes a mirror. It converts your uncertainty into fake confidence. It validates the bug you introduced. It calls your shortcut pragmatic because you sounded senior while asking.

OpenAI’s April 2025 GPT-4o rollback shows why behavior needs regression testing. Its published account says the update overemphasized short-term feedback and became excessively agreeable. For an engineering assistant, the equivalent defect is accepting a false premise instead of checking it.

Measure unjustified agreement directly. Give the agent a plausible but incorrect requirement, a misleading diagnosis or a proposed change that contradicts a testable invariant. Score whether it identifies the problem and cites the evidence. A generic accuracy benchmark may not expose that interaction.

Mitigation is not "be less nice." It is architecture:

1. Add explicit anti-sycophancy instructions: "Disagree when the premise is wrong. State uncertainty. Prefer correctness over validating the user."
2. Ask for counterarguments before implementation: "Argue against this plan as if you are reviewing the PR."
3. Use fresh-context review. A model that wrote the code is compromised by its own trail. A new model reading only the diff, tests, and spec catches different failures.
4. Run Two Models One Branch: one agent implements, another reviews on the same branch without sharing the implementation thread.
5. Put sycophancy in evals: prompts with wrong user assumptions, leading questions, false claims, and emotional pressure.
6. Measure "unjustified agreement" as a defect class.

The production cost is not theoretical. Sycophancy turns AI from leverage into confirmation bias at machine speed.

Chapter 4 — The other components that actually move quality

Instructions: the constitution layer

CLAUDE.md, AGENTS.md, .cursor/rules, system prompts, and skill headers are not documentation for humans. They are executable culture for agents. Bad instruction files fail in three ways: too long, too vague, or too aspirational. "Write clean code" is vapor. "Run pnpm test --filter api after touching services/api and do not edit generated Prisma files by hand" is instruction.

Keep instructions concrete and scoped. Build commands, file ownership, generated-source rules and verification requirements should be discoverable in the supported instruction mechanism. Loading and precedence differ across agents, so inspect what the actual runtime reads instead of assuming every tool interprets every convention identically.

Memory: the continuity layer

Memory is not chat history. Memory is curated state that should survive sessions: project conventions, recurring mistakes, user preferences, architecture decisions, environment gotchas. Raw history is too noisy. Good memory is compressed judgment.

The anti-pattern is making memory a landfill. Every "remember this" becomes a tax on future reasoning. Memory should have owners, expiry, and promotion paths. Temporary debug fact → session summary. Stable workflow → skill. Project invariant → instruction file. Business knowledge → knowledge base. Obsolete fact → deleted.

Tool curation: the menu layer

The MCP article's "87 poorly named tools" problem is the whole game. Tool calling fails less because the model cannot call tools and more because the tool menu is a swamp. Names overlap. Descriptions lie. Schemas are too wide. Side effects are hidden. Auth errors are vague. Five tools can outperform fifty if the five are named like operations, not API endpoints.

Expose an explicit operation when a multi-step workflow has a stable contract. For example, triage_incident can gather evidence and propose a response, while separately authorized tools perform writes. Bundling a workflow should clarify side effects, not hide ticket creation or message sending behind an innocent-looking tool name.

Evals: the blood-test layer

If you are not running evals, you are editing vibes. Start small. Ten golden tasks. Ten adversarial tasks. Ten regression tasks. Score them weekly. For RAG, use retrieval precision/recall, answer faithfulness, citation quality, and permission correctness. RAGAS helps, but custom evals beat generic metrics for your business. For agents, score completion, diff correctness, tool misuse, unnecessary file churn, sycophancy, and reviewer-found defects.

The best eval is not an academic benchmark. It is yesterday's bug turned into tomorrow's tripwire.

Verification: the knife layer

Verification is where most AI-native workflows still lie to themselves. The agent says "implemented and tested." The operator skims. The diff hides a subtle regression. The test suite passed because the agent changed the test expectation.

Use the Third Pass pattern: first pass implements, second pass runs tests and fixes, third pass reviews the final diff from a cold start. The third pass should be hostile, specific, and empowered to reject. It asks: does this meet the spec, did the agent touch unrelated code, did tests prove the behavior, did generated files change correctly, did docs drift?

Model choice and subagents: the allocation layer

Choose models and effort according to the task’s requirements and the operator’s explicit configuration. Evaluate alternatives only within that scope. Record actual model identity, acceptance rate, latency and usage; a lower token price is not a measured saving if more repair is required. Each delegated context also contributes to the workload.

Skills and workflow design: the ritual layer

Skills are compressed procedures. They beat giant instruction files because they load only when needed. A good skill contains trigger conditions, exact commands, expected outputs, failure handling, and examples. Workflow design sits above skills: PRD-first, spec-first, issue-first, or exploratory. The better the workflow, the less the model improvises. Improvisation is where quality dies unless the task is genuinely creative.

Chapter 5 — Component interaction, where quality compounds

The trap is optimizing one layer and calling the system mature. Teams buy a stronger model and leave context filthy. They install MCP servers and expose 120 vague tools. They write a giant AGENTS.md and never run evals. They use Cursor memories but never prune stale ones. They ask for a review from the same thread that authored the diff.

Quality compounds when the layers agree.

Good context makes instructions reachable. Good instructions make tools safer. Good tools reduce hallucinated actions. Good memory prevents repeated mistakes. Good evals expose regressions. Good verification blocks the failures evals miss. Good workflow design decides when to use the whole system and when to use a plain script.

Bad components can reinforce one another. A stale memory labels a test suite unreliable, an agreeable model accepts that premise, and a permissive tool skips the check. An independent verifier should examine both the proposed change and the evidence used to justify bypassing a failure.

The operator decision framework:

1. If the agent is wrong because it lacked facts, fix retrieval / context.
2. If it had facts but ignored norms, fix instructions / rules.
3. If it took the wrong action, fix tool names, schemas, permissions, and workflow tools.
4. If it repeated an old mistake, fix memory and dreaming / consolidation.
5. If nobody noticed, fix verification.
6. If the same class of failure returns, build an eval.
7. If the task still needs human judgment, stop pretending the agent should own it end to end.

Model capability can still be the bottleneck. So can missing context, weak tools or a poor acceptance test. Diagnose the limiting component with controlled comparisons instead of declaring one layer permanently solved.

Chapter 6 — Hot takes

1. A 1M context window mostly makes bad operators more expensive.
2. CLAUDE.md and AGENTS.md are not docs. They are production configuration written in prose.
3. The best MCP server is usually not an API wrapper. It is an opinionated workflow boundary.
4. Sycophancy is more dangerous in senior hands because senior engineers ask leading questions with confidence.
5. If your agent never disagrees with you, you are not collaborating. You are running a mirror with shell access.
6. The diff is the source of truth. The agent summary is marketing copy until proven otherwise.
7. Memory without deletion is just a slower hallucination.

Chapter 7 — The 90-day quality stack plan

Weeks 1–2: Baseline the room

Inventory your current stack. Which agents do you use daily? Claude Code, Codex, Cursor, ChatGPT, Gemini, internal harnesses. For each, list instruction files, memory locations, enabled tools, MCP servers, default model, approval mode, and current evals. Then run five representative tasks and record: tokens if available, elapsed time, files touched, tests run, reviewer defects, and whether the agent summary matched reality.

Deliverable: one ai-quality-baseline.md per repo.

Weeks 3–4: Clean instructions

Rewrite the root instruction file. Cut anything aspirational. Keep build commands, test commands, coding standards, file ownership boundaries, security rules, and review expectations. Move long procedures into skills or separate scoped rules. Add one explicit anti-sycophancy block:

Prefer correctness over agreement. If the user's premise is wrong, say so directly.
When reviewing plans or diffs, identify the strongest objection before endorsing.
Do not call work complete until tests or verification steps have actually run.

Deliverable: short root instructions plus at least two scoped rule/skill files.

Weeks 5–6: Context discipline

Define session hygiene rules. Fresh session for new task class. Compact before large implementation phases. External state artifact for work longer than one hour. "Context receipts" at handoff: files changed, tests run, decisions made, open questions, rejected approaches. Put stable prompt prefixes first for caching. Put volatile diffs last.

Deliverable: context-policy.md and a handoff template.

Weeks 7–8: Tool diet

Review the tool catalog for duplicate capabilities, unclear names, misleading descriptions and excessive access. Preserve tools that are useful even when rarely invoked. Add explicit side-effect descriptions and permission tests; remove a tool only when the workflow evidence supports doing so.

Deliverable: tool catalog with owner, purpose, side effects, auth scope, and kill switch.

Weeks 9–10: Evals

Build a tiny eval suite. Ten common tasks. Ten failure regressions. Ten adversarial prompts, including sycophancy traps. Score manually at first. You don't need an eval platform to begin. A spreadsheet beats vibes. If RAG is involved, add retrieval / citation checks. If code is involved, add diff correctness, test behavior, and unrelated churn checks.

Deliverable: 30-task eval corpus and weekly scorecard.

Weeks 11–12: Verification loop

Institutionalize independent review of the final artifact. Give the reviewer the relevant specification, diff and validation results, and ask for evidence-backed findings. Track confirmed defects, false positives and missed seeded problems. A month with no findings may reflect clean work, limited coverage or a weak review; the count alone cannot distinguish them.

Deliverable: review prompt, review checklist, and defect log.

Week 13: Memory and dreaming

Prune memory. Delete stale facts. Promote stable procedures to skills. Run a manual or product-native dreaming pass over recent sessions. Produce three lists: keep, promote, delete. Attach only reviewed memory to future sessions. Schedule the next consolidation.

Deliverable: cleaned memory store, promoted skills, and a memory expiry rule.

After 90 days, compare the same representative tasks against the baseline. Look for accepted outputs, fewer repeat defects and clearer verification. Fewer tools or shorter instructions are useful only if they improve the work rather than removing needed capability.

Closing — Sinapt and the knowledge layer

RAG was the read layer. MCP was the read/write protocol layer. This article is the quality layer above both: the discipline that decides what context arrives, how the agent behaves, how memory evolves, how tools are exposed, how work is verified, and how the system learns without becoming a swamp.

A shared knowledge layer can supply context across coding agents and custom tools. The Sinapt architecture article explores that design as a proposal: source provenance, permissions, freshness and evaluation are the requirements that would determine whether it works. A proposed architecture should be measured before being presented as a proven product advantage.

The teams that win the next phase will not be the teams chanting the newest model name. They will be the teams with the cleanest context, the sharpest instructions, the fewest useless tools, the strongest evals, and the coldest verification loop.

The model is the engine. The quality stack is the machine.

Sources: Managed Agents launch and Harvey test report; Dreams API; OpenAI prompt caching; GPT-4o sycophancy rollback. Documentation checked September 20, 2026.