brain/
conceptartificial-intelligence

Agent harness

Notes

Vintage: 2026-03–08. Primary definitions are Anthropic (2 Apr 2026) and LangChain (10 Mar 2026) as synthesized in 2026-08-27-best-agent-harnesses-for-programming. Capability and cost numbers are snapshots of that pass, not the current state of the world.

Agent harness

One-line summary: The software around a model — loop, tools, context/memory, sandbox, hooks, and orchestration — not the model itself. In 2026 this layer is sold both as a product packaging (Claude Code, Codex CLI) and as a reusable SDK.

The insight

A programming agent in 2026 is Agent = Model + Harness. The product (Claude Code, Codex, Cursor) is one packaging of a harness. The reusable substrate is now also sold as an SDK: Anthropic’s Claude Agent SDK, OpenHands’ Software Agent SDK, and OpenAI’s Codex runtime with permission profiles and AGENTS.md. That is a different grain from an IDE recap.

Harness design is deciding what belongs in the scaffolding and what to remove as models improve. The same lab, in the same month-range, both adds multi-agent orchestration for long-horizon apps and prunes context-reset scaffolding that became dead weight on Opus 4.5. The consistent rule in this source is “complexity must earn its keep,” not a universal more-or-less.

Evidence

  • From 2026-08-27-best-agent-harnesses-for-programming: Anthropic (2 Apr 2026) defines an agent harness as “the software scaffolding around a model: the loop, tools, context management, and guardrails that turn raw intelligence into a working agent.”
  • From the same source: LangChain (10 Mar 2026) states the same boundary as “Agent = Model + Harness” and “if you’re not the model, you’re the harness,” listing system prompts, tools/skills/MCP, filesystem/sandbox/browser, subagent orchestration, and hooks (compaction, continuation, lint).
  • From the same source: Anthropic’s Claude Agent SDK is “the same tools, agent loop, and context management that power Claude Code.”
  • From the same source: Claude 3.5 Sonnet hit 49% SWE-bench Verified with only bash + a text editor; skills, programmatic tool calling, and the memory tool are compositions of bash/editor.
  • From the same source: Anthropic contrasts context resets (clean slate + handoff file; needed for Sonnet 4.5 “context anxiety”) with compaction (summarize in place; enough for Opus 4.5+).
  • From the same source: two official Anthropic orchestration patterns — (1) strip the harness and let the model orchestrate via code exec (BrowseComp 45.3% → 61.6% when Opus 4.6 filtered its own tool outputs); (2) add a planner/generator/evaluator multi-agent harness for long-horizon apps.
  • From the same source: solo Opus 4.5 built a broken game maker in 20 minutes for $9; the full planner/generator/evaluator harness ran 6 hours for $200 and produced a playable app. Cost and “best” are not the same axis.

Fetched arXiv abs (Sep 1–2 window) — existence only

From 2026-09-02-grok-com-ai-news-digest-2026-09-02-fable-5-1-astra-atlas-g20. Only papers whose abs pages were fetched are cited. Recap-named titles without a fetched abs stay recap-only.

  • From the same source: arXiv:2609.01481 — Harness-of-Harness (HoH); abstract reports average relative gain of 52.25% after three iterations on named benches. Multi-day autonomous software-development harness. Attach as a paper that exists; not a rewrite of this page's Anthropic/LangChain definitions.
  • From the same source: arXiv:2609.01567 — SAGE (Selective Agent Guidance via Entropy); agent guidance from imperfect VLM teachers.
  • From the same source: arXiv:2609.01222 — M-CPE / X-CPE against 12 real harnesses including Claude Code and Codex. Cited as a paper that exists. This wiki page does not reproduce attack procedures.
  • Not fetched (recap-only as named papers): a distinct 24h paper titled "LLM scientific law discovery"; "construct-validity issues in LLM agent safety evaluations."

Math-formalization harness (2026-09-04) — not a definition rewrite

Meta AIRA₃ swarm (2026-09-05) — model + harness pairs, not a definition rewrite

AA Terminal-Bench 4.0 harness swap (2026-09-07) — not a definition rewrite

Layer map (from this source)

LayerWhat the source says
LoopReAct-style plan → tool → observe → repeat (Anthropic, openhands). LangChain adds Ralph-style “don’t let the model exit” hooks.
ToolsLoad-bearing claim is bash + editor; typed tools stay for UX, observability, or hard-to-reverse actions. Codex: schema-first exec + separate permission engine.
Memory / instructionsCompeting file standards — see agents-md-vs-claude-md. Memory-folder lift on BrowseComp-Plus 60.4% → 67.2% (Sonnet 4.5); compaction scales Opus 4.6 to 84% on BrowseComp vs Sonnet 4.5 at 43%.
SandboxCodex: least-privilege policy language that refuses a command if the policy cannot be applied. OpenHands: container/remote isolation. Claude Code (fetched docs): permissions + auto-mode (a second Claude judging bash), not a full OS jail.
OrchestrationStrip vs add — see Contradictions. Dynamic workflows / ultracode are the productized add path in claude-code.
EvalsHarbor / Terminal-Bench 2.0 is the public harness-vs-harness table. See terminal-bench-harness-swing.

Contradictions / tensions

  • Add scaffolding vs strip it. Same lab, same month-range: long-running-apps shows a three-agent harness 20× more expensive and qualitatively better than solo; harness-design says prune dead weight because context-reset scaffolding for Sonnet 4.5 became useless on Opus 4.5; dynamic workflows then adds orchestration. Left open — not a universal more-or-less.
  • Vendor default vs eval-optimized harness. See terminal-bench-harness-swing and does-terminal-bench-harness-swing-transfer-to-daily-coding.
  • LangChain Deep Agents claim vs fetched TB table. LangChain claims a harness-only jump from top-30 to top-5 on TB 2.0; the fetched leaderboard has Deep Agents + GPT-5.2-Codex at 66.5% (rank 27).

Open questions

Sources

Related

Referenced by
Mechanisms
AIRA₃ swarm → private-set gold → claimed RSI pathHarness choice → Terminal-Bench score swing
Concepts
AGENTS.md vs CLAUDE.mdAI Coding BenchmarksAI Coding Tool Landscape (2026)Artificial Analysis Intelligence IndexAutomationBench-AAClaude formalizes Fermat's Last Theorem in Lean (Anthropic, Sep 4, 2026)CLAUDE.md as team knowledge baseMulti-Agent Code ReviewOpenAI Hugging Face incident report (August 2026)Parallel Claude workflowTerminal-Bench harness swingWhat AI research LLMs can and can't automate (the capability boundary)
Questions
Can LLMs choose the right research question to investigate (not just run the experiment)?Does the Terminal-Bench harness swing transfer to daily coding?
Entities
AIRA₃Andrej KarpathyAnthropicBoris ChernyClaude CodeClaude Fable 5.1 / Mythos 5.1Codex CLIOpenAIOpenHandsProve2MeWorld Labs Atlas