Build Walls Outside the Prompt — Harness Engineering

If part 3 split the roles, the next question is: who enforces the contract? Writing “don’t touch .env” in a prompt is not the same as a hook, CI job, or sandbox that physically blocks the action.

Takeaway: Harness engineering is not making the model smarter. It is building walls around the model—tools, permissions, verification, and tracing.

This is part 4 of Coding Is Conversation, after multi-agent roles.

What is a harness? (model vs surroundings)

OpenAI’s agent docs draw a clean line (Sandbox Agents):

The model reasons; the harness decides what it may do and what must be proven. A strong model in a weak harness still ships incidents.

Why “click Allow” does not scale

Anthropic reports that relying on per-turn human approval in Claude Code hit approval fatigue—users approved roughly 93% of prompts. They invested in containment (OS sandbox: workspace writes, network denied by default) and saw a large drop in permission prompts (How we contain Claude).

Lesson: probabilistic supervision (humans reading every prompt) loses to deterministic boundaries (sandbox, hooks, CI).

I used to hammer Allow until muscle memory outran judgment—including on a shell command that deserved a second look. After that, dangerous shells go through a deny hook; only boring commands auto-approve.

Four pillars of a harness

1) Tools — what the agent may hold

• Read-only search vs write vs network vs secret stores
• MCP/plugins on an allowlist
• Same axis as locking Research to read-mostly tools in part 3

2) Sandbox — where work runs

• No writes outside the workspace
• Network default-deny or allowlisted hosts
• Temp dirs / containers / vendor sandbox sessions

3) Permissions — when to page a human

• Human gate only for high risk (deploy, secrets, destructive schema)
• Automate low risk (format, test, lint)
• PapaCoder: agents upsert drafts; Admin publishes

4) Eval — how you prove “done”

• “I ran tests” vs CI exit codes
• Quality/accuracy gates (PapaCoder editorial: ≥85, accuracyFail blocks)
• Failures become fuel for the next loop (part 5)

Eval discussions often treat model + harness as one system. Failures that return after every prompt tweak are usually harness holes.

Drill: weekend mini-harness checklist

A. AGENTS.md — policy (still useful)

# Harness policy (human-readable)

## Forbidden without explicit human approval
- Editing `.env*`, credentials, or CI secrets
- `git push --force`, production deploy
- Dropping tables / destructive migrations

## Required before claiming done
- Run unit tests for touched packages
- No new network calls to unknown hosts

## Publish
- Agents may create drafts only; humans publish

B. Cursor Hooks — deterministic blocks

Cursor runs scripts around the agent loop via .cursor/hooks.json. Hooks can observe, block, or modify behavior (e.g. beforeShellExecution) over JSON stdio (Hooks).

{
  "version": 1,
  "hooks": {
    "beforeShellExecution": [
      { "command": ".cursor/hooks/deny-dangerous-shell.sh" }
    ]
  }
}

#!/usr/bin/env bash
# .cursor/hooks/deny-dangerous-shell.sh — illustrative
# Read JSON stdin; deny force-push, curl|bash, rm -rf /, secret paths
# Follow current Cursor hooks schema for allow/deny payloads

The point is a process, not a polite sentence in a prompt.

C. CI — exit codes over vibes

# .github/workflows/agent-gate.yml (sketch)
name: agent-gate
on: [pull_request]
jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pnpm install --frozen-lockfile
      - run: pnpm test --filter ./packages/core
      - run: pnpm lint

If the agent opens a PR but cannot merge red CI, that is a real harness.

D. Secret boundary — PapaCoder-style

• Agent sessions: INTERNAL_API_KEY for draft API only
• No DATABASE_URL for agents
• Publish requires Admin session

Not “the model is nice”—the key is missing.

Do this today

Failure modes

Signal for part 5 (loops)

If the harness is the wall, loop engineering is how many plan→act→observe cycles you run inside it—retries, WIP limits, escalation. That is next.

Series roadmap — Coding Is Conversation

1. Tools & pricing
2. Vibe → deploy
3. Multi-agent
4. This post — harness
5. Loop engineering
6. Graph engineering
7. PapaCoder field notes

FAQ

Q. Is “harness” a product name?
A. It is a pattern. Cursor hooks, Claude sandboxes, OpenAI sandbox agents, CI, and secret isolation are harness pieces.

Q. Is AGENTS.md enough?
A. To start. When it is ignored, add hooks, CI, and sandbox.

Q. Where is PapaCoder’s harness?
A. Draft-only Internal API, quality/accuracy gates, Admin publish, no DB for agents.

Sources

• OpenAI — Sandbox Agents
• Anthropic — How we contain Claude
• Cursor — Hooks
• PapaCoder editorial draft publish runbook

Closing

Part 3 asked who does what. Part 4 asks who is forbidden from what—and puts that forbid list in the harness, not the prompt. That is when an agent team starts to look like a product team.

Going forward, the gap may grow less from model upgrades and more from how fast you freeze failures into evals. Part 5 turns those failures into loops.