Same Model, Different Behavior: What Coding Agent CLIs Change Around the LLM
A research article on why the same LLM behaves differently in different coding agent CLIs. Research date: Aug 2, 2026. All URLs verified; prompt quotes are from the actual prompt files in the official repos.
TL;DR
Run the same frontier model in Codex CLI, Claude Code, Gemini CLI, or OpenCode and you get materially different behavior — different levels of proactiveness, different verbosity, different testing habits, different willingness to plan. The agent loop is identical (see the companion article coding-agent-loop-pattern.md); what differs is everything the harness puts around the model:
- The system prompt — content (identity, autonomy stance, verbosity rules, risk frameworks) and size (from ~350 tokens in Pi to ~6,000+ in Gemini CLI).
- Per-model prompts — Codex ships a different prompt per model generation; OpenCode ships 14 model-specific prompt files.
- Memory injection — AGENTS.md / CLAUDE.md / GEMINI.md hierarchies, auto-memory, skills, environment context.
- Inference parameters — thinking-token budgets and reasoning effort are harness-set knobs that swing same-model benchmark scores by 5-10+ points.
- Sub-agent orchestration — hidden
dispatch_agentloops (Claude Code), named sub-agents (Gemini CLI), child sessions (OpenCode), or none (Pi). - Context management mandates — whether the prompt tells the model to treat context as a precious resource, and how compaction/exit conditions are enforced.
Evidence is both direct (people editing one line of a system prompt and changing the behavior of the same model) and quantitative (benchmarks holding the model constant and varying only the harness).
Part 1 — Evidence: the harness changes the model's behavior
1.1 Direct evidence: prompt edits change same-model behavior
Gemini CLI — one line changes everything. Google Developer Advocate Daniela Petruzalek documented that the Gemini CLI system prompt contains the line Proactiveness: Fulfill the user's request thoroughly, including reasonable, directly implied follow-up actions. — and identified it as "the source of 80% of my problems": the same model would delete files when asked a rhetorical question ("why did you add this file?"). She fixed the behavior by editing the prompt file and overriding the system prompt via the undocumented GEMINI_SYSTEM_MD environment variable (https://danicat.dev/posts/20250715-gemini-cli-system-prompt/). Same model, different prompt line → different behavior. (Notably, the current Gemini CLI prompt in the repo has since rewritten this mandate to "Take reasonable liberties to fulfill broad goals while staying within the requested scope" — the prompt evolves, and behavior evolves with it: https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/prompts/snippets.ts)
Claude Code — prompt churn changes model behavior. Mario Zechner (creator of the Pi agent), who prefers running Claude models, complains: "The system prompt and tools also change on every release, which breaks my workflows and changes model behavior" (https://mariozechner.at/posts/2025-11-30-pi-coding-agent/). He maintains cchistory, a version-diff tool for Claude Code's prompt: https://cchistory.mariozechner.at.
Anthropic's own numbers — orchestration alone is worth 90%. In building Claude's Research feature, Anthropic found that a multi-agent system with Claude Opus 4 as lead agent + Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on their internal research eval — same model family, different harness architecture. They also report that "a prompt tweak might boost success rates from 30% to 80%," that token usage explains 80% of performance variance, and that "small changes to the lead agent can unpredictably change how subagents behave" (https://www.anthropic.com/engineering/built-multi-agent-research-system).
1.2 Quantitative evidence: benchmarks with the model held constant
Aider polyglot leaderboard — harness settings move same-model scores. Aider's benchmark holds the harness fixed and varies models; but rows for the same model under different harness-set parameters show the lever sizes (https://aider.chat/docs/leaderboards/):
| Same model, different harness settings | Pass rate | Well-formed edits |
|---|---|---|
| claude-sonnet-4, 32k thinking tokens | 61.3% | 97.3% |
| claude-sonnet-4, no thinking | 56.4% | 98.2% |
| gemini-2.5-pro-06-05, 32k thinking | 83.1% | 99.6% |
| gemini-2.5-pro-06-05, default thinking | 79.1% | 100% |
| gpt-5, reasoning high | 88.0% | 91.6% |
| gpt-5, reasoning medium | 86.7% | 88.4% |
| gpt-5, reasoning low | 81.3% | 86.7% |
| o3, reasoning high | 81.3% | 94.7% |
| o3, default | 76.9% | 93.8% |
| o3 (high) + gpt-4.1 architect/editor split | 78.2% | 100% |
Thinking-token budgets and reasoning-effort are harness-controlled parameters (the CLI decides them, or the user via CLI flags). Same model, different settings → 5-15 point swings, plus visible edit-format adherence differences.
SWE-Search — the same five models, different scaffold, +23%. SWE-Search applied an MCTS scaffold on top of standard open-source agents and demonstrated "a 23% relative improvement in performance across five models compared to standard open-source agents without MCTS" (https://arxiv.org/abs/2410.20285).
mini-swe-agent — >74% SWE-bench Verified with a 100-line agent and no tools besides bash. The SWE-bench team's minimal agent deliberately has "no tools other than bash — it doesn't even need to use the tool-calling interface of the LMs," a completely linear history, and subprocess.run for every action. The README frames it as "a system that puts the language model (rather than the agent scaffold) in the middle of our attention" — and it still scores >74% on SWE-bench Verified and "beats Claude Code and Codex on DeepSWE" (https://github.com/SWE-agent/mini-swe-agent). The same team's original SWE-agent paper had claimed the interface (tool design) was the key to agent performance; mini is the explicit retraction of that claim as models improved (https://arxiv.org/abs/2405.15793).
Terminal-Bench — minimalism is competitive. Pi (a ~350-token system prompt, four tools, zero safety rails) ran a Terminal-Bench 2.0 submission with Claude Opus 4.5 and placed competitively against Codex, Cursor, Windsurf and others (https://www.tbench.ai/, results: https://gist.github.com/badlogic/f45e8f6e481e5ab7d3a50659da84edaa). Terminal-Bench's own Terminus 2 agent — which just hands the model a tmux session, no file tools at all — "holds its own against agents with far more sophisticated tooling" (Zechner, https://mariozechner.at/posts/2025-11-30-pi-coding-agent/).
1.3 The prompts are public, and they differ measurably
The strongest resource comparing them directly is Daniel Vaughan's dissection of Codex CLI vs Gemini CLI vs Claude Code vs Pi (https://codex.danielvaughan.com/2026/04/19/system-prompts-compared-codex-gemini-claude-code/), cross-checked against the actual source files:
| Prompt | Location | Est. tokens |
|---|---|---|
| Codex CLI, GPT-5.1 full prompt | codex-rs/core/gpt_5_1_prompt.md in https://github.com/openai/codex |
~6,000 |
| Codex CLI, compact prompt (fine-tuned Codex models) | gpt-5.2-codex_prompt.md |
~1,900 |
| Gemini CLI, assembled | https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/prompts/snippets.ts | ~5,000-6,000 |
| Claude Code, core | bundled in npm @anthropic-ai/claude-code; not published by Anthropic |
~3,000-5,000 |
| Pi | https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/system-prompt.ts | ~350 |
| OpenCode, per model | 14 files in https://github.com/anomalyco/opencode/tree/dev/packages/opencode/src/session/prompt (anthropic.txt 8.2KB, gemini.txt 15.4KB, codex.txt 7.4KB, gpt.txt, kimi.txt, meta.txt, ...) | varies by model |
Measured total initial payload (prompt + tool schemas + context, from actual API traces): Codex CLI ~8,497 input tokens for a minimal session (measured by Alex Fazio on Codex CLI v0.114.0, https://gist.github.com/alexfazio/359c17d84cb6a5af12bac88fa1db9770); Gemini CLI ~13,055 tokens for two trivial math queries (https://github.com/google-gemini/gemini-cli/issues/3784); Pi <1,000 tokens. Community collections of extracted prompts: https://github.com/asgeirtj/system_prompts_leaks (61.9k stars), https://github.com/badlogic/cchistory.
Part 2 — The levers, one by one
2.1 System prompt content: the persona
Identity lines from the prompts (verified):
| CLI | Identity | Autonomy stance |
|---|---|---|
| Codex CLI | "You are GPT-5.1 running in the Codex CLI, a terminal-based coding assistant." | "Persist until the task is fully handled end-to-end within the current turn whenever feasible... assume the user wants you to make code changes" |
| Gemini CLI | "You are Gemini CLI, an interactive CLI agent specializing in software engineering tasks." | "Fulfill the user's request thoroughly" (older); today "Take reasonable liberties to fulfill broad goals while staying within the requested scope" |
| Claude Code | "You are an interactive agent that helps users with software engineering tasks." | "Don't add features, refactor code, or make 'improvements' beyond what was asked." |
| Pi | "You are an expert coding assistant operating inside pi, a coding agent harness." | No autonomy instructions at all — "delegated to the model's training" |
Behavioral consequences documented by users: Gemini CLI's older proactiveness line made it perform "directly implied follow-up actions" — deleting files mid-question (https://danicat.dev/posts/20250715-gemini-cli-system-prompt/); Codex's prompt fights "AI slop" explicitly ("Avoid collapsing into 'AI slop'..."); Claude Code's prompt enforces YAGNI ("Three similar lines of code is better than a premature abstraction") and a structured risk taxonomy ("Measure twice, cut once"). Full philosophy comparison in https://codex.danielvaughan.com/2026/04/19/system-prompts-compared-codex-gemini-claude-code/.
Divergences with direct behavioral effect (per Vaughan's comparison + primary files):
| Dimension | Codex CLI | Gemini CLI | Claude Code | Pi |
|---|---|---|---|---|
| Verbatim vs concise output | tiered rules (tiny/medium/large changes) | "<3 lines of text output per response" | "If you can say it in one sentence, don't use three." | silence (nothing to reinforce verbosity) |
| Testing stance | "Don't add tests to codebases with no tests" | "ALWAYS search for and update related tests" + "Validation is the only path to finality" | nothing in core prompt | nothing |
| Risk/approval behavior | adapts to approval mode; dirty-worktree rules | "Do not take significant actions beyond the clear scope... ask for confirmation first" | "Executing Actions with Care" section; confirm destructive ops | none (YOLO by default) |
| Recovery rules | — | 3-strike rule: after 3 failed fix attempts, stop and propose a different approach | — | — |
| Context economy | (harness-level compaction) | "your own context window is your most precious resource"; context-efficiency mandate with worked examples | auto-compaction | no compaction at all (issue #92) |
2.2 Per-model prompts
- Codex CLI: separate
.mdprompt per model generation — full prompts for base GPT models (~24K chars) and compact prompts for fine-tuned "Codex" models (~7K chars) that already know the tools. You candiffprompt evolution across generations in the repo (https://github.com/openai/codex/tree/main/codex-rs/core). Unknown/local models fall back to a ~276-line default prompt; users can fully replace base instructions viamodel_instructions_fileinconfig.toml— with the warning that deviating "will likely degrade model performance" (https://github.com/openai/codex/blob/main/codex-rs/config/src/config_toml.rs). - OpenCode: ships 14 prompt files — one per model family (
anthropic.txt,codex.txt,gemini.txt,gpt.txt,kimi.txt,meta.txt,trinity.txt,beast.txt,copilot-gpt-5.txt...) plusplan-mode.txt/plan.txtoverlays (https://github.com/anomalyco/opencode/tree/dev/packages/opencode/src/session/prompt). Same CLI, different model → different system prompt. - Copy culture: OpenCode's
anthropic.txtis, per Zechner, "a cut-down version of the original Claude Code prompt they copied" — the repo retains ananthropic-20250930.txtsnapshot of the source (https://mariozechner.at/posts/2025-11-30-pi-coding-agent/). This is why behaviors feel similar across "Claude-style" CLIs: they inherit Claude Code's persona. The wren.wtf critique of OpenCode makes the same point (https://wren.wtf/shower-thoughts/stop-using-opencode/).
2.3 Memory and context-file hierarchies
| CLI | Context file | Behavior |
|---|---|---|
| Codex CLI | AGENTS.md (cross-tool standard, 60,000+ repos, 25+ tools) | hierarchical root→leaf; direct instructions override; 32 KiB cap |
| Claude Code | CLAUDE.md + auto-memory (.claude memory files) |
"Compact Instructions" section survives compaction |
| Gemini CLI | GEMINI.md + hierarchical memory (global ~/.gemini → project → subdirs) | context precedence order; cannot override Core Mandates |
| Pi | AGENTS.md; SYSTEM.md can fully replace the prompt | because the default prompt is tiny, a good SYSTEM.md "shifts the agent's personality more dramatically than in any other tool" |
Memory is a behavioral lever in both directions: project files teach conventions, and Gemini CLI even maintains a per-project private memory index (~/.gemini/tmp/<hash>/memory/MEMORY.md) that the agent can edit itself (visible in https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/prompts/snippets.ts). Portability matters too: AGENTS.md works across 25+ tools; CLAUDE.md/GEMINI.md lock you in.
2.4 Inference parameters (harness-set knobs)
- Thinking-token budgets and reasoning effort move same-model scores by 5-15 points (aider data above). The CLI (or its config) decides defaults: Codex sets per-model defaults; Gemini CLI historically allowed
--thinking-tokens; aider surfaces them as flags. - Different CLIs hitting the same provider API even send different request shapes: Codex uses the Responses API statelessly with exact-prefix prompts for cache hits; Pi's pi-ai layer must paper over provider differences (thinking traces as
<thinking>blocks, signed event-stream blobs, no tool-call streaming from Google) (https://mariozechner.at/posts/2025-11-30-pi-coding-agent/).
2.5 Sub-agent orchestration
| CLI | Mechanism | Visibility |
|---|---|---|
| Claude Code | dispatch_agent tool, concurrent, stateless, read-only sub-agents; depth-limited |
"zero visibility into what that sub-agent does" (Zechner); prompts unpublished (https://simonwillison.net/2025/Jun/2/claude-trace/) |
| Gemini CLI | named sub-agents in the prompt (codebase_investigator, cli_help, generalist, browser_agent) + "strategic orchestrator" mandate | delegated work "consolidated into a single summary" |
| OpenCode | subagents = child sessions via Task tool | session files visible |
| Pi | none — "no dedicated sub-agent tool"; spawning itself via bash is the workaround | full output observability |
Anthropic's research post is the canonical argument that orchestration is a performance lever, not just an architecture choice: subagents "provide compression" and "separation of concerns," and the multi-agent layout beat single-agent Opus 4 by 90.2% (https://www.anthropic.com/engineering/built-multi-agent-research-system).
2.6 Loop-level behavior mandates
- Exit conditions: Pi's agent loop has "no max steps or similar knobs... The loop just loops until the agent says it's done" (https://mariozechner.at/posts/2025-11-30-pi-coding-agent/); OpenCode forces text-only summarization when
stepsis exhausted (https://opencode.ai/docs/agents/); Claude Code stops auto-compacting with a thrashing error to avoid infinite loops (https://code.claude.com/docs/en/how-claude-code-works). - Compaction philosophy: automatic summarization (Claude Code), automatic
/responses/compact(Codex), hidden compaction agent (OpenCode), none (Pi — Zechner reports cramming "hundreds of exchanges" into one session without it). - Context-efficiency in the prompt itself: only Gemini CLI's prompt teaches the model context economics (estimating-context-usage heuristics, parallel reads to save turns,
wait_for_previoussequencing rules) — behavior shaping via instructions, not just harness mechanics.
Part 3 — Which levers matter most
| Lever | Magnitude of effect | Evidence |
|---|---|---|
| System prompt content (autonomy/verbosity/testing stance) | Highest, most user-visible | Petruzalek's one-line fix; Vaughan's philosophy table; prompt sources |
| Sub-agent orchestration | Very high on hard/long-horizon tasks | Anthropic +90.2%; token-usage-80%-of-variance |
| Inference parameters (thinking budget) | High (5-15 pts on same benchmark) | aider leaderboard rows |
| Prompt size / token overhead | Medium-high economics; critical for local 32K models | Fazio ~8.5K baseline; gemini-cli #3784 13K; Pi <1K |
| Per-model prompts | Medium; avoids wasted tokens, tunes behavior | Codex compact prompts; OpenCode 14 files |
| Memory/context-file hierarchy | Medium; compounds across sessions | AGENTS.md/CLAUDE.md/GEMINI.md ecosystems |
| Exit conditions/compaction | Low-medium; mostly cost & robustness | pi issue #92; opencode steps |
Practical takeaways for users: the fastest behavioral lever is the system-prompt override each tool exposes — Codex model_instructions_file (https://github.com/openai/codex/blob/main/codex-rs/config/src/config_toml.rs), Gemini GEMINI_SYSTEM_MD / GEMINI_WRITE_SYSTEM_MD (https://danicat.dev/posts/20250715-gemini-cli-system-prompt/), Claude Code CLAUDE_CODE_SIMPLE=1 (https://codex.danielvaughan.com/2026/04/19/system-prompts-compared-codex-gemini-claude-code/), Pi SYSTEM.md — plus writing AGENTS.md/CLAUDE.md/GEMINI.md, which every vendor ranks above their own defaults in precedence.
Caveats: no public benchmark isolates model × CLI in a controlled matrix (the earlier benchmark review reached the same conclusion: all leaderboards score agent+model combos). Pi's Terminal-Bench run used a different model (Claude Opus 4.5) than its competitors, so it proves minimalism is viable, not that harness alone explains the gap. The cleanest model-held-constant evidence remains aider (settings), SWE-Search (+23% across five models), and Anthropic's internal evals.
Conclusion
The loop is the same; the model can be the same; the CLI is the difference. Every coding agent CLI is a prompt + memory + orchestration + inference-settings wrapper around a general model, and each wrapper encodes a bet about what a good software engineer is: Codex bets on persistence and anti-slop aesthetics, Gemini on orchestration and validation ("Validation is the only path to finality"), Claude Code on restraint ("don't add features beyond what was asked"), Pi on the model's own RL training ("there does not appear to be a need for 10,000 tokens of system prompt"), and OpenCode on copying the winner and letting per-model prompts do the tuning. The system prompt, in Vaughan's phrase, "is the constitution of a coding agent" — and like constitutions, they reveal what their authors fear most: Codex fears blandness, Gemini fears inaction, Claude Code fears overreach, Pi fears the constitution itself.
References (all verified Aug 2, 2026)
Primary prompt sources - Codex CLI GPT-5.1 system prompt (raw): https://raw.githubusercontent.com/openai/codex/main/codex-rs/core/gpt_5_1_prompt.md · prompt dir: https://github.com/openai/codex/tree/main/codex-rs/core - Gemini CLI prompt assembly: https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/prompts/snippets.ts · repo: https://github.com/google-gemini/gemini-cli - Pi system prompt & post: https://mariozechner.at/posts/2025-11-30-pi-coding-agent/ · repo: https://github.com/badlogic/pi-mono - OpenCode prompt files: https://github.com/anomalyco/opencode/tree/dev/packages/opencode/src/session/prompt
Behavioral evidence - Petruzalek, "Proactiveness considered harmful?" (Gemini CLI): https://danicat.dev/posts/20250715-gemini-cli-system-prompt/ - Vaughan, "System Prompts Compared: Codex CLI vs Gemini CLI vs Claude Code vs Pi": https://codex.danielvaughan.com/2026/04/19/system-prompts-compared-codex-gemini-claude-code/ - Anthropic, "How we built our multi-agent research system": https://www.anthropic.com/engineering/built-multi-agent-research-system - Simon Willison on claude-trace and Claude Code's unpublished prompts: https://simonwillison.net/2025/Jun/2/claude-trace/
Benchmark / quantitative - Aider polyglot leaderboard: https://aider.chat/docs/leaderboards/ - SWE-Search (MCTS scaffold, +23% across 5 models): https://arxiv.org/abs/2410.20285 - SWE-agent paper (interface mattered then): https://arxiv.org/abs/2405.15793 - mini-swe-agent (>74% SWE-bench Verified, bash-only): https://github.com/SWE-agent/mini-swe-agent - Terminal-Bench (Stanford × Laude): https://www.tbench.ai/ · pi's TB2 results: https://gist.github.com/badlogic/f45e8f6e481e5ab7d3a50659da84edaa - Codex token baseline (Alex Fazio): https://gist.github.com/alexfazio/359c17d84cb6a5af12bac88fa1db9770 - Gemini CLI prompt-size issue: https://github.com/google-gemini/gemini-cli/issues/3784
Prompt archaeology / extraction - system_prompts_leaks (61.9k stars): https://github.com/asgeirtj/system_prompts_leaks - cchistory (Claude Code prompt version diffs): https://cchistory.mariozechner.at · repo: https://github.com/badlogic/cchistory