·6 min read·Ofir, Marshal Team

The Coding Agent Bake-off: How to Compare Claude Code, Codex and Gemini CLI Without Redoing Security Each Time

Claude Code vs Codex vs Gemini CLI: run a fair bake-off without restarting the security review for every new coding agent. Approve the environment once.

OWASP's Top 10 for LLM Applications describes the risks of an LLM-driven agent in terms that are deliberately not tied to any one vendor: excessive agency, supply chain, prompt injection. That framing is a hint about how to run agents in an organization. The risks belong to the category, not to the brand on the binary.

Most engineering orgs are not treating them that way. They are running an informal bake-off between coding agents, Claude Code this quarter, Codex for one team, Gemini CLI for another, Cursor for the people who like it, and every new contender restarts the security review from zero. New permission model, new config format, new telemetry, new questions for the security team, new weeks of waiting.

That is backwards, and it is expensive in the one currency that matters here: developer momentum. If the controls that actually enforce security sit outside the agent, you approve the environment once, and the choice of agent becomes what it should be, a productivity preference, not a security event.

Several coding agent tiles, Claude Code, Codex, Gemini CLI, Cursor and more, all feeding into one shared control-plane band that applies the same credential, egress, policy and audit rules, then out to shared destinations

Agent choice keeps changing, and that is fine

New models ship every few weeks, and each one shifts which agent feels best for a given kind of work. Developers notice, and they have strong preferences that are not arbitrary: an agent that fits one person's workflow genuinely makes them faster.

Fighting that churn is a losing game. The healthy goal is not to standardize on one agent forever. It is to make switching cheap, so the org can follow the frontier without paying a tax every time. The thing that makes switching expensive is almost never the agent itself. It is everything you bolted onto it to make it safe.

The security review that never ends

Here is why a per-agent review does not scale. Each agent brings its own permission model, its own configuration format, and its own idea of what it logs. So a review done for one does not transfer to the next. You redo the credential questions, the egress questions, the "what can it run" questions, and the "what does it record" questions, from scratch, for a tool that does the same job as the last one.

Multiply that by the pace of releases and you get a security team permanently a step behind the developers, who are, predictably, already running the new thing on a personal setup while they wait. The review that never ends does not actually control anything. It just delays the paperwork.

There is a subtler cost too. Because each review is bespoke, the controls end up slightly different per agent: one has egress logging wired up, another does not, a third logs to a place nobody reads. You are left unable to compare two agents on security at all, because they are not secured the same way. Standardizing the layer below the agent is what makes a like-for-like comparison possible in the first place.

Approve the environment, not the agent

The way out is to move the controls to the layer that does not change when the agent does.

Credentials, egress, action policy and audit are properties of the environment the agent runs in, not of the agent. Put them there, and they apply identically whichever binary is inside: the agent never holds the keys, whichever agent it is; the same verb-and-target policy governs all of them; the same audit lands in the same storage you own. Now a new agent is a new tenant of an environment you already approved, not a new security project. The review you do once is the review that keeps counting.

This is also the honest case for a control plane over per-agent configuration: not that any one agent is unsafe, but that securing each separately does not compose.

Running a fair bake-off

Once the environment is fixed, the bake-off becomes a real experiment instead of a security debate. Hold everything except the agent constant: the same repositories, the same tasks, the same policies and guardrails, the same environment. Then measure what you actually care about.

Good metrics are about outcomes, not vibes: time to a merged pull request, how much review rework each agent's output needed, and developer preference gathered honestly. Resist the urge to publish a benchmark number you cannot reproduce. A fair comparison is one another team could rerun and get close to the same answer.

What to tell developers

The message to developers is short and it is the reason this approach holds. They keep their tool. The guardrails come with the workspace, not with the agent, so choosing Claude Code over Codex over Gemini CLI is their call to make on speed and fit. Security is not asking them to give up the agent they like. It is asking only that the work happen inside the environment, which is the part they stop having to think about.

One operational note worth making to platform teams rather than developers: the controls sitting beside the agent should add little enough overhead that nobody is tempted to route around them. The goal is that the safe path is also the path of least resistance.


Every new coding agent should be a productivity decision, not a security review. Move credentials, egress, policy and audit outside the agent, into an environment you approve once, and you can run a fair bake-off between Claude Code, Codex, Gemini CLI and whatever ships next month, letting developers pick the fastest tool while the guardrails stay exactly the same underneath.

This is the developer-experience end of the argument the series opened with: the false choice between banning agents and letting them run →


Marshal runs any supported coding agent under one environment, with the same credentials, egress, policy and audit for all of them. Book a demo →

securityai-agentsdeveloper-experienceplatform-engineering

Marshal runs AI agent sessions in isolated Kubernetes pods: credential isolation, audited egress, per-verb tool policy.

Book a demo →