Agentic AI Analysis

Agentic engineering: how leading developers code with AI agents

We compared how 15 engineers who code with AI agents work, from Matt Pocock to Simon Willison. Where they agree, where they split, and what to copy.

Table of eight engineers: Mitchell Hashimoto runs one agent and reads code thoroughly, Addy Osmani three or four, Matt Pocock one subagent per ticket, Simon Willison several, Peter Steinberger four to ten, Thorsten Ball five or six and once 22, Kieran Klaassen parallel runs overnight, and Steve Yegge 20 to 30, never reading the code
Graphic: Mavday. Source: the engineers' own blogs, talks and podcasts

Engineers who code with AI agents agree on two things. Plan with the agent before it codes, and give it a way to check its own work.

Almost everything else is up for argument.

We read what 15 of them have published since mid-2025: blog posts, repos, talk and podcast transcripts. They split on how many agents to run, whether to read the code, whether rules files and skills help, and whether to mix models. This is agentic engineering as these practitioners describe it, and what we’d copy.

What agentic engineering means

Simon Willison, who writes a guide called Agentic Engineering Patterns, puts it simply: “I use the term agentic engineering to describe the practice of developing software with the assistance of coding agents.”

The term beat Willison’s own suggestion, “vibe engineering”. He noted in February 2026: “It looks like the term “Agentic Engineering” is coming out on top for this now.” Peter Steinberger had used it in an October 2025 post.

Andrej Karpathy, who coined “vibe coding”, draws the line between the two. In the edited transcript of his Sequoia Ascent talk, posted in April 2026: “Vibe coding raises the floor. Agentic engineering is about extrapolating the ceiling.”

Where they agree

Plan before the agent writes code. Mitchell Hashimoto, who created the Ghostty terminal, wrote: “Creating a comprehensive plan interactively with an agent is a really important first-step for anything non-trivial.” Karpathy: “People have to be in charge of the spec and plan.”

They don’t agree on how. Boris Cherny, who created Claude Code, wrote in a January 2026 thread on X, quoted by InfoQ: “If my goal is to write a Pull Request, I will use Plan mode, and go back and forth with Claude until I like its plan.”

Steinberger thinks “Plan mode feels like a hack that was necessary for older generations of models”. Armin Ronacher, who created Flask, plans in a Markdown file instead: “I never really used plan mode.”

Give the agent a way to check its own work. Hashimoto again: “If you give an agent a way to verify its work, it more often than not fixes its own mistakes and prevents regressions.”

Willison is blunter: “Automated tests are no longer optional when working with coding agents.” Cherny wrote in the same thread that Claude “tests every single change I land to claude.ai/code using the Claude Chrome extension.”

Judgement is the scarce part now. Addy Osmani: “Writing got cheap, understanding didn’t.” Kieran Klaassen, who runs Cora at Every: “So the bet is implementation is only getting cheaper and judgment is not.” Karpathy repeats a line from a post he keeps thinking about: “You can outsource your thinking, but you can’t outsource your understanding.”

Matt Pocock’s method, step by step

Table of Matt Pocock’s six-step workflow: grill-with-docs to align, to-spec, to-tickets, implement with TDD, code-review with two subagents, then pr and retro

Matt Pocock's main flow, from his mattpocock/skills repo. Graphic: Mavday. Source: mattpocock/skills README and AI Hero skills changelog v1.3

Matt Pocock, the TypeScript educator behind AI Hero, publishes his whole process as skills. His mattpocock/skills repo had more than 299,000 GitHub stars on 21 October. Version 1.3, released on 4 October, sums up the premise: “Writing code got cheap. Reading it did not.”

The flow starts with the agent interviewing him until they share one design, then writing a spec and splitting it into thin end-to-end tickets. Each ticket is built test first. A review runs in a fresh session, then the agent writes the pull request, and a retro turns mistakes into checks.

He has dropped Claude Code’s plan mode for that first step: “It doesn’t ask enough clarifying questions, it jumps to conclusions, and it rushes to create an asset.”

He tells people never to run /init to generate a CLAUDE.md: “Never run it. If you find one that was auto-generated, delete it.” Steering goes into short skills instead, and he calls most of them himself.

He is wary of piling them up. In a recorded talk for the AI Engineer World’s Fair, posted in June 2026, he warned of “another version of hell, which is skill hell.”

The first split: how many agents at once

At one end, Hashimoto: “I’m not [yet?] running multiple agents, and currently don’t really want to.” He has one agent working in the background for “maybe” 10% to 20% of his day. Kent Beck, the creator of extreme programming, wrote in April 2026: “Nobody wants agents. Nobody wants agent swarms.”

Osmani sits in the middle: “My ceiling for a typical session is somewhere around three to four threads depending on complexity.” His rule for autonomy: “The autonomy level should follow the verification process, not the task name.”

At the other end, Thorsten Ball, who works on the Amp coding agent, said on Amp’s podcast in September 2026: “Don’t wait for your agent. You’re wasting time. Let them wait for you.” He once had 22 running. Steve Yegge wrote in January 2026 that he could run 20 to 30 “productively, on a sustained basis”.

Long unattended runs divide them too. In September 2026, Ronacher wrote up letting a model run for 35 hours, and he didn’t like the result: “I honestly do not need an agent to run for 35 hours on a single prompt.”

The second split: whether to read the code

Abstract illustration of a glowing stack of sheets sending three streams of light through gates and blocks, with lines looping back to the stack

An illustration of one plan feeding parallel tasks, each passing checkpoints that loop back. Not a picture of any real tool. Illustration: Mavday, AI-generated

Hashimoto: “Please don’t ever ship AI-written code without a thorough manual review.” Osmani: “I only merge or ship code after I’ve understood it.”

Willison has moved to the middle. On a podcast he wrote up in May 2026, he said: “I’m not reviewing every line of code that they write anymore, even for my production level stuff.” In August he added: “Eyeballing every line of code has never been the most effective way to validate a change to a piece of software.”

Ball asks agents for evidence instead: “Give me irrefutable proof that this works.” Steinberger: “I don’t read the boring parts of code.”

At the far end, Klaassen said in his World’s Fair talk: “I haven’t written a single line of code this year, and maybe I haven’t even looked at most of it.” Yegge wrote of his Gas Town setup: “I’ve never seen the code, and I never care to, which might give you pause.”

A first-hand case for reading it comes from Dex Horthy of HumanLayer, who tried not to. His team went “lights-off” in July 2025.

HumanLayer later wrote: “After running a lights-off factory for five months and getting drowned in unmaintainable slop, we took a step back, reset, and revised our advice: read the code”. His explanation: models are trained against tests that pass or fail, and “there is no penalty for eroding codebase maintainability”.

The third split: rules files and skills

Everyone keeps some instructions for the agent. They disagree on how much.

Hashimoto builds his from mistakes: “Each line in that file is based on a bad agent behavior, and it almost completely resolved them all.” Ghostty’s top-level AGENTS.md is about 40 lines. Steinberger’s is far bigger: “My Agent file is currently ~800 lines long and feels like a collection of organizational scar tissue.”

On skills, Jesse Vincent, who built the Superpowers skill set, wants them to load on their own: “The whole point of skills, in my book, is that they should auto trigger when they’re needed.” Pocock prefers to call his by hand.

Geoffrey Huntley, who invented the “Ralph” agent loop, has turned against much of it. In October 2026: “Many people spend their time tuning the agent itself: skill packs, AGENTS.md, MCPs. They’re probabilistic. There’s no guarantee they’ll do anything.”

He uses pre-commit hooks that fail when the agent breaks a rule. Osmani agrees on the principle: “if a rule must always hold, encode it in a test, hook, or permission rather than leaving it as prose the model might lose.”

Table of four studies: SkillsBench found curated skills raised pass rates from 33.9% to 50.5%; SWE-Skills-Bench found 39 of 49 coding skills gave no improvement; Demystifying Agent Skills found use precision fell from 29.6% with 5 skills to 3.3% with 100; Evaluating AGENTS.md found context files gave no general gain at over 20% more cost

Four 2026 preprints on skills and context files. Graphic: Mavday. Source: arXiv 2602.12670, 2603.15401, 2608.14036 and 2602.11988

The research leans their way on size. SkillsBench found curated skills raised the average pass rate from 33.9% to 50.5%. But SWE-Skills-Bench, which tested 49 public software engineering skills, found “39 of 49 skills yield zero pass-rate improvement”.

The fourth split: one model or several

Cherny, in his January thread as quoted by VentureBeat: “I use Opus 4.5 with thinking for everything”. Huntley has reached the same view: “Pick a model that works and lock in. Don’t change every week. Switching models is like firing a developer and hiring a new one”.

Others mix vendors, mostly for review. Willison in July 2026: “I’ve started habitually having Anthropic’s best model review OpenAI’s work and vice versa”. Osmani: “Four copies of one model is a single reviewer with a larger invoice”. Pocock mixes sizes within one family, using a bigger model to review than to implement.

The evidence is thin. One October 2026 preprint found a top-tier reviewer from another vendor was not significantly different from the same model in a fresh session. Combining one of each caught more planted errors than two fresh-session reviews, 56.7% against 42.7%.

Who changed their mind

Table of changed views: Andrej Karpathy from calling agent code slop to mostly programming in English; Dex Horthy from lights-off to read the dang code; Simon Willison from calling cross-model review absurd to doing it habitually; Geoffrey Huntley from AGENT.md as the heart of the loop to deleting skills and AGENTS.md every few months; Steve Yegge from reusable Gas Town to giving up on reusable harnesses

Five practitioners whose views shifted between 2025 and 2026. Graphic: Mavday. Sources: Dwarkesh Podcast; The Decoder; HumanLayer; simonwillison.net; ghuntley.com; yegge.ai

Karpathy moved a long way. In October 2025 he told the Dwarkesh Podcast: “For now, autocomplete is my sweet spot.” By late January 2026 he wrote on X, as quoted by The Decoder: “I really am mostly programming in English now”.

Yegge has gone the other way on tools. “Gas Town was intended to be reusable, but I only ever wound up using it to build itself,” he wrote in August 2026.

Our take

Copy the common ground first. Plan with the agent before it writes code, and give it a check it can run on its own: tests, types, a browser.

Start with one agent. Add a second when you’re regularly waiting on the first, and tie how much you let it do alone to how well you can check its work. Osmani’s rule is the right one.

Read the code that would be expensive to get wrong. Horthy’s five months of slop is the clearest evidence here, and it’s first-hand.

Keep rules files short and written from real mistakes. Turn any rule that must hold into a hook or a test.

Try a second model as a reviewer, but start a fresh session first. That’s free, and in the study above a top-tier model from another vendor did no significantly better.

Remember who is talking, too. Cherny built Claude Code, Ball works on Amp, and Steinberger joined OpenAI in February. Their setups are also their products’ best case.

Coding agents need guard rails as well as process. See what happened when two OpenAI coding agents weakened misalignment monitoring. And for how one lab serves its own models, see DeepSeek’s open-source DeepInfer.

Get the builder’s briefing

Twice a week: the AI news worth knowing, from new models and agents to the money behind them. No hype, no spam.

We’ll email you to confirm before the first issue. Unsubscribe any time. See our privacy policy.

Sources

48
  1. What is agentic engineering? · Simon Willison
  2. Vibe engineering (with February 2026 update) · Simon Willison, 7 Oct 2025
  3. Vibe coding and agentic engineering · Simon Willison, 6 May 2026
  4. More than just code review · Simon Willison, 22 Aug 2026
  5. sqlite-utils and Fable · Simon Willison, 5 Jul 2026
  6. First run the tests · Simon Willison
  7. Sequoia Ascent 2026 · Andrej Karpathy, 30 Apr 2026
  8. Andrej Karpathy interview · Dwarkesh Podcast, 17 Oct 2025
  9. Andrej Karpathy now codes mostly in English · The Decoder, 27 Jan 2026
  10. mattpocock/skills (README) · Matt Pocock / GitHub
  11. Skills changelog v1.3 · AI Hero, 5 Oct 2026
  12. Never run claude /init · AI Hero
  13. Plan mode introduction (update note) · AI Hero
  14. Building great agent skills (talk transcript) · AI Engineer World's Fair, 29 Jun 2026
  15. Full walkthrough: workflow for AI coding (workshop transcript) · AI Engineer Europe, 24 Apr 2026
  16. My AI adoption journey · Mitchell Hashimoto, 5 Feb 2026
  17. Non-trivial vibing · Mitchell Hashimoto, 11 Oct 2025
  18. Ghostty AGENTS.md · Ghostty / GitHub
  19. How I use Amp · Amp (Thorsten Ball)
  20. Raising an Agent, season 2 episode 5 · Amp podcast, 25 Sept 2026
  21. The creator of Claude Code just revealed his workflow · VentureBeat, 4 Jan 2026
  22. Claude Code creator's workflow · InfoQ, 10 Jan 2026
  23. Just talk to it · Peter Steinberger, 14 Oct 2025
  24. Shipping at inference speed · Peter Steinberger, 28 Dec 2025
  25. Peter Steinberger (episode 491 transcript) · Lex Fridman Podcast, 12 Feb 2026
  26. What is plan mode? · Armin Ronacher, 17 Dec 2025
  27. Astra for coding: why are we doing this again? · Armin Ronacher, 7 Sept 2026
  28. Why software factories fail · Dex Horthy / HumanLayer
  29. HumanLayer resources · HumanLayer, 4 Sept 2026
  30. Agency · Geoffrey Huntley, 13 Oct 2026
  31. Ralph · Geoffrey Huntley, 14 Jul 2025
  32. Genie lessons: nobody wants agents · Kent Beck, 23 Apr 2026
  33. Superpowers · Jesse Vincent / GitHub
  34. Open Source Ready, episode 36 (transcript) · Heavybit, via blog.fsck.com, 23 Apr 2026
  35. Parallel agents and cognitive load · Addy Osmani, 7 Apr 2026
  36. My AI coding workflow · Addy Osmani, 4 Jan 2026
  37. Agentic code review · Addy Osmani, 15 Jun 2026
  38. Audit your agent files · Addy Osmani, 27 Aug 2026
  39. Agentic autonomy levels · Addy Osmani, 2 Jul 2026
  40. The era of compound engineering (talk transcript) · AI Engineer World's Fair
  41. Compound engineering camp: every step from scratch · Every, 13 Mar 2026
  42. Welcome to Gas Town · Steve Yegge, 1 Jan 2026
  43. The shape of things to come, part 1 · Steve Yegge
  44. SkillsBench · arXiv, 14 Jun 2026
  45. SWE-Skills-Bench · arXiv, 16 Mar 2026
  46. Demystifying Agent Skills · arXiv, 14 Aug 2026
  47. Evaluating AGENTS.md · arXiv
  48. Cross-model code review study · arXiv, 1 Oct 2026

Mr B

Co-founder & AI Engineer

Mr B is a co-founder of Mavday and an AI engineer, closely involved in agentic engineering: building software with AI agents. He writes about how that works in practice, and about the wider AI industry.

Work with us on a tutorial for your product

Related reading

See all

Latest