--- slug: production-readiness-cliff type: antipattern summary: "When an agent-built app crosses the \"looks done\" line long before the \"is production-ready\" line, leaving a polished UI over absent or broken backend behavior." created: 2026-06-09 updated: 2026-06-09 related: vibe-coding: relation: related note: "Vibe Coding is the authoring posture that accepts code you don't understand; the Production-Readiness Cliff is the deceptive output that posture tends to produce." happy-path: relation: related note: "The cliff is what you find the moment you step off the happy path the demo walked." silent-failure: relation: related note: "The absent backend rarely errors loudly; it returns canned data and fails quietly until something downstream depends on it." fail-fast-and-loud: relation: prevented-by note: "Making the missing layer fail loudly on first contact is the fastest way to expose the cliff before a user does." smoke-test: relation: detected-by note: "A smoke test that exercises a real write-read-reload cycle is the cheapest probe that reveals a stubbed backend." exploratory-testing: relation: detected-by note: "Unscripted poking at the seams the demo avoided is how a reviewer finds the edge of the cliff." acceptance-criteria: relation: violated-by note: "A demo that satisfies a glance but not the acceptance criteria has skipped the bar the criteria set." threat-model: relation: related note: "The security half of the cliff (absent auth, exposed secrets, missing authorization) is exactly what a threat model would have flagged." benchmark-mirage: relation: related note: "Benchmark Mirage is the evaluation-side twin: surface impressions outrun substance on the leaderboard, while the cliff is the same gap inside the shipped artifact." --- # Production-Readiness Cliff > **Antipattern** > > A recurring trap that causes harm — learn to recognize and escape it. *An agent-built application crosses the "looks done" line long before it crosses the "is production-ready" line, and the gap between them is invisible until something real touches it.* You've seen the demo. A prompt goes in, and a few minutes later there's a running app: a clean interface, working navigation, forms that submit, a dashboard with charts. It looks finished. The reflex is to believe it, because for most of software history a UI this polished could only sit on top of a working system. That reflex is now wrong. The polish and the substance have come apart, and the distance between them is the cliff. ## Symptoms - The app demos beautifully and breaks the moment you do something the demo didn't. The signup flow works on stage; reload the page and your account is gone. - Data doesn't survive. You create a record, navigate away, come back, and it's vanished, because nothing was ever written to a real store. - There's no login that means anything. Either auth is missing, or it's a form that accepts any input and gates nothing. - The "API" is the front-end talking to itself. Network calls return hardcoded fixtures, and there's no server behind them. - Secrets are in the client bundle, or the repository, or both. Keys that should never leave a server are shipped to the browser. - Two people using it at once corrupt each other's state, or the app simply assumes it will only ever have one user. - Nobody can say how you'd deploy it, roll it back, or tell whether it's healthy. There are no migrations, no logs worth reading, no plan for the day it falls over. ## Why It Happens Agents are trained, evaluated, and rewarded on what a reviewer can see. A front-end is visible in seconds: a human glances at it, a screenshot captures it, a demo sells it. A backend is invisible by design: persistence, authorization, concurrency, and observability are exactly the parts you don't watch happen. When the reward signal favors the visible layer, the model gets very good at the visible layer and learns that the invisible one is optional. This isn't a hunch. A 2026 benchmark (SWE-WebDevBench) evaluated AI app-builder platforms as if each were a small software agency, scoring their output across 68 metrics. It found the cliff everywhere. No platform cleared 60% on engineering quality. Security scores stayed under 65% against a 90% target. Concurrency handling fell as low as 6%. The benchmark's authors named the specific shape directly: "visually polished UIs mask absent or broken backend infrastructure." A separate 2026 study of realistic production iOS tasks found the best of 22 agent-and-model configurations completing one task in eight. The visible competence and the operational competence are different numbers, and the second one is much lower. There's a human half too. A demo that looks done feels done, and a thing that feels done is hard to keep scrutinizing. The polish doesn't just hide the gap; it actively lowers your guard about whether to look for one. This is the trap [Vibe Coding](vibe-coding.md) sets from the authoring side: when you accept output you don't understand because it appears to work, you inherit a system whose missing parts you've never seen. ## The Harm The cost lands late and lands hard. You ship the agent's "working" app, or you greenlight a build on top of it, on the strength of a demo. The missing backend doesn't announce itself; it surfaces as a [Silent Failure](silent-failure.md) (data that doesn't persist, an auth check that was never there, a race condition between two users) at the worst possible time, in front of a real person, in production where fixes are most expensive. Then someone has to find the edge of every cliff and build the missing half. The benchmark put a number on this with its Effort-to-Fix metric: the developer-hours needed to bring a generated app to production quality, and that hidden cleanup burden varied enormously across platforms. The work didn't disappear when the demo looked done. It was deferred, uncounted, onto whoever inherits the app. Inheriting a half-built system you didn't write is slower than building it yourself, because first you have to discover what's missing. ## The Way Out Stop trusting the glance. The cliff is invisible to a thirty-second look precisely because that's the look it was optimized to pass, so the way out is to probe the layers the demo can't fake. **Run a write-read-reload cycle.** The cheapest possible [Smoke Test](smoke-test.md): create a record, close the tab, reopen it, and confirm the record is still there. If it isn't, there's no backend, and everything else is theater. This one probe separates a real app from a convincing front-end in under a minute. **Make the missing layer fail loudly.** Adopt [Fail Fast and Loud](fail-fast-and-loud.md) as your probe. Point the app at a backend that doesn't exist, send malformed input, pull the network mid-request. A real system errors in a way you can read; a stub returns its canned answer and tells you nothing, which is itself the signal. **Walk a reviewer's checklist for the invisible half.** Before you trust an agent-built app, confirm each of these is present, not assumed: a real persistence store with [migrations](migration.md); [authentication](authentication.md) and [authorization](authorization.md) that actually gate; [secrets](secret.md) kept server-side, never in the client bundle; [concurrency](concurrency.md) handling for more than one user; [observability](observability.md) good enough to debug a failure you can't reproduce; and a [deployment](deployment.md) path with a [rollback](rollback.md). Each absent item is a section of the cliff. **Do some [Exploratory Testing](exploratory-testing.md) at the seams.** Unscripted poking at the parts the demo carefully avoided (the second user, the empty state, the malformed input, the back button) finds the edge faster than any test plan, because the demo's author already knew which paths to walk. The bar already exists; the demo just skipped it. [Acceptance Criteria](acceptance-criteria.md) and a definition of done name what "production-ready" requires for *this* app. Hold the agent's output to that bar, not to the bar of "did it run in the demo." > **💡 Tip** > > Before you accept an agent-built app, write one sentence describing what it does when no longer in the demo: where the data lives after a reload, what stops an unauthorized user, what happens with two users at once. If you can't answer from looking, you're trusting the glance. Go run the write-read-reload cycle first. ## How It Plays Out A solo founder uses an app-builder to stand up an internal tool for tracking customer onboarding. It's genuinely impressive: a kanban board, drag-and-drop cards, a clean filter bar, status badges. She shows it to her two-person ops team on Monday and they start using it. By Wednesday they're confused: cards one person moves don't move for the other, and a card someone archived reappears the next morning. There is no shared backend. Each browser is holding its own copy of the board in local state, and a "refresh" silently resets it to the seeded demo data. The board was never a tool. It was a picture of a tool, and the cliff was the line between the two. She rebuilds the persistence and the multi-user sync herself, which is the half the demo never showed. A staff engineer is asked to review an agent-generated service before it goes to staging. The diff is large and tidy: handlers, routes, a data layer, even tests. The tests pass. He almost approves it on that basis, then runs one experiment instead. He points the service at a database that isn't running and sends a request. Nothing fails. The endpoint returns a 200 with a plausible-looking object, because the "data layer" is a module of hardcoded fixtures and the tests assert against those same fixtures. The whole thing is a closed loop that never touches a real store. He sends it back with a single requirement: an integration test that exercises a real write and read against a real database, in CI, before any further review. He's installing the gate the agent's polish was built to slip past. ## Sources - The empirical shape of the cliff — no platform above 60% on engineering quality, security under 65%, concurrency handling as low as 6%, and the finding that "visually polished UIs mask absent or broken backend infrastructure" — comes from *[SWE-WebDevBench: Evaluating Coding Agent Application Platforms as Virtual Software Agencies](https://arxiv.org/abs/2605.04637)* (2026), which scored AI app-builder platforms across 68 metrics, including the Effort-to-Fix measure of developer-hours to reach production quality. - The corroborating production-task gap on realistic mobile work — the best of 22 agent-and-model configurations completing roughly one task in eight — comes from a companion 2026 benchmark of agent-and-model configurations against real iOS engineering tasks, circulating in the agent-evaluation research community in 2026. - The framing of the gap between apparent completeness and operational correctness draws on the long testing tradition that separates the "main success scenario" from its exceptions, formalized in Alistair Cockburn's *[Writing Effective Use Cases](https://openlibrary.org/works/OL2706111W/Writing_effective_use_cases)* (2001); the cliff is what you find when the exceptions the demo omitted turn out to be the whole job. --- - [Next: Code Review](code-review.md) - [Previous: Happy Path](happy-path.md)