Code has an eval function, UX doesn't: what a short, structured brief about users, data and workflows changed in an AI review of a generated app.
Coding agents made apps easy to build but hard to trust. A few prompts get you screens, routes and a working backend in minutes. What they don’t give you is any confidence that the result is usable, and nobody notices until a real user clicks through.
The code underneath is held to a much higher bar: unit tests, type checkers, CI and benchmarks tell us on every commit whether it works. Functional correctness has an eval function. UX quality is still missing one.
TL;DR
- We reviewed eight screenshots of one AI-generated app twice, with the same model, instructions and UX principles. The only difference: Review B also got a short, structured brief about the app, its users and their workflows.
- Across both reviews we found 13 distinct issues: seven found by both, two only by the review without context, and four only by the review with context. The context-aware review also judged issues differently, checking whether screens were true, not just clear.
- Good UX context is not volume. It is a few signals across four layers (business, user, data and workflow), and the same context that sharpens review can steer generation itself.
Green builds, broken user experiences
There is no npm test for “a user can finish their task without getting lost.” Lighthouse and axe cover a thin, DOM-level slice. Everything else (whether states are handled, whether data reads correctly, whether a flow makes sense for the person it is built for) is evaluated by a human looking at a screen.
That was tolerable when humans wrote UI slowly. Vibe coding removes the bottleneck on generation, but not on evaluation. Today the eval function for a generated app is the user, clicking around and typing another prompt: “the empty state is blank,” “format the amounts,” “clicking reject does nothing.” Each correction is a manual test case that is never written down and never re-run.
Take an invoice approval tool generated by an AI agent. Its build is green, its API tests pass, Lighthouse scores well and axe reports zero violations. Then a finance manager opens it:
- Amounts render as raw floats (
125000.5) instead of formatted currency. - A filter with no results shows a blank area with no message.
- There is no loading state, so the table flashes empty before data arrives.
- Rejecting an invoice gives no confirmation and no feedback.
- The approval screen shows vendor, amount and date, but not the purchase order or remaining budget, which is what an approver actually decides on.
The first four are rules a model can learn once and apply everywhere, and newer models increasingly do. The fifth is different: nothing is broken, and the same screen would be fine for another user doing another job.
Why existing checks miss the important failures
Across generated apps, failures like these tend to fall into six categories. The first three can sometimes be caught without knowing what the app is for. The last three can’t.
| Category | What it looks like | Detectable without context? |
|---|---|---|
| Flow breaks | Buttons with no handler, routes that dead-end, forms that submit to nothing | Partly |
| Cross-screen inconsistency | Separately generated screens use different components, spacing or labels for the same thing | Partly |
| Missing states | No loading, empty, error or partial-data state, or one generic spinner for everything | Partly |
| Data-semantic mismatch | Currency as floats, ISO timestamps shown raw, nulls rendered as “null”, wrong locale | No |
| Persona mismatch | View-only users see Edit, Approve and Delete buttons they aren’t allowed to use | No |
| Intent drift | Asked for a comparison tool, got two separate product pages | No |
The right-hand column is the crux. The failures that make a user say “this isn’t what I asked for” sit at the bottom of the table, and they can’t be found by inspecting the UI alone. A test needs an expected value to assert against, and for these failures the expected value isn’t in the code, the prompt or any design rule. It lives in the context of who the app is for.
The experiment: same app, two reviews
We ran a simple experiment on an app generated by a coding agent: an inventory-planning dashboard. A planner opens it each morning to decide which items to reorder, how many, and by when, then turns those decisions into purchase orders.
We took eight screenshots of the app and reviewed them twice, with the same model, the same instructions and the same UX principles.
- Review A got no context. It had to work out what the app was for from the screens alone.
- Review B got a short, structured brief about the app, its users and their workflows.
Review B’s brief followed this template:
# ux_specification: template used in the experiment
app: <app name and a one-line description>
industry: <the industry or domain the app serves>
business_problem: <the problem the app exists to solve, in one or two sentences>
target_user:
role: <who opens the app>
data_literacy: <how comfortable they are reading data>
frequency: <how often they use it, and when>
context: <where they use it: device, setting, other tools open>
measured_on: <the outcomes they're judged by>
primary_goal: <the one thing the user must get done>
workflows:
A: <the main task, from start to finish>
B: <the second most important task>
actions_and_constraints:
- <the key action the app must make easy>
- <what the app deliberately doesn't do>
- <how key numbers are calculated, and where they fall short>
What changed when the review had context
The two reviews spotted many of the same issues. What changed was how they judged them. We saw three shifts.
From “is it clear?” to “is it true?” Review A checked whether the screen explained itself. Review B checked whether the screen was accurate. Both flagged a score shown on every row as undefined. Only Review B knew the score was a category average rather than item-specific, so it recommended marking those rows and keeping them out of the confidence filter.
From polishing to placing. Review A improved elements where they sat. Review B decided where they belonged. Review A praised a “last updated” label on a minor panel. Review B moved it to the main table, since every due date there is calculated from today’s data.
From what the screen shows to what the system does. Review A judged controls by how they looked. Review B knew what happened after the click. A forecast-adjustment control looked like it worked, but the system never used the adjusted value. So instead of a layout tweak, Review B recommended labeling it display-only.
Across both reviews we found 13 distinct issues: seven were found by both, two only by Review A, and four only by Review B. Both reviews reached the same overall score, but only Review B’s list told the team where to start.
This was one app and a single run of each review, so the exact findings will vary from run to run. The pattern and the difference in findings are what we’re pointing to, not the exact numbers.
What good UX context actually means
Review B’s brief was a short, structured file, yet it changed what the review could see. So what made it work?
Not volume. Loading everything you know about an app into a prompt isn’t context; it’s noise with a label on it. Useful context is the right few signals, on top of a shared baseline.
The baseline is a set of UX principles: rules every app should follow regardless of who it’s for, like clear hierarchy, handled states and visible feedback. These are the checks a model increasingly gets right on its own. Everything specific to the app comes from four kinds of context:
- Business context: what the app is for, and what it deliberately isn’t for. A clinical tool and a retail tool shouldn’t be held to the same standard just because both are dashboards. Stating what’s out of scope matters just as much, because it tells the review which gaps are intentional and which screens promise something the system doesn’t do.
- User context: who opens the app, what they’re allowed to do, how often they use it, and how fluent they are. A daily power user and a weekly glance-checker need different interfaces, even on the same data.
- Data context: the shape and meaning of what’s on screen (dense time series, sparse categories, large tables) and how the key numbers are calculated. Without it, a review can say a number is unclear, but not whether it’s true.
- Workflow context: where the app sits in someone’s job, and which decision each screen supports. A single glance-and-decide moment calls for something very different from a tool someone works inside for hours.
These aren’t four checklists to satisfy one at a time. They converge on a single judgment, and none is sufficient alone. Business context without user context tells you the industry but not who is looking at the screen. Data context without workflow context tells you what the data looks like but not how much attention someone can give it.
In our experiment, each shift in Review B traced back to one of these layers:
| Context | What it gave Review B | The shift it caused |
|---|---|---|
| Data | How the row score was calculated | From “is it clear?” to “is it true?” |
| Workflow | Which screen carries the daily ordering decision | From polishing to placing |
| Business | What the system deliberately doesn’t do | From what the screen shows to what the system does |
Put simply, good UX context describes the moment a screen is used: who is looking at it, what they need to decide, which numbers they rely on to decide it, and what the app won’t do for them. If a line of context doesn’t change what belongs on a screen, where it sits or how it’s labeled, leave it out.
From review to generation
Our experiment reviewed an app that already existed, but it made one thing clear: context changes what a review can see, and the same gap exists when an agent generates the UI in the first place. Review still happens after the fact. The app gets generated, reviewed and patched, and then the loop runs again, with each patch risking new drift in something that was previously fine.
Look at what made review useful: context about the business, the users, the data and the workflow. That is exactly what the generator should have been working from all along. Once the UX-relevant parts of that context are distilled into a ux_specification file, none of it is specific to review. In review, it’s the expected value a screen is checked against. In generation, it’s part of the input.
Used in both places, the loop tightens: review findings stop being one-off corrections typed into a prompt and start pointing at what the context or the generator missed. That suggests four directions:
- Context-first generation. The
ux_specificationbecomes the contract the generator starts from, confirmed with the user up front, instead of something reconstructed after the app exists. - UX principles as constraints, not checks. If every list must have loading, empty and error states, the generator shouldn’t be able to emit one without them. A rule enforced at generation time never needs to be reported as a finding.
- Review as a critic inside the loop. The same context-aware review runs on each screen as it’s generated, so issues get caught and fixed in seconds rather than after a full build and review cycle.
- Tests generated from context. The workflows described in the context become end-to-end tests that ship with the app, so the expectations survive every future edit, whether made by a human or an agent.
Where this could go
We started with a gap: code has an eval function, and UX doesn’t. One experiment doesn’t close that gap, but it showed where the missing piece lives. It isn’t in a better model or a longer checklist. It’s in the few sentences that tell a reviewer, or a generator, who a screen is for and what it has to get right.
The interesting part is not context as a description of the app, but context with enough shape to steer UX choices. An app shouldn’t just get reviewed toward fitting its business, users, data and workflow. It should start out already pointed that way. We’ll share more as this work develops.
If you’re building data apps and want them to fit the people who use them, book a discovery call to see how we approach it at RapidCanvas.
