UAT sign-off you can defend: go/no-go verdicts built on evidence

Why UAT sign-off ends in “works on our side”

Picture a Monday, three days after handoff: your operations lead opens a teammate invitation on her phone, and the Join button sits off-screen. The agency says it works on their side, the final invoice goes on hold, and the UAT sign-off turns into an argument.

User acceptance testing (UAT) was supposed to prevent this. But the agency tested on a desktop, logged in as the owner, and their QA sheet says “Invites: pass”, with no device, no role and no date.

Then the screenshot attached to the sign-off email turns out to show a real customer’s email address.

The bug is real, but the dispute is about evidence: no shared scope, no agreed rule for what a failure means, and proof that leaks data.

We built a prototype release verification tool, a command-line runner that drives Chrome, to fix this. Its rules apply to any custom software acceptance.

What UAT sign-off templates miss

Search for UAT sign-off or a software handover checklist and you find templates: criteria, a severity scale, a named approver. Those help, but none make the evidence trustworthy.

The common thread is timing.

A release verdict is only as strong as the rules you agreed before you saw the result.

Go/no-go criteria agreed before the run

The Test Plan is the agreement both sides approve before the run: a JSON file naming exactly three workflows, such as “sign up and confirm an account”, “invite a teammate” and “place a test order”.

Every workflow runs in desktop Chrome and in an emulated mobile viewport (1440 by 900 and 390 by 844 in our example plan): six slots. Each slot starts with fresh browser sessions, so login state never carries over.

Each check also carries a failure policy, and both sides approve it before the first click:

• No-go: “New account dashboard loads”. If it fails, the release stops.
• Conditional-go: “Join button is visible on mobile”. If it fails, you ship with a known issue on record.

Under this plan, the off-screen Join button from the opening is caught before handoff and recorded as a known issue, instead of freezing the final invoice.

From there, the verdict is arithmetic. A finding is a check that ran and saw something other than agreed. A blocked slot could not finish: staging went down, a credential expired, or a button or field the script needed to use could not be found.

INCONCLUSIVE is the important one. A blocked slot is never counted as a finding, so a run that could not finish never reaches the client as their bug. It is never hidden either, so a flaky run cannot become a false GO.

When UAT fails: retests that cannot drift, and cleanup that proves itself

Scope drift often happens on the retest. A “flaky” step gets deleted, and the second pass covers less than the first.

A fingerprint is a short code computed from the plan’s exact contents: change one step or one policy and the code changes. Every run stores a SHA-256 hash of the validated, normalized plan.

A retest is accepted only when:

• The source is a complete initial run, not an earlier retest, from the last seven days.
• Its fingerprint matches the current plan exactly.
• Its summary matches its own results when recomputed, so a summary hand-edited without its results is rejected.
• None of its cleanup was left unverified.

The retest then reruns only the failed or blocked workflows, but on both profiles. A mobile fix can break desktop, and we want to see it.

Cleanup is part of the test

Workflows that create data must declare how to remove it. Cleanup is armed the moment the creating step is attempted, and it runs even if later steps fail. A half-created order is still an order.

Cleanup must end in a zero-count check, such as “test order was removed: count 0”. A visible or text check must come first, because a page that failed to load also contains zero orders.

Screenshot evidence that protects customer data

Typical advice on how to mask sensitive data in test screenshots is to box the elements you remember to list. Anything you forgot, like a chat widget in an iframe, lands in the PDF your client forwards.

Verizon’s 2025 breach report found that third-party involvement in breaches doubled to 30%. An agency holding screenshots of your staging data is a third party.

So our capture fails closed, refusing the picture when in doubt. Before each screenshot, the runner:

1. Hides every input, textarea, select, editable region and element marked for redaction (a forced display: none), so form fields never appear in the image.
2. Walks the full page tree, including web components’ hidden sub-trees (shadow roots), and blocks capture if any iframe, embed or component host is unmarked.
3. Takes a viewport-only PNG, then confirms the same elements are still attached and hidden and nothing new appeared.
4. Re-evaluates the check before and after capture. If the outcome changed, it deletes the image and blocks the slot.

That last step keeps the evidence honest. A screenshot showing “passed” next to a failed result is worse than no screenshot.

Everything else visible on the page is captured, including values the app echoes back, so plans run on synthetic data only.

Text output gets the same care: plans refer to passwords by name, fields that look like secrets are rejected, and secret values are scrubbed from the run file and reports before saving.

How we fence in automated tests against staging

A browser script on client staging can still send a real email, call a live payment provider or follow a redirect into production.

An allowlist is a guest list for network traffic: if an origin (a site address such as staging.example.com) is not on it, the browser’s request never leaves the machine.

The allowlist only governs the test browser. If staging’s own server sends email or charges cards, switch those integrations to sandboxes before the run.

During the run, any popup, new window or download blocks the slot, and the page’s origin is rechecked after every action. API checks are read-only (GET or HEAD), capped at 1 MB, and raw responses are never stored.

Each run is built in a pending folder and published only after the report renders and every lock is released, so a half-written report never reaches a client.

Tradeoffs, and a UAT sign-off checklist you can use tomorrow

The narrow scope is deliberate, and it has costs:

• Preparation is the real work. A run needs stable selectors, repeatable synthetic data and temporary credentials, or you get honest INCONCLUSIVE reports and little else.
• Three workflows is a hard limit. More flows, browsers or native apps are a new scope, never a quiet addition.
• Embedded content is out of reach. The runner cannot act inside iframes or shadow DOM, so a hosted checkout widget is beyond what it can test.
• Our own tests stop short of a real browser. The 19 regression tests use a simulated browser, so every run starts with a live smoke test in Chrome.

Next, we would run the real-browser checks in our build pipeline (CI) and delete delivered evidence when the agreed retention period ends. Gates that cannot be faked also shape how we keep AI coding agents in check.

If you are commissioning an app from an agency, or handing one over, book a free 30-minute discovery call. Bring your next release. We will pick its three riskiest workflows with you and mark each check no-go or conditional-go, so “done” is agreed before testing starts.