Requirements Validation and User Testing: Prototypes, Usability, and A/B
Turning "what we think users want" into "what users actually need and can use" requires validation. Nielsen Norman Group points out that even the best designers cannot design a good-enough experience without observing real users — the only way is to test. This article covers four methods — prototype testing, usability testing, A/B testing, and interviews — and when to use each.
Why User Testing
The goals of usability testing usually include: identifying problems in the design, uncovering opportunities to improve, and learning about target users' behavior and preferences. Modern interfaces involve huge numbers of variables, and the human brain adds even more. The total number of combinations is enormous — the only way to get the UX right is to test it, through iterative design driven by observation of real users.
The Three Elements of a Usability Test
A typical usability test has three core roles and elements:
- Facilitator: gives instructions, answers questions, and asks follow-ups, while avoiding leading questions that could influence the participant's behavior.
- Tasks: realistic activities the participant might perform in real life. Task wording matters enormously — small errors can cause misunderstanding or prime the participant.
- Participant: a realistic user (or someone with a similar background or needs), often asked to "think aloud" — narrating actions and thoughts while performing tasks.
A typical test script does not need to be long — 4-6 tasks, ordered the way users would actually do them. For a cart checkout flow, for example:
| Task | Wording | What to watch |
|---|---|---|
| 1 | "Find the jacket you added to your cart yesterday" | Navigation and search path |
| 2 | "Enter your address on the checkout page" | Form comprehension and error messages |
| 3 | "Change the delivery method to next-day" | Discoverability of options |
| 4 | "Complete payment (use the test card)" | Trust cues and friction on the pay page |
Note that task 1 says "the jacket you added yesterday", not "click the cart icon" — the latter hands the participant the answer and tells you nothing about the real navigation path. Every detail of task wording can shift the results, which is why NN/g keeps stressing that tasks should mirror real life.
Qualitative vs. Quantitative
Qualitative testing collects insights, findings, and anecdotes about how people use the product — best for discovering UX problems and the most common form. Quantitative testing collects metrics such as task success and time on task — best for establishing benchmarks. For a typical qualitative study of a single user group, NN/g recommends five participants to uncover most common problems.
Remote vs. In-Person
Remote tests usually require less time and money. There are two types: remote moderated tests work like in-person sessions via screen sharing, while remote unmoderated tests let a testing tool take over the facilitator's role, delivering tasks and collecting metrics and recordings at scale. In-person sessions are better for observing body language and real environments. For AI features, use the methods in AI App Virtual User Testing to cover scenarios at scale with virtual users.
Prototype Testing and A/B Testing
Prototype testing validates information architecture and interactions before coding, using paper or clickable prototypes — low cost, fast iteration, and the first gate of requirements validation. A/B testing runs quantitative comparisons of two variants after launch, such as button copy, pricing display, or onboarding flow. They complement each other: prototype testing answers "is the solution usable," while A/B answers "which solution is better." Before writing test scenarios, first capture requirements as User Stories and Acceptance Criteria so tasks and acceptance align.
Here is how the four methods line up:
| Method | Question it answers | Typical sample | Cost | When |
|---|---|---|---|---|
| User interview | What users want and why | 5-8 people | Low | Discovery phase |
| Prototype testing | Is the solution usable | 5 people | Low | Before coding |
| Usability testing | Where the current design fails | 5 people/round | Medium | Every iteration |
| A/B testing | Which variant is better | Thousands to tens of thousands | Depends on traffic | After launch |
A concrete validation loop
Say a team is adding an "auto-reconciliation" feature to an accounting SaaS. The first round is five user interviews, which confirm that "manual reconciliation is exhausting" is a real pain point — but users do not understand the phrase "auto-reconciliation". What they say is "make the system balance the books by itself", and that changes the feature's name. Round two uses a clickable prototype with five users, and it surfaces the expectation that results appear within 10 minutes of importing statements, while the team had planned a 30-minute batch job — so the async progress messaging gets reworked. After launch, an A/B test on the "reconciliation results" page shows the new summary card lifts clicks on "what to do next" by 21%.
The takeaway: every round answers one concrete question, and the question comes from the previous round's findings. Validation is not a one-off review meeting; it is a small loop embedded in the development cadence.
Reference: Nielsen Norman Group, Usability Testing 101 https://www.nngroup.com/articles/usability-testing-101/
Cost and ROI
Simple "discount usability" studies are inexpensive — a conference room and modest participant incentives, completed in about three days (plan, test five users, analyze, and turn findings into redesign recommendations). Elaborate studies — competitive, international, multi-group, eyetracking — can run into hundreds of thousands of dollars. Yet even high-cost research usually has positive ROI, because fixing problems early costs far less than rework after launch.
16IDC perspective
For website-building and SaaS teams, build a validation loop in this order: interview and prototype to validate requirements -> usability test to validate usability -> A/B to keep optimizing after launch. Do not chase sample size in every round — five users, think-aloud, and a clear improvement list are enough to keep each iteration closer to users. For the full analysis workflow, return to the Requirements Analysis category on this site.
Source: https://www.nngroup.com/articles/usability-testing-101/