Working Surfacelink · 14 Nov 2026
05
A ship slides down its slipway with a wind vane, a row of small gauges and a trailing log line already fitted, while a figure seen from behind on the quay watches the needles, the whole scene drawn with wide empty margins on every side.
Nielsen Norman Group

Research-In-The-Loop: How UXR Adapts to Development at AI Speed

Tony Alicea · 13 November 2026

How-to · artifacts · collaboration · Evals before the build

Read the original
Takeaway

Tony Alicea of Nielsen Norman Group proposes that researchers add a research section to each feature's spec, so coding agents build the measurement along with the feature.

Summary

Tony Alicea, writing for Nielsen Norman Group, a user experience research and training firm, argues that AI coding agents now let teams release features faster than researchers can study users. His answer, which he calls research-in-the-loop, has research state what a release must measure before the build. Agents build that measurement with the feature, and researchers study the results to shape the next requirements.

Key points
  • His example is a help-desk product that lets departments pass tickets directly to each other. Its research section states the expected outcome, signs it worked, signs of harm, what to track and when to interview people.
  • A coding agent reading that section can build the event logging, a dashboard for researchers and a send-back question along with the feature.
  • Alicea warns that the section is not a complete evaluation design. It names what to measure but cannot show the feature caused a change, which may need a comparison group and an observation period.
  • Qualitative research stays: on a regular schedule, when release data points to a group worth studying, and before decisions that are hard to reverse, such as pricing.
Implication

The author suggests a product team try it for one release: add a short research section to the spec and have the coding agent build the tracking it names. He presents the approach as a proposal with open questions, not a tested method.

Plays
Design

Write a research section into a feature's spec naming the expected outcome, signs of success and harm, and what to track, so the coding agent builds the tracking.

Design

When release data shows one group behaving unexpectedly, such as a department sending back many tickets, schedule interviews with people from that group.

Derived by Working Surface from the article; more in the Playbook. Source line: The researcher is accountable for asking the right questions.

Source issue

14 November 2026: as AI agents produce more code and tests than people can inspect, teams are writing down which work gets checked closely, by whom or by what, at what cost, and how they will know it worked.