Decide what would change your mind
Before recruiting anyone, write down what result would cause you to alter the plan. If no realistic finding would change anything, the research is theatre and the budget is better spent elsewhere.
This also prevents the common outcome where inconvenient findings are reinterpreted afterwards until they agree with the original intention.
Ask about the past, not the future
"Would you use this?" produces polite agreement and predicts nothing. People are genuinely bad at forecasting their own behaviour, and they are also trying to be helpful to you.
Ask instead what they did last time: what they were trying to achieve, what they actually used, what went wrong, and what they did next. Recalled behaviour is far more reliable than predicted behaviour.
Watch rather than ask, where you can
Give someone a realistic task and observe. Where they hesitate, what they misread, and which route they take are all more informative than any opinion they could offer.
Resist helping. The moment you explain the interface, you have stopped learning anything about whether it explains itself.
You need fewer participants than you think
For usability problems, a handful of participants surfaces the large majority of issues, and additional sessions mostly repeat what you have already seen.
Larger numbers are needed for measuring preference or estimating rates across a population — a different kind of study with different methods. Do not use a small qualitative sample to claim a percentage.
- Recruit people who match your actual users, not colleagues
- Run sessions one at a time so you can adjust between them
- Have someone from the build team watch at least one session live
- Record, with permission, so findings can be evidenced rather than recalled
- Write up within a day, while the detail is still available
Separate observation from interpretation
"Four of six participants tried to click the section heading" is an observation. "Users expect headings to be clickable" is an interpretation. Keeping them distinct in the write-up lets others check your reasoning rather than only accepting your conclusion.
Report what contradicts you
Research that only supports the existing plan is the clearest sign of a poorly run study. Report the findings that were unwelcome, prominently, because those are the ones with the highest value.
A team that punishes inconvenient findings will get research that stops producing them, and will keep paying for it.