Quality Assurance Labs
QA Testing

Usability Testing — Watch Real Users, Not Test Scripts

Senior QA Engineer6 min readPublished Updated

Users don't file usability bugs. They just leave. The friction that frustrates them never reaches your bug tracker — it just shows up as churn. Here's how to run usability testing that catches the bugs automation can't.

Hands reviewing interface wireframes with user models
#usability-testing#UX-testing#user-research#product-quality

Usability bugs are invisible to normal testing. The button works — it just isn't where users expect it. The flow completes — it just takes 3x longer than it should.

None of these show up in functional tests. None appear in automated regression. They only surface when a real human tries to use your product for a real task.

What usability testing actually tests

Findability — Can users find the feature?

Clarity — Do they understand what to do next?

Efficiency — How many steps does it take?

Error recovery — What happens when they make a mistake?

Trust — Do they feel confident completing the task?

Moderated vs unmoderated

Moderated: researcher watches + asks questions. Highest quality.

Unmoderated: user completes tasks alone, screen + voice recorded. Scalable.

How to run a session

Define the task — "sign up and connect payment" not "test signup"

  • Recruit real users matching your base
  • Give them the task, not instructions
  • Stay silent — the struggle is the data
  • Record everything (screen, voice, expression)
  • Ask non-leading questions after

What we typically find

  • Buttons that don't look clickable
  • Forms with unnecessary fields
  • Confusing error messages
  • Missing confirmation feedback
  • Internal naming used in UI

Metrics to track

  • Task success rate
  • Time on task
  • Error rate
  • Satisfaction score
  • Think-aloud friction

When to run usability testing

  • Before major changes
  • After new features ship
  • When metrics decline without obvious cause
  • During redesigns
  • Quarterly

Common mistakes

  • Testing colleagues instead of real users
  • Leading users toward the "right" answer
  • Skipping recruitment screening
  • Not recording sessions
  • Ignoring findings

Key takeaways

  • Usability bugs don't show in functional testing
  • Watch real users attempt real tasks — don't guide them
  • Mix moderated (deep) and unmoderated (scalable)
  • Track task success, time, and error rate
  • Run usability testing quarterly, not once

Further reading

About the author

Senior QA Engineer →

Senior QA Engineer · Quality Assurance Labs

Notes from the lab.

Testing, engineering and growth — delivered to your inbox.

Need a usability testing sprint? Book a call

Let's talk →