Unmoderated Usability Testing vs Moderated Usability Testing: Which is Right for You?

Imagine that you’re about to launch a new feature on your company’s app that will finally make your product competitive with top ranking brands in the app store. While your product team believes the app is bound for greatness, you’re not so sure about how your users are going to feel. The problem? You needed a large-scale usability study completed yesterday.

What can you do to ensure you’ve addressed all of your user’s pain points by speaking to as many as possible in a very short amount of time? You can conduct unmoderated usability testing quickly, going into the launch with more confidence than ever.

What is unmoderated usability testing and how can your organization use it to better understand customers to drive insights and outcomes? In order to ensure that this method of user experience (UX) research is right for you, let’s explore the key benefits, differences between unmoderated and moderated usability testing, and factors you should consider before taking on your next unmoderated usability test.

What is Unmoderated Usability Testing?

Unmoderated usability testing essentially means that there is no one present during the research project or study. Participants of the study answer questions at their own pace, on their own time, and can participate from any location. This type of study is typically conducted by having a pre-set test that is given to participants after they meet the specified criteria. Unmoderated testing is most useful where a particular environment or context is critical to the outcome of the study.

Examples of where unmoderated usability testing can be used include:

• When a quick and specific functionality needs to be tested
• When two or more products or interfaces need to be compared
• For benchmarking studies comparing competitor products
• For comparing different iterations of your product

Key Benefits of Unmoderated Usability Testing

There are many benefits to using unmoderated usability testing. As mentioned, when user and customer feedback is gathered remotely, the possibilities are endless. Key advantages and characteristics include:

• Time Saving. UX teams can expect lower turnaround time compared to moderated usability testing, with projects that can be completed in just a few days rather than weeks when scheduling, travel logistics, multiple moderators, etc. are eliminated entirely.
  Compared to in-person moderated usability testing, teams not only cut down on time, but budget spend as well.
• Scalable. Unmoderated testing is extremely useful in testing large sample sizes, especially for quantitative studies that only require pointed, simple responses. Because this testing method can be executed anytime and anywhere, you can recruit more representative samples from anywhere in the world much more quickly than through moderated testing.
• Iterative. When the feedback loop is quick and adaptable to people’s responses, product and UX teams can get feedback at every stage of product development to ensure they continue on the right path. Teams that iterate not only deliver more relevant results to key audiences, but they move their entire organization towards becoming more agile.

Unmoderated vs. Moderated Usability Testing

Perhaps the biggest (and most obvious) difference between unmoderated usability testing and that of moderated is that there is not a testing professional present.

In a moderated study, a moderator can nudge participants to share or speak up, ask questions to make sure participants are following along, and generally read body language in response to questioning. Moderated research also empowers teams to dig deeper into responses and uncover why they answered a question a certain way.

In addition, moderators can share confidential prototypes and other proprietary materials to gather feedback around concept and product testing.

Although unmoderated usability testing does not require hands-on involvement from a moderator, UX and research teams should have dedicated team members on hand throughout a study to monitor responses and review the feedback as it comes in, ensuring that responses are being submitted correctly.

Here are some best practices to keep unmoderated usability testing on track sans moderator:

• Give clear instructions. Make sure that tasks are written out clearly; otherwise, respondents could make mistakes that would discount the results. For example, if your task requires having respondents give credit card details, providing fake credit card details to show as an example would help streamline your instructions.
• Use questions with tasks. Include questions after each individual task to be sure that the tasks are completed correctly.
• Avoid bias. You can avoid creating bias by showing respondents just one question at a time so as not to influence how they answer other questions in a sequence.
• Identify objectives before using a platform. Before choosing the right platform for your unmoderated usability project, identify the objectives you wish to achieve.
• Set clear expectations. Remember that participants won’t receive any assistance on the study from a facilitator – make sure that tasks are clear, user friendly, and that help is always available.
• Keep participants engaged. Interesting, thought-provoking and short prompts are ideal for this mode of usability testing, which will only increase the likelihood that your participants will complete all the tasks you need for your study.

Conclusion

Now that you understand the key differences in unmoderated usability testing and moderated usability testing, how do you know what is the right fit for your next project? Here are a few questions that you may want to answer before you make that decision:

• Are you short on time and budget resources? If so, unmoderated testing may be your best option.
• Do you need a large sample size quickly? You can scale up more easily with unmoderated feedback as opposed to the scheduling and logistics of moderated usability testing.
• What is the complexity of the project? Will you have confidential or sensitive information that you prefer not to share through an unmoderated prompt? Moderated testing may be your best solution — learn about Discuss mobile screen sharing capabilities here.
• Are you looking for quick, iterative feedback or larger scale studies of people’s lifestyles, ongoing attitudes, or other longer term studies? Unmoderated usability testing lends itself to lightweight research that is fast and efficient.

Unmoderated usability testing can be extremely useful in terms of scaling user and customer feedback. In comparison with moderated, unmoderated usability testing enables participants to answer questions on their own time, and in their own environment.

To conduct unmoderated video surveys, diaries, and more, Discuss offers Self Captures, integrated into our single, comprehensive People Experience Platform to help you make decisions faster and smarter.

Click here to speak with a Discuss Expert to determine which approach is best for you, and to better understand our range of capabilities that support user testing.