How to Perform Usability Tests for CRO
When businesses have highly specialized users or when internal teams are having trouble understanding how and why users interact with a flow, I will conduct a usability test.
Overview
Usability testing is a form of user research in which I will ask users to complete specific website tasks and monitor their behavior remotely or in person by a CRO professional. In usability tests, users are asked several questions about their experience to inform the CRO professional about where there are issues in the existing design and how I can alter it to address user needs more comprehensively. The A/B test ideas generated from these studies are often highly effective.
Planning
It is vital to set clear goals for a usability test to help clarify the process and the research questions the study will attempt to answer. The usability test plan should outline the tasks I will ask the user to accomplish and all pre-task and post-task questions. The client should approve the plan before I begin recruiting for my test.
Recruiting
CRO conducts usability testing with users or peers unaffiliated with the client. However, if I am seeking the experiences of highly specialized users, or users who are more technical, I will use a third-party testing service such as Lyssna or UserTesting.
Note: When proposing a usability test to a client with a research platform such as Usertesting.com, ensure to discuss the cost and payment plan with the client beforehand.
When conducting research without using a third-party testing platform, I will obtain informed consent from my users.
Conducting Usability Tests
Scheduling and Recording Sessions
When conducting a usability test, I will schedule the interview session with each participant at least two days before (longer is better) to ensure that users have enough time to complete the test scenario and have plenty of time for questions. A typical usability test session will run for at least 30 minutes, but 45 minutes is often preferable for a more comprehensive study.
These sessions can be conducted remotely or in person. In either case, recording the session for later observation and additional note-taking will be important. If using a video chat software such as Zoom, I will ask the participant to download the app on their desktop or mobile device, whichever device I want to have them use in the usability test. I’ll also have them check their phone/computer settings to allow for screen sharing.
When conducting the test, I will begin recording through Zoom and have the participant share their screen before starting the user experience flow. This way, I can see their screen throughout the process.
Pre-Task, Task, and Post-Task
Before having the participant begin the actual usability test tasks, I ask any pre-task questions that will provide me with the necessary background on user habits. Once the user addresses those questions, I’ll walk them through the test instructions and any scenarios prepared.
At this point, I will have the user attempt to complete the tasks outlined.
As users attempt to complete the scenario’s tasks, I will not react to their actions on screen. I’ll also refrain from feeding them information to assist them, but I will always be available if they have questions. I take notes about the behaviors they demonstrate in addition to their spoken thoughts.
I follow up with any relevant post-task questions once the user completes the task (or gives up), and ask as many follow-up questions as needed to make sure that I fully understand participant responses. Before closing out the session, I will ensure that the user doesn’t have any follow-up comments or thoughts as often off-the-cuff banter can bring the best insights.
Analyzing the Results
Identifying Problems
Usability test results are best analyzed through the lens of trends. I will look for patterns in user responses and quotes from my notes or Zoom recordings that corroborate those patterns. I’ve noted that past clients have responded well to direct quotes and screenshots of specific moments in usability tests because these demonstrate behavior in real time.
The trends identified should center around elements that create friction for the user. These can be elements that did not meet user expectations, should have been more transparent for users, or were conversion barriers. It is also helpful to identify trends in unanswered user questions and user priorities when seeking information.
Note: Though friction points are the best source of optimization ideas, points of delight can also be impactful to focus A/B tests on. I will look for content and on-page elements that users respond well to, and communicate that to my client.
Defining Solutions
From the trends identified in the study, I’ll begin to generate potential solutions to user friction points and other ways to incorporate findings into a CRO roadmap. This is similar to how I conduct a CRO Assessment.
Presenting Your Findings
Usability test results should be shared alongside CRO roadmap recommendations when possible. I will schedule an hour-long meeting with the client; 30 minutes on usability test results and 30 minutes on recommended next steps.