User Research/ Testing
User Interviews
After the market and competitive analysis has been conducted, I dive into initial user interviews that will ultimately fold into my overall user research.
Test Script Example
Process
I start the process by creating a test script and recording release/NDA forms so that users understand how we will be using this data and what to expect during the interview.
Purpose
I start the process by creating a test script and recording release/NDA forms so that users understand how we will be using this data and what to expect during the interview.
Test Script
Hi,_______________. My name is__________________, and I’m going to be walking you through this session today.
Before we begin, I have some information for you, and I’m going to read it to make sure that I cover everything.
You probably already have a good idea of why we asked you here, but let me go over it again briefly. We’re asking people to look at current features from our website , so we can see whether these features work as intended. This session should take about 30-45 minutes.
The first thing I want to make clear is that we’re testing this website, not you. You can’t do anything wrong here. In fact, this is probably the one place today where you don’t have to worry about making mistakes.
As we go, I’m going to ask you as much as possible to try to think out loud. Please say what you’re looking at, what you’re trying to do, and what you’re thinking. This will be a big help to us.
Also, please don’t worry that you’re going to hurt our feelings. We’re doing this to improve our product, so we need to hear your honest reactions.
If you have any questions as we go along, feel free to ask them. I may not be able to answer them right away, since we’re interested in how people do when they don’t have someone sitting next to them to help. But if you still have any questions when we’re done, I’ll try to answer them then. And if you need to take a break at any point please let me know.
With your permission, we’re going to record what happens on the screen and our conversation. The recording will only be used to help us figure out how to improve this feature, and it won’t be seen by anyone except the people working on this project. It also helps me because I don’t have to take as many notes.
There are few people from the our team who will be observing this session. They are here only to observe to better understand how people are using this product.
At this time, I am going to send over a NDA form and the recording permission form. Please check your email for the HelloSign documents.
-ACTION BREAK-
-Confirm the user has signed the permission form and the nondisclosure form
-Start the screen recording
-END ACTION BREAK-
Do you have any questions so far?
User Question Example
Process
Every interview question I ask is designed to measure a specific objection or assumption to better define the problem statement. After I formulate the questions, I document what the question measures and what to watch for in the user's response. Once I conduct the user interview and take my notes, I map the data to find patterns and then prepare my user interview report.
Purpose
To guide the interview process while assisting in data collection. This layout makes it easier to assess patterns in the data.
A/B Testing: Intake Form
Process
In this status page A/B test example, I was interviewing users to determine what status badges would cause the user to take immediate action. I also tested the copy to determine what aligned most with user expectations.
Purpose
The purpose of A/B testing is to determine what design is easier to use. The beta testers provide valuable insight to what processes they prefer to use and why, helping us determine what designs reduce the cognitive load.
Interview Findings
The assumption for the badge design was that we needed to keep it short and clean. The user interviews showed us that "Attention Needed" was more likely to trigger the user to take action. "On Track" left the users feeling more at ease that their tax returns were getting filed on time. All financial advisors determined that "Complete" was not sufficient because complete could mean the tax return was completed and ready for review or that the tax return was filed. This insight helped determine that the status badge needed to say "Tax Return Filed" instead of "Complete." It was also decided that when a tax return has been completed but not filed, it is still considered "In Progress."
Another unexpected discovery during these interviews was that that the design needed to have two columns for client names due to joint filings.
Sample A
Sample B
Final Design
A/B Testing: Find an Advisor Profile
In this A/B test example, I was interviewing users to determine what profile was more effective in displaying a financial advisor's information. This test was conducted with financial advisor clients to determine what layout was more inviting and digestible.
Findings
Process
The findings from user interviews showed that having a banner in the top right corner, like shown in example B, was more impactful than having an explanation of the match shown in example A. However, the remaining layout in example A was more favorable than example B because it was more clean-cut.
User Testing: Prototypes
I follow the same cadence in questioning when testing prototypes or final designs. Unfortunately, these interviews can't be shown outside of my current organization due to signed agreements.
Analyzing Company Data
Besides testing and user interviews, I also pull data from our organization when conducting user research. This includes our annual benchmarking survey, membership data, portal analytics, and product analytics
