User Research/ Testing

User Interviews

After the market and competitive analysis has been conducted, I dive into initial user interviews that will ultimately fold into my overall user research. 

Test Script Example

I start the process by creating a test script and recording release/NDA forms so that users understand how we will be using this data and what to expect during the interview. 

Test Script

 

Hi,_______________.  My name is__________________, and I’m going to be walking you through this session today.

 

Before we begin, I have some information for you, and I’m going to read it to make sure that I cover everything.

 

You probably already have a good idea of why we asked you here, but let me go over it again briefly.  We’re asking people to look at current features from our website , so we can see whether these features work as intended.  This session should take about 30-45 minutes.

 

The first thing I want to make clear is that we’re testing this website, not you.  You can’t do anything wrong here.  In fact, this is probably the one place today where you don’t have to worry about making mistakes.

 

As we go, I’m going to ask you as much as possible to try to think out loud.  Please say what you’re looking at, what you’re trying to do, and what you’re thinking.   This will be a big help to us.

 

Also, please don’t worry that you’re going to hurt our feelings.  We’re doing this to improve our product, so we need to hear your honest reactions.

 

If you have any questions as we go along, feel free to ask them.  I may not be able to answer them right away, since we’re interested in how people do when they don’t have someone sitting next to them to help.  But if you still have any questions when we’re done, I’ll try to answer them then.  And if you need to take a break at any point please let me know.

 

With your permission, we’re going to record what happens on the screen and our conversation.  The recording will only be used to help us figure out how to improve this feature, and it won’t be seen by anyone except the people working on this project.  It also helps me because I don’t have to take as many notes.

 

There are few people from the our team who will be observing this session.  They are here only to observe to better understand how people are using this product.

 

At this time, I am going to send over a NDA form and the recording permission form.  Please check your email for the HelloSign documents.

 

-ACTION BREAK-

-Confirm the user has signed the permission form and the nondisclosure form

-Start the screen recording

-END ACTION BREAK-

 

Do you have any questions so far?

 

OK.  Before we look at these features, I’d like to ask you a few questions.

 

What was your experience like using the XY Tax Solutions intake form last season? 

If you could pick out one thing that you would have like to have seen on the intake form last season, what would it be?

 

OK, great, we can now start looking at the features.

 

-ACTION BREAK-

Go to the Google Sheets document, “August Usability Testing/User-XYTS Intake form” follow all questions, fill out observation sheet per user

 

-ACTION BREAK-

Thank you for completing this testing with us.  Please give me a few minutes to make sure my team doesn’t have any follow-up questions they’d like me to ask you.

-Message the team to see if they have any follow up questions

-ACTION BREAK-

 

It doesn’t appear that my team has any follow-up questions, so I am going to end this session with a few questions of my own.

 

1. What were your overall impressions or thoughts about the features you have seen today?

2. If you could give one piece of constructive feedback about this intake form, what would it be?

2.Do you have any questions for me, now that we’re done?

 

-ACTION BREAK-

-If an incentive was promised, remind them that it will be given to them.

-Stop the screen recorder and save the file.

-Thank them for their help, and escort them out of the office.

-ACTION BREAK-

User Question Example

Every interview question I ask is designed to measure a specific objection or assumption to better define the problem statement. After I formulate the questions, I document what the question measures and what to watch for in the user's response. Once I conduct the user interview and take my notes, I map the data to find patterns and then prepare my user interview report.

A/B Testing: Intake Form

In this status page A/B test example, I was interviewing users to determine what status badges would cause the user to take immediate action.  I also tested the copy to determine what aligned most with user expectations.

Interview Findings

The assumption for the badge design was that we needed to keep it short and clean. The user interviews showed us that "Attention Needed" was more likely to trigger the user to take action. "On Track" left the users feeling more at ease that their tax returns were getting filed on time. All financial advisors determined that "Complete" was not sufficient because complete could mean the tax return was completed and ready for review or that the tax return was filed. This insight helped determine that the status badge needed to say "Tax Return Filed" instead of "Complete." It was also decided that when a tax return has been completed but not filed, it is still considered "In Progress."

Another unexpected discovery during these interviews was that that the design needed to have two columns for client names due to joint filings.  

Sample A

Sample B

Final Design

A/B Testing: Find an Advisor Profile

In this A/B test example, I was interviewing users to determine what profile was more effective in displaying a financial advisor's information.  This test was conducted with financial advisor clients to determine what layout was more inviting and digestible.

Findings

The findings from user interviews showed that having a banner in the top right corner, like shown in example B, was more impactful than having an explanation of the match shown in example A. However, the remaining layout in example A was more favorable than example B because it was more clean-cut.

User Testing: Prototypes

I follow the same cadence in questioning when testing prototypes or final designs. Unfortunately, these interviews can't be shown outside of my current organization due to signed agreements.

Analyzing Company Data

Besides testing and user interviews, I also pull data from our organization when conducting user research. This includes our annual benchmarking survey, membership data, portal analytics, and product analytics