Everything You Need to Know to Start User Testing

INTERSECT helps businesses create and deliver exceptional digital products that marry business objectives and user needs. Learn more.

If you aren’t user testing, you aren’t building a great product.

User testing is how companies validate what they want to put in market (or what they already have) with who will actually be using it. Without it, products are really only a collection of guesses and (possibly misguided) intuition on what a customer might want.

Building a strong product means understanding your product can always be better. That means listening to your customer – a lot. If you can manage to do that, you aren’t just optimizing the experience for the sake of it. Product improvements can be rooted back to tangible positive business outcomes. Creating a more positive experience can even lead to a higher willingness to pay – 81% of customers indicate they will pay more for a better customer experience.1 If customers are having a better experience, they are more likely to refer services, which decreases your customer acquisition costs. Less friction can mean higher conversion rates, a higher average spend, and an increase in your customer’s total lifetime value.

If you’re ready to take your product to the next level, we can guide you through our process for agile and manageable user testing. Here’s what we’ll cover in this article:

  • What’s the end result of user testing?
  • Where do I start?
  • Will this really be a simple task?
  • What kind of user testing should I do?
  • When should I do user testing?
  • Who should I test with?
  • How do I set up a simple user testing plan?

What’s the end result of user testing?

As user testing is conducted, you’re looking to validate 3 major pieces of your product:

    1. Design and UX (User Experience) – Can customers navigate your experience effectively and use your features successfully?
    2. Content – Does the product have the information customers are looking for? Does the content you’ve used improve & add to the experience?
    3. Capability – Does the product do what customers expect it to? Did you get the minimum viable product (MVP) right? Should your roadmap change?

Looking at these 3 pieces builds an understanding of what’s going right and reveals opportunities that have been missed or have developed over time. The outcome of user testing is being able to confidently answer these major questions:

Where do I start?

The best guideline we can offer is to start with a goal for your user testing. During the Strategy Discovery phase of a product, you’ll determine a product goal, which is an aspirational vision that aligns everyone on what this product is trying to accomplish. For example, a product goal could revolve around bringing transparency & simplicity to employee benefits. User testing should clarify if the product is meeting this goal and where it’s falling short. One layer below the aspirational vision should be high-level strategies that support this goal – if these strategies are achieved, then you will be successfully working toward this goal.

An example of a product strategy could be ‘Transparent & Digestible Information’. Benefits can be confusing, and if you’re trying to provide clarity, this is a big strategy. Going one layer deeper leads to the features or tactics you will use this to meet this strategy – like an Overview Dashboard or a registration flow. User testing can then be pulled in to build objectives for each of these major tactics that are a core piece of your strategy.

Will this really be a simple task?

The return on investment for user testing can be pretty impressive. You only need to speak to 5 customers to recognize 85% of major interface problems. Speaking to only 12 customers will turn up 99% of problems.2 This doesn’t mean you can skip steps like QA, which stress-test the build of the product and find any bugs in its engineering. It means major experience issues can be spotted without needing to invest tons of resources to testing. Once you’ve set your objectives and a simple user testing plan (outlined later in this article), you’re on your way to an awesome product.

What kind of user testing should I do?

User testing can take on many different forms – but it comes down to any activity that will help you better understand how your customer interacts with your product. This can fall on a spectrum roughly mapped out as “Quantitative & Impersonal” on one end and “Qualitative & Personal” on the other. Within this spectrum, you can plot things like Analytics and A/B testing, mouse tracking, remote interviews (conducted over technology vs in person), and in-person interviews. Both quantitative and qualitative information is critical as they often support each other.

At Intersect, we typically suggest analytics, in-person interviews, and A/B testing. Analytics become a pillar for recognizing possible usage patterns (i.e. drop off, feature interaction). These patterns can then be clarified in qualitative interviews. Once clarified, different solutions can be implemented with small groups of user (A/B testing) and the best solution can be rolled out.

If you want to get the most out of your interviews, doing them in-person is strongly encouraged. Being there physically gives you insights that are just not possible with remote testing. Mainly, in-person interviews make it possible to create a comfortable environment, ensure the customer is in the right mindset and isn’t distracted, and observe hand movements or facial expressions that can further be noted or explored.

When should I do user testing?

A good rule of thumb is to test early and often, from prototype (if it isn’t in market) to any other stage before large changes to the product will be made. The earlier you test, the less you risk moving in the wrong direction, which means you won’t waste a lot of resources building the wrong product. This is an example of user testing fitting into a product’s development cycle:

Who should I test with?

To understand who will offer the most insight when testing, you’ll have to think about every different group that could be interacting with this product. Don’t forget that customers can also be broken down into different personas, so you may want to ensure you’re getting perspectives not only from different audiences (new vs. current customer) but also different kinds of each (millennials new to the product vs. well-established starter families). You should be getting at least 2-3 people per audience. Here are some examples of different kinds of audiences that could be a part of user testing:

How do I set up a simple user testing plan?

Congratulations on making it to the last part of the process! A user testing plan is a great way to make sure everyone who touches the product knows what’s going on. It acts as a guideline for how you are choosing to approach user testing. There are four major sections to this plan. Here they are:

    1. The Basics
    2. The Goal, underlying Objectives & Tactics
    3. Measuring Success in User Testing
    4. Possible Scripting & Questions/ Tasks to Ask

The questions we’ve covered so far about user testing fit in really well with our process for developing a user testing plan. Come back next week to understand the last piece of the puzzle for conducting user testing successfully.