Research Options Defined: Usability Testing

As applications and platforms continue to get more complicated, it is increasingly important to understand what end users think, feel and do when they use your application or website.

Most of our work at Crux Collaborative revolves around creating or optimizing complex processes that are behind a login. We approach these projects using a different methodology than we would a public, content-based site and we never take a one-size fits all approach when it comes to user research.

It’s important to choose the research methodology based on the type of information you are looking to collect.

This is the third in a series of articles that focuses on the pros and cons of various research methods and how to use them to effectively meet your objectives.

In this article we’ll be covering usability testing.

What is it?

At its core, usability testing is a qualitative research methodology that provides insight into how users interpret content and functionality on a website or application. Usability evaluations are a fantastic tool for diagnosing issues with the interface, identifying potential feature enhancements, and gaining insight into the attitudes and perceptions of your target audience.

How it’s done?

There are a number of different ways to approach a usability test and there are many different methodologies and tactics that can be used to evaluate user behavior. We tend to break our evaluations down into two broad categories.

1. Usability as an evaluation tool (for something that already exists)
2. Usability as a design step (when we are creating something new)

Baseline usability enables us to evaluate the efficacy of core components of an application and how determine if the site is meeting our client’s overall business objectives. We will generally invite 6-8 respondents from each of our clients’ key audiences to participate in the evaluation.

Many of our clients will identify up to 5 audiences at the beginning of the project, but we rarely end up talking to 40 respondents. Instead, we reduce the number of audiences by focusing on broad characteristics, rather than granular details. After 4-6 sessions, trends start to emerge and we begin to see where users are having trouble.

In a given day of research, we typically facilitate 8 one-on-one sessions of 45-60 minutes in length. Following the research itself, we analyze the findings and deliver a detailed report, often with visual examples of how to fix problems that surfaced during the evaluation.

Sometimes a client comes to us looking to evaluate a specific process that they know isn’t working for their web site or application. In cases where we are evaluating a limited set of functionality, we can take a leaner approach and reduce the cost of the study. Typically, for these focused usability studies, we deliver a high-level report with a bullet-list of actionable changes.

Usability testing as a design step is one of the most valuable service we offer to our clients. It differentiates us from almost all of our competition, elevates the quality of the final product, and protects our clients’ users from frustration (not to mention reducing calls to the support center).

It is especially helpful during an extensive redesign or if a new application is being created from scratch. Checking in with end users at the right time in the design process can help to inform the overall direction, eliminate extraneous functionality, help prioritize features for future releases, and resolve usability issues before a costly implementation. We typically test a small prototype with a group of target users and deliver a high-level actionable report.

Getting potential clients to agree to this step sometimes takes some convincing. They are hesitant to invest the budget needed to complete the research and reluctant to take time to gather feedback from their users.

We explain to potential clients that checking in with users as a design step is an integral part of our process. We ask them to think of design as a hypothesis and research as an opportunity to validate and refine that hypothesis. We uncover unexpected breakdowns and make substantial improvements to the interface after every single design-step usability test. The question is simple—would you prefer those issues are addressed during a research study or by frustrated end users?
Design-step usability for applications that handle especially complex tasks, in secure environments, with elaborate requirements, is a cost-saving measure, not an additional expense.

When to use it?

A baseline usability test is very helpful at the beginning of a redesign project to help determine what is working and what is not. It also helps the team better understand the target user’s mental model, expectations and desired functionality.

Conducting a one off usability evaluation to improve a specific set of features or drive more successful outcomes is also valuable. It takes the guesswork and assumptions out of play and provides a deeper understanding about how your user base is evolving and using your site or application.

Design-step usability testing is best done on a small tactical prototype (10-15 pages) before any implementation has occurred. Making adjustments to the user experience at this stage is far more cost effective than doing it either post launch or during the implementation.

Advantages

• You will learn more chatting with target end users than making assumptions about what you think they want.
• You will learn something new every single time you run an evaluation.
• Watching a user tackle a complex process will force you to better understand what functionality best matches the business objectives you have and how to make the application more efficient to meet them.
• Prioritization of features for future implementation to help with long term cost planning and budgets.
• It’s easier to build a case for making big changes when you can demonstrate how difficult a process is and how it’s impacting the business.

Challenges

• Recruiting the right people who represent your target users
• Not having enough functionality to test
• Trying to test too much with too many people
• Making strategic decisions about functionality and value based on one person’s opinion

Why we’re good at it

Our business was founded as a usability practice and everything that we do is based on providing an optimal experience for end users while meeting our clients’ business objectives and goals. We have been conducting evaluations with end users for over 15 years and we have tested a wide range of sites, concepts, and applications. We have created a proven process that fosters collaboration between our team and the client team.

Our recommendations produce results because these insights are rooted in 15 years of talking to end users and identifying the patterns that cause breakdowns in the user experience.

We can coordinate a recruitment effort to find and schedule participants for both broad and very specialized markets and we have the ability to scale our approach based on our client’s goals and budget.

If you need help determining whether a usability evaluation is the best fit for your project, talk to us. We’d love to share more about the way we approach user research and how it can benefit you and your users.