Usability has been around for a lot of years. Some people trace its growth from the early 1900s when Frederick Taylor published his “Principles of Scientific Management” in which he described time and motion studies. Today, usability is one of the basic building blocks of creating a great user experience. If a product or service is unusable, then your customers will not stick around waiting for improvement. This is especially true in the age of agile and lean development methodologies. One of the primary principles of agile development, for example, is rapid and continuous contact with the target user.

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. (Agile manifesto)

The only way to know for certain if you are satisfying the customer is to contact them. If you are delivering on a continuous basis, you need to be continually asking them. Usability is one method used throughout much of the software development industry to gather rich and full information. Unlike online surveys which are typically answered by those who either love or hate the product or telemetry which tells you what the behavior your users are attempting but lacks the “why” behind the behavior, usability provides the illuminating insight that will help move the product forward.

Usability does have drawbacks. Usability can be costly in terms of time, depending upon the target profile of your user base. It can also be costly in terms of money if you want a very clean, rigorous result. However, doing regular usability studies while a product is in development reduces costly churn. Early testing allows you to discover things that don’t work before they get committed to code.

In my experience, the primary determinant of the usefulness of a usability study is the product team’s investment. Some teams see usability as a checkbox to complete on the road to shipping a product. More than once I have watched teams hire a researcher to run the study, sit through the report presentation and vow to prioritize the findings with the product backlog. Then late in the cycle, the usability bugs are de-prioritized as the product team attempts to hit its glide path to ship.

The temptation to slip usability bugs is understandable — there is usually an overabundance of bugs to squash and there is always documentation on how to overcome the usability issues. Despite the issue of whether the target users will actually read the documentation. In truth, a prepared team can assimilate the usability feedback with hardly a pause in the normal work streams. Further, early testing can reduce the number of bugs that the team has to fix at the critical end-game.

Here are a few things that a team can do to be ready to incorporate usability sessions into their work stream without major disruptions:

1) Prepare!

Consider whether you need to do a study in the first place.

Participants can be costly (after recruiting and gratuities) and so are researchers. However, doing an expert review is vastly cheaper than taking a raw prototype in the lab. Usually, an expert review is only an hour or two of the researcher’s time, but it can save you thousands of dollars in the lab. It is not that the usability researcher has a better opinion, but they have a different perspective. They leverage psychology and HCI (Human-Computer Interaction) theory to see where issues could arise. They can usually suggest changes that will eliminate some of the most egregious usability issues before you let your target users see it. An experienced researcher will have seen hundreds, if not thousands, of users interacting with software, so they can bring that experience to bear in novel software experiences. An expert review will identify any big usability issues that could hide other critical usability issues, making your usability study vastly more informative.

Know what you want to test and share existing data.

PM, Marketing, Product Planning (and CSS if you have already shipped the V1 product) all have some contact with customers. This is a great source of data that can help define the protocol for a great usability study. What are the top-of-mind questions that have arisen in conversations with customers? What are the most common CSS calls? Those are great sources of research questions the researcher can use to craft tasks. Using this data can ensure that you are not just making changes to the product with new features, but also improving the quality and satisfaction for your customers.

What data did you use to come up with your designs?

The design is never created in a vacuum. Giving your researcher the background of where the designs originated can help them understand where you started. Knowing the origins of the design will helps the researcher understand the inherent bias of the design decisions. Bias is not a bad word here, everyone is biased by their experiences. Making assumptions is almost a requirement at the very beginning of product development. Letting your researcher know what the assumptions are can help them ask questions that will challenge whether those assumptions are accurate or not. This can be hard on the team, and some tightly held opinions may have to get sacrificed for the betterment of the product. However, it is better to discover those in the lab than in the post-shipping customer forums where the court of public opinion can cause user abandonment and give fodder to your competitors.

Know who you are building for so that you don’t confound your results.

It is critical that the profile, meaning the unique description of behaviors and knowledge of your target users, is explicitly defined and ratified by all stakeholders. Early in my career, I performed a study on a profile defined by a PM only to discover in the report presentation, after running the study, that the marketing team had a completely different definition of the target audience. It wasn’t that the results were wrong, but changing the product to suit the users we tested would not have put the product where the market opportunity was the most promising.

2) Participate!

Attend as many usability sessions as possible.

It may seem redundant. You are hiring an expert so why not just wait for the report? However, by sitting through at least a few studies, you gain a visceral understanding of how big an issue is and how the users reacted to it. You might find an inspired fix after seeing the behaviors that the participant attempted when they ran into the issue.

Talk to the researcher after each session.

I have run studies where I called out an issue after just one participant demonstration. I have also downplayed an issue where multiple participants demonstrated a behavior. In the first case, the issue was of such significance that the user would never be able to recover, which would have resulted in severe data loss. In the latter, the background of those specific participants was not an exact match for that scenario, so their failure was due to inexperience in the space, rather than the lack of information in the UI. We also verified that the tasks we ran were not feasible by a novice, something that was a dispute within the team. A researcher’s experience can help keep you from chasing red herrings but also spot camouflaged pitfalls. Debriefing will ensure the best chance of capturing all the important feedback and getting it in the final report.

Take your own notes.

Again, this may seem redundant, but sometimes a new nugget appears that your researcher missed in the hectic lab environment. You have a unique experience with the product and may spot a new piece of data because of that. In the debrief, compare notes and talk about what you have seen. It can also save you from following a red herring and making changes to the product when it is not needed.

Log your own bugs.

First, I have to say that you should follow the bug logging plan of your team. However, as a general recommendation, if you see something, say something. Logging bugs will ensure that no issue falls through the cracks. Later in triage, bugs can be reconciled as duplicates. The researcher typically won’t get to it until after the final report. Another reason to file bugs as you see them is to ensure that the bug contains all of the relevant information for your team to understand. A consultant researcher or a vendor may not know all the bug conventions of your specific team. A team member writing the bugs ensures that the issue is categorized and described correctly, so it is triaged in the most efficient manner.

3) Follow up!

Attend the report presentation.

Typically, the researcher will have some suggestions about how to improve the product/service. However, they may not be the subject matter expert, so their suggestions may not work. The suggestion might not be feasible due to technology constraints, timeline constraints, or business reasons. The researcher might not be privy to all of those factors, so your attendance and participation at the presentation will help to create better bug entries and more efficient solutions. Participating in the report presentation will help the team take the right information back to their desks.

Make sure you are completely clear about the recommendations.

Recommendations are based on insights the researcher has gleaned from their notes, user behavior, psychology, sociology, product heuristics, etc. You can improve the outcomes by asking for clarification, demonstration or even illustrations of the recommendations. The product team will be doing the work to fix the product based on the feedback. However, the researcher may or may not be working on the product or feature at that point. It’s important to understand the design implications of the recommendations. Otherwise, it is difficult to fully weigh their importance when compared to the rest of the bugs.

Incorporate usability findings and recommendations into already existing bugs.

One of my professors in graduate school told me that psychology was “the science of rediscovering the obvious”. If the product is in development for longer than a heartbeat, then it probably has a test team already filing bugs against it. A proficient test team is probably already filing usability bugs. Bugs logged from usability session and usability bugs logged from the test team are a bit different from one another. It doesn’t matter how good a test team is, they really just have a hunch that a target user may experience the issue. The usability data provides the support for that hunch. Linking the test bug with the usability study findings will help prioritize the right issues. It also helps identify the right bug categories which allow for more accurate cataloging of other issues.

Conclusion

The success of a usability study relies on the skill and expertise of the design researcher you hire/use. However, as a stakeholder in the product outcome, you can influence the outcome for the better by following these tips. This post is based on usability work I have done both as an internal employee at large software corporations and as a consultant. However, I have found that I still learn from every new engagement. Every new usability session has something to show me. I would like to hear from other teams about what you would like to see from your usability research? Alternatively, if you are a practitioner, what tips am I missing from my list above?


Don’t miss future updates. Sign up to our newsletter to get great content and insights right in your inbox every other week.