Measuring usability metrics: 6 test metrics & free scorecard

How to test the usability of a system

In this section, we’ll be going over some of the top usability metrics you can use to improve user satisfaction right out of the gate, allowing you to send the product into the world with confidence. Below this section, you’ll find a scorecard to help with these metrics.

What are the different usability testing metrics?

There are three main categories in usability testing that all of the other metrics are born from. It’s necessary to determine which of these is going to be the best measure of success for your product and then from there go deeper into the subcategories of specific tests in your usability evaluation.

The total number of tests you run directly affects what you learn, but a word of warning; be mindful of going overboard on the people taking the tests. If you’re doing so many tests that it all becomes a blur, you’ve gone too far. And people only want to answer so many questions before they feel they’ve given you enough information. You’ve likely done a survey like this where you felt it went on for long enough just to find out you’re only 36% complete. Find the sweet spot.

Once you’ve locked in your tests, it’s onto specific UX metrics. We’re going over the ones we feel work the best — but there are plenty more out there. Just bear in mind there is a difference between usability tests and usability metrics.

Task success

Here we can test the number of errors that occur within a process by way of a numeric score. This one’s pretty basic but it gives a very real indicator of how easy something is to do without outside assistance.

What is task success?

It’s measured from 0 to 1 and given a % out of 100. 0 is a fail, 1 is a pass, divided by the number of attempts. The completion rate can tell you in the simplest terms if something works or doesn’t. It may have made sense to you in development but maybe was unclear to the rest of the world (i.e., the people who have to use and spend money on the product). This is what we call formative vs. summative usability testing.

Why is task success important?

The task completion rate is very important for one glaring reason; if people aren’t completing a task, they’re getting nowhere. Furthermore, if they’re getting nowhere, they’re unlikely to keep spending money on your product or recommend it to others. They’ll find another one they actually enjoy using. And fast.

Number of errors

The number of errors experienced is somewhat of an offshoot of task success but provides further information. The users may have completed the task successfully, but how much trouble did they have doing it? Task success tells us that they did complete it but you need to know the number of errors because even a few errors can be a major deterrent. The number of errors test measures ease of use based on the error rate of a specific task.

What is the number of errors test?

Calculating the number of errors can be done in a few ways. As with task success, it can be divided into a percentage, but you can also just look at the raw numbers. No math is needed if it takes someone 10 tries to get something done. The answer there is pretty clear.

Why is the number of errors important?

Think about what you consider too many attempts before you say forget it and move on? Two? Three? Five? The reality is some people don’t have the patience for one error let alone five. So getting it down to zero is great, but everybody has a different way of processing information so some errors here and there are ok. The majority should be error-free though — getting it done the first time, in a perfect world, anyhow.

Task time

Getting task time down is vital. For the most part, if a site takes longer than 2-3 seconds to load a page, people will leave (yes, for real). We’ve all done it here and there so this shouldn’t be a surprise but when you see it in writing, it’s quite shocking. Because of that, getting task time down to a minimum is of the utmost importance. People have "stuff" to do. It isn’t always about how long it takes the page to load. The task itself just shouldn’t be a burden to people and their valuable time.

What is task time?

Task time, as it suggests, tells us how long it takes to complete a task and satisfy the completion rate. While all of these require user research, it’s particularly important for task time. Taking into account other metrics can help determine how to get this number down. Fewer errors mean faster completion.

Why is task time important?

As mentioned, if people can’t get it done quickly and easily, they leave. Task time can help gauge if people will stick around to get it done or if it needs improvement. If they leave, they’re taking their sale with them.

Single ease question (SEQ)

Now we’re getting into the questionnaire bits where information is more granular because of the specific things being asked. It’ll still be determined by a number but you’re asking a question instead of determining if someone is completing something or not. These are simple yet highly effective tests to get info from the largest number of people. It isn’t too hard to answer a quick question on your way out the door, right?

What is a single ease question?

The best example of this is when you’ve completed a purchase, you’ve likely seen the question, how was your experience? Sometimes it’s a number system; sometimes, it’s the five emoji faces, from a frowny face to a happy face. Some sites even make this a double, and they may ask you to write a little something to elaborate, but you get the idea.

Why is the single ease question important?

Time, efficiency, answer rate/participation, etc. You can get a lot of responses from a single question. UX researchers enjoy these simple tests, and they’re very efficient. Although quick and simple, we can gain a lot from it. A score to look for would be in the 4.8 range or higher.

System usability scale (SUS)

Your next step and a great option is the system usability scale. The big brother to the single ease question. A neatly packaged bunch of ten questions graded by ten points each.

What is the system usability scale?

The questions are given ten points each and added up. Ordinarily each question is marked with a score of 1-10 or 1-5 (then multiplied by 2) for a total score of 0-100. This is where things really become clear and a lot of data comes in. Ten questions really aren’t too bothersome but can provide a lot of insight into the minds of those who use the product. Here we go further into detail about the experience and what people were hung up on, or what they found pleasant.

Why is the system usability scale important?

This is relevant because the amount of data that can come from a simple ten-question system is surprisingly vast. It gives people the ability to elaborate on things without having to write out why. It’s an invaluable tool that people can use with just ten clicks. The truth is, not everybody’s a poet, and writing out answers can be off-putting to some folks. This is the best way to get the most with the least effort. But don’t think that means you can cut corners. The questions need to be worth it, or else the data is garbage. You’re looking for a score of 68 or better on the end result for the usability to be considered successful.

Single Usability Metric (SUM)

The single usability metric is a benchmark that combines other metrics to give a single score. This is a trusted metric as it has a strong theoretical foundation.

What is the single usability metric?

This metric provides a score based on a couple of previous metrics that are then standardized so they can be combined, divided, and compared. This one admittedly is complicated, but there are SUM downloads that you can pop right into Excel or Google Sheets. For a full-on explanation of the process, further information can be found in this article over at UX Collective.

Why is the single usability metric important?

As mentioned, it standardizes different sets of data and combines them for an overall score. The individual ones are still useful but putting everything together can give you another angle. And since it’s in Excel or Google Sheets, it can be automated. We previously mentioned being mindful of overloading the people doing the testing; the beauty of this is that it’s more data without shoving more work in people’s faces. More data never hurt anybody.

Start gathering usability test metrics with this free usability scorecard

Usability scorecards are a great way to understand how users interact with your product and identify areas that need improvement. Available for free, Atlassian’s usability testing scorecard can be downloaded here. Use it as is or as a template to suit the needs of whichever tests and metrics you choose.

Using your usability test metrics results

Taking all of these usability metrics into consideration can genuinely make a world of difference to your product. Every bit of data should be combed over and used in some way. Of course, not everything will affect the masses, so maybe one person’s issue with this color or that isn’t exactly life or death. Still, you may need to change it up if you’re getting critiques about your fondness of a page covered in Baker-Miller Pink (think drunk tanks and college football opposing team locker rooms).

There’s a truly massive amount of data out there that’s actually really easy to get and can be relatively inexpensive. Even the more expensive options are well worth it, though, because real-world feedback on a product before it hits the market can be the difference between bestseller and cellar dweller. Would you rather have 20-50 people give feedback to help shape the direction of a product or have 20 million people hate it?

Our conclusion on measuring usability metrics

Usability testing, and the metrics that come from them, should be carried out by anyone who truly cares about the success of their products. The feedback may not always come wrapped in a bow but it’s much easier to hear it from a small few than read savage reviews in the App Store after you launched. Naturally, there are more options for metrics than written about in this particular article and you should consider them all and what they’re best used for. Whether it’s qualitative data or quantitative data, you can use it all until the wheels fall off. But skipping over it, well, not the ideal move if your goal is to create the best product possible. When asking ‘what is UX design’, your answer should include confirming that the design functions at 100% for all.

For further UX design information and for those who learn best through visuals, YouTube UX design creators have endless amounts of content — and best of all, it’s free!

FAQs