Website and Mobile Application Usability Testing: The Secret of the Number 5
January 10th, 2017
The role of product testing from the point of user experience (also known as UX) is on the rise. More and more attention is paid to how easy the application or service is for the people to use. This question became as important (if not more so) than the way the app looks design-wise.
At the same time, there is still a lot of confusion in regards to the UX testing.
It’s one thing to internally deep-test the product, but only “average user” testing would show how the app would behave in real-world situations.
Some companies move toward simplifying the UX processes to the maximum. Some companies ignore the UX testing altogether, thinking it would require a significant investment of resources and time.
How many resources does it really require?
The best and most efficient, in terms of results, would be a series of small tests conducted at the same time by five users.
But before we dive deep into the meaning of 5, let’s see why usability testing is important.
User experience is a personal, private experience, which the user receives when s/he interacts with each aspect of the product or service.
The concept itself has been described by the scientist Don Norman in the early 1990s when he served as a vice president of the Apple.
"User experience" encompasses all aspects of the end user's interaction with the company, its services, and its products. // Don Norman
Norman later expanded this concept, highlighting the combination of components required to achieve a high-quality user experience. These components include engineering, marketing, graphic and industrial design, as well as interface design.
Contrary to the popular belief, the UX is not limited to the visual interface of your product. (That’s UI, and we discuss the difference here.) Its quality depends on how user-friendly is your product for an average user to understand it and master it. It even depends on the feelings your product causes while the person is interacting with your website or app.
UX designers are responsible for ensuring that the company provides a product or service that meets customers’ needs and enables them to easily achieve the desired result.
Usability testing is convenient to do using interactive prototypes.
If you use prototyping approach,
-
You save a lot of time and money because your developers aren’t heavily involved in the process (only the UI/UX designer). According to UsabilityGeek, it can be up to 100 cheaper to change a prototype than a coded feature.
-
You get to identify the errors before they are a part of your actual product
-
You get to make changes in the UX without drastic changes to the product’s code (and dependencies, etc.)
There are many online tools such as InVision, for example, that you can use for creating your prototype.
Usability testing is a top priority to ensure that the product will be well-accepted by the users and not ignored. Despite popular misconceptions about the scale of this process, in fact, like we have mentioned above, you need a test group of five people. The results will be sufficient to ensure that the product offers users a quality experience.
So, what’s up with number 5?
Number 5 (in terms of quantity of test users) can be explained mathematically.
Taking into account that the average likelihood of errors arising during product testing is 31% (data from MeasuringU), using five testers will identify 85% of the issues with the interface.
Three testers will help you get the information about almost 65% of the problems. With four users, the number of issues identified will reach 75%.
Once you cross the threshold of 5 users and start adding more people, the percentage of revealed problems will stop rising so quickly. Six users will give you 90% of issues, 8 - 95%, and 12 - 99%.
This decrease is explained by the fact that the user experience of each tester overlaps others’ experience. In fact, each new tester will detect the same mistakes that you found out about from other people, while the number of newly discovered problems decline with each added user.
People, of course, differ from each other. Their user experience is also different and they will be able to identify problems that were not noticed by their predecessors. But to avoid ineffectiveness, a team of five members is the best solution for testing UX.
Another thing to keep in mind: do not make an exclusive emphasis on the quantitative indicators of users’ feedback. It is more critical to pay more attention to the qualitative issues - revealed deficiencies, which make the product uncomfortable for users and battling negative aspects of the app or website.
The feedback you’re looking for should be giving you insight on how the app will behave in the hands of an average user (and how an average user will feel using it).
For example, your test users found some inconsistency in the color scheme on the home page in the registration field.
Instead of only asking for “impersonal” quantitative assessment of the color scheme quality (for example, on a scale of 1 to 10), it is much better to ask the users to explain what the problem is.
It may happen that the blue color used in the navigation bar on the homepage is too similar to the blue “Continue” button on the registration screen.
You can see why finding a solution in the latter case would be much easier and efficient than in the first case, where you just know that there’s something wrong with the color scheme.
What else do you need besides the testers?
To get the most of out of your five testers requires a specific set of skills. Your demeanor and respect to the users are key factors here.
None of the test users will give honest answers in a situation that’s difficult for them. Therefore,
- Do your best to eliminate pressure
- Try to avoid asking leading questions
- Abstain from criticism
It is recommended to do the survey in a form of an informal conversation, clearly explaining the goals of the meeting from the very beginning. Your sincere interest is the user’s main motivation to share his/her personal experiences from interacting with your product.e
It is also recommended to have the UX designers and/or developers as observers of the study (or have it available for them in a form of video) because they would better understand why the feedback was given and what emotions it caused.
Notes:
It should be noted that error probability of 31% is an average in the industry, but the number greatly depends on the kind of product you have as well as the maturity of the product. If you have been working on it for years, then the likelihood of errors might be down to 5%. Or, it can be up to 50%.
However, the number 5 in regards to a number of testers is still considered the “gold standard.” Having five user-testers will give you the best ratio between the issues found and cost and duration of the testing process.
Need UI/UX design and development for your project?
Instead of P.S.
Whatever you do, however many users and testers you ask, just make sure you avoid UX fails like below :)
