Usability expert Jakob Neilson said that most of the lessons you get from usability testing can be obtained from only 5 users.
Some people think that usability is very costly and complex and that user tests should be reserved for the rare web design project with a huge budget and a lavish time schedule. Not true. Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.
Long term plan: multiple series of tests
He recommends doing iterative testing, using 3 studies:
The curve clearly shows that you need to test with at least 15 users to discover all the usability problems in the design. So why do I recommend testing with a much smaller number of users?
The main reason is that it is better to distribute your budget for user testing across many small tests instead of blowing everything on a single, elaborate study. Let us say that you do have the funding to recruit 15 representative customers and have them test your design. Great. Spend this budget on 3 studies with 5 users each!
I’ve just finished my first series of usability tests with 7 users:
- Arwen: 40s
- Susan: 60s
- Andrew: 60s
- Ken: 70s
- Marilyn: 70s
- Harold: 70s
- John: 80s
This first round of testing I focused on retirees because:
- They are more likely to have an extra one+ hour to dedicate towards the testing process
- They are more likely to discover the types of problems that affect many older readers, which is a big demographic for newspapers and other media.
- If a desktop design works for older users, it likely works for younger users who use desktop.
Next Phase:
Two of my first round users tested mobile and two tested phone. The second phase of testing should better test mobile devices.
How I Tested:
1) Reading:
I directed users to a testing page and gave all users 2 articles to read without instructing them on how they should read the articles.
There instructions were to read them as they usually would and narrative what they are thinking as I watched over their shoulder.
Lesson (Reading):
I found that some users are wary of viewing videos. One problem I observed is that when a user clicks on a video, they may not be able to tell whether the video is about to play or whether they need to click again.
Sometimes the first click only selects the video and a second click is required to play the video. But sometimes the player is buffering and they click a third time, which causes the player to pause, rather than play.
I plan to solve this problem by creating a dedicated “play” button. The “play” button will change to a “pause” button when the button is pressed or when the video is pressed. This requires the OnChange event to recognized the players state and toggle the state..
2) Writing:
I also had most of them test the process of creating new citations using the WordPress demo area.
Lesson (Writing)
My oldest user (in their 80s) was able to do the reading, but not the writing, because it requires the ability to copy text from one tab and navigate and paste into the WordPress tab.
This is no different than the requirement for making a link in WordPress, but needs to be named as a limitation.