Keith Richards wasn't using a website when he famously declared his inability to "get no satisfaction" - in fact he was half asleep - but if he had been online, he would have made a great case for usability testing. Satisfying our users is what it's all about, isn't it? In this edition of Newsability, we deconstruct the concept of satisfaction (did you know there are two fundamental types?) and tell you how to measure it from one angle.
Jump to: Article summary | Usability tip
Think about the things that satisfy you the most - a good meal is probably somewhere on the list, maybe finishing a challenging job... aaand we'll stop there. But I'm willing to guess that the last time you used a good website, you didn't push back from the desk, put your feet up, cross your arms behind your head and reflect on your satisfying experience. Truthfully, satisfaction isn't something we ordinarily associate with day-to-day tasks - so why do we wait with bated breath to see the "satisfaction" ratings testing participants and survey respondents give to our sites? Let's discuss what satisfaction actually means, and then look at some tools to help build one type of customer satisfaction survey.
Defining satisfaction
It's important to note that "satisfaction" in a testing context can refer to one of two things:
- The fulfillment or gratification of a desire, need, or appetite.
- Pleasure or contentment derived from such gratification.
http://www.thefreedictionary.com/satisfaction
Usually what we're measuring in usability testing is definition-a: fulfilling a need or desire. There's a strong temptation to associate user website satisfaction with definition-b: fulfilment plus pleasure or contentment. We've usually put a lot of effort into the sites we build and we want people to be happy using them!
The problem with this is that it's really hard, and essentially irrelevant, to measure definition-b satisfaction (pleasure) when people are using a website to find a piece of practical information such as finding a bus route or viewing a utility bill. While we don't want our users to leave feeling negatively - we don't want an unusable, irritating experience - it's not realistic to expect that people will feel any notable sense of pleasure in accomplishing tasks like these. Other sites, such as e-commerce sites, enable definition-b satisfaction through streamlined, relevant access to products or services, but don't really provide definition-b satisfaction as standalone interfaces.
Two levels of satisfying people
A recent survey of online companies found that Facebook, the second most-visited website in Australia and the world (500 million global users and growing), has an "abysmal" user satisfaction rating. People are irate about Facebook privacy policies and frequent changes to the site. Yet people come in droves to see what their friends have to say. Does Facebook just have a captive audience? Maybe they were just good enough to wrestle the social network crown from Myspace, and now they're big enough to get away with irking users. Could a well-designed social networking site topple Facebook? Doubtful - as where else can you find such a large portion of your friends? I'd argue that Facebook does fine with definition-a (fulfilling a need), but does a poor job with definition-b (pleasure). But because there are such huge barriers to doing a better job than Facebook with definition-a satisfaction, they get away with it.
When we look at Apple's customer satisfaction ratings, we see a more intuitive picture. People are really definition-b when it comes to Apple's PC products. This happiness with Apple in general might be why their stock value has gone through the roof in the past 5 years.
Measuring satisfaction
This illustrates an interesting truth about measuring user responses to our sites. Highly competitive, commercial products or sites are going to have to shoot for delivering pleasure, while informative, utilitarian sites should stay realistic and strive for streamlined need fulfilment. Facebook gets away with definition-a satisfaction because they fulfil a need (access to lots of friends in an instant) that is really hard for others to provide.
Most websites don't really need to shoot for definition-b. Usability testers have known this for some time; there are several classic tools with cool acronym names like SUS, WAMMI, SUMI, CSUQ and QUIS to use as starting points when measuring definition-a satisfaction. They all revolve around a set of questions that can be combined to construct an index score. While some of them ask a few definition-b leaning questions about enjoyment or aesthetics, these tools mainly focus on efficiency, understandability, and perceptions of difficulty using the site.
By asking research participants a set of questions about their experiences with an interface, we can see how well these people perceive that the tested interface provides definition-a satisfaction - need fulfilment.
SUS - System Usability Scale
The SUS (PDF) is a simple survey of Likert scales. The creator, John Brooke, intended the SUS to be a quick and dirty benchmark of software usability.
WAMMI - Website analysis and measurement inventory
WAMMI is a paid user survey service that gives your site a score based against a database of hundreds of other sites that have used the same survey. The WAMMI questions are onlineand are a useful starting point for building your own survey questions. If you build your own, you won't have access to the database, but you can use your results to benchmark against iterations or site improvements.
SUMI - Software usability measurement inventory
SUMI differs from the previous surveys in that it's 50 questions long. Instead of rating items on a 5-10 point Likert scale, users simply agree or disagree with a statement. The benefit to this approach is that respondents don't have to think about quantifying the degree of their opinion; SUMI makes up for the lack of detail in individual answers by asking lots of questions. The SUMI questionnaire is also a very useful starting point for writing your own custom survey. Like WAMMI, SUMI is a paid-for service.
These three survey approaches have been around for a long time. They are highly practical questionnaires originally designed to deliver usability ratings for software systems. Because they come from a time when software was classically horrible to use, the bones of these surveys are well-suited to get at definition-a satisfaction. There are some other old-school usability indexes such as QUIS, or CSUQ that can be examined as well.
Conclusion
At the end of the day, a large portion of web sites should really be targeting definition-a satisfaction - need fulfilment. A realistic understanding of the level and type of engagement that users will have with your site should let you know what type of satisfaction you can reasonably hope to provide to your users.
Getting the definition of satisfaction wrong can be costly. If you're a definition-a site and you're trying really hard to give people the glow of pleasurable, definition-b satisfaction, you're going to spend a long time spinning your wheels and might even try too hard, making your site visible when it should be invisible to the user. And on the flipside, if you need to be providing definition-b satisfaction but you simply meet a need without the correspondent afterglow, someone else is going to steal your lunch.
Measuring definition-b satisfaction is another story entirely, but we'll save that for another article. 😉
Summary
If you really just didn't want to read the whole thing, here's what we said:
- There are two types of "satisfaction":
- Definition A: having a need met (usability);
- Definition B: feeling pleased, happy, or sated.
- Not all sites need to strive to give pleasure to users in order to satisfy them.
- Sites that provide mundane information or enable low-engagement tasks will do better by meeting needs quietly.
- There are several classic survey tools that act as a starting point for building surveys and questionnaires to measure Definition A satisfaction.
- Sites that provide services, have strong competitors, or work with high-engagement information need to meet Definition B satisfaction.
- We hope to talk about measuring Definition B satisfaction in a future article!
Usability tip
When asking for information from users, tell them why
When testing e-commerce sites, application forms, or basically any other site that requires users to create an account and provide personal information, we almost always hear the question from participants: "Why do they want me to tell them that?!" While we won't answer during a usability test (we ask them why the think they need to provide the information) the problem is apparent and surfaces frequently. If the user doesn't understand why they are asked to give you information, then you should probably explain yourself because they will have lost trust in you.
Think about it - if you went up to a cashier at a department store and they asked your religious affiliation, or how many kids you have, or your home town, you'd probably be a little confused if not irritated. The same holds true for interactions with websites. Although users are not interacting with a person directly, they are highly aware that they are providing information to other people. It's just common courtesy to explain your motives. You'll also engender trust and increase your credibility.