Usability testing is different

Usability testing is done for roughly the same reason as other kinds of testing: To discover new knowledge about the system under test. In this case knowledge about how users work with the system.

But Usability Testing is a very different discipline from ordinary system testing.

In my opinion, the one thing that makes usability testing different from system testing is, that it never discovers absolute facts about the system. 

Instead, a usability test will only say something about how the system works in relation to someone – and this someone is a person – or persons. And as you have probably experienced several times in your life, real persons aren’t absolute, predictable, or static – they’re quite dynamic and you never really know what to expect. Usability testing is a bit like integration testing against a remote system, which keeps changing, even while you test it!

Another aspect is that it’s much more important, yet also more difficult, to describe the context in which the system is to be used and the test is executed in.

I’ll illustrate that with a basic example: Many corporations implement systems for internal use which are not really user friendly, but since the employees get paid for using them, they don’t mind this – and once they’ve used the system for a few weeks, the system may have become an efficient work tool. An opposite example is most computer games, which are deliberately designed to be inefficient, but are usually easy to learn and use. These two systems or applications may be used on the same day by the same person, but in different situations, of course.

Setting the context is not always possible, though. For example is that most people will only respond to things which they understand: Chances are that if you had tested Facebook on users in 1995, they wouldn’t like it at all because they would not understand what it was to be useful for.

Essentially, I feel that the tester must be much more conscious about what he is doing and how it is affeting the test results. I actually believe that the description of the context itself and the way you describe it is as influential on the results you get, as the system and the users themselves.

Yes, testing usability can be very challenging!

7 thoughts on “Usability testing is different

    1. andersdinsen 2010-11-30 — 12:12

      Thanks a lot for the reference to your articles. That’s a interesting debate you’re opening there.
      Engineering is an applied art, there’s little doubt about that, and we’re always deeply buried in the fundamental problem that we’re inventing the problems we’re solving as we go along.

  1. Nicely put.

    Regarding this statement: “Usability Testing is a very different discipline from ordinary system testing”

    I wonder if a word like “skill” or “task” would be a better substitute for “discipline”. On one hand, other non-functional areas like Performance, Reliability, or Security could be treated the same way, so it’s really a testing skill…

    But on the other hand, I think I see software testing in general growing into specialized roles, where usability tester could be one.

    Anyway – nice post. Look forward to reading more.

    1. andersdinsen 2010-11-30 — 11:59

      Thanks for your comment 🙂

      I think usability testing has traditionally been done mostly outside of the testing community and that it’s a new trend to combine the two. It probably depends on the project which is the best approach.

      I see your point about using the words “skill” or “task”, I think I used “discipline” to highlight that I tend to see usability testing to be more of a CHI-engineering task, than a testing task.

  2. I agree fully regarding usability that “it never discovers absolute facts about the system.”
    But I think the same goes for other system testing as well.
    A system is reliable to someone, the functionality is adequate to someone etc.

    Usability testing is different because it is about usability, and also because it is more obviously subjective.

    1. andersdinsen 2010-12-03 — 17:15

      Very good point, Rikard.
      I think I meant “objective”, not “absolute”, thanks for clarifying that!

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this:
search previous next tag category expand menu location phone mail time cart zoom edit close