Think aloud

Just another weblog


leave a comment »

Reviewing the audio tapes gave me opportunity to observe my own behavior during a test.  I also noticed a few things about my test user’s behavior, more specifically their behavior and thought patterns. Peculiarities that I should be aware of, the next time I do interviews.  I can’t let those things get in the way of the information I need to get.

Here’s what I noticed users do:

Users don’t finish their sentences

I wanted to cut a few quotes out of the audio tapes, to put in my report. I listed a few of them in my notes, like:

“Well I would never look for the Print layout function there, I’d expect it in the right corner”.

Perfect quote, right? Except, when I look it up in the audio tapes, what’s actually said is something more like this:

“Hmmm yeah.. (pause).. I don’t… eh… this function, I don’t think… I don’t see.. Here might be bet- but I guess the date is in that spot already.. hmms”.

Leaving me very little to work with. My coworker in press relations says it’s very common for people not to finish their sentences. They give media training to prominent figures to teach them things like that: to speak in complete sentences, and to give remarks that are easily quotable.

Of course my test users didn’t receive any media training, and if they did, they might not be as useful at test users. But up until I listened to those tapes, I never noticed how many people don’t finish their sentences. Now I notice it more and more.

Users don’t answer the question

At some point during the interview I asked someone “Do you find you use one medium for a specific kind of news?” and the test user went on to explain to me that he usually watched the TV news at night and surfed the internet in the morning. A few times also, I asked a question, and test users kept on answering the previous question. I guess I should have given them more time for each question, but there’s only so much time to go around!

I have to keep in mind that some part of the message always falls away, even in a two-point telephone game. A few times I was quick enough to notice the user wasn’t answering the question, and posed the same question again in a slightly different way. A few other times I didn’t catch it at all, and only noticed it when it was far too late.

Users have a hard time expressing themselves

I had a few hard questions in my interview. Questions like:

  1. “Do you use a particular kind of medium for a specific kind of news?”
  2. “What sort of links do you usually click on in an e-mail newsletter?”
  3. “Why is staying updated on current events so important to you?”

Exploratory questions whose answers are very interesting, giving subtle cues to their personality and to their needs. But they’re very hard for users to answer, requiring a lot of self-reflection. As I mentioned above, I sometimes noticed that two questions on, my users were still thinking about the first question they were asked! I

t might be good to make these questions easier and less abstract. It will require more work on my part, but the results might be better. For example:

  1. What kind of news do you watch on TV? And on the internet? And on the radio?
  2. (handing over an e-mail with 5 different kinds of links as identified in previous research) Which of these links appeal to you? Can you explain why?
  3. How do you use the news information that you gain? What else do you do with it? Which of those things is most important to you? Why?

Users constantly apologise for their behavior

“Yes, I overlooked that menu. It’s probably because I’m not wearing my glasses. I’m going to get some next month”
“That navigation was confusing but that’s probably just me, I normally only use the search bar”
“I missed that link but I’ve never been to this website before, so that’s normal, right? I should really look at this website more”.

In four interviews I heard countless excuses as to why it was their fault if something went wrong, definitely not the website’s fault. I wonder why. Is it because I didn’t stress enough that 1) it wasn’t my website, and 2) we’re testing the website, and not them? I certainly tried to explain those things very thoroughly.

Or is it just human nature to think “I’m not a typical human, they don’t need to take me into account”. I wanted to explain that a website should also work for badly-sighted people, for search bar-navigators, and for people who visited a website for the first time.  But I didn’t, I didn’t have the time and it wasn’t what I was there for.

Users will point out problems you never even thought of

In my very first test, my user got to a lower importance news page and said “wow, this page is horrible”.
“What makes you say that?”, I asked.
“Well I like to quickly scan the  titles to see which news items are of interest me, and here the titles are the same size and type as the rest of the text, I can hardly make them out”.

He was right, of course, and I felt pretty stupid for missing it in the first place. It wouldn’t be the last time a user pointed out something I never considered. Or thought of a solution or a new feature that I hadn’t considered yet.

Users will bend over backwards to help you out

Sure, they get a nice 20 euro voucher to spend in everyone’s favorite electronics store. They may come to your office for the money, but once they’ve arrived, they’ll do whatever they can to help you out. They’ll dilligently search through a webpage for five minutes for a feature that’s been styled to look like a disclaimer (good job, design guy!), they’ll answer difficult questions they normally wouldn’t even think about, they’ll scour their mind for any information that they think might help you. They wait patiently while you sort your task scenarios, they speak their mind eventhough they have cameras and audio recorders pointed at them.

Maybe there’s very little you can say about “the typical test user”. But I can sure say that “the typical test user” is certainly a very nice guy.

Users will prove you wrong

The newssite I’m testing has little quotes from famous people on the bottom of each page. “How stupid!”, I thought. “Totally unprofessional and not fitting to our image. I’ll put it in the test and we’ll get this thing sorted quickly”.

Except every test user I’ve had so far thinks they’re fun. Has no problem with them. Don’t think they detract from our image. One of them went so far as to say: “They’re fun, I don’t see how anyone could be bothered by them! You’d have to be a pretty sad person to find something like that objectionable”.

Sometimes what you expect to happen is nowhere near what will actually happen. That’s quite a grateful moment, because it’s when you realise you’re doing all this work for a very good reason.


Written by Jules

June 5, 2009 at 8:47 am

Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: