Here’s what I noticed users do:

Users don’t finish their sentences

I wanted to cut a few quotes out of the audio tapes, to put in my report. I listed a few of them in my notes, like:

“Well I would never look for the Print layout function there, I’d expect it in the right corner”.

Perfect quote, right? Except, when I look it up in the audio tapes, what’s actually said is something more like this:

“Hmmm yeah.. (pause).. I don’t… eh… this function, I don’t think… I don’t see.. Here might be bet- but I guess the date is in that spot already.. hmms”.

Leaving me very little to work with. My coworker in press relations says it’s very common for people not to finish their sentences. They give media training to prominent figures to teach them things like that: to speak in complete sentences, and to give remarks that are easily quotable.

Of course my test users didn’t receive any media training, and if they did, they might not be as useful at test users. But up until I listened to those tapes, I never noticed how many people don’t finish their sentences. Now I notice it more and more.

Users don’t answer the question

At some point during the interview I asked someone “Do you find you use one medium for a specific kind of news?” and the test user went on to explain to me that he usually watched the TV news at night and surfed the internet in the morning. A few times also, I asked a question, and test users kept on answering the previous question. I guess I should have given them more time for each question, but there’s only so much time to go around!

I have to keep in mind that some part of the message always falls away, even in a two-point telephone game. A few times I was quick enough to notice the user wasn’t answering the question, and posed the same question again in a slightly different way. A few other times I didn’t catch it at all, and only noticed it when it was far too late.

Users have a hard time expressing themselves

I had a few hard questions in my interview. Questions like:

1. “Do you use a particular kind of medium for a specific kind of news?”
2. “What sort of links do you usually click on in an e-mail newsletter?”
3. “Why is staying updated on current events so important to you?”

Exploratory questions whose answers are very interesting, giving subtle cues to their personality and to their needs. But they’re very hard for users to answer, requiring a lot of self-reflection. As I mentioned above, I sometimes noticed that two questions on, my users were still thinking about the first question they were asked! I

t might be good to make these questions easier and less abstract. It will require more work on my part, but the results might be better. For example:

1. What kind of news do you watch on TV? And on the internet? And on the radio?
2. (handing over an e-mail with 5 different kinds of links as identified in previous research) Which of these links appeal to you? Can you explain why?
3. How do you use the news information that you gain? What else do you do with it? Which of those things is most important to you? Why?

Users constantly apologise for their behavior

“Yes, I overlooked that menu. It’s probably because I’m not wearing my glasses. I’m going to get some next month”
“That navigation was confusing but that’s probably just me, I normally only use the search bar”
“I missed that link but I’ve never been to this website before, so that’s normal, right? I should really look at this website more”.

In four interviews I heard countless excuses as to why it was their fault if something went wrong, definitely not the website’s fault. I wonder why. Is it because I didn’t stress enough that 1) it wasn’t my website, and 2) we’re testing the website, and not them? I certainly tried to explain those things very thoroughly.

Or is it just human nature to think “I’m not a typical human, they don’t need to take me into account”. I wanted to explain that a website should also work for badly-sighted people, for search bar-navigators, and for people who visited a website for the first time.  But I didn’t, I didn’t have the time and it wasn’t what I was there for.

Users will point out problems you never even thought of

In my very first test, my user got to a lower importance news page and said “wow, this page is horrible”.
“What makes you say that?”, I asked.
“Well I like to quickly scan the  titles to see which news items are of interest me, and here the titles are the same size and type as the rest of the text, I can hardly make them out”.

He was right, of course, and I felt pretty stupid for missing it in the first place. It wouldn’t be the last time a user pointed out something I never considered. Or thought of a solution or a new feature that I hadn’t considered yet.

Users will bend over backwards to help you out

Sure, they get a nice 20 euro voucher to spend in everyone’s favorite electronics store. They may come to your office for the money, but once they’ve arrived, they’ll do whatever they can to help you out. They’ll dilligently search through a webpage for five minutes for a feature that’s been styled to look like a disclaimer (good job, design guy!), they’ll answer difficult questions they normally wouldn’t even think about, they’ll scour their mind for any information that they think might help you. They wait patiently while you sort your task scenarios, they speak their mind eventhough they have cameras and audio recorders pointed at them.

Maybe there’s very little you can say about “the typical test user”. But I can sure say that “the typical test user” is certainly a very nice guy.

Users will prove you wrong

The newssite I’m testing has little quotes from famous people on the bottom of each page. “How stupid!”, I thought. “Totally unprofessional and not fitting to our image. I’ll put it in the test and we’ll get this thing sorted quickly”.

Except every test user I’ve had so far thinks they’re fun. Has no problem with them. Don’t think they detract from our image. One of them went so far as to say: “They’re fun, I don’t see how anyone could be bothered by them! You’d have to be a pretty sad person to find something like that objectionable”.

Sometimes what you expect to happen is nowhere near what will actually happen. That’s quite a grateful moment, because it’s when you realise you’re doing all this work for a very good reason.

• Not asking open questions
  I wrote down a lot of open questions in my interview plan. One could be : “What do you think of this website?” . But on the audiotapes I catch myself saying: “What do you think of this website? Do you think it’s interesting or boring or something else?” With every open question I’ve added some possible answers, thereby steering the entire response into one direction. The user will now answer the question with a scale of boring-interesting in mind, potentially disregarding his own spontaneous thoughts because they don’t fit in this spectrum.
  What I should do: Just ask the open question the way it’s on your page and let the user formulate his own answer. If he doesn’t understand the question, then you can expand on it a bit.
• Not leaving room to think
  A few times in the audiotapes the test user finishes his answer and there’s a short pause. Then both I and the test user start talking again. Often then the user stops and I keep talking. Cringeworthy.
  What I should do: I should really count to three before I start talking again.
• Not staying on topic
  One remark a user made reminded me of a similar short story, that I then related. Why? I don’t know. It just came out of my mouth. It didn’t break the flow of the interview but when i listen to the tapes again it feels out of place, or at least a waste of valuable interview time.
  What I should do: stay focused on your task.
• Not being detached
  My first test user was a very attractive young man. The audio replay shows a lot more expressiveness in this interview than in the other ones. I am ashamed.
  What I should do: No really, just stay focused on your task

Now, with the irrefutable audio- and videotaped evidence before me, I see I still have very much to learn. And I’m eager to start. But we’ll take it one day at a time, and this day I still have to finish writing up the interview and test of testuser3.

I’m not one to follow achievement experts. I do have a copy of the 7 habits in storage somewhere, but it was a gift and I kept it for the inscription. The advice seems solid, though. An hour a day might not turn you into a powerhouse of knowledge, experience and wits. But you’ll undoubtably become more knowledgeable and better at what you do. Here’s me, testing that theory. Sorting my cards and thinking aloud.

I may not have much to say. I’m not an expert. But if I don’t start the journey, I won’t be reaching any fun destinations either.