Think aloud

Just another weblog

Interview mistakes

with 2 comments

mistakesLast friday, my first four test users came in one by one, and I sat them down for an interview and a usability test after that. I was quite content with my own performance, but now that I’m looking through the audio- and videotapes, I see mistake after mistake. Don’t get me wrong, I still got a lot of information out of these interviews. A little too much, when I see how much I still have to process, and how lovely the weather outside is. But I could have gotten more if hadn’t been making these errors. I’ll note them here so they will seep into my conciousness and I’ll have a chance to work on them:

  • Not asking open questions
    I wrote down a lot of open questions in my interview plan. One could be : “What do you think of this website?” . But on the audiotapes I catch myself saying: “What do you think of this website? Do you think it’s interesting or boring or something else?” With every open question I’ve added some possible answers, thereby steering the entire response into one direction. The user will now answer the question with a scale of boring-interesting in mind, potentially disregarding his own spontaneous thoughts because they don’t fit in this spectrum.
    What I should do: Just ask the open question the way it’s on your page and let the user formulate his own answer. If he doesn’t understand the question, then you can expand on it a bit.
  • Not leaving room to think
    A few times in the audiotapes the test user finishes his answer and there’s a short pause. Then both I and the test user start talking again. Often then the user stops and I keep talking. Cringeworthy.
    What I should do: I should really count to three before I start talking again.
  • Not staying on topic
    One remark a user made reminded me of a similar short story, that I then related. Why? I don’t know. It just came out of my mouth. It didn’t break the flow of the interview but when i listen to the tapes again it feels out of place, or at least a waste of valuable interview time.
    What I should do: stay focused on your task.
  • Not being detached
    My first test user was a very attractive young man. The audio replay shows a lot more expressiveness in this interview than in the other ones. I am ashamed.
    What I should do: No really, just stay focused on your task

Written by Jules

May 31, 2009 at 2:24 pm

Posted in Uncategorized

2 Responses

Subscribe to comments with RSS.

  1. Don’t worry, even seasoned testers make mistakes! The key is to keep doing it, keep reviewing tapes (painful, I know), and work on extinguishing the bad behaviors while increasing the frequency of good behaviors.

    A tip that I’ve learned is to mentally treat the test session as “not a social interaction”, while still expressing empathy and maintaining a warm but professional demeaner. It takes some practice.

    One other thing: there are times when you’ll just be cringing in sympathy if a user is struggling particularly hard. It’s going to be difficult to get used to it, and you’ll want to help them out. But you will get used to it, and you’ll come to appreciate the good data you get from these situations.

    Paul Sherman

    June 3, 2009 at 4:26 pm

    • Thank you for your encouragement! Yes, stepping outside of the social interaction behavior is very hard. It’s certainly a learning experience. Fortunately I can also see a bit of progress from the first interview to the fourth.

      I followed you back to, it looks very interesting. I’ll have a closer look tomorrow. There’s only so much you can learn from reading but for now I can use all the information I can get my hands on.


      June 3, 2009 at 7:29 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: