The Power of In-Person Interviews

It’s 8:00am on a Summer Tuesday morning and we are getting ready to conduct a day of user research with a client looking to get some feedback on their website.

The lab is ready, our test plan is ready, and we have 8 people recruited to come in to our offices, use the site, and give us their feedback. All of the respondents are listed on the schedule by first name and last initial. They all made it through the recruiting screener criteria, and they’ve all agreed to come to our office and receive a small incentive to help us understand how to make our client’s site better. Before anyone arrives, I wonder what they’ll be like in person, what insight they will bring and how many things we will learn that we did not expect.

Typically we have some background about our participants. In this case we know their income range, their profession, their age, and gender. We also know that they answered all the questions in our screening process and fit the demographics our client was looking for.

Who is Sue S.

Our first respondent is Sue S. She is a 42-year-old administrative assistant with a household income of  $45,000-60,000. Up until the point she walks into the lab she is a faceless user of our client’s site– her only trace being a few clicks on their analytics report. The moment she arrives, Sue becomes a representative for the thousands of users like her that visit this site to find information that could impact the health and well-being of their families. Clearly, we have a lot to learn from her.

Upon arrival, Sue signs the video release forms, fills out a questionnaire and accepts a cup of coffee. She is busily working on her iPad when I introduce myself.

“Hi, you must be Sue. Thanks for coming in today.” 

She was worried about being late for the session. Her daughter forgot her lunch at home and she needed to make a special trip to school. Sue doesn’t like driving in traffic and rarely comes to this part of town. She made a test run the previous night to see how long it would take to get to our office.

We begin the study. I notice when she opens one particular page she grimaces.

“You just made a face. Tell me about what you were thinking. Is this what you expected to see?”

“Well, yes and no. The pictures are so contrived, its like they are trying way too hard to show every possible type of person and be politically correct. It just seems so fake to me. It makes me not trust them and I just don’t like that.”

“What kinds of images would you like to see instead?” I inquire

“I’d get rid of them all together. I am trying to figure out how to login. I don’t want to see all of this marketing fluff.” 

We continue working through the tasks.

“Oh, I like this a lot.” 

She says as she moves her cursor over a table that shows key information for several items in the middle of the screen.

“It would save me a lot of time if more sites did this. I have some vision loss in my right eye and it helps that I can see these items compared this way.” 

I am not a use case. I am a human.

Sue is not a use case. She is not a scenario, a persona or part of a journey map. She is a real person who needs to get something important done using a complicated web application.

In person, one-on-one interviews help us capture intricacies and behavior patterns that we may not pick up over the phone or with an online testing methodology. It allows us to collaborate with our client’s end users, connect with them as humans and gain deep understanding about what is important to them.

In general, people tend to share a lot more information when someone is asking the questions in person. It becomes a conversation instead of a task list. We notice things that we wouldn’t if we were conducting an online study or working remotely- and we can follow up on those things in real time. It’s also much easier to ask a follow-up question and get examples to support what people are talking about. As a facilitator, I can ask them to show me what might make the experience better or tell me about another website that may be doing a better job. It is also common for people to share their views on the company, vent about something that happened when they called customer service, or why they switched to a competitor—it’s market research and usability all rolled into one.

The Benefits of In Person One-On-One Interviews

We find that one-on-one time with a skilled facilitator gives people an opportunity to participate in a direct way, and they have a greater buy-in to the results. There are pitfalls to watch out for. In some cases, participants feel that they are being tested instead of the other way around. It is important to help them to feel relaxed during the session. When we are screening participants, we ask questions about how comfortable they will be sharing their opinions openly. We remind the respondent to speak out loud during the session so we can understand what thoughts are going through their head.

In Minnesota, we have a reputation for being nice. “Minnesota nice” can hinder a respondent’s ability to provide honest feedback during a site evaluation. Negative feedback and ideas for improvement are exactly what we are looking for, so we strongly encourage participants to be direct.

One of the biggest advantages of in person interviews is that we can pick up on nonverbal cues from the respondent. They often demonstrate any discomfort, stress or problems during the session through grimaces, nervous fidgeting and other body language. I often will say to the respondent. “I am sensing that this is making you uncomfortable. Can you tell me a little more about that?” Sometimes it’s these moments that provide the most insight into an experience and often the questions we ask are spontaneous and not part of the approved test plan. It is a primary reason why we request some time in advance with the site or application (especially if we have not designed it) to better understand its intended business goals and user objectives. Having this knowledge helps us improvise during a session and dig into topics that may not have considered when we drafted the test plan.

Free Exploration

While having a set of tasks is a key component of research, we also like to see what someone does with a site or application on their own. Typically we will give people a few minutes at the beginning of the session to get oriented and explore the site or application. We look for trends in where they tend to click first and what comments they initially make about the overall design and visual approach. That first impression says a lot about their expectations for the overall experience.

Participants presented with a complicated home pages tend to demonstrate body language that indicates stress and anxiety.

Marketing experts will tell you to reduce the number of clicks—this was a “best practice” in our industry for years. It’s what led to those complicated and overcrowded home pages that cause so much anxiety. We’ve observed time and time again that the number of clicks does not correlate with user satisfaction or confidence. Rather, if users feel confident in the path they are taking, they would prefer multiple clicks vs. scanning an overwhelming set of options.

Equally as fascinating is watching when they forget there is someone watching. We generally end our session with a follow up questionnaire. It gives us the ability to come into the observation room and see if our clients have any specific questions for the participant. I like to say…

“I am going to have you fill out this short questionnaire and see if any one in the back has any questions for you. If you finish before I get back, feel free to explore the site a bit on your own.” 

Observing them explore the site unprompted give us insight into the resources and functionality that they are interested in that are outside of the scope of our test plan.

User research has always been at the core of what we do at Crux Collaborative. We build a research step into each key phase of the project. We are constantly advocating for checking in with target users to better understand how they perceive and use a site or application. Any user research is usually better than making assumptions based on no research. We have found that it is safest to never assume that we know what the users will think, feel and do. The most effective way to learn about end users (the humans that visit your site) is to collaborate with them and talk to them in person.

If you are looking to gain deep insight into the behavior of your target users or if you are trying to understand how to get more engagement with a specific process or functionality on your site, we would love to chat.