Mahtab: Hello and welcome to episode 16 of The Crux of It. I am Mahtab Rezai. I am the principal and CEO of Crux Collaborative. We are a user experience consulting firm specializing in regulated industries. For this episode I am joined by my colleague Rebecca: Grazzini. Hi, Rebecca.

Rebecca: Hi Mahtab.

Mahtab: Today we are going to be talking about usability for mobile apps. Let's just give a 101.

Rebecca: Absolutely. So, an app is going to be a separate piece of software that you download from an app store, versus going into a browser on your phone and looking at a web based experience.

Mahtab: Yeah. So sort of the difference between if you were to open Safari and go into the web based version of Twitter versus downloading a Twitter app.

Rebecca: Correct.

Mahtab: So, there's certain things about apps when you’re going to be preparing to conduct research about them. The first is that there's actually different versions of them because there's different operating systems of phones.

Rebecca: Correct.

Mahtab: So we typically stick with just testing two version, an iOS version and an Android version.

Rebecca: Mm-hmm (affirmative).

Mahtab: But we see a whole variety of phones, a whole variety of reactions.

Rebecca: Yes.

Mahtab: To the app. But that's a key thing to note is that it actually is when we're talking about usability testing for apps, especially if something isn't a prototype, that it can get tricky.

Rebecca: Yes.

Mahtab: So let's talk about prototypes and sort of faking up an app experience in a mobile device.

Rebecca: Yes. So we frequently have people who don't have an app built, not launched in an app store. So, they create a prototype of it and there are tools like InVision out there that will allow you to create a prototype and launch it on a phone. What we typically do is have a device in each of the two operating systems, an iPhone and an Android device, that we download those prototypes onto, or launch them, so that people can use them on our devices.

Mahtab: So we start early in the recruiting process in making sure that we are asking the recruiting company to get us a variety of users on different types of devices.

Rebecca: Yes.

Mahtab: But we also then validate or verify that when they arrive in their pre-questionnaire so that we can hand them ... we can start by handing them the type of device that they prefer and are used to using.

Rebecca: Exactly.

Mahtab: But also there's different design patterns. In an iOS experience, everything from menus, to select buttons, to radio buttons, to all sorts of things behave slightly differently than in an Android experience. So being familiar with how different operating systems basic UI elements operate. Where on the screen they live. At the top, the bottom. Where things come from, how the keyboard interacts. Those types of things are critical for building a prototype – because one mistake we've seen is people will build a prototype that emulates a web-based experience or maybe emulates only iOS.

Rebecca: Right.

Mahtab: Then while you can put that prototype on an Android device or you can put that prototype on an iOS device, it still is not going to give you really accurate data for how that's going to behave in the real world.

Rebecca: Exactly. Yeah, we frequently see people designing apps for iOS initially, and that's a common way to develop apps because it's easier to launch them in one app store versus multiple. Then you try to test on an Android device, which has a native back button, and inherently the menus and the navigation will work differently for someone using an Android based device than an iOS.

Mahtab: Yes. When you have superimposed something that looks and behaves in a way it won't look and behave in the real world, you're not getting really valid findings.

Rebecca: Correct.

Mahtab: So being aware of native design patterns and how those devices operate is one key thing when prepping for prototype testing.

Another way of doing it that we actually prefer to do is to have participants bring their own devices.

Rebecca: Yes.

Mahtab: To the session. So again this starts in recruiting where we make sure that the participant has a mobile phone, that they use apps, that they're comfortable bringing their own device, but then we, over the years, have learned a number of valuable lessons about good and not so good ways of conducting these studies. So first let's talk about preparing the participants for what is going to happen.

Rebecca: Yes. So making sure that they're aware that we're going to ask them to use their own device. If we need them to have a particular app that they may not have, to have them download it in advance so they don't have any issues with connectivity.

Mahtab: Or permissions from the app store. If they have family sharing or they forgot the password. Whatever it is.

Rebecca: Whatever we can do before we get them to walk into our office to make sure that they have their phone, they have the app that we're going to test, and they're set up and ready to go.

Mahtab: Then over the years we've learning it's also important to say things like, "Please charge your phone."

Rebecca: Yes.

Mahtab: Please be aware that we are recording your phone so if there's anything you would like us to not see, please either have that closed, down, or off of your home screen.

Rebecca: Yes.

Mahtab: So those are some things. But there's also other things we've learned that have been surprises to us along the way. One thing is how much people customize their devices and the look and feel of their devices. Everything from the size of the font and the icons, to the font itself – where someone can select to override whatever font the designers have picked.

Rebecca: It's crazy to see your app in script font.

Mahtab: Yeah. Because somebody just likes it that way. Or just whatever it is that they've done is that there are many, many ways for people to customize their own device and people do customize their devices. So be prepared to see your app in a way that you had not anticipated seeing the app.

Then let's talk about access to data and WiFi versus using data.

Rebecca: Yes. So, it's not uncommon for people to have their data throttled. We all have cell phone plans and some of them don't give us as much data as we might use. So when someone comes in and their data is being throttled, they'll do something on a screen and it won't respond particularly quickly. People expect things to happen quickly. So, it's always interesting to see in the lab when people see nothing happens and then they just leave. They go to another screen. So, make sure that you're prepared with micro-interactions for those moments when something is not loading quickly, and also have WiFi available. If you can get people on WiFi, you can see the ideal behavior of the app in the speed that you'd want it.

Mahtab: Yeah. What you said about performance. I mean I think that's what you really see show up in app usability study, which is that something when you're on blazing fast, corporate WiFi, just loads and reacts and behaves in a certain way, and when someone's on a throttled data plan, behaves very differently. As you said, people and users don't know what it's supposed to do. So when they take an action and nothing happens, they think that the action either happened or that it's not going to. But they have very little patience for sitting there for even more than three to five seconds to wait and see what happens. They're on to the next thing. So that's also a really great learning for project teams to watch as they’re waiting for the piece of feedback to come or the next piece of data to come and someone's already gone.

Rebecca: It gives them a great opportunity to see those moments that you don't think of in the design process where you can add some context, some understanding that a thing is happening, and please wait, we'll be with you in a moment.

Mahtab: Yes. Absolutely.

Then the other thing I think when you're having people use their own devices, particularly with live apps that are available, is the invaluable insights from seeing users interact with their own data, with real data.

Because I think there's - with prototypes, there's always this hurdle where we're having this conversation of - we've tried to pick some gender neutral name. “Let's imagine your name is Pat Smith. You're married. We've picked a gender neutral spouse, someone named Alex”. But it's really hard if what they're looking at isn't their own information for them to place themselves and to keep themselves in context as they're going through an experience.

Whereas, when it is their data, their spouse, their info, their home address, they're really better able to give you authentic feedback in the context of an experience. So that's the other fabulous thing when you have the opportunity to test a live app, to be able to do.

So- recognizing that in the course of a study, the phone is behaving like the phone always behaves. So we're with them for 45 minutes to an hour and in that time, they get news alerts, they get texts from their kids, they get texts from their spouse. An app tells them that someone liked their photo or commented on something. So, the constant interruptions by other aspects of the device. They get phone calls. Whatever it is -because it is a phone after all. So recognizing that you're going to be not just managing your app and your tasks when you're dealing with people's real devices in real time, but also everything else that's going on. Again, that's wonderful context because you realize how very limited an amount of their attention you have at any given point.

Rebecca: Exactly.

Mahtab: All right. Well, I think that's all we wanted to say today about mobile and native usability testing. Let us know if you have any feedback about what we shared. If you have other questions or topics you'd like us to answer or discuss.

Our contact information is on our website.

Thank you for listening, you can find us on SoundCloud, iTunes, Google Play, Stitcher and on