Last week we ran a usability study of a recently released mobile website. Since testing on a mobile device is a bit more difficult than on the desktop, I wanted to share a couple of details about our setup and what we learned.
Avoid complicated camera rigs.
There are lots of great tools for recording a usability session on the desktop, but most aren’t designed for testing on a mobile device. So how to reliably capture a session? You may have seen examples of people attaching a camera to a phone in order to record a user’s taps. We evaluated a few similar options, but they all seemed too intrusive. These setups constantly reminded users they were being recorded. In addition, most added a lot of extra weight to the device making it very difficult for a participant to use the device naturally.
We found that mounting a high resolution HD camera on a tripod high above the participant’s shoulder allowed us to adequately record a session. For some users, it took a bit of panning and zooming to follow their hands as they used the device, but most required very little adjustment of the camera. And with a standard zoom, we were able to get a surprising amount of detail. This allowed us to stay out of the way — after a few minutes, most users forgot they were being recorded.
Let participants use their own device.
Avoiding a complicated camera rig had another advantage — it allowed us to let participants use their own devices. We found allowing people to use their own phones really helped make them feel more at ease. However, with people bringing in their own devices, we weren’t able to set anything up beforehand. So, in order to quickly get started and to avoid asking participants to type in large URLs on a tiny keyboard, we created short URLs for the website being tested and texted them to users. We asked permission first, but everybody was more than comfortable with the idea. Of course, it helped that we were testing a publicly available, production website.
Making participants as comfortable as possible.
Any usability session can be awkward for participants, and a mobile device study is no different. Participants are often required to sit in a sterile room with a one way mirror while a camera is pointed at them and someone is asking them dozens of questions. We found that both of the tips described above helped to ensure people were comfortable. The more comfortable people are, the more natural they’ll act — and the better feedback they’ll provide.