Other disadvantages

Some people will say that not being able to run the study in a controlled environment, such as a lab, is a disadvantage of remote methodologies, but we would argue that it provides a more natural context to the study. How likely is it that the user of an app will always have a completely quiet environment, with perfect lighting, free of any interruptions when they are using it? No one lives in a vacuum: the phone will ring, someone will be at the door, the participant’s child will want attention, and so on. If you remember the definition of usability testing, it states that the interface is tested by “representative users”. You can only really judge whether an interface is easy to use when it is being used by the people whom the products target in order to achieve a goal the users would actually have and in the environment that the users would naturally be using the interface in.

Another disadvantage of the remote methodologies that we try to turn into a benefit is that users might run into issues when installing or setting up the interface to be tested. When this setup is identical to how the actual product would be set up, this is valuable feedback regarding how easy it is to get started with the product under testing. However, when the study is based on early prototypes, their installation might require additional time and effort in order to get the participant set up correctly, and this is admittedly more difficult to do remotely than it would be if both parties were present in the same lab together.