Following over a year of research and development by a cross-departmental team, MoMA Audio+ mobile guide debuted in July 2013, replacing the handheld audio guides that the Museum had been distributing for many years. MoMA Audio+ enables visitors not only to listen to audio commentaries and read interpretive texts about works of art, but also offers the ability to take and share photographs, locate works of art and facilities, and e-mail yourself a record of your visit using the My Path feature.

A visitor in the galleries using the MoMA Audio+ mobile guide. Photo: Martin Seck

Testing and assessment has been key to the process of developing MoMA Audio+. Built using a method called agile development, in the months leading up to launch, the development team released and tested functional versions of the app as it was being built. Based on testing and feedback, we could iteratively implement improvements to the user interface and features to better enhance visitor experience. After launch, we conducted assessment from a variety of perspectives: visitors returning the mobile guide were interviewed about their experience, Antenna staff distributing the mobile guide were surveyed, and user-testing sessions were conducted.

MoMA Audio+ mobile guide distribution desk. Photo: Martin Seck

How Does User Testing Work?

Nine individuals were recruited to participate in user testing. Participants represented a mix of ages and genders, as well as varying interests, work experiences, and technical abilities. User testing took place in December over the course of three mornings in the American Modern exhibition. Three participants were scheduled for each 60- to 90-minute session, and for every participant there was a lead facilitator and a cofaciliator. The lead facilitator guided the participant through the directed-testing instrument, prompted and asked for clarification as needed, and took notes. The cofaciliator also prompted as needed, asked questions, and took notes. For each session, there was also at least one additional staff person floating between groups and observing. Participants were asked to think out loud and voice any questions or concerns as much as much as possible during the testing.

A user-testing participant with facilitators in the galleries. Photo: Claire Huddleston

So, what did we learn?

A lot! I’ve been working in museum evaluation for a few years and am constantly amazed by the amount and depth of information that even a small group of user-testing participants can provide. While some of the insights offered confirmed our own thoughts about the mobile guide, there were also plenty of things they pointed out that we did not realize were barriers to a smooth user experience. Along with critical feedback, it was also great to hear what people enjoyed about using MoMA Audio+.

Here are some of the overall findings from the user testing sessions:
What did users really like about MoMA Audio+?
• The amount and variety of content available for them to access
• The high quality images presented on the device
• The camera feature, My Path feature, and other functions that allowed them to personalize and share their experience at MoMA

What aspects of MoMA Audio+ did users find difficult or confusing?
• Locating the How To use MoMA Audio+ instructions
• Understanding and locating all the layers of content and features available
• Locating specific artworks using the map

We really appreciated all the critical feedback these user testing participants offered because it helps us find ways to improve on what we currently offer visitors. Of course, we also loved that 89% (N=9) felt that MoMA Audio+ is something that would enhance their experience in the galleries. A few comments from participants included:
• “I feel that MoMA Audio+ enhances what I came to see here. Thousands of things on view and this helps me to shape my visit.”
• “It added another dimension. Nothing replaces the act of looking, but knowing more about what I’m seeing enriches the experience. This achieves that goal. Devices are not too heavy, not burdensome, not ugly, nice MoMA graphic.”
• “Usually, I just want to be with the art and wander around but I really like the ability this device gives you to document what you see during your visit. Personalizes the experience”

Using the MoMA Audio+ mobile guide to document a performance in the galleries. Photo: Martin Seck