Usability Testing Protocol

Allianz Life Insurance - Product Development Process Maturity

Over ten months, I worked with Allianz to establish an internal usability testing protocol. The tests were focused on new self-service features being produced by multiple scrum teams. With the input of Product Owners, Customer Experience professionals and UX Designers, we launched a series of successful usability tests despite many logistical hurdles.

The Problem

In 2018, Allianz had been releasing many new features to their website both for policy holders and financial professionals licensed to sell Allianz products. Though there was some iteration within the design process of these new features, there was almost no usability testing involved. Despite the strong desire to bring usability testing into the design process, there were still many roadblocks as we worked to embed testing within the existing workflow.

The Approach

In order to establish a viable testing process, I began by partnering with those who drove new feature production and had conducted early research. I then worked to understand the current feature team workflow to ensure I did not slow down or interrupt current processes. Lastly, I wrote a test template to encourage rapid and iterative testing; to provide a method of tracking improvements; and to create a way to efficiently document, synthesize and share the results.

Multi-Team Partnerships

The first step in creating a usability testing program was to learn what research was already being conducted, how features received the green light to move to the design phase and who could support the generation of effective test goals. I determined that the most synergetic collaboration would be between the Customer Experience team, Product Owners of the features being produced and User Experience Designers.

Building and maintaining these partnerships were not always smooth or straight forward. There was a lot of up-front education on what exactly usability testing was how it differed from other types of testing (such as UAT, NPS, etc). When these partnerships were working well, they were some of the most productive collaborations I’ve had in my career so far.

Usability Testing Process

Our next step was to determine how usability testing could fit into existing processes. To build trust with our business partners and to strengthen our multi-team partnerships, we needed to make sure testing would not slow down any current workflows.

Over two weeks, I worked with a designer who was a part of the feature teams. He helped me understand the existing processes and brainstorm on how testing would best support them.

After a process was determined, we got right into producing our first usability test. We took each new feature and broke it up into sections. This allowed each test to remain under 15 minutes (an important requirement as we were not offering participants any financial incentive). We created goals for each test and then began composing questions and tasks.

The test platform we used was Userzoom. Having this tool allowed us to work very quickly: tests could be duplicated and reworked, results could be exported to many different applications and a preview of each test could be shared with our business partners for approval.

In the fall/winter of 2018, we were able to test three different features. Our process improved with each subsequent effort and soon we were able to deliver key insights to the Product Owners. In some instances, these findings brought new momentum by offering answers to design questions that had forced projects to become stagnant and deadlines to be missed.

In early 2019, we lost our budget for Userzoom — one of the many hurdles faced in establishing a usability testing program. Both myself and other UX Designers on our team continued to use practical methods to test their designs. These ranged from Accessibility Audits with Front-End Developers, leveraging secondary research and using ‘lo-fi’ usability testing (shown below). Although these methods were less conclusive than a usability test supported by powerful software, they absolutely improved our work by illuminating previously unseen design errors.

Managing Results and Recommendations

Being able to clearly communicate insights uncovered through usability testing was crucial to cementing relationships with business partners. This also aided rapid and iterative testing: once one insight was shared, product owners wanted to pursue that line of inquiry further in hopes of improving designs.

The results of usability tests were shared in Confluence. Having results presented in a simply stated and easily sortable way allowed business partners to quickly refer to a specific insight in meetings and ongoing design discussions. Using Userzoom’s data visualization, results to a given question were shown in multiple ways. This helped construct a compelling qualitative narrative of participants’ experience with an existing design.

In some instances, test participants were easily able to complete a task…

Other times, the design proved to be a bit more problematic…

Next Steps

To advocate for further usability testing budget, I worked with my manager to create a presentation for the AVP of Digital Solutions. This presentation documented our successes with the feature teams, gave a brief overview of the ROI of usability testing and provided actionable recommendations on how to incorporate testing into the broader digital team at Allianz.