Collaborative work to get the results you need – a simple UX test
Early user research and card-sorting for an entertainment platform.
Sep
10
2019
THE CHALLENGE
Evaluate the usability of a redesign of an in-train entertainment platform. We needed to find out if users understood the content and navigation structure of the new design, and which homepage and navigation version they preferred.
THE APPROACH
We proposed ten task-based, one-on-one, 45-minute sessions that included card sorting with a mobile prototype.
THE OUTCOME
Users liked the new design and had a clear preference for homepage and navigation design. The navigation structure and category labels generally fit participants’ mental model, however, we identified aspects for improvement.
Why conduct a user experience test? – Study background & objectives
On long-distance train rides and special train models our client offers customers a free onboard entertainment platform. The platform can be accessed via login to the onboard network and used on smartphone, tablet, or laptop. Riders can watch movies, listen to audiobooks, play games, or read news. Additionally, riders can access service information about their journey, offers of the onboard restaurant, and construction notifications.
According to our client, the platform had already reached a very high customer satisfaction level. However, to continue to improve the user experience and meet evolving user expectations, they redesigned the platform to maintain the level of satisfaction.
The homepage and navigation structure was the focus of the redesign. To align the development process with user needs and to collect feedback, our client wanted to conduct a user experience test with representatives of one of their predefined personas.
How we approached the project – One-on-one interviews & card sorting
Our client wanted to test different design versions of the homepage and navigation against each other. We worked closely with the design team to align the prototype for the interviews. We rearranged prototype content and pictures in order to build a scenario with which we could conduct the sessions.
In addition to the explorative interviews, we decided to use qualitative card sorting to understand if the current information structure fit the mental model of the users.
We had 50% of the participants conduct an open card sort and 50% conduct a closed card sort. We designed the study this way because our client wanted to get insights into current navigation structure and how it fit users’ mental model, but also wanted to know if the categories were labeled in a way that made sense.
„Card Sorting allowed us to identify patterns and structures in how participants would expect to find content on the platform.“
Collaborative results workshop – a rich knowledge pool for optimization ideas
We asked the project team to note their observations during the interviews as a basis for the results workshop. Having clients observe the interviews and capture their thoughts helps to identify possible issues and areas for improvement. In the workshop, we discussed and sorted the observations, noted positive and negative findings, and came up with recommendations for optimization – as a collaborative effort with the project team.
The physical outcome of the workshop was a photo protocol of the workshop results and a high-level summary of the main findings and recommendations. With the protocol, we created a highlight video showing the main findings, issues, and voice of the participants (opinions, expressions, etc.). Highlight videos together with the results deck can create empathy for users and a better understanding for optimization measures.
„The different perspectives from client team and UX experts together create a rich knowledge pool for optimization. Bonus: the client is actively engaged in the field and analysis phase.“
Example Workshop results for the first-use scenario (anonymized) | open card sorting result | UX test setup
THE PROJECT LEAD
Tabea Daunus
Tabea is one of our Young professionals with 5 years of experience in running user research studies. Being a certified Medical Devices Usability Expert (TÜV) her main interests are human factors and healthcare studies. Tabea is also responsible for the quality management within the company.
