Smartphone digital biomarker app usability testing with patients

A pharmaceutical manufacturer was planning to use a smartphone digital biomarker app in a clinical trial. Before the trial began, uintent tested the usability of the app with intended users – patients in different stages of the disease. The research revealed several usability issues that were addressed by the manufacturer before the app’s launch in the clinical trial.

Jun

2019

THE CHALLENGE

A pharmaceutical manufacturer was developing a new compound for the treatment of a neurodegenerative disorder and aimed to test it in a clinical trial. In order to gather additional endpoints for the trial, they wanted to use a smartphone app to collect sensor data. The app had a game look and feel and consisted of several tests to assess specific disease symptoms. The data captured via the app was to help the manufacturer assess changes of patients’ abilities over an extended period (during the trial), and therefore potentially assess the effectiveness of the treatment. In order to ensure the app would capture appropriate data during the clinical trial, it had to be tested with patients prior to launch to ensure they could use the app as intended.

THE APPROACH

uintent recruited 20 patients diagnosed with different stages of the neurodegenerative disease, accounting for four intended user profiles, and organized a usability test in London, UK. Each session lasted approximately 60 minutes, during which patients were asked to use the smartphone app and play the “games”. For each session, patients completed an initial comprehension assessment of the instruction, they were then asked to complete the test. For some sessions, alternative versions (e.g. with several difficulty levels, or different parameters) were tested to identify the best design and/or baseline settings.

THE OUTCOME

The research revealed several usability issues with the app. Additionally, we showed that the instructions were not understood by all participants and were sometimes a source of confusion. Based on these results, uintent provided recommendations for the manufacturer to address before the app’s launch in the clinical trial. Additionally, uintent gave recommendations on the suitability of the different tests for the various patient types, which allowed the manufacturer to personalize the app to patient’s individual abilities and needs.

The importance of recruiting the right patients for the usability test.

As our client wanted to test the smartphone app before actual use in a clinical trial, it was important to conduct the research with the intended app users. The user profiles were defined based on the requirements of the clinical trial, which led to four participant profiles.

Patients with this neurodegenerative condition can potentially have issues traveling by themselves, which could be problematic for interviews conducted at a central location studio. For this study, in-home interviews were not recommended as we wanted to ensure a similar setup across all interviews. Participants were asked to come to the interview with a study partner (family, friend, caregiver). Those who were not willing or could not come accompanied were offered transportation to and from the study location.

Based on all study requirements and limitations (time, budget), uintent created a recruitment screener reflecting as closely as possible the clinical trial requirements. Recruitment of all participants was performed by our preferred medical recruitment partner who is experienced in recruiting patients (and their sub-groups) from this indication, in London. Several methods were leveraged (panels, patient advocacy groups, physician referral, etc.) and we achieved full recruitment.

Before the interview, participants were asked to complete a clinical anchor assessing different types of cognitive abilities. The results helped to calibrate the research sessions (avoid floor/ceiling effects) and increase granularity of disease severity for analysis. Results from the sessions and from the clinical anchor were analyzed together.

„[…] a close collaboration between our clients and the project team [is] key.“

The right team for the project.

Interviewing patients with a neurodegenerative disease can be delicate. We have experience asking questions the right way with patients of all ages, and for many sensitive medical conditions (respiratory, oncology, hematology, autoimmune, neurodegenerative, neurological, and developmental, etc.). In order to ensure that these sensitive patients are approached adequately, we strive to have all interviews done by uintent project members. For this project, it was very important to be gentle with patients, as a loss of cognitive functions can be overwhelming, and having patients think and reflect about their disease can be very sensitive.

Regarding the project, as for every project, a close collaboration between our clients and the project team was key. It was especially important for this project, as the interviews’ sensor data, collected via the app, had to be sent to our clients’ servers after each session day for analysis. This ensured the tests settings could be adjusted based on the results (calibration of the tests). As the moderator for the sessions was also the uintent project manager, quick adjustments could be made to the sessions after the interview day. Also, at the beginning of the fieldwork, we observed that baseline settings for several of the app’s tests did not work out and we could react quickly and adjust them between two interviews, after discussing with our on-site client observers.

THE PROJECT LEAD

Matthias Gasser

After receiving his MsC in Psychology in 2009, Matthias started working in the Market research industry and moved into UX and human factors research shortly after. Matthias’ main focus at uintent is health and early user research.

Send Matthias an e-mail