Formative Evaluation, Healthcare, Product Development

Beyond formative and summative evaluations

In the healthcare space, there is a big focus from our clients on formative and summative evaluations, because they are legal requirements to get the 510(k) / CE mark approval of their product, so that it can be marketed. When a formative evaluation is conducted, the expectation is to identify and improve usability issues and user experiences during development.

Often, formative evaluations show that the product has issues that could have been prevented before even reaching the formative stage. There are several steps before conducting formative evaluations that, in our opinion, are of utmost importance to design a good product that is actually ready for formative testing. Even though some upfront investment is needed, ensuring that these steps have been followed ultimately saves time and money, by reducing the amount of effort to correct the product during formative evaluations, and can potentially reduce the need to re-evaluate the product in another/several rounds of formative evaluation – basically, the goal is to have a well-designed product when entering the formative evaluation phase, and to have the lowest amount of changes made to it after the formative evaluation.

 

The different steps that, in our opinion should be performed before any formative evaluation of a product are:

  • Understand who the end users are – this is important for new products for which there is no clearly defined end user population yet, or when there is a need to assess whether certain users could become end users, e.g. “can severe Rheumatoid Arthritis patients handle a new type of autoinjector”. A variety of methods can be used to assess this, ranging from simple telephone interviews, to face to face interviews, to focus groups etc.
  • Understand who the competitors are, and how their products are perceived by end users. As above, several methods can be used to assess this.
  • Understand the end users’ needs (what would be the ideal experiences) to highlight gaps and opportunities – what are the use cases, the pains and gains, the preferences, the most important features etc. As above, several methods can be used to assess this.
  • Understand how end users might work together and how it can impact product design (for example in an operating room, how do the different specialties interact with similar products) – this can be achieved with the observation of user experiences in a real-world setting. The outcome is an overview of the people, touchpoints, interactions, and emotion drivers that define the experience. Photos and videos of users interacting with the product help understand the reality.
  • Understand the user environment and key contextual factors potentially needing to be considered in product design.
  • Strategy and Design workshops
  • UX Design

 

However, clients’ needs also need to be part of the equation, as one team’s needs and focus might not reflect another team’s needs and focus. Strategy and design workshops can be conducted to iron out the product design requirements and define clear guidelines upfront. These workshops can be conducted with a variety of relevant stakeholders and are usually informed by previous research or already available sources from the client. The output could be strategy / journey maps guiding new developments as well as first interaction and design concepts to inform the translation into usable concepts.

Once the end user requirements, and the client’s requirements are clear, it is time to start the design of the product. Iterative task-based usability testing should be performed to refine design concepts, to ensure that the product going into formative testing is as refined as can be, based on the initial input from the users and the client.

 

Conclusion

The product that needs to be tested in formative evaluations can only be as good as the sum of the previous designs and developments. All steps that come before the formative evaluation are therefore critical and should never be rushed. Making sure the product takes into account both user and client needs is paradigm to ensure a smooth formative and summative evaluation process and a good final product.

Author

Jonathan Singh

Jonathan has more than 11 years of experience in the health industry. His main focus at uintent is Human Factors work, running human factors studies (formative and summative) for all types of medical devices. He has an in-depth knowledge of industry standards and regulations (e.g. FDA guideline, AAMI HE75, IEC 62366 and EU MDR) and Usability processes. He also has extensive experience in digital biomarkers (sensor-based endpoints) and companion health apps. Jonathan sits in our Swiss office in Basel.

Go back

Datenschutzeinstellungen

Wir nutzen Cookies auf unserer Website. Einige von ihnen sind essenziell, während andere uns helfen, diese Website und Ihre Erfahrung zu verbessern.
In dieser Übersicht können Sie einzelne Cookies einer Kategorie oder ganze Kategorien an- und abwählen. Außerdem erhalten Sie weitere Informationen zu den verfügbaren Cookies.
Group Essenziell
Name Contao CSRF Token
Technical name csrf_contao_csrf_token
Provider
Expire in days 0
Privacy policy
Use Dient zum Schutz der Website vor Fälschungen von standortübergreifenden Anfragen. Nach dem Schließen des Browsers wird das Cookie wieder gelöscht
Allowed
Group Essenziell
Name Contao HTTPS CSRF Token
Technical name csrf_https-contao_csrf_token
Provider
Expire in days 0
Privacy policy
Use Dient zum Schutz der verschlüsselten Website (HTTPS) vor Fälschungen von standortübergreifenden Anfragen. Nach dem Schließen des Browsers wird das Cookie wieder gelöscht
Allowed
Group Essenziell
Name PHP SESSION ID
Technical name PHPSESSID
Provider
Expire in days 0
Privacy policy
Use Cookie von PHP (Programmiersprache), PHP Daten-Identifikator. Enthält nur einen Verweis auf die aktuelle Sitzung. Im Browser des Nutzers werden keine Informationen gespeichert und dieses Cookie kann nur von der aktuellen Website genutzt werden. Dieses Cookie wird vor allem in Formularen benutzt, um die Benutzerfreundlichkeit zu erhöhen. In Formulare eingegebene Daten werden z. B. kurzzeitig gespeichert, wenn ein Eingabefehler durch den Nutzer vorliegt und dieser eine Fehlermeldung erhält. Ansonsten müssten alle Daten erneut eingegeben werden.
Allowed
Group Essenziell
Name FE USER AUTH
Technical name FE_USER_AUTH
Provider
Expire in days 0
Privacy policy
Use Speichert Informationen eines Besuchers, sobald er sich im Frontend einloggt.
Allowed
Group Analyse
Name Google Analytics
Technical name _gat,_gtag_UA_120284862_1
Provider Google LLC
Expire in days 730
Privacy policy https://policies.google.com/privacy
Use Cookie von Google für Website-Analysen. Erzeugt anonyme statistische Daten darüber, wie der Besucher die Website nutzt.
Allowed