Beyond formative and summative evaluations

Often, formative evaluations show that the product has issues that could have been prevented before even reaching the formative stage. There are several steps before conducting formative evaluations that, in our opinion, are of utmost importance to design a good product that is actually ready for formative testing. Even though some upfront investment is needed, ensuring that these steps have been followed ultimately saves time and money, by reducing the amount of effort to correct the product during formative evaluations, and can potentially reduce the need to re-evaluate the product in another/several rounds of formative evaluation – basically, the goal is to have a well-designed product when entering the formative evaluation phase, and to have the lowest amount of changes made to it after the formative evaluation.

 

Different steps in formative evaluations:

The different steps that, in our opinion should be performed before any formative evaluation of a product are:

• Understand who the end users are – this is important for new products for which there is no clearly defined end user population yet, or when there is a need to assess whether certain users could become end users, e.g. “can severe Rheumatoid Arthritis patients handle a new type of autoinjector”. A variety of methods can be used to assess this, ranging from simple telephone interviews, to face to face interviews, to focus groups etc.
• Understand who the competitors are, and how their products are perceived by end users. As above, several methods can be used to assess this.
• Understand the end users’ needs (what would be the ideal experiences) to highlight gaps and opportunities – what are the use cases, the pains and gains, the preferences, the most important features etc. As above, several methods can be used to assess this.
• Understand how end users might work together and how it can impact product design (for example in an operating room, how do the different specialties interact with similar products) – this can be achieved with the observation of user experiences in a real-world setting. The outcome is an overview of the people, touchpoints, interactions, and emotion drivers that define the experience. Photos and videos of users interacting with the product help understand the reality.
• Understand the user environment and key contextual factors potentially needing to be considered in product design.
• Strategy and Design workshops
• UX Design

 

However, clients’ needs also need to be part of the equation, as one team’s needs and focus might not reflect another team’s needs and focus. Strategy and design workshops can be conducted to iron out the product design requirements and define clear guidelines upfront. These workshops can be conducted with a variety of relevant stakeholders and are usually informed by previous research or already available sources from the client. The output could be strategy / journey maps guiding new developments as well as first interaction and design concepts to inform the translation into usable concepts.

Once the end user requirements, and the client’s requirements are clear, it is time to start the design of the product. Iterative task-based usability testing should be performed to refine design concepts, to ensure that the product going into formative testing is as refined as can be, based on the initial input from the users and the client.

 

Conclusion

The product that needs to be tested in formative evaluations can only be as good as the sum of the previous designs and developments. All steps that come before the formative evaluation are therefore critical and should never be rushed. Making sure the product takes into account both user and client needs is paradigm to ensure a smooth formative and summative evaluation process and a good final product.