Workshops are commonly used in the field of toxicology and chemical risk assessment to gain consensus and understanding between actors, often on contentious and difficult topics. Organising successful workshops is challenging. This is especially true when a workshop brings together individuals with potentially conflicting roles, expertise, values, and areas of focus. In consensus processes, it is critical that diverse perspectives are appropriately and effectively considered. Researchers, therefore, must frequently address and overcome the challenges of workshop organisation.

The “Collaboration to Harmonise the Assessment of Next Generation Evidence” (CHANGE) is an initiative to design system-level interventions for bringing forward the date of effective use of New Approach Methodologies (NAMs) in regulatory toxicology (Mathisen et al. 2024). A primary component of the CHANGE methodology is the purposeful engagement of a diverse range of actors and interest-holders in a series of workshops to identify what factors might matter as it relates to uptake of NAMs (“Explore”), agree on what actually does matter from this range of factors ("Reflect”), and finally agree what should be done to address these factors ("Design") (see Fig. 1).

Fig. 1

Illustration of the three-phase approach to design system-level interventions

In this article, we reflect on the first CHANGE workshop (“Explore”). The purpose of this workshop (‘Explore’) was to create a qualitative dataset of the experiences and observations of participants in the regulatory toxicology system (Fig. 1). This dataset is to be used for the development and refinement of theories about how the regulatory toxicology system functions around NAMs, ultimately informing the design of system-level interventions that should promote the effective use of NAMs.

In this article, we present reflections on what we, as a team, consider to have been important to the successful implementation of the first CHANGE workshop. We focus on what methodological approaches we consider fundamental to the success of our workshop, why we took these approaches, and how we implemented these in practice. We also set out a series of recommendations to support other researchers in organising future workshops.

We have structured our reflections under five themes:

1.

Community engagement and ownership, for securing buy-in to the results of the process.

2.

Incentives for participation and engagement, to promote involvement of as broad as possible a range of interest-holders.

3.

Environment of transparency, respect and confidentiality, to enable participants to speak freely and contribute information.

4.

Iterative approach to data collection, to help ensure contributions were reflected on and validated.

5.

Inclusive participation and global outlook, to help ensure that a wide range of views are represented in participant contributions.

This article is complemented by a companion article ‘How to organize a successful toxicology workshop? A participant perspective on the first CHANGE workshop, June 2024’ (Diemar et al. 2025). That article reports on the participants' perspective on the first CHANGE workshop held in Oslo (June 2024). To make the companion pieces as objective as possible, the articles have been written independently with neither author team given access to the other’s work.

Community engagement and ownership

To ensure the results of CHANGE meet the communities’ needs and provide a usable end-product, principles of participatory co-design were used in the design and development of our approach. This supported the integration of interest-holder knowledge and expertise, establishing a shared understanding of the project's aims and scope and promoting ownership and buy-in. We believe if an individual is involved in the design and development of an intervention then they are more likely to take up or act on the results. This approach to problem formulation and intervention design has also been shown to be an effective tool for establishing trust and meaningful collaboration (Bradwell and Marr 2017; Durose and Richardson 2015).

Core to our methods of participatory co-design is a tiered approach to participation (Fig. 2). We used this approach to secure interest-holder engagement and support, and to manage the co-design and co-participation process.

Fig. 2

Tiered approach to participation

At the outset of the project, we established a Convening Committee to ensure the project was on a firm initial footing. The Convening Committee had several tasks: to ensure the uniqueness and value of the project; integrate CHANGE into existing and ongoing work; and establish the strategic Advisory Board. The idea was for the Convening Committee to use its broader network connections to help ensure greater diversity of participation in the Advisory Board and maximise the value and uniqueness of CHANGE, than could be achieved through the Project Team alone. For continuity, members of the Convening Committee became the founding members of the CHANGE Advisory Board.

While we designed the Convening Committee to initiate the project, we designed the Advisory Board to provide strategic guidance and support to the Project Team. The Advisory Board consists of 27 members with a diverse range of institutional, sectoral, and geographic expertise. We consider the Advisory Board to be a broadly representative cross-section of the interest-holder groups we need to engage for CHANGE to be a success. Advisory Board members have a high degree of involvement in the project's active progression and development, meeting the project team monthly. This helps ensure continuity, whilst also helping to make people feel valued and involved in the project's development.

Incentives for active participation and engagement

To ensure CHANGE achieved active engagement and involvement of relevant interest-holders, at the first in-person workshop we provided different incentives for participation, each requiring different degrees of commitment and time. We considered incentives such as arenas for dialogue, networking and information flow; opportunities to learn new perspectives and engage across silos and disciplines; and possibilities for publication and co-authorship to be fundamental to effective in-person engagement.

The main data collection component of the workshop was a series of small and large group discussions. Guidance themes (e.g., ‘Prediction versus protection’; ‘Silos’) were used to initiate and inspire the discussions and encouraged participants to reflect on and think about their own experience of the system, to generate anecdotes. Each guidance theme was tested through an online–pilot discussion with members of the Advisory Board to ensure it was suitable for the generation of anecdotes. Each guidance theme was introduced by a workshop participant in the form of a short talk and was followed by small (6–8 people) and large (20–25 people) group discussions.

Discussions were designed to allow sufficient time to engage in in-depth discussion whilst preventing cognitive fatigue. Simultaneously, we planned for enough time and space to allow participants to attend to their urgent upcoming tasks outside the workshop (e.g., respond to emails). We considered it important to the active engagement and involvement of participants, as well as their ability to listen, learn, and critically reflect that a ‘no laptop’ rule was enforced throughout the workshop sessions.

In addition to audio recording, during the small group discussions, facilitators and participants documented anecdotes on a pre-printed template by hand, which participants could refine at the end of the session. This helped facilitate continued engagement and ownership, as participants were able to contribute anecdotes and ideas without having to be an active participant in the discussion. This was particularly useful given how verbal contributions can only be made one at a time, which acts as a major rate-limiter on the total number of contributions that can be made per given time interval in a small group discussion. It also supported inclusive participation and engagement by providing an alternative mode of contribution for participants who were less comfortable sharing anecdotes verbally in a group setting to be included.

Participants were randomly allocated to every discussion group to avoid the generation of participant sub-groups, whilst maximising the diversity of perspective (be that sector, role, institution, region) and expertise in each group. This allowed participants to engage with different perspectives and points of view whilst also encouraging social interaction and networking.

Environment of trust, transparency, and respect

To ensure we collected the data we needed to achieve the goals of the first workshop (Fig. 1) it was crucial that we built a workshop structure that supported our approach to data collection. As the nature of the data collected in the workshop was focused on exploring participants' personal experiences of how the system worked, we considered it essential that participants felt able and willing to share honest and open reflections. This required an environment of trust, transparency and respect (Spatz 2000; Pennington 2008; 2011). Having the competence and tools in place to protect the participants’ privacy and exhibiting values to respect the participants’ individual wishes was fundamental to promote trust among participants (Earle et al. 2007).

Actions were taken ahead of the in-person and online workshops to ensure participants were comfortable. This included sending detailed information to all participants in advance of the workshop on rules of confidentiality, pseudonymisation, and data management. In addition, the workshop followed Chatham House rules (i.e., participants are free to use the information received, but neither the identity or affiliation of any speakers, nor that of any other participant, can be revealed).

We also wanted to create welcoming social spaces for participants to engage in. Providing opportunities and spaces for informal networking, collaboration and socialising is considered vital to building trust and strengthening relationships between participants (Spatz 2000). At the in-person workshop in Oslo, this focused on regular coffee and refreshment breaks in shared ‘social’ spaces outside of the workshop rooms, poster sessions' where participants could reflect and engage in conversation, and evening meals taken together to encourage networking and relationship-building outside of the formal environment of the workshop. High quality foods and drinks complemented these spaces.

Iterative approach to data collection and refinement

The purpose of collecting anecdotes was to enable exploration of how people in the regulatory toxicology system experience the system, helping understand how participants make sense of the world around them and how this supports their judgment and decision making (Craik 1943; Johnson-Laird 1986). We used group-based elicitation methods to collect and then refining anecdotes from participants about their experience of working in the regulatory toxicology system, using group-based elicitation.

To support the discovery and development of ideas at the workshop, we designed an iterative approach to data collection that encouraged refinement and reflection (Fig. 3). Data was collected in the form of anecdotes (i.e., candid stories and accounts of working in or around the regulatory toxicology system). Each participant engaged in four small group discussions and two large ‘fishbowl’ discussions. Participants then collectively prioritised anecdotes they thought were interesting, surprising or that they wished to explore more before diverging into small intermediary groups for in-depth exploration of prioritised issues.

Fig. 3

Schematic representation of our approach to data collection. Small-group work emphasises creative activity and brainstorming of anecdotes. Large group work emphasises consolidation of ideas and building consensus

In the small group discussions, participants were encouraged to share anecdotes of both positive (i.e., what worked well, best practices) and negative (i.e., what did not work well, lessons learned) experiences of working within the regulatory toxicology system. As the small group discussion was focused on sharing personal experience, the facilitators' role was key to ensuring all voices had the opportunity to be heard. The facilitators focused on encouraging individuals to participate, rather than being the ones having the ‘best’ or ‘right’ answer. We believe that this approach to “storytelling” helped foster an environment of open-mindedness, curiosity and respect, whilst encouraging participants to reflect on their own experience and viewpoint. In many instances, the discussions were free flowing and organic, meaning that one anecdote often led to another.

The large group discussions focused on bringing the group together; to consolidate, merge or share ideas; and to refine or generate further anecdotes. Here we used a “fishbowl” approach to anecdote generation (see Fig. 4). In this approach, participants in the outer circle are encouraged to listen and critically reflect on in-depth core discussion between engaged participants in the inner circle. Only those in the inner circle are allowed to speak. If a participant wishes to say something they must enter the inner circle by taking a free seat, or by asking someone to move to the outer circle by lightly tapping their shoulder.

Fig. 4

Visual representation of the large group ‘fishbowl’ exercise

The fishbowl methodology is a useful tool for ventilating, generating, and sharing ideas through collective cognition and diversity, and fostering dynamic participation (Pennington 2008, 2011), whilst also preventing dominant voices or messaging. Whilst largely self-organising once the discussion begins, pro-active facilitation was used to encourage a larger process of dialogue, contribution and flow of movement between the circles. This included encouraging people who, through body language, looked like they wanted to step forward and ensuring those that were not stepping forward felt comfortable to do so. Fishbowls are considered particularly useful in introducing a novel and fun approach to data collection and turned out to be a highly positive and productive experience.

Inclusive participation and global outlook

The CHANGE in-person workshop was designed to bring together different actors from across the regulatory toxicology system to maximise success and sustainability of long-term solutions. We categorise the regulatory toxicology system to consist of four main constituent parts. These include data generation (the research and testing that generates primary data for decision making); data analysis (the evidence review and risk assessment processes that make sense of the primary data for the purpose of decision making); data use (the policy and decision making that uses the analyses of scientific evidence about risk, and other data, for making decisions) and data impact (the direct and indirect effects on the population or environment of policies and decisions). We consider the inclusion and participation of these groups important to further promote the exchange of experience between traditionally disconnected regions, provide maximum opportunity for cross-pollination of ideas and to identify interventions that can be adapted or applied in the context of local conditions (Mathisen et al. 2024).

To ensure that all parts of the regulatory toxicology system were represented, CHANGE took a purposefully cross-sectoral and international approach to workshop participation. We chose to selectively invite individuals whom we considered possessed the relevant expertise and understanding from a diverse range of sectoral, institutional and geographical perspectives. Only participants that were invited were able to attend the workshop.

Historically it has been challenging to involve risk managers, NGOs and participants from low- and middle-income countries and those from regions where English is not widely spoken. We supported participation with travel funding, though we did not have finances to support everyone who wanted to participate. To support the inclusion of those who were unable to attend the in-person workshop in Oslo, we organised a series of follow-up online workshops.

Participation from certain parts of the system including those affected (i.e., impact) and those making the decisions (i.e., data use) turned out to be hard to achieve, despite efforts to actively balance participation, undertake follow-up online workshops and provide funding or travel support.

While we deem our approaches to workshop design, community engagement, facilitation and data collection to be largely successful, we do not feel like we achieved a sufficiently global participation. Regions where regulatory toxicology communities are smaller or under-resourced, impacted groups, communities that are geographically distant, and do not speak English, were underrepresented in the first CHANGE workshop.