Home > Vol 16, No 2 (2022) > Kerr

Integrated Co-design: A Model for Co-designing with Multiple Stakeholder Groups from the ‘Fuzzy’ Front-end to Beyond Project Delivery

Jeremy Kerr*, Michael Whelan, Okansa Zelenko, Keely Harper-Hill, and Clare Villalba

Queensland University of Technology, Brisbane, Australia

As co-design is increasingly applied across multiple sectors, there is a greater need for practical guides to support extended, low-cost and diverse implementation. This paper proposes a model, integrated co-design, developed by research through design, which provides a process for conceptualising and implementing co-design projects aiming to collaborate with multiple stakeholder groups. It supports a holistic application of co-design across all stages of a project, from the initial scoping and key ideation phases to beyond project completion. The model was iteratively developed through the design of inclusionED, a nationwide learning and teaching community-based platform aiming to implement research findings of the Autism CRC in schools across Australia. Within this project, multiple stakeholder groups, including teachers, specialist educators, students on the autism spectrum, parents, and policy-makers, were engaged throughout the design process and in ongoing resource development. The integrated co-design model provides a series of strategies for multiple stakeholder group engagement and participation, including methods for designing for co-design, facilitating blended co-design, applying asynchronous co-design, and working with participants requiring customised collaborative experiences to support their involvement, including co-designing with people with cognitive difficulties and trauma.

Keywords – Co-design, Design Methods, Knowledge Translation, Participatory Design.

Relevance to Design Practice – The integrated co-design model provides a practical tool and series of guidelines for practitioners and researchers to plan, develop and execute co-design projects in complex scenarios where multiple stakeholder groups are involved. The model can be applied across different sectors of industry and for diverse project aims.

Citation: Kerr, J., Whelan, M., Zelenko, O., Harper-Hill, K., & Villalba, C. (2022). Integrated co-design: A model for co-designing with multiple stakeholder groups from the ‘fuzzy’ front-end to beyond project delivery. International Journal of Design, 16(2), 75-90. https://doi.org/10.57698/v16i2.06

Received October 22, 2020; Accepted June 19, 2022; Published August 31, 2022.

Copyright: © 2022 Kerr, Whelan, Zelenko, Harper-Hill, & Villalba. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content is open-accessed and allowed to be shared and adapted in accordance with the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.

*Corresponding Author: jeremy.kerr@qut.edu.au

Jeremy Kerr is an academic and researcher in the School of Design at the Queensland University of Technology (QUT). His research focus is the exploration and development of design and design frameworks to advance community capacity building and self-advocacy. Central to his work is applying and innovating co-design methodologies to foster collaboration and to include the authentic voice and creative input of stakeholders in the development of solutions to complex problems. He has worked extensively with community and government organisations and with health services in the areas of mental health and well-being, informal and formal education, children’s health, intercultural communication, and with the disability sector. His work has involved leading co-design collaborations for projects with CARE International, Bravehearts, Education Queensland, the Woolworths Centre for Childhood Nutrition Research, the Autism CRC and Children’s Health Queensland Hospital and Health Service (CHQ), amongst other organisations.

Michael Whelan is an Associate Professor at QUT Australia, a writer, musician and diversity advocate. He has led numerous interdisciplinary creative arts projects working with young people with autism spectrum conditions, students from Indigenous backgrounds in regional communities and young people from refugee backgrounds. In addition to his interdisciplinary live arts practice and autism research, Michael has written and directed a range of short films on topics ranging from post school transition for young people with autism to youth at risk and bullying. His short film on social inclusion and autism for Australian schools titled “What are you doing?” was premiered at the United Nations in New York for World Autism Day in 2013. Along with colleagues from QUT, Michael co-authored two books, Research Approaches to Supporting Students on the Autism Spectrum in Inclusive Schools and Supporting Students on the Autism Spectrum in Inclusive Schools (Routledge) in 2021.

Oksana Zelenko is a design academic and researcher specializing in applications of participatory design for eHealth. Her area of expertise is in interaction and visual design for e-health applications with a specific focus on using participatory and co-design methods for design, development and evaluation of interactive health promoting technologies promoting youth mental health and wellbeing. She has had a leading role in design research across a series of national and international projects, resulting in a wide range of novel technologies, ranging from mobile health apps to education and capacity-building platforms, contributing as part of large transdisciplinary teams. She is co-editor of Design and Ethics: Reflections on Practice published by Routledge.

Keely Harper-Hill is the Research Associate for the Enhancing Learning and Teaching education research program of the Autism CRC at QUT Australia. An experienced speech-language pathologist and educator with values of integrity, inclusion and kindness, she makes strong, positive contributions to organisations including those with a clinical or education setting. She is committed to ongoing professional development and life-long learning with a Masters in Education (Special Education: Autism) with distinction from the University of Birmingham and a PhD (Speech-language Pathology) from The University of Queensland. In 2021 she co-authored two books, Research Approaches to Supporting Students on the Autism Spectrum in Inclusive Schools and Supporting Students on the Autism Spectrum in Inclusive Schools (Routledge) which showcase the work of the Education Program of the Autism CRC.

Clare Villalba is an experienced service designer and researcher. Her PhD research is situated at the intersection of design, health and innovation and explores why preventable health conditions still affect people today, looking closely at the preventable eye disease, diabetic retinopathy. She collaborates with the people at the centre of healthcare challenges to facilitate meaningful change and enable wellbeing, working closely with the Southern Queensland Centre of Excellence for Aboriginal and Torres Strait Islander Healthcare. Clare also led the human-focused innovation stream of the Eye Health Innovation Strategy being implemented by the Medical and Healthcare Robotics Team at Queensland University of Technology. She has a passion for working with communities and seeking out opportunities to contribute to a world where individuals and communities are able to shape the systems and services that influence their lives.

Introduction

As co-design and participatory design processes are increasingly applied across diverse sectors including business, health, government and education, with resulting success (Greenhalgh et al., 2016; Lal & Adair, 2014; Steen et al., 2011), the value in engaging users and diverse stakeholders in design processes is further understood and appreciated. Co-design continues to emerge as a key methodological foundation through which complex, transdisciplinary problems and challenges are addressed (Smith et al., 2017). Despite this, models and design principles for complex, multiple, and diverse stakeholder collaboration is limited. This paper addresses this gap by describing the development of an integrated co-design model, which provides of a series of approaches and strategies to address wicked problems (Australian Public Service Commission, 2007; Rittel & Webber, 1973). This model was iteratively developed through a process of research through design (Zimmerman et al., 2010) in the creation of inclusionED, a national online platform to support diverse learners within Australian education systems.

The design of inclusionED emerged from a transdisciplinary initiative aiming to construct a learning ecosystem based on autism education and aspiring to embed co-design and multiple stakeholder collaboration at its core. The initiative was instigated by the Cooperative Research Centre for Living with Autism (Autism CRC), the world’s first national, cooperative research effort focused on autism, located in Australia. Autism is a cognitive and communication condition estimated to affect from one in 150 people (Australian Institute of Health and Welfare, 2017) and within Australia, most schools have students on the autism spectrum enrolled, including in government, Catholic, and independent school jurisdictions. The Australian Autism Educational Needs Analysis (Saggers, Klug et al., 2016) highlighted the urgency for teachers, throughout all educational sectors in Australia, to enhance their knowledge and understanding of autism to best support students and improve learning outcomes (Saggers, Carrington et al., 2016). In developing a national learning ecosystem in this area, it is envisaged over 400,000 educators across Australia can gain direct access to specialist knowledge about autism including strategies to support its practical implementation. The inclusionED project aimed to transfer the Autism CRC’s significant findings into accessible, pedagogically sound professional learning (PL) resources and to disseminate these through a custom-made knowledge translation process.

The foundation of co-design was established for this venture to ensure services offered directly aligned with the needs of users. Solutions developed this way are more likely to succeed than those imposed or introduced by experts alone (Lin et al., 2011; Orlowski et al., 2016; Zelenko et al., 2021). Appropriately, the co-design foundation aligned to the initiative’s educational philosophy of inclusivity, which emphasises all learners are present and equally supported within mainstream schools (Jiménez et al., 2007). Developing this foundation entailed applying collaborative design to the entire project cycle and engaging multiple and diverse stakeholder groups on a state and national level. Stakeholder groups included classroom teachers, special needs teachers, policy-makers (with varying jurisdictions) and representatives from partner organisations, along with students on the autism spectrum and their parents. Co-design entailed scoping this design challenge with these stakeholders at the undefined and messy front-end of the project—what has been referred to as a fuzzy front-end (Herstatt & Verworn, 2004)—as well as maintaining a design dialogue with them until the project’s completion where a community of users could be established.

While there was an expectation that co-design would maximise relevance and avoid some of the common pitfalls associated with the introduction of technological systems (see, for example, Davis et al., 2014; Lin et al., 2011; Orlowski et al., 2016), it posed unique challenges. This included the feasibility of engaging all identified groups in meaningful ways, within budget constraints, as well as integrating competing stakeholder values within project outcomes. In addition, there was a challenge in developing appropriate and respectful approaches for collaborating with students on the spectrum.

In undertaking this formidable project, research through design was used as a research method to progressively discover knowledge through the design work (Herriott, 2019). Through this, the integrated co-design process model was developed, representing a unique approach to planning for the incorporation of multiple stakeholder groups throughout a project and to applying the diverse approaches needed for this to succeed. This model was iterated through a reflective process as design occurred (Zimmerman et al., 2010). In the following section, design exemplars informing this process model are outlined, and then an overview of the model-in-action and results of its application are presented. Finally, in the discussion, a refined conceptual model is proposed, based on this study, for potential future use by practitioners.

Background: Co-designing with Multiple Stakeholder Groups

The starting point for iterating a holistic co-design approach involved identifying design principles and strategies based on prior, successful design initiatives involving diverse, multiple stakeholder groups in co-design processes, along with specific collaboration approaches for working with young people with autism. In the case of the former, while co-design has gained momentum, details of the particular processes for integrating multiple stakeholder groups remain minimal. Literature provides many descriptive examples where one key stakeholder group is engaged (typically primary end users) or examples of multiple stakeholder groups being involved but with limited operational details.

A study outlining the process is Ospina-Pinillos et al. (2018), which describes the engagement of multiple stakeholder groups (both young people and youth health professionals) to develop a mental health eClinic for young people. Each group was regarded as a core end user, and a co-design process was devised to give agency to both groups. Twelve participants from each stakeholder group participated in separated workshops where similar activities were undertaken, and the same facilitators were used. Mixed stakeholder sessions then followed for further project development. Bowen et al. (2013) also undertook a similar design process to develop better outpatient services for older persons, with similar workshop activities undertaken by both health staff and patient groups. Shared themes across both groups were then critical for final project development.

Beyond the health sector, Woelfer et al. (2011) outline utilising four diverse stakeholder groups within a co-design process to improve safety for homeless youth. This entailed working with groups of direct stakeholders (homeless youth) and indirect stakeholders (service providers, police officers, and community members). Project workshops were customised for each group, though similar approaches and generative activities (including reflective sketching and scenario activities) occurred across all. This strategy facilitated multiple perspectives to be more easily integrated when developing a final solution.

Wake and Eames (2013) engaged multiple stakeholder groups, consisting of children/students, teaching staff and community members (including professional practitioners) to build an eco-classroom model. In this case, the authors advise planning not only for multiple stakeholder engagement but to consider the scheduling of each group’s participation appropriately. Further, the value of planning stakeholder engagement at the start of a project is emphasised by Vines et al. (2013). The authors also advise engaging the stakeholder groups in multiple forms in a single project and considering stakeholder participation at different levels across the design cycle.

A limitation in adopting such stakeholder involvement models, however, is its impact on project expenses and time. Sanders et al. (2010) observed face-to-face participatory design can be time and cost-intensive and identify video-conferencing technologies as an alternative. Despite the potential in co-designing with different stakeholder groups via such technology, its application as a replacement for face-to-face workshops remains minimal.

Näkki et al. (2011) have identified how virtual co-design can be effective when applied before, during, and after face-to-face sessions. While the authors emphasise face-to-face engagement is vital in project development, well-considered and applied technology is proposed as an option for meaningful off-site participation. The authors suggest holding mixed workshops, consisting of face-to-face and virtual participants. This entails the use of video-conference technology (such as Skype, Zoom, etc.), alongside text-based communication tools, document sharing and a process of microblogging sessions. The authors found this combined strategy helped create a sense of presence over distance for off-site participants.

Customising Co-design Processes: Working with Students on the Autism Spectrum

Additional considerations are necessary when co-designing with stakeholder groups requiring specialised engagement strategies. This includes working with children (Langridge et al., 2017) or people with communication, cognitive, and behavioural issues (Francis et al., 2009). In such cases, approaches require consideration of a group’s unique needs, abilities, and predispositions. Francis et al. (2009) identify key challenges in working with people with autism, for example, as a heightened fear of failure as well as a difference in motivation, imagination and communication. In such cases, there is the potential for participant distress if workshop activities are not appropriately structured or facilitated. Strategies such as understanding behaviours indicating participant discomfort, providing adequate times for tasks and offering careful and meaningful explanations are suggested. Holone and Herstad (2013) have outlined further design principles for working with diverse groups, particularly children with severe disabilities. The authors emphasise patience in sessions and allowing for additional timeframes for activities. Using elements of fun and play within sessions has also been proposed as an engagement approach for this group (Wilson et al., 2019; Makhaeva et al., 2016).

Benton et al. (2012) offer further guidelines in co-designing with children with autism, including supporting children’s cognition difficulties by organising and sequencing activities using constant direction, as well as using visuals to scaffold and support tasks (for example, visual timelines and drawing templates). To reduce participant anxiety levels, it is suggested co-design sessions occur in familiar environments such as a participant’s school, and involve a teaching staff member who knows the student in the sessions.

Method for Integrated Co-design

The approaches outlined in the literature were incorporated into a robust process for the inclusionED initiative. In establishing the initiative upon a foundation of co-design, initial project development involved the planning and designing of an extended co-design process. This entailed identifying the key stages of project creation and application, which eventually became known as the design, develop, and implement stages, and then developing a series of co-design and participatory practices involving a range of stakeholder groups throughout these. A diagram outlining the final project structure and involvement of multiple stakeholder groups across the project’s full cycle is shown in Figure 1. This represents the integrated co-design process model for the entire project. The diagram shows different forms of stakeholder collaboration (identified in colours) interspersed and informing key design phases (identified in greys).

Figure 1. inclusionED co-design process.

As indicated in this diagram, within the project, following preliminary project planning and a workshop pilot (Phase 1), a series of in-person and mixed (or blended) co-design stakeholder workshops were undertaken to inform conceptual design approaches and prototype development. These initial co-design sessions (Phase 2) consisted of a series of nine workshops attended by a total of 28 participants. Separate sessions were undertaken with: 1) classroom teachers and specialist support teachers; 2) students with autism; 3) parents of students with autism; and 4) education organisation policy-makers and provider representatives. Teacher participants came from government, Catholic, and independent school sectors across the states of Queensland, NSW, Victoria, Tasmania and Western Australia, and participants in the policy-maker groups came from one state-based and three national organisations. This diversity was necessary in order to ensure the resulting platform was responsive to the needs of different education systems embedded across Australia’s states and territories. The inclusion of interstate participants led to the use of the blended workshop model which consisted of on-site participants alongside online participants connecting via video-conferencing software. Many participants in this early phase then participated across phases 5, 7, and 8.

The following sections outline the full phases of this custom-made design process, highlighting the key methods and approaches in use.

Design Stage

Preliminary Stages: Co-designing Planning, Pilot Workshop & Preliminary Activities (Phase 1)

Prior to engaging stakeholders in formal workshops (in Phase 2), three key activities were undertaken. Firstly, the research team iteratively developed the overall co-design structure, which entailed scoping relevant user and stakeholder groups and developing a series of participatory processes across the entire project timeline. While this structure further evolved as the project progressed, a holistic blueprint occurred at this point. Following this, a pilot co-design workshop with teachers, the primary user group for the platform, was undertaken to test and refine the overall toolkit design for front-end co-design sessions in Phase 2. The pilot included five teacher participants, who were recruited through the research team’s network. Once completed and a better understanding of key participant needs and preferences was known, the researchers created a pre-workshop introductory video for Phase 2 participants and distributed this via a YouTube link. The video outlined the project aims and introduced the workshop facilitator and project team. The delivery approach was informal and friendly to set the tone for non-hierarchical, collaborative workshops (see Figure 2). Significantly, the video also served as an early provocation for themes explored in the workshops, technology-based learning, and communication design preferences.

Figure 2. Images from the pre-workshop video sent to co-design participants.

Co-design Workshops: Teachers, Students with ASD, Parents & Policy-makers (Phase 2)

All nine initial co-design workshops with four different stakeholder groups were delivered as a series of linked creative activities consisting of generative methods (Sanders & Strappers, 2012) involving participants documenting reflections and ideation through visual mapping. This aligned approach was taken to deliver consistency in types of data produced across workshops. In some cases, participants were invited to creatively respond to findings from other workshops, for example, where policy-makers were presented with outcomes from workshops with students with ASD and teachers. A mixed/blended model, consisting of on-site and online participation, was applied in workshops with both teacher and policy-maker stakeholder groups where it was not possible for all participants to attend in person. Workshops were held in person at the university campus and concurrently as a virtual workshop through Connect, an interactive online video display system which allowed participants to show workshop artefacts and visual maps when discussing them (see Figure 3). Co-design workshops with students with ASD and parents of students with ASD occurred either at the students’ school or the location of a youth group centre students attended, both familiar locations for participants as per Benton et al. (2012).

The workshop design was informed by principles of strength-based/alutogenic health promotion with participation led by the individuals’ perceived strengths (contrasting with deficit-based models, which target identification and management of risk factors). Activities were developed as creative play, with vibrant visual prompts and tactile making. Facilitation was upbeat and promoted a sense of fun, building upon the pilot approach and avoiding participants finding the design process boring (Ehn, 1992).

Figure 3. A blended co-design workshop linking on-site and online participants.

A series of three separate co-design workshops with teachers and special needs teachers were held, with nine participants in total. These utilised the blended approach, and all three sessions followed the same format of three linked creative activities with themes of exploring your best learning experience, imagining how this would occur in the near future, and designing approaches for learning in the near future.

Concurrently, two co-design workshops with teenage students on the autism spectrum were undertaken with four participants in total. The aims of these workshops were complementary to those held with teachers and other stakeholder groups, as students co-designed how their social presence (Ovaska et al., 2008) was represented across the teachers’ learning ecosystem and how the experience of autism was to be presented in terms of visual depiction and verbal/textual framing.

Strength-based design principles and a sense of play were accentuated in student workshops. Strategies applied were informed by Francis et al. (2009), Holone and Herstad (2013), Wilson et al. (2019), and Makhaeva et al. (2016). Customisation for this workshop occurred through framing activities around a series of visual personas (Hanington & Martin, 2012), which represented teenagers with autism. This was developed to enable third-person discussions of portrayal, perception, and experience where themes could be explored without directly asking participants to recount potentially upsetting and triggering personal experiences. Participants were asked to become a film director and direct a short video based on a selected persona, from a mixed group of student personas displaying ASD behaviours. Persona toolkit material from the workshop is displayed in Figure 4. Along with the researcher who acted as facilitator for all nine co-design sessions, an autism specialist/psychologist attended the student sessions to offer additional support and facilitation.

A separate workshop was also held with parents of young people with autism, consisting of four parents representing four families each with a different child that was either in high school or had recently completed high school. This workshop mirrored the three-step structure of the teacher workshops.

Figure 4. Example of a visual persona and task slide from workshops with students with autism.

Finally, three workshops were undertaken with policy-makers and professional learning providers (two of which were conducted in a blended format). These comprised of 11 participants drawn from the Autism CRC’s 53 participant organisations and partners based around Australia. These workshops followed the same model as the teacher and parent sessions but were also designed to indicate feasibility issues relating to proposed learning approaches and preferences raised in other co-design workshops. Here, participants were encouraged to identify perceived obstacles and barriers and iterate solutions to these.

Data Analysis (Phase 3)

Following the co-design workshops, researchers not directly involved in them collated and analysed all the resulting data, which consisted of visual maps and workshop transcripts. Data was analysed using visual and textual thematic analyses (Given, 2008). Each stakeholder group was analysed separately, with at least two researchers being involved at each stage. Initial coding was based on the PACT framework (Benyon et al., 2005) with researchers developing a colour coding system to reflect each of the PACT components, which was iteratively refined as analysis progressed. This process is displayed in Figure 5. A process of affinity diagramming (Hanington & Martin, 2012) was then undertaken to give a high-level view of the optimal system for each stakeholder group, and from these, a macro-level affinity diagram was compiled to identify recurring themes between stakeholder groups or themes that were unique to the individual groups. Findings were summarised in a final map that encompassed key themes and priorities for all stakeholders.

Figure 5. Researchers undertake coding and data analysis across visual maps and workshop transcripts.

Prototype Design (Phase 4)

After data analysis, which identified a series of key design principles as well as user preferences, a service mapping of these findings was produced, followed by an iterative user journey map (Hanington & Martin, 2012). The process for design at this stage can be considered a combination of logical approaches and intuitive and designerly responses to identified principles. This led to a resolved interactive prototype, which was taken into the next project stage.

Develop Stage

Participant Feedback (Phase 5) and Further Prototype Development (Phase 6)

The proposed learning ecosystem prototype was presented to all co-design workshop participants for feedback via a user walk-through YouTube video shared via email. After viewing the video tour, participants were asked to complete an online survey consisting of closed and open-ended questions (Phase 5). Participant feedback then directly informed the second iteration of an interactive prototype (Prototype 2), which contained expanded content (Phase 6).

Multiple Stakeholder Feedback (Phase 7)

Prototype 2 was presented to a larger sample of teachers and autism educators to provide evaluation and feedback. This involved presentations at two showcase events, the national 2018 Autism Spectrum Australia (Aspect) Autism in Education Conference and the Brisbane Catholic Education (Springwood) Support Teacher Inclusive Education forum. In both cases, a live digital survey instrument, Mentimeter, was used for quantitative audience feedback via participants’ mobile phones. The researchers presented multiple user journeys across the prototype to test both the pedagogical integrity and design functionality. Participants were asked 19 questions across each presentation, and the results were published immediately on the screen to the audience, allowing for the groups to discuss and reflect on the findings. Semi-structured discussions occurred with groups at the end of each presentation.

Implement Stage

Platform Build with Ongoing Participatory Design: Establishing an Advisory Committee of Stakeholders and User Testing (Phase 8)

Following mixed stakeholder feedback, the formal build of the platform began. Integrated into this stage was an ongoing co-design component with stakeholders for further content and design development. This occurred through establishing a formal advisory committee consisting of multiple stakeholder group members. Participants elected to be involved through responding to calls made on the Autism CRC newsletter emails and Facebook page, as well as invitations to participants in prior co-design sessions and conference and presentation forums.

Participatory involvement in the committee occurred asynchronously via email, with selected sub-groups of members being assigned evaluation and creative feedback tasks related to particular content and designs being developed for the platform. Alongside collaborating with the Advisory Committee, on-site user testing also occurred with volunteer teachers in multiple states using a think-aloud protocol (Hanington & Martin, 2012).

Results

Within this section, findings and outcomes at different stages of the project are outlined to demonstrate the full application of the integrated co-design model. This is followed by a reflective evaluation of the process model.

Design Stage

Research findings from front-end stakeholder co-design workshops directly informed the development of Prototype 1. This prototype then became the focus of further participatory design with stakeholders, leading to Prototype 2 and the final project build. Through data analysis of the initial workshops, researchers identified participants’ key priorities, needs and preferences. From this, researchers extrapolated a series of key themes, which formed core design principles informing design:

• Strong foundation: A priority for a learning ecosystem is for it to be built upon a proven and explicit evidence base.
• Efficient discovery: It is essential for a learning ecosystem to provide the correct resources for a specific situation as soon as it arises.
• Supported implementation: For new teacher/classroom practices to be successfully engaged and implemented there needs to be a series of supports integrated into the system design. This can include videos with experienced teachers demonstrating the practices in action using the resources provided, as well as a practical guide format to step teachers through the iterative process of planning, implementation and reflection.
• Learning community: A community of learners is necessary for a learning ecosystem to thrive. Teachers desire to learn alongside peers with similar skill levels in a supportive environment, and there is a preference for sharing resources and classroom practices, as well as active, ongoing discourse around teaching approaches.

Further Initial Findings and Outcomes

An additional, definitive finding from this analysis was that all adult participants reported that teachers were required to meet the needs of a wide range of diverse learners, some of whom had needs similar to those students with autism. Thus, the implications and potential impact of the learning ecosystem is far broader. This finding led to an initial framing of the learning system as the Diverse Learners Hub and then later inclusionED, supporting diverse learners. These titles indicate the applicability of the system’s teaching/classroom practices to a diversity of students as opposed to only those with ASD.

In parallel with the finding described above, students with ASD expressed a strong preference for avoiding language and terminology directly referring to autism. Rather, a descriptive approach to student behaviour and abilities displayed in the classroom, not necessarily attributable to their diagnosis, was advocated for. This firmly established a need for the learning ecosystem to move beyond the dichotomy of special needs/disability and normal/mainstream students. In doing so, it was seen that the system could be transformative in terms of altering the mindset of other educators, and the broader community, and how they perceive learners in classrooms.

Prototype Ideation and Design

All findings related to core design principles, representation, and media preferences informed the development of a series of user journey maps for the learning platform and then a Future Service Map (displayed in Figure 6), all of which were used to develop wireframe prototypes. The iteratively developed Future Service Map displays a resolved design showing a teacher engaging in an online platform/learning ecosystem, as they select a teaching strategy and subsequently plan, implement and reflect.

Figure 6. Future service map for the platform developed from key findings. (Click on the image to see a larger version.)

A clickable digital prototype for an online platform was then created based upon the future service map, directly integrating the core design principles and findings that emerged as a priority across the co-design workshops. The prototype homepage is displayed in Figure 7.

Figure 7. Diverse Learners Hub Prototype 1.

In the prototype, the design principle of Strong Foundation was applied by documenting the underpinning research which informed teaching practices. Summaries were presented through infographics, and an accordion interface made additional research project details and findings easily accessible, along with links to relevant Autism CRC Knowledge Centre research reports and published journal articles. This mixed content allows for users to have a multi-layered, self-directed experience, wherein their accessing and understanding of the evidence base is dependent on their own needs and motivation. The design principle of Efficient Discovery was applied through the multi-functional design of the site search engine, which is used to identify relevant teaching practices to issues and problems. The search capacity for the database is also developed as usage continues. A further application of efficient discovery is the site’s ability to allow users to save what they are doing and return directly to the content.

The design principle of Supported Implementation was applied in the structural design and delivery model for all teaching practices. These practices form the bulk of content across the learning platform. All practices are presented in an identical, scaffolded format that takes a user through their application step-by-step and allows for reporting to and feedback from the learning platform. The scaffolded approach consists of five phases—explore, plan, act, reflect, and share—for all practices. Further, much practice content is video-driven, using short vignettes ranging from 30 seconds to five minutes, applying a show, don’t tell approach. These prototype design elements are displayed in Figure 8.

Figure 8. A classroom practice interface, with tabs to steps and short showcase videos.

Finally, the design principle of Learning Community was applied through users having an option to create a site profile and contribute directly to discussions based around specific practices. Users have the option of a profile being based on a real-world identity or pseudonym, and this flexibility promotes broader usage. Online discussions are designed to include the researchers who have developed practices, and they also have their own profiles in the system. The system also integrates formal recognition of practice application to teaching, with the generation of official certificates when accomplished.

Develop Stage

Participant Feedback

Following viewing video demonstrations of how the prototype operates (displayed in Figure 9), participants were able to provide survey feedback. In all survey responses, the platform design was positively assessed and validated by participants. The sole critical feedback entailed minor cosmetic changes to interface elements (e.g., a larger, direct link to the home page from each practice page, etc.). The multi-modal nature of the content, consisting of videos, text and images, which aligned with the diverse learning ethos of the platform, was highly praised. One respondent noted,

I am dyslexic so would prefer to watch an illustration of a practice video with the printed resource in front of me so that I can stop and start the process and make notes etc. for when I use the strategies myself.

Respondents strongly praised the structural design of the teacher/classroom practices, the contribution of the site and its depiction of diversity. Here a respondent noted,

I think it allows for diverse learning to be at the forefront of planning and the idea of a site dedicated to this adds value and emphasises the importance of catering to the needs of all students.

Figure 9. Screenshots from the Prototype 1 demonstration and evaluation video.

The positive feedback led to the development of the second prototype, which integrated further content from Autism CRC research into the tested format, with minor adjustments.

Multiple Stakeholder Feedback

As noted, the second interactive prototype was presented to a series of critical audiences. In two separate interactive showcase events, the 2018 Autism Spectrum Australia (Aspect) Autism in Education Conference (30 educators and parents) and the Brisbane Catholic Education (Springwood) Support Teacher Inclusive Education forum (80 participants including teachers and autism specialists), the live digital survey instrument, Mentimeter, was used.

Significantly, there was a clear alignment of participant responses across both events. Both groups indicated very strong satisfaction with the design of the platform and that it was a much-needed resource they would use often. The most suggested recommendations, which came from both groups, were for further use of authentic classroom demonstrations and student perspective videos in the practice’s media. The consistency in positive responses served as the final validation of the prototype design and feedback from the sessions was directly integrated into the build of the final version of inclusionED.

Implement Stage

Following this mixed stakeholder feedback, the formal build of the platform began. At this stage, the advisory committee was established to support ongoing stakeholder engagement in content and design development. Over 120 teachers and specialist educators volunteered for the committee and members provided feedback and input to iterative platform material asynchronously and via email. Committee members contributed heavily by reviewing media options and crafting the language used to best support teacher instruction. On-site user testing also occurred prior to a soft launch and provided direction for changes to the final tab system for all practices. Here the design was improved for usability across devices, and the labelling of steps was made more explicit to better direct users. Final designs for the inclusionED platform, now accessible online, are displayed in Figures 10 and 11.

Figure 10. Co-designed Netflix interface for users to explore content.

Figure 11. A co-designed teaching practice with a revised, scaffolded implementation format (right).

Reflections on the Process Model

The outlined results demonstrate the variety of outcomes which can be generated throughout the Integrated Co-design process and how these directly inform the following phases within it. The successful development of inclusionED, which was subsequently launched by Australia’s Minister of Education and is now being actively engaged and utilised in classrooms across Australia, as well as the USA, UK, China, and India, provides a body of evidence of the viability of this experimental co-design approach and its ability to solve wicked problems.

A summary of activities and outcomes in applying the integrated co-design model is presented in Figure 12. This includes the project timeline, along with key deliverables at critical points in stakeholder engagement and co-design. Within the grey sections, the red text indicates key co-design findings that defined project direction, while those in grey text indicate primary areas where multiple stakeholder collaboration provided a further key role.

Figure 12. inclusionED co-design process with key outcomes. (Click on the image to see a larger version.)

The initial process model’s success and the learnings from the project are integral to translating it to a model for future projects. The foundations of a future-focused concept model are outlined in the following discussion.

Discussion

Following developing and testing the integrated co-design process model, a subsequent phase has been to synthesise the project process model into a flexible model, which is transferable to different future scenarios and can guide designers. Entailed in this is seeing the applied success of the process model as a stepping-stone to theory generation (Zimmerman et al., 2010) and that a theory for design can be generated to improve the practice of design.

This subsequent iteration of the model serves as both a planning tool and a method to better ensure authentic and productive collaboration with multiple and diverse stakeholder groups across projects. The integrated co-design concept model provides a way to conceptualise, develop and apply an extended co-design process throughout a project, from initial scoping and ideation to ongoing collaboration. A summary of the concept model is presented in Figure 13. Notably, this iteration contains 4 overarching stages. The model’s key components are discussed in the following sections.

Figure 13. Integrated Co-design Concept Model.

1. Design for Co-design (3 Pre-Design Phases)

Within this Integrated Co-design concept model, a distinct stage, Design for Co-design, has been added to emphasise the necessity in specifically designing the co-design process itself, separately, before beginning the overarching project design process.

Co-design Planning

Of the 3 phases that make up the Design for Co-Design stage, the most integral is Phase 1: Co-design Planning. This entails addressing the two main challenges teams using co-design face: whom to involve and how to do this.

The integrated model prompts a full consideration of all potential stakeholder groups, either direct or indirect, for inclusion in a project. A scoping of participant groups is required, with consideration of each’s value and/or need in participating in a project. The model emphasises engaging all identified groups in the front-end of the design process, where problem scoping and concept ideation occurs to maximise their contribution.

For front-end engagement, a workshop format is likely to be preferred due to its established effectiveness and ability for real-time discussion and creative activities. Decisions can be made, however, whether these should be on-site, blended, online or a combination of types, and this is likely dependent on the background of participants and the project budget. It is recommended stakeholder groups remain separated in this phase to avoid issues arising from conflicting viewpoints and perceived hierarchies.

As aims for co-design workshops are identified, the project team iterates the tools, processes and techniques to use within sessions to best place participants at ease, build rapport, help understand lived experiences and support creative thinking. Essential in designing co-design is also considering the scheduling of early stakeholder workshops to best support a project. This was seen in the positioning of the policy-maker and educator provider sessions immediately following initial teacher co-design sessions in the inclusionED project. This allowed for the critical lens this group provided to be utilised as much as possible, across what was emerging from prior sessions.

In planning this holistic co-design process, the concept model can operate as a template to iterate the overarching structure including and beyond the front-end workshops. Consideration can be made about which stakeholder groups will be involved not just in the front-end, but in subsequent key phases and how.

Pilot Workshop/s

Following planning the overall co-design approach, prior to formally starting, is the opportunity to test the more experimental collaborative methods and to also engage stakeholder groups to plan aspects of future participation. These occur in pilot workshops. This allows a team to have any outstanding questions and uncertainty addressed, with pilot results informing the next iteration of a co-design plan. As well as testing experimental workshop methods these sessions may be used to understand specific stakeholder views and receive feedback on aspects of project design. This might include views on project framing, appropriate language to use, and identifying preferences for future collaborative methods (i.e., asynchronous or synchronous, types of technology used, etc. in later stages).

Preliminary Activities

This optional phase encompasses a range of forms of engagement with stakeholder groups, with all intended to serve an important role in supporting the project and participants before any formal, collaborative design occurs. Entailed here may be introducing the project and team to participants, prior to workshops occurring, potentially through videos or other media forms. This can be an effective strategy in developing a positive relationship with participants and heightening excitement prior to a project beginning. In the inclusionED project, the playful YouTube video sent to workshop participants set a specific tone for the collaboration. By the research team presenting themselves as people first and foremost, it dispelled a level of the hierarchy, and it presented the values and intention of the project in an upbeat and entertaining way.

Notably, an option also exists here to send introductory activities to specific participant groups, which may then support their involvement in workshops (e.g., a pre-workshop reflective exercise). The use of such activities, however, is very much determined by both the background of the participants and the nature of the problem being addressed.

2. Design

Front-end Co-design Workshops with Multiple Stakeholder Groups

As noted, the integrated co-design model prioritises optimal stakeholder engagement at the front-end, the benefit of which was seen explicitly in the inclusionED project. Essential to the platform’s success on a variety of levels (teacher engagement, organisational alignment, representation of youth, etc.) was the initial co-design sessions with different, key stakeholder groups, some of which were direct and others indirect (Woelfer et al., 2011). This elicited diverse domain expertise and ensured all relevant perspectives were considered and informed the initial project direction. The advantage of this approach is that it considered broader issues and potentially opposing viewpoints at project commencement, thus the design was less likely to be impeded or derailed further on. Importantly, early multiple stakeholder engagement also allowed for validated commonalities and complementary findings to strengthen the initial project vision and direction.

Despite these benefits, this type of front-end engagement can be complex and complicate a project’s direction due to the diversity and volume of data generated. To address this, the inclusionED team applied overarching design principles of continuity and consistency in conceiving all co-design workshops. This entailed researchers devising sessions with similar themed activities, generating similar forms of data. As a result, the data was easily comparable and synthesised into design outcomes. Significant differences in workshop design—what could be considered a customised approach—only occurred where it was clearly beneficial to deviate and in response to the needs and abilities of a specific stakeholder group (for example, collaborating with students with autism).

Blended & Virtual Co-Design Workshops

As noted, in employing front-end workshops, the use of blended and/or virtual co-design workshops, instead of conventional on-site ones, should be considered. A blended or virtual approach means participation is no longer limited by geographic location, allowing for a wider spread of participants and this can also greatly reduce project costs. Within this study, all blended co-design workshops proved equally successful in terms of design outcomes and participant feedback, as those held exclusively on-site.

While guidelines to support teams in delivering blended and virtual/online co-design workshops are still limited, this study demonstrates the viability of workshops beyond standard on-site approaches, particularly blended ones. In delivering such workshops, it was noted that it proved highly advantageous to have several workshop participants located on-site with the facilitator, rather than sessions consisting of only online participants and a facilitator. It was observed that an on-site presence served to ground sessions and orientate participants through the creative activities. On-site participants essentially operated as task demonstrators for online participants, with the facilitator able to interact directly with them and instruct on correct approaches (as seen in Figure 3). The success of the blended co-design model within this study has seen Author One apply the format across further projects and develop further approaches in online co-design workshops (Kelly et al., 2019).

Customizing Co-design Approaches

Of particular note, workshop design might not only entail adapting existing approaches from literature, but applying this knowledge to design entirely new methods for collaboration. An example of this within the inclusionED project was developing the use of visual personas as a co-design tool for collaborating with teenagers with autism. This approach was seen as a viable method for participants to reflect on lived experiences without provoking distress. Personas were designed to be visually engaging and used elements of cartoon imagery to create empathy and connection (McCloud, 1994). The cartoon design appeared to keep the overall tone of sessions light-hearted, even though serious issues were broached. The success of visual personas in this project indicates their potential as a co-design tool in other contexts, particularly where a stakeholder’s direct reflection may not be desired because it could provoke distress or trauma. This mindset of innovating and developing new methods is essential within this co-design model, especially to ensure the needs of participants are met and that particularly the voices of marginalised stakeholder groups are heard. Developing new, customised methods can assist a project to reach its full potential and avoid insubstantial engagement and tokenism.

Data Analysis

Following the front-end workshops is the process of analysing all artefacts and transcripts generated from sessions. Recommended here is to utilise researchers/analysts that have not been part of co-design sessions, as well as the facilitator who is more familiar with the nuances of what occurred in workshops. Both researchers and facilitators should start by working independently and iteratively, and established approaches such as thematic analysis can be used here alongside affinity diagramming as a process for concept building/solutions.

Prototype 1

The outcomes of the analysis are then used to inform and work towards a design prototype or prototypes that can be shared with all participants for concept validation and further feedback. These prototypes should reflect the combined findings from engaging the different groups and be developed through an iterative design process. A suggested approach to progress through this phase may be to hold design workshops where the core project team examines findings and develop prototypes together. These can also occur with the presence of external designers if additional expertise is required. The scale of prototypes should be considered, so overarching key conceptual approaches are captured, however, the design is not unwieldy in terms of participants being able to give feedback. Restricting the scope here will also be cost-effective. At this point, proof-of-concept videos may be a particularly user-friendly option to highlight prototype design/s to co-design participants.

2. Develop

Workshop Participants Feedback

This entails a check-in process with all co-designers from the different stakeholder groups. The aim is not simply to endorse a proposed solution, but to use the prototype as a provocation for further design directions and refinement. At this point, the project team is testing its interpretation of data and designerly responses with all participants.

A key recommendation here to avoid overburdening participants is to use asynchronous methods to gather feedback. In the inclusionED project, an effective approach was to use video and email to engage workshop participants and then allow them to provide their thoughts in their own time (in this case via an online closed and open-ended survey).

Prototype 2

After analysing responses from workshop participants, the proposed design is further developed into a more detailed, scaled-up prototype, integrating changes responding to feedback (both significant and/or minor). This prototype should more directly reflect the final project outcome (i.e., move beyond a proof-of-concept format), so that it may be scrutinised in far greater detail by representatives of stakeholder groups.

Multiple Stakeholder Feedback

This encompasses a final level of project-defining feedback, prior to building the final outcome. At this point, the team should identify avenues and opportunities to maximise stakeholder feedback, with the understanding that the larger this occurs at, the more certainty and clarity there can be for a proposed design solution. A consideration here should be made of potential online and in-person, synchronous and asynchronous stakeholder engagement methods. Different approaches may also be used for different stakeholder groups. Feedback should be specifically sought from representatives of stakeholder groups not previously aligned with the project, so they evaluate the proposed design without prior knowledge or vested interest in the outcome. It is suggested that a team might look at having mixed stakeholder groups in synchronous sessions, as a method to increase the scrutiny of design at this critical phase and bring to light any issues or barriers. An engagement method which proved valuable in the inclusionED project was the team’s development of interactive showcases, which were presented at conferences and other events that attracted mixed stakeholder groups.

4. Implement

Refine & Grow with Stakeholder Input

The final stage encompasses the production process for the design solution and collaborating with representatives of stakeholder groups in diverse ways to directly serve the outcome. In some cases, this may entail ongoing collaboration with stakeholders on a project once it is released in the public sphere (i.e., beyond project delivery).

Entailed in this stage for inclusionED was formal user-testing with direct stakeholders, as well as the establishment of a community of co-designers to contribute to finalising specific design aspects. This community operated as a formal network of volunteers, referred to as the inclusionED advisory committee, and members contributed to the platform through assigned independent, asynchronous activities. The advisory committee was structured so that participation was flexible and different members were engaged in different participatory processes to ensure no member was overly burdened. Establishing this committee structure and giving formal accreditation to participants proved successful in encouraging people to donate time for further collaboration on the platform. Again, at this stage, there is an onus on teams to develop new, agile methods for collaboration.

Future Directions

The integrated co-design model is recommended for teams as a way to streamline and manage the complicated process of building a project with a co-design ethos, and to prioritise authentically engaging diverse stakeholder groups. This model has potential for use across health, education, business, government, and service sectors where there is an aim to solve complex issues with communities, and it has potential for use in the development of objects, platforms, services, or systems.

Presently, Author One is continuing to explore the model’s potential across multiple research initiatives within the Woolworths Centre for Childhood Nutrition Research, where he is embedded in 4 long-term projects, all of which are based on a foundation of co-design. The model is currently being applied to address complex problems relating to food insecurity in low socio-economic families, and food literacy for schoolchildren, amongst others. In utilising the model, new approaches in the use of visual personas, blended and online co-design workshops and for asynchronous co-design are being developed and applied, therein extending its scope. Within all projects, the model has provided a valuable tool to think through, design and apply co-design involving diverse, multiple stakeholder groups.

Conclusion

This paper has proposed an integrated co-design model to holistically apply co-design with multiple stakeholder groups to solve wicked problems, and outlined how this was applied across a multi-year project. The series of outcomes outlined in this paper not only substantiate the model as a design approach for complex co-design, but demonstrates how this process operates in a pragmatic way. Further, the study translates its process into a concept model for future projects, which aim to incorporate the lived experiences and creative input of diverse stakeholders into the design of most suitable solutions.

The integrated co-design model involves adopting extended, low-cost and diverse methods of co-design across all stages of a project, from conception to beyond its implementation. It supports and prioritises collaborating with multiple, diverse stakeholder groups in innovative, authentic and agile ways, responding directly to participants’ needs and contexts. Integral to the model, supporting both participants and budget constraints, is applying collaboration options beyond standard in-person workshops, a long-established cornerstone of co-design, and utilising blended and online co-design approaches, as well as asynchronous collaboration. In doing so, it seeks to inspire researchers in the ever-growing field of co-design to pursue new possibilities.

Acknowledgements

The authors acknowledge the financial support of the Cooperative Research Centre for Living with Autism (Autism CRC), established and supported under the Australian Government’s Cooperative Research Centres Programme.

References

1. Australian Institute of Health and Welfare. (2017). Autism in Australia. Retrieved August 20, 2020, from https://www.aihw.gov.au/reports/disability/autism-in-australia
2. Australian Public Service Commission. (2007). Tackling wicked problems: A public policy perspective. Retrieved August 20, 2020, from http://www.enablingchange.com.au/wickedproblems.pdf
3. Benton, L., Johnson, H., Ashwin, E., Brosnan, M., & Grawemeyer, B. (2012). Developing IDEAS: Supporting children with autism within a participatory design team. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2599-2608). ACM. https://doi.org/10.1145/2207676.2208650
4. Benyon, D., Turner, P., & Turner, S. (2005).  Designing interactive systems: People, activities, contexts, technologies. Addison-Wesley.
5. Bowen, S., McSeveny, K., Lockley, E., Wolstenholme, D., Cobb, M., & Dearden, A. (2013). How was it for you? Experiences of participatory design in the UK health service. CoDesign, 9, 230-246. https://doi.org/10.1080/15710882.2013.846384
6. Davis, M. C., Challenger, R., Jayewardene, D. N. W., & Clegg. C. W. (2014). Advancing socio-technical systems thinking: A call for bravery. Applied Ergonomics, 45(2), 171-180. https://doi.org/10.1016/j.apergo.2013.02.009
7. Ehn, P. (1992). Scandinavian design: On participation and skill. In P. S. Adler & T. A. Winograd (Eds.), Usability: Turning technologies into tools (pp. 96-132). Oxford University Press. https://doi.org/10.1093/oso/9780195075106.001.0001
8. Francis, P., Balbo, S., & Firth, L. (2009). Towards co-design with users who have autism spectrum disorders. Universal Access in the Information Society, 8, 123-135. https://doi.org/10.1007/s10209-008-0143-y
9. Given, L. (2008).  The Sage encyclopedia of qualitative research methods. Sage.
10. Greenhalgh, T., Jackson, C., Shaw, S., & Janamian, T. (2016). Achieving research impact through co-creation in community-based health services: Literature review and case study. The Milbank Quarterly, 94(2), 392-429. https://doi.org/10.1111/1468-0009.12197
11. Hanington, B., & Martin, B. (2012). Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers.
12. Herriott, R. (2019). What kind of research is research through design. In Proceedings of the IASDR conference of the international association of societies of design research (pp. 1707-1716). Manchester Metropolitan University Press.
13. Herstatt, C., & Verworn, B. (2004). The ‘fuzzy front end’ of innovation. In T. Durand, D. Probert, O. Granstrand, A. Nagel, A. B. Tomlin, C. Herstatt, & H. Tschirky (Eds.), Bringing technology and innovation into the boardroom: Strategy, innovation & competences for business value (pp. 347-372). Palgrave. https://doi.org/10.1057/9780230512771_16
14. Holone, H., & Herstad, J. (2013). Three tensions in participatory design for inclusion. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2903-2906). ACM. https://doi.org/10.1145/2470654.2481401
15. Jiménez, T. C., Graf, V. L., & Rose, E. (2007). Gaining access to general education: The promise of universal design for learning. Issues in Teacher Education, 16(2), 41-54.
16. Kelly, N., Wright, N., Dawes, L., Kerr, J., & Robertson, A. (2019). Co-design for curriculum planning: A model for professional development for high school teachers. Australian Journal of Teacher Education, 44(7), 84-107. http://dx.doi.org/10.14221/ajte.2019v44n7.6
17. Lal, S., & Adair, C. E. (2014). E-mental health: A rapid review of the literature. Psychiatric Services, 65(1), 24-32. https://doi.org/10.1176/appi.ps.201300009
18. Langridge, R. P., Smith, S. P., Smithers, K., & Southgate, E. (2017). Participatory design with children and young people: An annotated bibliography. Retrieved August 20, 2020, from http://dice.newcastle.edu.au/DRS_5_2017.pdf
19. Lin, M., Hughes, B. L., Katica, M. K., Dining-Zuber, C., & Plsek, P. E. (2011). Service design and change of systems: Human-centered approaches to implementing and spreading service design. International Journal of Design, 5(2), 73-86.
20. Makhaeva, J., Frauenberger, C., & Spiel, K. (2016). Creating creative spaces for co-designing with autistic children. In Proceedings of the 14th participatory design conference (pp. 51-60). ACM. https://doi.org/10.1145/2940299.2940306
21. McCloud, S. (1994). Understanding comics: The invisible art. Harper Perennial.
22. Näkki, P., Bäck, A., Ropponen, T., Kronqvist, J., Hintikka, K., Harju, A., Pöyhtäri, R., & Kola, P. (2011). Social media for citizen participation: Report on the somus project. VTT Publications.
23. Orlowski, S., Matthews, B., Bidargaddi, N., Jones, G., Lawn, S., Venning, A., & Collin, P. (2016). Mental health technologies: Designing with consumers. JMIR Human Factors, 3(1), e4. https://doi.org/10.2196/humanfactors.4336
24. Ospina-Pinillos, L., Davenport, T. A., Ricci, C. S., Milton, A. C., Scott, E. M., & Hickie, I. B. (2018). Developing a mental health eClinic to improve access to and quality of mental health care for young people: Using participatory design as research methodologies. Journal of Medical Internet Research, 20(5), e188. https://doi.org/10.2196/jmir.9716
25. Ovaska, S., Leino, J., & Räihä, K. J. (2008). Studying user interest in social media sites. In A. Karahasanović & A. Følstad (Eds.), The NordiCHI 2008 workshop. New approaches to requirements elicitation & how can HCI improve social media development? (pp. 64-71). TAPIR Akademisk Forlag.
26. Reilly, D., Voida, S., McKeon, M., Le Dantec, C., Bunde-Pedersen, J., Edwards, K., Mynatt, E. D., & Mazalek, A. (2010). Space matters: Physical-digital and physical-virtual codesign in inSpace. IEEE Pervasive Computing, 9(3), 54-63. https://doi.org/10.1109/MPRV.2010.22
27. Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155- 169. https://doi.org/10.1007/BF01405730
28. Saggers, B., Carrington, S., & Harper-Hill, K. (2016). The Australian cooperative research centre for living with autism (Autism CRC): Supporting improved educational outcomes for students on the autism spectrum. CAISE Review, 4, 66-85.
29. Saggers, B., Klug, D., Harper-Hill, K., Ashburner, J., Costley, D., Clark, T., Bruck, S., Trembath, D., Webster, A., & Carrington, S. (2016). Australian autism educational needs analysis— What are the needs of schools, parents and students on the autism spectrum? Cooperative Research Centre for Living with Autism.
30. Sanders, E. B-N., Brandt, E., & Binder, T. (2010). A Framework for organizing the tools and techniques of participatory design. In Proceedings of the 11th biennial participatory design conference (pp. 195-198). ACM. https://doi.org/10.1145/1900441.1900476
31. Sanders, E. B-N., & Stappers, P. J. (2012). Convivial design toolbox: Generative research for the front end of design. BIS.
32. Smith, R. C., Bossen, C., & Kanstrup, A. M. (2017). Participatory design in an era of participation. CoDesign, 13(2), 65-69. https://doi.org/10.1080/15710882.2017.1310466
33. Steen, M., Manschot, M., & De Koning, N. (2011). Benefits of co-design in service design projects. International Journal of Design, 5(2), 53-60.
34. Tollyfield, R. (2014). Facilitating an accelerated experience-based co-design project. British Journal of Nursing, 23(3), 136-141. https://doi.org/10.12968/bjon.2014.23.3.136
35. Vines, J., Clarke, R., Wright, P., McCarthy, J., & Olivier, P. (2013). Configuring participation: On how we involve people in design. In Proceedings of the the SIGCHI conference on human factors in computing systems (pp. 429-438). ACM. https://doi.org/10.1145/2470654.2470716
36. Wake, S. J., & Eames, C. (2013). Developing an “ecology of learning” within a school sustainability co-design project with children in New Zealand. Local Environment: The International Journal of Justice and Sustainability, 18(3), 305-322. https://doi.org/10.1080/13549839.2012.748723
37. Wilson, C., Brereton, M., Ploderer, B., & Sitbon, L. (2019). Co-design beyond words: ‘Moments of interaction’ with minimally: Verbal children on the autism spectrum. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1-15). ACM. https://doi.org/10.1145/3290605.3300251
38. Woelfer, J. P., Iverson, A., Hendry, D. G., Friedman, B., & Gill, B. T. (2011). Improving the safety of homeless young people with mobile phones: Values, form and function. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1707-1716). ACM. https://doi.org/10.1145/1978942.1979191
39. Zelenko, O., Gomez, R., & Kelly, N. (2021). Research co-design: Meaningful collaboration in research. In A. Blackler & E. Miller (Eds.), How to be a design academic: From learning to leading (pp. 227-243). CRC Press.
40. Zimmerman, J., Stolterman, E., & Forlizzi, J. (2010). An analysis and critique of research through design: Towards a formalization of a research approach. In Proceedings of the conference on designing interactive systems (pp. 310-319). ACM. https://doi.org/10.1145/1858171.1858228

		International Journal of Design is a peer-reviewed, open-access journal. All journal contents are allowed to be shared and adapted in accordance under the Creative Commons Attribution 4.0 International (CC BY 4.0) License.