4_3201

Practice-Centered Design of an Anesthesia Medication Template to Reduce Medication Handling Errors in the Operating Room

Axel Roesler 1,*, Eliot B. Grigg 2, Lizabeth D. Martin 2, Faith J. Ross 2, Magnus Feil 1,3,
Sally E. Rampersad 2, Charles M. Haberkern 2, and Lynn D. Martin 2

1 Division of Design, School of Art + Art History + Design, University of Washington, Seattle, USA
2 Seattle Children’s Hospital, Seattle, USA
3 The Design School, Herberger Institute for Design and the Arts, Arizona State University, Tempe, USA

This case study examines the design process of the Anesthesia Medication Template (AMT), a low cost, standardized layout template for the staging and identification of syringes on top of the anesthesia cart in the hospital operating room. The project was conducted in a participatory design framework as co-design between physicians and designers. Led by anesthesiologists and immersed in the work setting as practice-centered design, design development addresses individual, environmental and organizational aspects of the hospital as high-stakes work setting. The layout of the AMT was developed based on observational studies and a cognitive analysis of medication management workflows, conducted by medical practitioners. The physical design and product development of the template reflects best practices in Interaction Design, Visual Communication Design and Industrial Design. The design was incrementally refined using a series of iterative prototypes that were evaluated with expert practitioners in simulation studies and in the work setting. The final design of the template was implemented and monitored in a long-term study. Four years of data since the introduction of the AMT document significant improvements in medication handling safety and the AMT has been adapted for use at other medical centers across the United States..

Keywords – Practice-Centered Design, Co-Design, Participatory Design, Interaction Design, Industrial Design, Medical Design.

Relevance to Design Practice – The case study offers a detailed view on how design can contribute to improving medical practices. It documents the effective, close collaboration between designers and practitioners in the design of cognitive support for high-stakes work tasks as situated design activity, immersed in the work setting under design. The study results could be of interest or applicable to design practices.

Citation: Roesler, A., Grigg, E. B., Martin, L. D, Ross, F. J., Feil, M., Rampersad, S. E., Haberkern, C. M., & Martin L. D. (2019). Practice-centered design of a medication template to reduce medication handling errors in the operating room. International Journal of Design, 13(1), 53-68.

Received January 20, 2018; Accepted March 27, 2019; Published April 30, 2019.

Copyright: © 2019 Roesler, Grigg, Martin, Ross, Feil, Rampersad, Haberkern, & Martin. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

*Corresponding Author: roesler@uw.edu

Axel Roesler is an Associate Professor and Chair of the Interaction Design Program in the Division of Design, School of Art + Art History + Design at the University of Washington. He received his Ph.D. in Cognitive Systems Engineering and M.F.A in Industrial Design from the Ohio State University and his Diplom in Industrial Design from Burg Giebichenstein, University of Art and Design Halle, Germany. His research focuses on workplace design in high-stakes settings and new interaction models in AR/VR. Recent research projects explore new interaction design concepts for the commercial flight deck, design frameworks for line of sight interaction in augmented reality and new interaction models for mobile and wearable computing.

Eliot B. Grigg received an MD from George Washington University. He then went on to complete an internship and residency in Anesthesiology at the University of Washington. He further completed a fellowship in Pediatric Anesthesiology at Seattle Children’s Hospital and later joined the faculty. He is currently an Assistant Professor of Anesthesiology and Pain Medicine at the University of Washington and specializes in anesthesia for spine surgery and pediatric pain and is the Medical Director of sedation. His research focus is healthcare workplace design and quality improvement. He is a nationally recognized expert in medication safety. 

Lizabeth D. Martin is an Assistant Professor in the Department of Anesthesia at Seattle Children’s Hospital where she is a pediatric anesthesiologist and Associate Director for Quality and Safety in the Division of Pediatric Anesthesiology. She received her M.D from the Dartmouth Medical School and completed her anesthesia residency at the University of Michigan. She trained in pediatric anesthesiology and safety and quality improvement at Seattle Children’s Hospital. Her research interests are system-based strategies that decrease the likelihood of errors in the anesthesia workflow and improvements in the anesthesia workspace to minimize the risk of medication errors and improve patient safety in the fast paced and high-risk operating room environment.

Faith J. Ross is an Assistant Professor and pediatric cardiac anesthesiologist at Seattle Children’s Hospital and the University of Washington. She studied medicine at the University of Connecticut and completed her anesthesia residency at the University of Pittsburgh Medical Center. She trained in pediatric anesthesia and pediatric cardiac anesthesia at Seattle Children’s Hospital and the University of Washington. Dr. Ross participated in the analysis and implementation of the AMT currently in use at Seattle Children’s Hospital. Her other research interests include muscle oximetry in congenital heart disease and the impact of malnutrition on outcomes after pediatric heart surgery. Dr. Ross is a director of the resident anesthesia rotation and develops educational resources for anesthesia trainees.

Magnus Feil is an Assistant Professor for Industrial Design at the Herberger Institute for Design and the Arts at the Arizona State University. He received his MFA in Industrial Design from The Ohio State University and a Diplom (FH) from the Hochschule für Gestaltung, Schwäbisch Gmünd, Germany. In 2017, Magnus was recognized as Honors Faculty by Barrett, The Honors College at Arizona State University. His research interests are focused on the critical examination, design and development of product systems to facilitate more meaningful interactions between people and technology with an emphasis on design as storytelling, re-envisioning of foundation design education, design for social good and design for tangible interactions.

Sally E. Rampersad, MB FRCA is an Associate Professor and Attending Anesthesiologist at Seattle Children’s Hospital (SCH) and the University of Washington. In addition to her clinical work, Dr. Rampersad is the Director of Quality Improvement (QI) for the Department of Anesthesiology and Pain Medicine. In this role, she has overseen projects to reduce infection rates, to increase the safety of medication handling and administration by anesthesia personnel and projects aimed at increasing the ability of anesthesia personnel to respond to emergencies. Dr. Rampersad is an instructor for Pediatric Advanced Life Support and regularly teaches nursing staff about emergency protocols. Wake up Safe is a patient safety organization, run by the Society for Pediatric Anesthesia (SPA). SCH were one of the founding members and Dr. Rampersad has been SCH’s Wake up Safe representative from the start. She serves on Quality and Safety committees at the SPA and at SCH.

Charles M. Haberkern, MD, MPH. is a Professor in the Department of Anesthesiology & Pain Medicine; Adjunct Professor, Department of Pediatrics, University of Washington School of Medicine. He received his BA from Yale University in 1970, his MD from Columbia University, College of Physicians & Surgeons in 1974 and his MPH from Harvard University in 1997. Boards: Pediatrics (1978); Neonatal-Perinatal Medicine (1979); Anesthesiology (1986); Pediatric Anesthesiology (2013). His professional interests are Pediatric-Neonatal Perioperative Medicine, Safety and Quality of Perioperative Care, and Health Services.

Lynn D. Martin, M.D., M.B.A. is the Medical Director, Continuous Improvement and Innovation at Seattle Children’s. He is a Professor of Anesthesiology and Pediatrics (adjunct) at the University of Washington School of Medicine. He received his M.D. from the University of Washington and his M.B.A. from the University of Tennessee. He completed residencies in both Pediatrics (Phoenix Children’s Hospital) and Anesthesiology (Johns Hopkins University), followed by fellowships in pediatric anesthesiology and pediatric critical care medicine at Johns Hopkins University. He is board certified by the American Board of Anesthesiology with specialty certification in pediatric anesthesia and the American Board of Pediatrics with specialty certification in pediatric critical care medicine. Dr. Martin has served as the president of the Society for Pediatric Anesthesia. His original research interests involved conventional and non-conventional forms of mechanical ventilation and more recently focus on quality improvement, patient safety and operative outcomes research.

Introduction

Design in the medical setting is situated in a high-stakes, expert work setting that is practice driven. Critical aspects for design reside in medical practice that unfolds in a physical and social work environment. Practice is driven by the professional and social aspects of collaboration, enabled by technology and constrained by the organizational structure of the workplace. New design needs to integrate into existing work operations that rely on effective collaboration across the medical team, are fast paced, high workload and high risk. In this high-stakes work setting, experts with extensive domain knowledge, skill sets and experience rely on sophisticated technology to do their work (Hollnagel & Woods, 2005). Designing this technology requires deep insight into practices in the work setting that cannot be obtained by designers without close collaboration with medical practitioners during the design process.

This case study captures the design process of the Anesthesia Medication Template (AMT), a standardized layout for the staging and identification of syringes on top of the anesthesia cart in the hospital operating room. The AMT was designed in a collaboration between physicians at Seattle Children’s Hospital and faculty of the Division of Design at the University of Washington to improve the safety of anesthesia medication management in the operating room.

The AMT is a low-cost medication layout template that is placed on the top of the anesthesia cart, the work place of the anesthesiologist at the head of the operating table. The design of the AMT is the result of a detailed examination of anesthesia management practices in the workflows between anesthesiologist, patient and anesthesia cart, where medications are staged for use during surgery. The aim of our design team was to design the AMT from the perspective of the practicing anesthesiologist to serve as a cognitive aid during medication selection and administration.

Access to the work environment and support and design collaboration with practitioners within the design setting are critical to enabling designers to acquire domain understanding and gather insight into practice in high-stakes design projects. Immersion of the design work into the work setting by establishing a collaborative design process between practitioners and designers on site is an important requirement when we compare design in high stakes settings with conventional commercial design contexts where designers work externally and conduct field studies such as ethnographies (Wasson, 2000; Dourish, 2006) or contextual inquiries (Beyer & Holzblatt, 1997) to acquire expertise in the design setting from the outside. During a design ethnography process, the designer, ideally under guidance of an anthropologist, observes the design setting in the field with the goal to understand the activities and interactions of people in their everyday life settings and to capture interesting interactions and inspire design. Design ethnographies have been successfully applied to identify design opportunities in everyday environments, but design ethnographers tend to separate observations from the design process and are constrained by the subject position of the ethnographer in the relationship with the observed setting (Dourish, 2006). Both designers and observers separate themselves from the design situation under observation. This can lead to a bias of the design ethnographer as they observe through the lens of a need for design where practitioners in the field under design often do not see the need for things to change (Norman, 2013).

In high-stakes environments such as medical settings, design ethnographies are further limited by the pressures and demands of the work under observation. Practitioners who work on risk-prone operations in saturated environments such as the hospital operating room can perceive the presence of observers in the work setting as disruptive. Traditional designers are often not prepared to conduct observations in these settings, because they are not trained in the operations under observation or are not familiar with the social and organizational structure of the work setting. These situated design aspects include roles, hierarchy of decision making, responsibilities and limits of operation (Hutchins, 1995a).

In technical ethnographies using field observation techniques that were developed for critical operations in technology driven environments (Hutchins, 1995b; Suchman 1987, 2002), the ethnographer has domain knowledge and is trained in the psychological, social and organizational aspects of the work setting under observation. Additional process tracing techniques such as cognitive task analysis, protocol analysis and other retrospective techniques (Hollnagel & Woods, 2005; Klein & Militello, 2001; Ericcson & Simon, 1993) can be incorporated to study work in context. However, in this more structured observational research setting, the observer’s task is again to capture and describe, not to intervene. The observer stays distant.

Because of this distance, structured field observation techniques are external design research techniques that produce insight and inspiration for design, but do not involve the observed practitioners actively in the design work. The resulting line of separation between designers and practitioners can prevent direct conversations in the work setting that explore the potential of design interventions. Although this separation does not often cause problems in design ethnographies in everyday settings where the designers bring a similar everyday knowledge to the site as the subjects under observation, it can be critical in high stakes settings where practitioners engage in complex procedures with nuances that are often undetectable by non-experts (Roesler & Woods, 2007).

For the design of the AMT, we applied an integrated design approach. Practitioners and designers conducted observation and design development together, embedded in the work setting under design (Suchman, 2002). In the design field, integrated design approaches include participatory design (Bjerknes, Ehn, Kyng, & Nygaard,1987; Greenbaum & Kyng 1991; Sanoff, 2007; Kensing & Greenbaum 2013; Luck, 2018) and co-design (Sanders, 2000; Sanders & Stappers, 2008; Holmlid, Mattelmäki, Sleeswijk Visser, & Vaajakallio, 2015). Developed in Scandinavian design practice in the 1970s, participatory design focuses on responsible and appropriate design development in work settings by bringing all stakeholders in the design together to identify the possibilities and constraints of the design. Although the concept of design participation is widely accepted, its realization has often turned out to be difficult, in part due to divergent interests and different levels of power among the various stakeholders in the design development process (Luck, 2018).

It is critical how the design is initiated. When the design initiative comes from within the design setting, cooperation between practitioners and designers has a better foundation. The equal footing of designers and practitioners enables design collaboration. Co-design extends the collaboration between stakeholders and designers past the identification stage in participatory design. In co-creation, designers and stakeholders in the design collaborate in creating a desirable future by both identifying promising design change and developing design concepts together (Lee, Jaatinen, Salmi, Mattelmäki, Smeds, & Holopainen, 2018). However, co-design is constrained by the complexity of the design setting and the corresponding training of the designers. With conventional design education backgrounds that focus on the design of consumer products, the designers might not be prepared to work with practitioners in expert domains to acquire the insight into the design context that is required to responsibly innovate existing practices. The designers might be met with skepticism by the practitioners in the work setting who are aware of the dangers of miscalibrated design and its potential to interrupt ongoing work practices.

In the following design case study, we report on a participatory design process that was adapted to the specific design conditions of a high stakes work setting in the hospital operating room.

Practice Centered Design

Practice is a situated activity in a social, physical and organizational context that needs to be considered when we envision, realize and evaluate design interventions that will alter the work setting. Practice-centered design takes into account the expertise of the practitioners present, the dynamics of the work setting and the physical, social and organizational context of work to support understanding, reasoning, decision-making and action-taking in response to changing situations. This response is constrained by resources at hand at the time of need (people, expertise, technology and time).

The goal of practice-centered design is to realize design outcomes that will support practice at the intersection of people, technology and work in a meaningful way so that it makes sense for practitioners to integrate the new design into their ongoing practice. To do this, the designed artifacts need to be useful, understandable and usable to improve existing workflows so that the design will be accepted and used without getting in the way of established practice (Woods & Christofferson, 2002).

Practice-centered design has originated in Cognitive Systems Engineering, a Human Factors engineering field that studies innovation at the intersection of people, technology and work (Woods & Dekker, 2000). Cognitive Systems Engineering was established as a research field in response to the Three Mile Island nuclear accident in 1979 (Roesler, 2009) when the inappropriate design of the control room led to the overtaxing of the control room operators, resulting in a loss of control of the nuclear reactor during a mechanical failure. Comprised of psychologists, systems engineers, designers, and work studies and field observation experts, the Cognitive Systems Engineering community has played a major role in developing reliable control systems and resilient operations frameworks for high-stakes work environments in medicine, aviation, space exploration, military and process control.

Practice-centered design integrates structured observation and analysis of work settings in high-stakes environments with collaborative design techniques in a participatory design setting that brings practitioners, innovators and technologist together to develop new approaches to cognitive work (Roesler et al., 2005). Frameworks for this design approach have been developed in Cognitive Systems Engineering research areas such as Naturalistic Decision Making (Klein & Zsambok, 1996), Joint Cognitive Systems (Woods & Hollnagel, 2006) and Macrocognition (Klein &Wright, 2016). In the design community, Stappers and Norman (2015) recently proposed Cognitive Systems Engineering methods for the design of complex sociotechnical design challenges in Health Care and organizational design.

Case Study

Background

Medical devices are often developed outside of hospitals, based on new technologies that become available and which promise to improve work, but they are seldom developed from inside the medical work setting in response to the needs of medical practitioners. The new devices are then introduced into the work setting where they may interrupt work flows rather than improving them. Medical practitioners often do not advocate for the new systems because they were not involved in the design. They were not asked if the new equipment could be helpful and as a result they do not identify ownership of the new design and question why they should support it (Cook & Woods, 1996).

This design development model is decoupled from practice and has traditionally focused on the design of larger scale technology systems: life monitors, infusion devices and imaging technology. From the side of the medical equipment manufacturers, design strategies behind these devices address the hospital administration as customer and often not the needs of the medical practitioners because they do not make purchase decisions. Regulatory expense and complexity in health care are also hurdles to creating innovative devices or iterative improvements to existing ones. The industry-driven and technology-centered design and development approach for medical equipment makes it difficult for medical practitioners to become involved in early design stages where they could provide insight and critical input into what products should be developed. The model also falls short in examining the potential of design innovation for smaller scale design projects that address issues that emerge in day-to-day medical practice.

The issue that the present case study addresses had surfaced in 2011, when a series of medication handling incidents in a number of hospitals initiated a wave of internal medication handling safety evaluations at various hospitals around the world. One such internal evaluation of medication handling safety practices was conducted by physicians at Seattle Children’s Hospital to survey potential improvements in anesthesia medication work flows in the operating room.

The analysis of findings from their observations in the operating room and conversations with colleagues about anesthesia practices was driven by the hypothesis that vulnerabilities exist in current anesthesia medication management practice and that aspects in the routine work flow could be improved by integrating new forms of structure, thus making the work more observable and deviations more easily detected. Omission or swap errors would stand out so that they would be noticed before they could cause harm. One goal of this process was to identify leverage points for design intervention—areas where minimal change would yield significant positive impact.

In contrast to the large-scale projects initiated by medical suppliers from the outside, this was a design opportunity that was discovered by medical practitioners who had critically examined their own work practices, work environment and the organizational structure that had governed operations. The framing of the design opportunity at the time as a new form of structure that would guide medication handling pointed to a small-scale design intervention that that could be developed from within the medical work setting.

The composition of our design team with physicians and designers was crucial to the success of the design. Design through the lens of the practitioners was possible because the design of the AMT was initiated by anesthesiologists who had conducted the internal medication handling safety study. The physicians had championed the design project through the hospital organization and due to their organizational status as key expert practitioners, they were able to generate broad support to create a unique design environment that provided the design team with access to examine key aspects of anesthesia practice

We had previously collaborated on a successful medical design project (Grigg et al., 2013) and we had a strong mutual understanding of each other’s expertise. There was strong advocacy for design on the side of the medical experts in our team and there was expertise in the design for high-stakes environments in aviation, medicine and process control on the side of the designers who had secondary backgrounds in Cognitive Systems Engineering that complemented their Industrial Design expertise. The physicians on our design team contributed their expertise in anesthesia practice to the design task. They pointed the designers to interesting aspects in existing workflows, were able to detect nuances during observation that would go unnoticed by the designers and would reach out to the appropriate experts in their organization to address design questions. The anesthesiologists were the insiders of the processes under the design lens. The outsider perspective of the designers helped the practitioners to reveal previously unnoticed design opportunities in their practice by connecting the design work at hand with similar design situations in aviation and process control where design had provided cognitive support.

Our collaborative design process was driven by the cognitive and practice-oriented aspects of expert work: the conceptual models of expert practitioners, their knowledge, reasoning and decision-making strategies in light of promising and appropriate opportunities for change by design (Woods & Roesler, 2007). Applying a practice-centered design approach, our aim was to support ongoing practice in the work setting with the intent to support practitioners in what they are doing best.

In terms of the expertise and design techniques that the designers contributed to the team, the AMT was developed by a combination of Industrial Design, Interaction Design and Visual Communication Design techniques.

Table 1 illustrates how the different types of design expertise were combined and distributed to address the design requirements for the AMT that we encountered during the process. Design decisions were based on observations of practice in the medical work setting, direct design input from and co-design with practitioners, iterative development, testing and evaluation of the design in the work setting, and the application of best practices in design from similar high-stakes domains in aviation and process control.

Table 1. Design constraints, considerations and relevant design expertise areas for the design of the AMT.

Initial Framing of the Design Project

During this first stage of the design project, our team began to examine anesthesia medication workflows to identify promising areas for design improvement.

We studied the arrangement and use of medications on the anesthesia cart in the operating room. The anesthesia cart contains all medication preparation supplies and pre-drawn medication syringes for the anesthesiologist. In addition to airway management, the course of an anesthetic is largely determined by a carefully orchestrated sequence of medications, many of which have profound physiologic consequences and must be precisely dosed. This occurs in a dynamic and fast-paced operating room setting where there are often distractions and concurrent tasks for the anesthesia provider to perform. Before and during an anesthetic, providers prepare medications in plastic syringes that are arranged on top of the anesthesia cart. Currently, the manner in which providers organize and interact with medications on the anesthesia cart top is highly variable across institutions (Figure 1) and lacks visual design organization and rigorous input from human factors and interaction design.

Figure 1. Different medication arrangements on the anesthesia cart captured during field observation.
In the absence of a layout, anesthesiologists have developed their own medication arrangement schemes. These arrangements were handed down from the different medical schools, training and practice sites or are the result of personal work style.

These observations revealed variation in arrangement schemes and it became clear that developing a standard organizational structure for syringe arrangements on the cart top would be a promising design opportunity. The historic anesthesia cart and medication syringes are the result of unconnected industrial design efforts. The arrangement of the anesthesiologist and the patient, as well as the arrangement of the anesthesia cart and life monitor have been a standard in surgical practice for more than forty years (Drui et al., 1973). Some syringes will be used only once; others will be used multiple times throughout a case; still others sit largely unused over the course of a day of multiple cases and are only there in case of emergency. Syringes come in multiple sizes and are typically adorned with color-coded labels. The consequences of selecting the wrong syringe can be severe.

The design hypothesis was that a standardized medication layout on top of the anesthesia cart could reduce medication mix-ups and omissions. This design hypothesis helped to frame subsequent field observations in the operating room. The design project for the AMT was initiated. The team began to examine the anesthesia workplace as a fast paced and high stakes interaction setting with the intent to improve workflows by providing cognitive support for the appropriate selection of medications to reduce medication mix-up errors.

In the course of the following three years, the design and refinement of the AMT cycled through observation, ideation and realization in a quest for obtaining fit between design possibility and existing context. This process is a concurrent design activity that converges three types of expertise: expertise as practitioner in the field under design, expertise in design and expertise in product realization and integration into the field of work (Roesler et al., 2005). Figure 2 provides an overview of the different stages in the design process.

Figure 2. The design process moves from the 3 o’clock position in the diagram counter-clockwise.
Call-outs 1-9 refer to the various design stages described in the following.


Design of the AMT Based on an Analysis of Interactions

The design of the AMT was developed by carefully analyzing the interactions between anesthesiologists, anesthesia medications, anesthesia cart and patient. These interactions would frame the use of the AMT and in turn shape its physical layout for the arrangement of medications.

In March 2012, the anesthesiology team at Seattle Children’s Hospital initiated a design exploration of a unified arrangement of medications on the anesthesia cart. To test different layout concepts, a series of PowerPoint layout drafts were printed and laminated to serve as syringe arrangement templates at full scale. These layout sheets allowed the physicians to experiment with syringe arrangements on the cart top in an authentic work setting configuration in terms of height, viewing distance and lighting conditions. They were able to determine basic field arrangements, order and field sizes to accommodate the different size syringes. The early prototypes were used to facilitate feedback from other anesthesiologists. This stage of design benefitted greatly from the inside perspective of the anesthesiologists—leading to insights that were grounded in the culture of the work setting, encompassing social roles, tacit knowledge, individual preferences, personal styles and expertise in anesthesia. The prototypes facilitated conversations about the design as knowledge exchange between colleagues rather than a collision of insider knowledge and outsider insights that is typical when external design teams present to practitioners. Because the design project was owned by the practitioners internally, the design team was able to connect with other practitioners in the work setting, often impromptu and informally. This resulted in fast and direct feedback gathering based on existing relationships in the work setting that was grounded in professional respect and mutual trust. Insights included approaches to learn and modify distinct workflows and strategies to provide support structures for critical situations in the workflow.

Table 2. Interaction design constraints and design considerations.

In a progression of several prototypes, the team designed layouts to record their understanding of medication arrangement patterns that would be favored by individual practitioners, then the team extracted common patterns they had observed in various individual layouts (Figure 1) to develop a generalized template. The greatest challenge in this layout design effort was to make the simplest design possible to be most effective. Observations of typical workflows revealed that less visual clutter would result in a more reliable organization system. The team realized that in order to make the design widely applicable, they would have to provide free space so that this space could be used flexibly.

Early prototypes included dedicated fields for a variety of medications, many of which were commonly used. For example, the international standards organization ASTM International identifies a total of 13 medication classes, but the template that resulted from the team’s contextual inquiry only directly addresses 6 of them. This change of design requirements and constraints would have been difficult to push for an external design team, but because of their embedding in the work setting, the physicians had a clear case to make the design call for the simplified layout that is key to the final design concept. Half of the real estate on the template is free space to be used for medication preparation or any syringes with medications not identified in the dedicated fields. This provides anesthesiologists with adaptive space to create individual arrangements for low-risk medications. Future iterations intentionally removed additional fields for commonly used medications that are seldom problematic. Propofol, for example, is used in almost every case, but because of its distinctive white color and relatively safe side effect profile when used by an anesthesiologist, it does not require a dedicated space. Similarly, most anti-nausea medications do not need dedicated spaces because the likelihood and consequences of medication errors with them is relatively small. Conversely, high-risk medications (including muscle relaxants, vasopressors and opioid medications) as identified by the institute for safe medication practices (Institute for Safe Medication Practices, 2012) were designated a specific location on the template to sequester and standardize their location within the anesthesia workspace. Figure 4 shows the medication template design that would form the basis for the visual design detail work that followed.

For the detail design development of the AMT, the Seattle Children’s Hospital team partnered with faculty in the Division of Design at University of Washington. The intent of the detail design stages (positions 4-8 in Figure 2) was to provide the best design possible to support the work of the practitioners in the form of a design that is minimal, provides clear visual information, guides interactions and aligns with the anesthesiologists’ conceptual model of the placement and role of the various medications in the layout so that various types of medications can be distinguished during fast paced work situations. The aim for the design was to provide cognitive support, to aid the practitioners’ decision-making and support them in structuring workflows based on visual patterns and detect irregularities in these patters when medications are missing or misplaced.

Visual Design of the AMT Layout and Labelling

Based on the identified layout requirements (Figure 3), the team developed the first detailed visual design concepts of the template (Figure 4) with defined typography, line attributes, use of color and balanced visual layout of the medication arrangement fields. The design also took into account the presence, sizes and orientation of the 3-dimensional syringes on the cart top in relationship to the 2-dimensional layout template to optimize identification of color-coding and readability of medication labels (Figure 6). The detailed visual design defines the formal aspects of the template in order to create the desired hierarchical presentation of information to support decision-making.

Table 3. Visual communication design constraints and design considerations.

Figure 3. Before visual design: AMT draft developed by the research team at Children’s Hospital during the analysis of field observations and review of best practices for medication use and arrangement.

Figure 4. After visual design: Detail design of the first version of the AMT for the anesthesia cart top based on an interaction design analysis of use and readability and the layout draft shown in Figure 4.

Color-coding became a critical element in the visual design process. The background of the AMT is white in order to better contrast with the highly saturated colors. The ASTM color scheme used for medication identification in the hospital setting uses several colors at full saturation that are known in visual design to be difficult to balance in equal size areas, for example, yellow and blue. To resolve this issue, we opted for colored outlines rather than solid colored fields because different colors have a different presence when presented in areas of equal size (Albers, 2006). For example, only a small area of yellow is required to match the presence of a much larger field of dark blue (Itten, 1993). An earlier design variation (shown in Figure 5) illustrates the different look of solid color fields and the quite different visual hierarchy that is the result of this.

Figure 5. Design variation of the cart top template layout concept shown in Figure 5 with solid color fields. This figure illustrates the problematic contrast between color-coded fields that creates unwanted visual hierarchy; note how the yellow and blue fields dominate the layout, the problematic readability of the labels and the unwanted contrasts with the white background that emphasize field shape over field content.

In order to balance the visual organization of the cart top template at different levels of syringe placement density, the impact and presence of color has to be taken into account for both instances when the fields are empty versus when they contain syringes (Figure 6). The intent of the color-coded fields in the template is to match the colors of the medication labels present with the colors of the field frame. Mismatches should stand out visually as color contrasts (Christ, 1975; Ware, 2012). The syringes are white/translucent, minimizing the contrast between syringe body and background emphasizing the small color ring on the syringe label and supporting the detection of color contrasts if syringes are misplaced.

Figure 6. Syringe / template contrast and color label identification on a white background with colored field outlines.

If the fields were solid color, their background color would dominate the syringe labels that would be placed in the fields and differences would be difficult to detect. The more discrete colored outlines in the final template design create a better contrast with each other and focus the visual attention on the whitespace in the center of the field and with this emphasize the medication/syringes contained in the various fields. Here, Figure 4 shows a design concept variation while Figures 6, 8 and 9 show the final design.

We opted for a white template background despite initial concerns about light reflection issues from the strong operating room lighting because nuances of grey in the template background would likely create a specific kind of contrast between the brightness of the grey and the similar brightness levels in the cyan and yellow tones of the field outlines. This type of color contrast is known as simultaneous contrast (Ware, 2012; Itten 1993) and would produce a vibrating visual illusion where the colors meet, resulting in an unintended visual emphasis on this area of the template. The reflection of light from the white background through the transparent syringes provides additional emphasis on the color labels on the syringes.

Text labels provide a detailed identification of each medication field. Typeface and size of the labels were designed to be instantly readable from different distances typical in the work setting (Degani, 1992). The medication labeling uses the typeface Helvetica, considered by many designers as the best sans serif typeface in terms of objectivity and readability (Müller, 2002). For the label design of the AMT, Helvetica Light is used for standard labels and Helvetica Bold is used for emphasis. Font size for readability depends on the ratio between information area and reading distance (Aicher, 1988; Kunz, 1998; Hofmann, 1965; Müller-Brockmann, 1968). The same font size might be perceived as larger when placed in a small area and smaller when placed in a large area. Good fit is perceived at the right visual font size in the given area. The font size for the AMT was chosen based on reading tests from different distances at authentic template placement height and lighting conditions to optimize for pattern-based scanning of labels and quick readability in context with the medication field outlines. As a result of the careful adjustment of type size to template area and amount of text and the adjustment between label size and medication field outline weights, the font size in the final design is smaller than earlier iterations of the template (Figure 3), however the readability is better as the font size now appropriately relates in scale to the overall size of the template, the proportions of the various template fields and the viewing distance. The labels are placed in the outline of the field so that when a syringe is in place it does not obstruct the ability to read the label. Another difference in the typography in Figure 4 compared to Figure 3 is the abandonment of center alignment for label organization in favor of left alignment. Left alignment provides faster and more accurate scan-ability of text and labels as the eyes are drawn to a consistent start of lines in the left alignment of the paragraph (Aicher, 1988).

Working with the higher fidelity prototypes in the detailed visual design, we conducted further observations of the spatial workflow between anesthesiologist, anesthesia cart and patient to optimize medication arrangement. The overall strategy used to determine whether a medication required a dedicated space was borrowed from the reliability engineering Failure Mode and Effects Analysis (FMEA) framework where risk is a combination of probability, severity and the likelihood of detection (DeRosier, Stalhandske, Bagian, & Nudell, 2002; Martin, Grigg, Verma et al., 2017). This resulted in the arrangement of medications shown in Figures 8 and 9.

Industrial Design of the AMT

In this stage, we refined the 3-dimensional aspects of the AMT as the design would gradually evolve from a flat laminated template print to a 3-dimensional physical artifact on top of the anesthesia cart. We designed the form of the AMT in correspondence to various materials and fabrication processes that would allow us to produce the AMT in a small series production run.

In June 2013, after reviewing additional medication handling flows with the refined 2-D laminated template, we opted to introduce additional 3-dimensional hierarchical cues to the template design in the form of raised and recessed placement zones. Emergency medications, such as epinephrine, atropine and succinylcholine, although not commonly used, can have disastrous consequences if used incorrectly or not available when needed. Hence, they literally have an elevated position at the back of the template. Neuromuscular blocking agents and opioids can both have very morbid consequences and are fairly likely to be confused, so they both have dedicated spaces. Succinylcholine and the other neuromuscular blocking agents are not directly in front of one another because they use the same ASTM red color and could be confused even if given dedicated spaces. Distance of reach forms another hierarchical structure: The more likely a medication is to be used, the closer it is to the front of the Template and the closer and more accessible to the provider. Less frequently used medications are further towards the back.

Grouped fields create separation between different classes of medications. Empty fields carry associative meaning. The antibiotic field serves dual purposes: (1) to help avoid a syringe swap with other similarly sized syringes with white labels and (2) to serve as a reminder that antibiotics may have not been given when the space is empty. Even the lack of anything in the space can convey important information. The flexible space in the front of the template can be used as the practitioner chooses. This flexible space preserves individual staging strategies shown in Figure 3 while contextualizing them in the uniform layout. Thin guidelines encourage the orthogonal alignment of resources, but there are not physical restrictions to organizing materials. Large syringes (50ml or greater) can fit in this space, which can be used for medication preparation; it can even be used for other types of supplies other than syringes (e.g. needles, caps). The flexible space appears simple, but it actually is equally important as the rest of the template. It encourages user acceptance by allowing some flexibility, helps to reduce visual clutter and allows for the inevitable circumstances not anticipated by the designers of the template. Other versions of the basic template have modular pieces for some added complexity: a cardiac and transplant module adds fields for epinephrine, calcium and phenylephrine. Even with this variation, however, the high-risk area is removed, although the vast majority of the template remains the same so that the overall layout is consistent across all Templates in different locations. This standardization allows providers to become instantly familiar with every Template and the location of medications in any location, even when asked to assist a colleague in another room during an emergency or during routine breaks or handovers. Intentionally, only two types of templates are in use at Seattle Children’s Hospital to limit variation between different versions.

In order to function in a fast-paced health care setting, the template has to be durable, flexible and reliable. Durability is both a matter of providing adequate infection control and standing up to daily physical abuse. Flexibility is required both to function across multiple care settings and case types and also to accommodate multiple providers and different practice styles. Reliability is a matter of functioning equally well during long periods of minimal activity where monotony and vigilance are concerns to emergent situations where distraction and confusion are the primary challenges. In order to remain reliable over time, especially in an academic medical setting with frequently rotating inexperienced trainees, the template also has to be intuitive to the point that no training is required to operate the device and be designed such that it is difficult to use incorrectly.

The design of the template began as a 2-dimensional “shadow board”, but eventually became a 3-dimensional structure to accommodate movement of the cart and prevent misalignment of the medications as a result of vibrations during relocation of the anesthesia cart. The 3-dimensionality both provides an additional, tactile way of differentiating areas and it physically prevents users from aligning syringes incorrectly. The emergency medication enclave, for example, exists on an elevated plateau, which further distinguishes these high-risk, low-use medications from the rest of the template (Top-right corner of the template in Figure 8).

For infection control a 3-dimensional template has to be free from any sharp transitions, angles or small pockets that would be difficult to clean. It also has to withstand a variety of hospital-grade equipment disinfecting agents including Isopropyl Alcohol, Sodium Hypochlorite (“bleach”) and Benzalkonium Chlorides (common in germicidal wipes). It has to be made of a non-porous material that could harbor pathogens and be water and stain proof to avoid damage from medications and other spills. The template needs to be light enough to be easily transferred from one location to the next, but also substantial enough that it does not move around during use. Ideally, the template has a subtle texture to prevent syringes from moving around, but not such a pronounced texture that it would interfere with effective cleaning.

To meet the cleaning and durability requirements in medical settings, plastics and stainless steel are common materials used in health care. Steel is cost-prohibitive, heavy and difficult to manipulate, so plastics were the obvious choice for the template. Given the complex 3-dimensional structure of the template with multiple rounded edges and corners, the materials used had to be easily milled and assembled. The initial 3-dimensional prototype was a simple Acrylic Polymethyl Methacrylate (also known as Plexiglas) cut-out that created dedicated cells for syringes, but only had a single degree of depth (Figure 7). The Plexiglas template would be placed on top of an inexpensive 2-D print that was laminated.

Table 4. Industrial design constraints and design considerations.

Figure 7. Laser cut transparent acrylic overlay to prevent medications from shifting locations when the anesthesia cart is moved.

Figure 8. The final anesthesia cart top template as it is in use in the operating room.

The first true 3-dimensional prototype was made of painted wood, a material not appropriate for patient care. For the more complex final shape (Figures 9, 8 and 6), the next prototype was made with polyurethane (PUR). For a small batch production of this design, an aluminum version of the template was milled and used to make a silicone negative. Polyurethane was then poured into the silicone negative to create the final shape. The colored outlines around each cell were narrow channels filled with colored latex. The lettering was originally done with simple, transparent adhesive labels and later with vinyl letters, but neither proved robust enough to withstand frequent cleaning and disinfection of the template. Originally, the PUR version was covered in an aliphatic urethane coating (Wasser MC-Luster 100) for chemical resistance and to maintain a bright white color. Unfortunately, the coating did not prove abrasion resistant and began flaking off after a year, perhaps because plastic was not the ideal substrate for the coating. An intermediate version with uncoated polyurethane turned a yellowish tint over time and absorbed stains from various inks in the workplace, so PUR was abandoned in subsequent versions. The final template was made from Type 1 Polyvinyl Chloride (PVC) in three layers that were glued together (Figure 6, 8 and 9). This time lettering and colored outlines were all applied with screen printing. One concern in health care is the risk of exposure to plasticizers like BPA (Bisphenol A) and DEHP (Diethylhexyl Phthalate), especially in pediatric patients, so a type of PVC that did not contain these additives had to be used. The PVC has proven both chemical and abrasion resistant and has maintained a bright white color over months of use.

Figure 9. The final AMT shown with the Anesthesia Cart.

Testing and Evaluation of the AMT

During the design process, prototype iterations of the AMT were used in simulation and in “real world” conditions in the operating room to refine visual, physical and experience design, integrating expertise in anesthesia practice with expertise in interaction, industrial and visual communication design.

The first design evaluations were conducted with a simple series of tape outlines that experimented with the size, location and number of fields. The second design evaluation was based on laminated paper print-outs (Figure 3) and the third evaluation utilized an acrylic glass overlay on laminated paper print-outs of the refined visual design (Figure 4) to add some depth (Figure 7).

The final design of the AMT (Figures 6, 8 and 9) was the basis for a detailed evaluation in a clinical setting in two phases (Grigg, Martin, Ross et al., 2017): (1) forty-one anesthesia providers administered medications in two prospective randomized operating room simulations with or without the AMT and (2) around 200 providers prospectively self-reported medication errors from all anesthetizing locations during a 2-year period at Seattle Children’s Hospital, an academic, pediatric medical center.

In simulated emergencies, the odds of medication dosing errors using the AMT were 0.21 times the odds of medication dosing errors without AMT (95% confidence interval [CI], 0.07, 0.66), controlling for scenario, session, training level and years at training level. During the year after implementation of the AMT, the mean monthly error rate for all reported medication errors that reached patients decreased from 1.24 (95% CI, 0.85–1.79) to 0.65 (95% CI, 0.39–1.09) errors per 1000 anesthetics. The mean monthly error rate of reported swap, preparation, miscalculation and timing errors decreased from 0.97 (95% CI, 0.64–1.48) to 0.35 (95% CI, 0.17–0.70) errors per 1000 anesthetics (Grigg, Martin, Ross et al., 2017).

The practice of anesthesiology has never had a widely accepted standard for organizing medications during a surgical procedure and this is the first known attempt of its kind to create a sophisticated, 3-dimensional device based on well-established design principles. These evaluation results show that using design tools to standardize medications with the AMT is an effective, low-cost strategy with the potential to significantly improve patient safety through reducing medication errors by anesthesia providers. With now over four years of data available, the AMT has nearly eliminated syringe swaps and miscalculation errors at Seattle Children’s Hospital over its first 2.5 years in use and it has reduced the incidence of medication-related harm (Grigg, Martin, Roesler, & Ross, 2017). Such a sustained reduction in errors in the operating room is almost unheard of with a single, analog device. This has had an enormous impact on the overall safety of providing anesthesia at Seattle Children’s Hospital and the template has since seen been adopted at a number of other medical centers around the United States.

Conclusion

The AMT (Anesthesia Medication Template) is a cognitive aid that supports anesthesiologists in identifying, locating and applying the appropriate medications needed during an anesthetic in the operating room. The template improves the detection of erroneous medication arrangement and reduces the occurrence of medication dosage errors. It is a representation of the knowledge that resides in the relationship between anesthesiologist, patient and medications that are located on top of the anesthesia cart. By representing this knowledge in a physical product in the work setting, the AMT contributes to a safer anesthesia work environment.

The success of the AMT lies in the match between its medication layout design and the conceptual model for medication use that anesthesiologists internalize for the appropriate choice of medications, the properties of the various medications and their risk profile. The complementary design knowledge and understanding of anesthesia workflow could only be integrated by the close collaboration between designers and anesthesiologist and interdisciplinary teamwork in multiple phases of this project.

The practice-centered design approach enabled the design team to incorporate the situated and practice-driven nature of the design situation:

  • Individual/professional aspects of expert practice,
  • The physical environment,
  • The social/collaborative nature of team work in the operating room,
  • The organizational structure of the governing hospital organization.

Crucial to the access to these layers of the design situation was that the design of the AMT was initiated and owned by practitioners. Physicians at Seattle Children’s Hospital initiated the study and improvement process in their work setting. As insiders in the work setting, they were well familiar with its organizational structure and design, the hierarchical structure of decision-making that guided operations and workflows and the resources for innovation support that were available. As key stakeholders in the organization, they were respected as key-players ‘on the ground’ and were able to access all layers of the design project. They partnered with professional designers to link first hand domain insight with external design expertise. In contrast to traditional participatory design approaches that are initiated by upstream stakeholders, the physicians at Seattle Children’s Hospital initiated the design project from within their work setting.

On the basis of an accurate conceptual model of anesthesia work, the AMT has been refined in its visual, industrial and interaction design in order to maximize its adaptability and effectiveness. The AMT provides cognitive support that maps a conceptual model of medication layout into the workspace and integrates meaningful placements into a visual design that is based on three design principles: (1) the orthogonal and vertical organization of syringes in an asymmetrical grid, (2) standardization of location of syringes across sites, (3) using visual and tactile cues to differentiate between syringes, all with the goal of reducing the cognitive load of providers in the operating room. Much of the design process involved simplifying the overall design in order to make the most versatile and practice-friendly design, this included the detailed design of typography to optimize readability and fast scanning, color-coding and visual guides. Multiple iterations with continuous testing and refinement and close collaboration between designers and clinicians produced a practice-centered design that resulted in an effective tool, which makes the operating room safer, is intuitive, easy to learn in a high turnover staff environment and fits well into the existing anesthesia workflow.

Acknowledgments

This work was supported by a Laura Cheney Anesthesia Patient Safety Endowment Junior Investigator Starter Grant for 2013. Dept. of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA. 

References

  1. Aicher, O. (1988). Typography. Berlin: Wilhelm Ernst & Sohn Verlag für Wissenschaft und Architektur.
  2. Albers, J. (2006). The interactions of color. New Haven, CT: Yale University Press.
  3. Beyer, H., & Holtzblatt, K. (1997). Contextual design: Defining customer-centered systems. New York, NY: Morgan Kaufman.
  4. Bjerknes, G., Ehn, P., Kyng, M., & Nygaard, K. (1987). Computers and democracy: A Scandinavian challenge. Farnham, UK: Gower Publishing.
  5. Cook, R. I., & Woods, D. D. (1996). Adapting to new technology in the operating room. Human Factors, 38(4), 593-613.
  6. Christ, R. E. (1975). Review and analysis of color coding research for visual displays. Human Factors, 17(6), 542-570.
  7. Degani, A. (1992). On the typography of flight documentation (Contractor Report #177605). Moffett Field, CA: NASA Ames Research Center. Retrieved from https://ti.arc.nasa.gov/m/profile/adegani/Flight-Deck_Documentation.pdf
  8. DeRosier J., Stalhandske, E., Bagian J. P., & Nudell, T. (2002). Using health care failure mode and effect analysis: The VA National Center for Patient Safety’s prospective risk analysis system. The Joint Commission Journal on Quality Improvement, 28(5), 248-267. https://doi.org/10.1016/S1070-3241(02)28025-6
  9. Drui, A. B., Behm, R. J., & Martin, W. E. (1973). Predesign investigation in the anesthesia operational environment. Anesthesia and Analgesia, 52(4), 584-591.
  10. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Revised Ed.). Cambridge, MA: MIT Press.
  11. Grigg, E., Palmer, A., Grigg, J., Oppenheimer, P., Wu, T., Roesler, A., Nair, B., Ross, B. (2013). Randomized trial comparing the recording ability of a novel, electronic emergency documentation system to the AHA paper cardiac arrest record. Emergency Medicine Journal, 31(10), 833-839.
  12. Grigg, E. B., Martin, L. D., Ross, F. J., Roesler, A., Rampersand, S. E, Haberkern, C., Low, D. K. W., Carlin, K., Martin, L. D. (2017). Assessing the impact of the anesthesia medication template on medication errors during anesthesia: A prospective study. Journal for Anesthesia and Analgesia, 124(5), 1617-1625.
  13. Grigg, E. B, Martin, L. D., Roesler, A., Ross, F., & Martin, L. D. (October 23, 2017). Sustained reduction of medication errors with the anesthesia medication template. Poster presented at the Annual Meeting of the American Society of Anesthesiologists. Boston, MA. Abstract retrieved on April 12, 2019, from http://www.asaabstracts.com/strands/asaabstracts/abstract.htm?year=2017&index=15&absnum=4095
  14. Greenbaum, J., & Kyng, M. (Eds.). (1991). Design at work: Cooperative design of computer systems. Hillsdale NJ: Lawrence Erlbaum Associates.
  15. Hofmann, A. (1965). Graphic design manual: Principles and practice. Zurich, Switzerland: Arthur Niggli Verlag.
  16. Hollnagel, E., & Woods, D. D. (2005). Joint cognitive systems: Foundations of cognitive systems engineering. New York, NY: Taylor & Francis.
  17. Holmlid, S., Mattelmäki, T., Sleeswijk Visser, F., & Vaajakallio, K. (2015). Co-creative practices in service innovation. In R. Agarwal, W. Selen, G. Roos, & R. Green (Eds.), A guidebook to service innovation (pp. 245-574). London, UK; Springer-Verlag.
  18. Hutchins, E. (1995a). Cognition in the wild. Cambridge, MA: MIT Press.
  19. Hutchins, E. (1995b). How a cockpit remembers its speeds. Cognitive Science, 19(3), 265-288.
  20. Institute for Safe Medication Practices (2012). List of high alert medications. Retrieved January 19, 2018, from https://www.ismp.org/resources/results-ismp-survey-high-alert-medications-differences-between-nursing-pharmacy-and
  21. Itten, J. (1993). The art of color: The subjective experience and objective rationale of color. New York, NY: John Wiley & Sons.
  22. Kensing, F., & Greenbaum, J. (2013). Heritage: Having a say. In J. Simonsen & T. Robertson (Eds.), Routledge international handbook of participatory design (pp. 21-36). New York, NY: Routledge.
  23. Klein, G., & Wright, C. (2016) Macrocognition: From theory to toolbox. Frontiers in Psychology, 7, 54. https://doi.org/10.3389/fpsyg.2016.00054
  24. Klein, G., & Militello, L. G. (2001). Some guidelines for conducting a cognitive task analysis. Advances in Human Performance and Cognitive Engineering Research, 1, 161-199. DOI:10.1016/S1479-3601(01)01006-2
  25. Klein, G., & Zsambok, C. (Eds.) (1996). Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum Associates.
  26. Kunz, W. (1998). Typography: Macro- and microaesthetics. Fundamentals of typographic Design. Salenstein, Switzerland: Niggli Verlag.
  27. Lee, J. -J., Jaatinen, M., Salmi, A., Mattelmäki, T., Smeds, R., & Holopainen, M. (2018). Design choices framework for co-creation projects. International Journal of Design, 12(2), 15-31.
  28. Luck, R. (2018). What is it that makes participation in design participatory design? Design Studies, 59, 1-8. https://doi.org/10.1016/j.destud.2018.10.002
  29. Martin L.D., Grigg E.D., Verma S., Latham G., Rampersad S.E. (2017). Decreased Medication Errors in Pediatric Anesthesia Practice: Outcomes from a Failure Mode and Effects Analysis. 28370645 Pediatric Anesthesia, 2017 March 28
  30. Müller, L. (2005). Helvetica – Homage to a typeface. Zürich, Switzerland: Lars Müller.
  31. Müller-Brockmann, J. (1968). Grid systems in graphic design: A visual communication manual for graphic designers, typographers and three dimensional designers. Zurich, Switzerland: Arthur Niggli Verlag.
  32. Norman, D. A. (2013). The design of everyday things. New York, NY: Basic Books.
  33. Roesler, A., & Woods, D. D. (2005). Inventing the future of cognitive work: Navigating the ‘northwest passage’ in design. In W. Jonas, R. Chow, & N. Verhaag (Eds.), Proceedings of the 6th International Conference of the European Academy of Design. Bremen, Germany: University of the Arts. Retrieved from https://www.researchgate.net/publication/228668738
  34. Roesler, A., & Woods, D. D. (2007). Designing for expertise. In R. Schifferstein & P. Hekkert, Product experience—Perspectives on human-product interaction (215-237). Elsevier, Oxford, UK.
  35. Roesler, A. (2009). Lessons from Three Mile Island: The design of interactions in a high-stakes environment. Visible Language, 43(2/3), 170-195.
  36. Sanders, E. B.-N. (2000). Generative tools for codesigning. In S. Scrivener, L. J. Ball, & A. Woodcock (Eds.) Collaborative design (pp. 3-12). London, UK: Springer.
  37. Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. International Journal of CoCreation in Design and the Arts, 4(1), 5-18.
  38. Sanoff, H. (2007). Editorial: Special issue on participatory design. Design Studies, 28(3), 213-215.
  39. Stappers, P. J., & Norman, D. A. (2015). Design X: Complex sociotechnical systems. She Ji, 1(2), 83-106. https://doi.org/10.1016/j.sheji.2016.01.002
  40. Suchman, L. A. (1987). Plans and situated action: The problem of human-machine communication. Cambridge, UK: Cambridge University Press.
  41. Suchman, L. A. (2002). Practice-based design of information systems: Notes from the hyperdeveloped world. The Information Society, 18, 139-144.
  42. Wasson, C. (2000). Ethnography in the field of design. Human Organization, 59(4), 377-388.
  43. Ware, C. (2012). Information visualization: Perception for design (3rd ed.). San Francisco, CA: Morgan Kaufman.
  44. Woods, D. D., & Dekker. S. W. A. (2000). Anticipating the effects of technological change: A new era of dynamics for human factors. Theoretical Issues in Ergonomic Science, 1(3), 272-282.
  45. Woods, D. D., & Christoffersen, K. (2002). Balancing practice-centered research and design. In M. McNeese & M. A. Vidulich (Eds.), Cognitive systems engineering in military aviation domains (pp. 121-136). Dayton, OH: Human Systems Information Analysis Center.
  46. Woods, D. D., & Hollnagel, E. (2006). Joint cognitive systems: Patterns in cognitive systems engineering. Boca Raton, FL: Taylor & Francis.
  47. Woods, D. D., & Roesler, A. (2007). Connecting design and cognition at work. In R. Schifferstein & P. Hekkert (Eds.), Product experience—Perspectives on human-product interaction (pp. 199-213). Oxford, UK: Elsevier.