Prototyping Materials Experience: Towards a Shared Understanding of Underdeveloped Smart Material Composites

3_3255

Prototyping Materials Experience:
Towards a Shared Understanding of Underdeveloped Smart Material Composites

Bahareh Barati 1,*, Elvin Karana 1,3, and Paul Hekkert 2

1 Department of Design Engineering, Delft University of Technology, Delft, the Netherlands
2 Department of Industrial Design, Delft University of Technology, Delft, the Netherlands
3 Centre of Applied Research for Art, Design and Technology, Avans University of Applied Sciences, Breda, the Netherlands

Over the past years, product designers have been involved in collaborative developments of smart material composites early on in the development process, to showcase creative applications of them. In these projects, the way the material is presented to the development team and the extent to which its properties are defined affect how designers understand the potentials and boundaries of the material and envision product applications. In the context of a European project, Light.Touch.Matters, we studied the attempt of designers to understand and prototype underdeveloped composites of thin-film organic light emitting diodes and piezoelectric polymer. Arguing for a collaborative exploration of the unique experiences that such underdeveloped composites unfold, we elaborate on a challenge designers face in understanding and prototyping the experiential qualities, specifically, the dynamic and performative qualities. The paper presents our design approach and complementary tools to overcome this challenge. It further discusses the applicability and limitations of the proposed design supports in the context of collaborative materials development and outlines future research directions.

Keywords – Collaborative Materials Development, Design Tools, Experience Prototyping, Experiential Qualities, Smart Materials.

Relevance to Design Practice – Motivated by the emerging interests in designers’ contribution to materials development, the paper probes into the challenges of understanding an underdeveloped smart material composite. It presents a design approach and tools developed to support the interdisciplinary discussions of the composite’s potentials and boundaries.

Citation: Barati, B., Karana, E., & Hekkert, P. (2019). Prototyping materials experience: Towards a shared understanding of underdeveloped smart material composites. International Journal of Design, 13(2), 21-38.

Received March 25, 2018; Accepted July 13, 2019; Published August 31, 2019.

Copyright: © 2019 Barati, Karana, & Hekkert. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

*Corresponding Author: b.barati@tudelft.nl

Bahareh Barati is a postdoctoral design researcher at Delft University of Technology, responsible for integrating biodesign and biofabrication in design engineering education. In her Ph.D. research, Bahareh developed an approach, tools, and exemplars to unpack the potentials of smart material composites, and disseminated her work in various design and ACM conferences, such as CHI, and has published in international journals as well as exhibitions (Dutch Design Week). Bahareh received her M.Sc. in Integrated Product Design (cum laude) from Delft University of Technology and received the Best Graduate of Industrial Design Engineering Award from faculty in 2012. She is an alumnus of the University of Tehran, where she received her B.A. and M.A. in Industrial Design.

Elvin Karana is professor of Bio-based Art and Design at the Centre of Applied Research for Art, Design and Technology (CARADT) in the Netherlands, where she leads Material Incubator, a creative research lab that explores the potentials of materials from living organisms for design, and an alternative notion of the everyday. She received her Ph.D. in 2009 from Delft University of Technology, where she is also currently an Associate Professor at the Faculty of Industrial Design Engineering, leading the research group Materials and Fabrication Design. She has published in Materials and Design Journal, International Journal of Design, Journal of Cleaner Production, and Design Issues. She is the main editor of Materials Experience: Fundamentals of Materials and Design (Elsevier, 2014).

Paul Hekkert is full professor of form theory at Delft University of Technology. Paul conducts research on the ways products impact human experience and behavior, and leads the international project UMA (Unified Model of Aesthetics). He is co-editor of Product Experience (2008) and published Vision in Design: A guidebook for innovators (2011), a book that describes an approach to design and innovation. Paul is co-founder and chairman of the Design and Emotion society and former chairman of the executive board of CRISP, a national collaborative research initiative for and with the Dutch creative industries. He is also member of the Dutch Creative Council and scientific member of the Top Sector Creative Industries.

Introduction

In product design, traditionally, material considerations come at the very end of the design process where designers often act as passive recipients of materials with no big influence on material properties. While this still defines the majority of design projects, over the last decade, we can observe an increasing number of projects in which product designers are involved for the collaborative development of new materials (e.g., the European projects Light.Touch.Matters and Solar-Design). These pioneering projects are particularly interesting for the design research community due to the early influence that design can have on materials properties (Bergström et al., 2010). The contribution of designers to such upstream collaborative projects may range from exploring and showcasing the design possibilities of the new materials, to bringing market considerations and consumer perspectives to materials research (Nathan et al., 2012). However, as the materials are far from being integrated in products due to being underdeveloped, i.e., certain aspects of them, including their structure, fabrication, properties, and behavior, are either unknown or undefined, thus, understanding them and exploring their potentials and boundaries can be a challenge for designers. Given that the materials information flows between the disciplines of materials science and design, the challenge is as much about how these new underdeveloped materials can and should be communicated to the designers.

In this paper we focus on the development of smart material composites, which are composite materials that tightly integrate sensing, actuation, communication, and computation (McEvoy & Correll, 2015). Due to their intrinsic physical properties and built-in control mechanisms, smart material composites are capable of sensing their environment and responding to it in a specific, predetermined manner (Addington & Schodek, 2005; Spillman, Sirkis, & Gardiner, 1996). Adaptability, memory, and multiple functionalities are among smart characteristics that these composites bring to numerous possible applications in aerospace, civil engineering, biomedicine, etc. (Kamila, 2013). More anchored to product design applications, smart material composites are envisioned to blur the existing boundaries between the physical form and digital content of products (e.g., Ishii, Lakatos, Bonanni, & Labrune, 2012) and dramatically change the experiences of future interactive products (McEvoy & Correll, 2015; Nijholt, Giusti, Minuto, & Marti, 2012). The emotive, expressive, and communicative aspects of smart materials have attracted many practitioners and researchers from the broad fields of interactive art, design, and human-computer interaction (HCI) to use these materials in creating tangible and organic user interfaces (e.g., Coelho & Zigelbaum, 2011; Wakita, Shibutani, & Tsuji, 2009), responsive architecture (e.g., Lumina by Chin Koi Khoo; Penumbra project led by Richard Blythe and Paul Minifie), and expressive and communicative wearables (e.g., Chromat Adrenaline Dress in partnership with Intel).

In the early stages of materials development, direct experiences of the underdeveloped smart material composite might be substituted with verbal, graphical and/or numerical descriptions, and representations, for practical or/and strategic reasons (cf. Davis, Shrobe, & Szolovits, 1993). Despite being commonly used for communicating materials between materials scientists and engineers (Miodownik, 2007), these representations take no account of materials experiential qualities (Ashby & Johnson, 2003; Karana, 2009; van Kesteren, 2008), and have little to offer about the aesthetic, expressive, and performative qualities of the novel smart materials (Vallgårda & Sokoler, 2010). In addition, researchers have argued that high-level descriptions and representations of a technology, or black-boxing, as a strategy for reducing technical complexities can have counter-effects on designers’ attention to and understanding of its distinctive properties (Sundström et al., 2011).

To support their understanding and anticipation of the experiential aspects of materials, designers often rely on physical encounters with materials (Miodownik, 2007; e.g., materials libraries) and memories of previous experiences with them. The physical encounter with materials is argued to provide an often-forgotten way into technical and interdisciplinary discussion of materials (Miodownik, 2007; Wilkes et al., 2016). Physical material samples, however, can hardly scratch the surface when communicating the complex, temporal, and context-dependent functions and expressions of smart material composites. Further, for new smart material composites, experiential references and design precedents hardly exist (Bergström et al., 2010). Designers might as well find the common approach of relying on a priori knowledge and experiences of the existing materials and technologies insufficient, impractical or not as informative. Even the vocabulary commonly used for conventional materials might not be sufficient in capturing and expressing behavioral transformability (Parkes & Ishii, 2009; Rasmussen, Pedersen, Petersen, & Hornbæk, 2012) and temporal forms of smart material composites (Vallgårda, Winther, Mørch, & Vizer, 2015).

The main objective of this research is to explore how the prototyping tools used in design processes that depart from an underdeveloped smart material composite may support understanding, exploring, and communicating of the experiential qualities. The research was part of an extensive European (EU) research project, Light.Touch.Matters (LTM), which focused on the development of a specific group of smart material composites. The LTM materials are technologically viable thin and flexible composites of an organic light emitting diode (OLED) and piezoelectric polymer. In this project, material scientists and designers have joined forces to showcase innovative applications of the LTM materials and collaboratively develop these composites. Early on in the project, there was no sample of these composites and rather the underdeveloped composites were represented to the designers through their functions, for instance, piezoelectric polymer functions as a pressure and deformation sensor, and physical features, such as thin, flexible. Figure 1 shows a schematic representation of the components and the layered structure of the LTM materials used to communicate these composites. Based on such representations of the underdeveloped composites, the designers were asked to explore and showcase future application concepts.

Figure 1. A schematic representation of the components constituting the LTM materials.

The paper presents our design approach and the tools developed to support understanding, exploring, and communicating the experiential qualities of the LTM materials. We develop and motivate our approach and the tool through the analysis of: (1) an interview study with professional designers and a materials scientist involved in the LTM project; and (2) the prototyping tools and techniques developed or adopted by master’s-level students through five design processes departing from the LTM materials. Prior to the studies, the background on materials experience, smart material composites, and the existing approaches and tools for experience prototyping smart materials are presented. Next, we explain our design research methodology and present the findings from the interview and the student case studies. After introducing the proposed tools to the professional designers and materials scientist, we inquired about the applicability and limitations of such tools in the EU project. In the discussion, we bring together their feedback and our critical reflection of the tool and reflect on the generalizability of the tools to other smart material composites.

Background

Experiences of Smart Material Composites

Materials experience, as a notion, was first introduced by Karana (2009) to acknowledge the active role of materials in conditioning and influencing our experiences with and through materials. The notion points to the need for a more holistic understanding of the material potentials, informed not only by the technical properties but also by the experiential qualities elicited by them in human-material interactions (Karana, Barati, Rognoli, & Zeeuw van der Laan, 2015). Giaccardi and Karana (2015) specified that material experiences can be analyzed and articulated at four experiential levels: (1) sensorial, i.e., how the material is received through the five senses (e.g., hard, transparent), (2) affective, i.e., the emotions elicited in interaction with a material (e.g., surprising), (3) interpretive, i.e., the meanings assigned to the material (e.g., cheap-looking), and (4) performative, i.e., the actions involved in handling the material (e.g., pressing, knocking). Despite the power of language in capturing and analyzing material experiences (Giaccardi & Karana, 2015; Manzini, 1989), descriptive representations can never replace the need for hands-on experiences of materials in design (van Kesteren, 2008). Particularly, sensorial and performative qualities are bound to embody interactions and practices with and through materials (see, Dourish, 2001; Sennett, 2008).

The designers’ understanding of materials experience is not only tied to their thinking about materials, but also unfolds in their making practices with them (Ingold, 2009; Nimkulrat, 2012; Schön, 1983). The role of making and fabricating in understanding composite materials (Barbero, 2011) and digital materials (Löwgren, 2015; Solsona Belenguer, 2015) has been emphasized in the past, wherein Barbero explained that “there is no better way to gain an initial sense for composite materials than actually observing a composite part being fabricated” (p. 1). Laboratory experience and the tutorial videos showing and explaining how the composites are being fabricated are thus advised. Löwgren (2015) described his relationship with the material during the design process as palpating the material, with which he brings forth the corporeal component of the relationship. The experimental design approach as explained by Bergström et al. (2010) allows for probing into aspects of the potential or eventual expressiveness of the material.

Given that the dynamic and responsive behavior of the smart materials can only unfold over time, temporal form becomes an important element of materials experience (Mazé & Redström, 2005; Vallgårda et al., 2015). Temporal form of smart materials largely relates to the computational structure that enables and demands a temporal expression in the resulting design. In an experimental setting, Vallgårda et al. showed how temporal form of similar-looking compositions of textile material and actuators could elicit different experiences. The potential of this design space has been increasingly acknowledged in interaction design literature (e.g., Löwgren, 2009; Lundgren, 2013). Many artists have explored the expressive design space of temporal forms through their kinetic artwork and sculptures (e.g., Jean Tinguely, László Moholy-Nagy, Philip Beesley). There are numerous examples from stage, costume, and fashion design that take a particular interest in exploring the temporal form of materials, smart materials, and light-emitting technologies (e.g., Hussein Chalayan). Related work in the industry ranges from the BMW’s shape-changing car concept, to 3D mapping light projection (e.g., Mr. Beam), to one-off installations that combine shape-morphing surfaces and light projection (e.g., Parametric Space by Kollision, CAVI and Wahlberg in collaboration with Zaha Hadid Architects).

Prototyping Material Experiences

In product and interaction design, the role of sketching and prototyping in supporting the designers’ understanding of user experiences and exploring the design space has been long discussed (e.g., Buxton, 2007; Lim, Stolterman, & Tenenberg, 2008). Many researchers and practitioners have contributed to the field of prototyping, by developing new ways of prototyping (e.g., Greenberg & Boyle, 2002), comparing the existing techniques (e.g., Sefelin, Tscheligi, & Giller, 2003), and identifying the types (Lichter, Schneider-Hufschmidt, & Zullighoven, 1993) and anatomy of prototypes (Lim et al., 2008). Researchers have used theatre, theatrical performance, and acting out as a means to inform and generate new insights and ideas (Burns, Dishman, Verplank, & Lassiter 1994; Iacucci, G., Iacucci, C., & Kuutti, 2002; Oulasvirta, Kurvinen, & Kankainen 2003). Simple animations and Wizard of Oz techniques have been extensively relied on in making early representations of system’s interface in action (Buxton, 2007). Some researchers are in favor of a distinction between such sketches of a future product and its prototypes, arguing that they serve different purposes throughout the design process, in the transition from ideation to usability testing. Other researchers, however, have argued that prototypes can be as much for design-thinking enablers, and not just tools for evaluating design outcomes (Jones et al., 2007; Lim et al., 2008). Such prototypes are largely used for discovering problems and/or exploring new solution directions, rather than for proving solutions, and as such are a means of generative and evaluative discovery (Lim et al., 2008). According to Lim et al., prototypes are for traversing a design space and are purposefully formed manifestations of design ideas. The incompleteness of a prototype is appraised to be its primarily strength in enabling the designer’s traversal of a design space.

To overcome the complexity of exploring the design space of temporal forms that are not yet built, researchers have relied on a variety of prototyping techniques, including a combination of cardboard prototypes and tools to register the durations of action and reaction (Frens, Djajadiningrat, & Overbeeke, 2003) or a combination of music sheets to register the patterns of color change of an interactive piece of furniture and a graphical interface connected to an Arduino board (Nilsson, Satomi, Vallgårda, & Worbin, 2011). The latter work explores the temporal form as the last part of the application design, when decisions regarding the product category, shape, and patterns have already been made.

Buchenau and Suri (2000) coined the term experience prototyping, where prototypes aim at getting a sense of the real experiences and letting the designer reflect upon them before the product exists. The tools and techniques used for experience prototyping reinforce the attitude of actively experiencing the sometimes subtle differences between various design solutions. The prototype’s fidelity, however, remains an important issue, largely because fidelity is a matter of cost. A celebrated example of low-fidelity prototyping is paper prototyping (e.g., Snyder, 2003) but it can as much benefit from advanced technologies, such as augmented reality (Nam & Lee, 2003), due to providing a much cheaper alternative to developing the actual hardware. Some researchers have argued, however, that instead of deciding between low and high-fidelity prototyping, both may be combined in exploring different dimensions of design consideration (McCurdy, Connors, Pyrzak, Kanefsky, & Vera, 2006). It has been proposed that “the best prototype is one that, in the simplest and most efficient way, makes the possibilities and limitations of a design idea visible and measurable” (Lim et al., 2008, p. 7:3).

Researchers have argued the importance of experience prototyping in exploring and communicating the effects of materials (e.g., Bergström et al., 2010; Saakes, 2010). With the aim of bringing material considerations to the early stages of the design process, Saakes (2010) designed and developed a projection-based tool, called Skin V.2. The tool enables the design team to combine and manipulate physical and digital patterns, and project them real-time on the surface of early physical prototypes. Bergström et al. (2010) developed representations of an adaptive seat to support experiencing and building a common understanding of the context-dependent material expressions. Their approach involved a combination of real-scale low-tech prototypes of the seat placed in multiple households and experimental material samples to depict a process of wear-and-tear. The work of Saakes and Bergström demonstrates two different functions/applications of experience prototyping in the early development. The former allows for sketching and getting an impression of materials on full-scale mockups and the latter provides a contextualized representation of the material embodying a specific application.

Experience Prototyping of the LTM Materials

The literature suggests that the experience of smart material composites might be unique, particularly in relation to the temporal and context-dependent qualities of these composites (e.g., Bergström et al. 2010; Vallgårda et al., 2015). Accordingly, researchers have favored experience prototyping of specific applications to help the development team with an in-depth understanding of the unique experiential qualities of these composites and their further effects on existing use practices (Bergström et al., 2010). Prototyping specific applications, however, has limitations, particularly when it comes to exploring the potential of these composites across various applications and contexts of use. Experience prototyping physical and temporal forms of these composites in experimental settings may offer ways of understanding their expanded design possibilities beyond the limits of very specific applications. In order to identify the challenges of understanding and prototyping materials experience in the early development of smart material composites, we have conducted a set of design studies in the context of the LTM project (2013-2016).

Method

In order to investigate how product designers may approach the experience prototyping of the LTM materials and to identify the challenges on the way to understanding these underdeveloped smart material composites, we conducted two studies. Study #1 involved a semi-structured interview that was conducted to incorporate the experience and insights of professional designers who have participated in the LTM project. To that aim, we relied on purposive sampling (Palinkas et al., 2015) and included the LTM project’s materials scientist (8 years of experience), responsible for the development of the OLED component, to account for the complementary nature of the collaborative work. The materials scientist and three designers (6-14 years of experience) based in different European countries participated in the interview via Skype. The questionnaire consisted of a combination of structured and open-ended questions related to the participants’ personal experience of the LTM project, the challenges faced and support (e.g., tools, approaches, activities) used for understanding the LTM materials (see Attachment for the interview questions). The interviews took approximately 45-60 minutes. All data was transcribed verbatim.

Study #2 involved five semester-long (20 weeks) master’s student projects during the period of the LTM project, which provided us with the possibility of observing the prototyping activities of master’s-level design students. The study included four group projects and one graduation project that were defined with the same objective: to explore and communicate the design potential of the LTM materials. The information about the LTM materials was communicated to the students through the LTM project’s coordinator who is an experienced academic involved in teaching courses on various topics of design and manufacturing. The group projects were part of two curriculum courses: Interactive Technology Design and Advanced Embodiment Design. Each group was composed of six first or second year students. The method of data collection was through participant observation (Denzin, 2009), meaning that the first author was involved in the supervisory team of the projects as a coach and had the opportunity to frequently meet with the students and get updates about their activities on a weekly basis. The design activities were also documented by the design students in a written format complemented with photos and videos of the process.

In the following, first, the challenges faced by the professional designers in understanding the LTM materials and the mitigating strategies adopted by them are reported. Second, we focus on the tools and techniques our design students used for exploring and communicating the experiential qualities of the LTM materials.

Results

Study #1: Challenges and Mitigating Strategies of the LTM Designers

The interview results indicate that among the three designers (D1, D2, D3) only D2 had worked with underdeveloped smart material composites prior to the LTM project. Unlike the other two, D2 mentioned that the LTM project was not very different from their daily practice and they often work with new materials and technologies. This seemed to influence how they approached the uncertainties resulting from the lack of knowledge and definition in the LTM project. In addition to the prototypes of their proposed application concepts, D2 contributed to the project by creating a series of physical booklets that documented their material explorations with surface and light and integration of various sensors in soft materials, to name a few. She made a distinction between her design practice and “companies that are accustomed to working with more clear technologies,” and suggested that the latter might be approached when the gaps in materials understanding are bridged.

As expected, the way the underdeveloped smart material composite was introduced to the designers, through descriptions of its physical properties and sensing/actuating functions, was found insufficient and even deceiving. According to the designers, in the first round of concept designs, their imagination went far beyond what actually could be done to/with the composite.

D3: the design boundaries were not clear in the beginning until we started to push through.

D2: we had the idea of it (the LTM material) doing more different things, […] the limitations were not clear at least.

The ineffectiveness of using such abstract descriptions of the material was further explained by the material scientist (MS). According to him, despite the effort of the materials R&D team to communicate the material information, the designers ended up conceptualizing product ideas that either were not possible with the technology or did not push enough for its uniqueness.

MS: In the beginning we tried to inform the designers of the qualities and properties of the various materials. What we learn is that apparently we didn’t do well enough and that created a lot of confusion and also disappointment later in the project and also it created a lot of, let’s say, ineffectiveness in the sense that people spend a lot of energy into ideas which in the end when we saw them, we could immediately say that, well sorry, this is not going to happen.

In explaining the causes of their insufficient understanding of the boundaries, the designers blamed the mere mediated access to the material, as opposed to directly experiencing the material and feeling its resistance.

D2: I think it was very much connected to not having the material in the hands. We never saw the piezo material in action. the biggest hindrance was the difference between the imagination and the actual things.

D1: The moment you describe the [underdeveloped] material, it exists in the mind of the designers five years ahead of the material research. This misalignment of the scale is the opportunity in this project but also the difficulty to overcome.

It seems that the first round of concept designs following the initial introduction of the LTM materials enabled some of the predefined boundaries to surface. The designers presented posters of their product concepts to the material scientist, using refined descriptions, hand sketches, 3D renders, and story-boards. Some designers brought along material samples and simple animations to communicate particular qualities of their proposed applications. The designers indicated that discussions about the application concepts positively influenced their understanding of the boundaries. However, they agreed that it was later in the process when they tangibly interacted with the material samples and prototypes that they developed a better understanding of the potentials and boundaries.

D3: I think the first round of concepts was useful […] with the prototypes and actual samples that we realize that there were a lot more constraints that should have been told to us.

D1: until that point [when material samples were provided] we were talking about the properties in abstract form in graphs and descriptions […] the material scientists were not materializing their thoughts […] I think if anything designers are quite materialized [...] so that step to think material in a tactile way was quite a key point.

The designers emphasized that the dynamic and interactive aspects of the LTM materials made understanding of these material composites more challenging. They hinted that when combining the performative qualities of the LTM materials, the actual behavior of the composite could not be easily imagined.

D2: I think with the flexible OLED it was quite clear […] but I am not sure that I still got the real understanding of the piezo component. The visual way of understanding thing is quite easy for your imagination to make a bridge between having the electroluminescent film and super flat big OLEDs... [For the piezo] you have a piece of plastic in your hand but it is so much the dynamic and interactive properties, when it is manipulated.

The materials scientist further elaborated on a variety of tools and technologies they used to communicate the potentials and boundaries of the OLED development to the designers, including a document in which they addressed designer’s specific questions and explained some practical rule of thumb to be considered in application design, for instance if you want a bright OLED you should forget about mechanical flexibility. In acknowledgement of the designers’ emphasis on physical interaction with the material, the materials scientist considered how certain representations might have supported the designers’ understanding of the underdeveloped OLED by reflecting on parts of its properties.

MS: There were a number of ways, one very formalized and centralized way was this document, which was obviously not a big success. […] having these materials in your hand or having some representatives that reflect part of the materials properties was very important […] in the very beginning of the project we send around OLEDs, even if there were still rigid so they (designers) could see how do they look like […] you can learn a lot even if it is still not transparent, not flexible, if it is only a square and so on. […] we send semi-finished products or just plastic foils that don’t give light but have the mechanical properties of the OLED […] also broken OLEDs that we get out of our production that people can have a look at how to connect them but also how they look like in off-state.

To capture and represent the experiential qualities of the eventual composite, the design professionals relied on two main approaches, namely physical prototyping using materials that resemble the descriptions of the LTM materials and Arduino-based prototyping using ready-made transducers.

D1: The basic technology we used is what everyone did, which is Arduino-based programming trying to figure out the interaction.

D2: We always do very physical very early, to try to understand different things […] they were not actually prototypes, but to resemble the materials.

The designers also invested in systematic explorations of the interaction and expression possibilities of the LTM materials, which related to their application concepts. Documenting their explorations as sketches and sample books, they tapped into the discussion of how understandable those were for the materials scientists.

D2: We also did this systematic. For instance, we had this investigation on surfaces and light […] we had all these different materials to investigate how light looks like through them. […] for us putting (different investigations) together in small sample books was a way to communicate different qualities.

Unlike the interactive sample book format, the 2D design sketches of the possible interactions hardly transcended the disciplinary barriers between designer and materials scientist when it came to discussing the dynamic behavior of the underdeveloped composite.

D1: We made maybe hundred little sketches of different interactions on a poster, grouping them to touch interaction, physical bend twist, scale of interaction like palm-size or body size or wind […] but in terms of communication it (the poster) wasn’t clear, the way we (designers) talked about the interaction (among ourselves), did not work for material scientists. They don’t read sketches.

The challenges, as reported above, mainly originated form the underdeveloped state and dynamic qualities of a new smart material composite, and the need for communicating it between the two disciplines. However, the disciplinary gaps between designers and materials scientist seemed to impact beyond causing a representation and communication challenge of the interactive and performative qualities. According to the designers participated in the interview, the disciplinary differences were as much present in how designers and materials scientists thought about the future of the material and its development. Grounded in their in-depth knowledge of the material, material scientists took control over the eventual development of the underdeveloped composite, while the material knowledge gaps remained unbridged between the two disciplines.

D3: The material scientists already had a clear understanding where they wanted to take the material. What they were going to do with it. And I think at the beginning it was very much looking at how can our design influence the material. Towards the end, due to time constraints, it was more like: this is what we are going to be doing with this material and come up with concepts for it.

Study #2: Design Representations Developed by Design Students

Our observations and the students’ self-reports revealed that a variety of tools and approaches were used and developed in the design processes with the LTM materials. Our previous publications (Barati, Karana, & Hekkert, 2015; Barati, Karana, Hekkert, & Jönsthövel, 2015; Barati, Karana, & Hekkert, in press) report on the observed design activities, explorations, and tools at length. In this study, we focused on how the design students took account of exploring and communicating the LTM material’s possible physical and temporal forms. To identify an initial set of design aspects that prototyping LTM materials’ physical-temporal forms might exhibit, we considered the interactions between the three elements of material, light, and time (Figure 2). This further provided a basis for clustering and analyzing the students’ prototyping attempts and specifying the gap that the proposed tools intend to bridge.

Figure 2. An ontological deconstruction of the LTM materials used in analyzing the prototyping of their physical-temporal form.

The element of material underlines the importance of physical embodiment in investigating the expressions of computational objects (e.g., Mazé & Redström, 2005; Jong & Stolterman, 2012). Particularly for the LTM materials, integration of the piezoelectric component in flexible substrates allows for an extended repertoire of physical interaction, while the formal/spatial relations between the two material components are under-defined. Integration of light into curved displays (e.g., Papillon by Disney Research) and flexible substrates adds additional experiential dimensions to the experiences of the physical body. Light as an experiential element has been discussed across various disciplines, such as theatrical design and more recently textile and interaction design (e.g., Franinović & Franzke, 2015; Jansen, 2015). Accordingly, we specified three spaces based on the interactions between the material and light elements in prototyping Luminescent Materials, and their individual interactions with time in prototyping Performable Structures and Dynamic Light, respectively. In addition, a fourth space was specified where the three emergent spaces overlap as Physical-Temporal form of the LTM materials.

Our findings suggest that the students represented and prototyped the LTM materials more or less within the boundaries of the three emergent spaces.

Prototyping Luminescent Materials

A variety of approaches and techniques were used to consider the interactions between the material structure and light elements, focusing mainly on bricolages of available physical and digital materials (Hazlewood et al., 2013). While in the early explorations, the two elements were not often combined in a single unit, the final prototypes aimed at realizing the experiences of both elements as an integrated whole (Figure 3). Figure 4 shows two ways of using LEDs for creating surface illumination. The actual fabrication of these composites was implemented towards the final stages of concept design and embodiment design (Figure 5). These detailed representations reveal how the designers invested in experience prototyping surface lighting, creating designs that could not be simply achieved using point light sources such as LEDs. Their strategies to achieve this quality through integrating a larger number of LEDs, however, resulted in stiffer material structures, which were inevitably less representative of the material’s tactile and performative qualities. In the final prototypes of the LTM product applications, the fidelity trade-offs were favored over the visual qualities of the luminescent material rather than the tactile and performative qualities.

Figure 3. The integration of light and material elements in three final prototypes:
Yoga mat (left), CPR trainer (middle), punching bag (right).

Figure 4. Two ways of making thin and flexible light-emitting surfaces with LEDs:
using LED strips and translucent material sheets on top (left) and using side-emitting LEDs and engraved acrylic (right).

Figure 5. LED edge lighting allows for a thinner structure in comparison to using top diffusers:
the engraved acrylic triangles in combination with the LEDs (left), and the final light-emitting sample as a proof of concept (right).

Prototyping Performable Structures

Structural movement offers a way of looking at the LTM materials as under-specified structures that require pressure and deformation to activate. A performable structure is a relationship between the parts of the composite that enables certain dynamic movements (Niedderer, 2012) and encourages certain performances with and through the composite. An example of material structures that exploit the dynamic movement of sheet materials is action origami (i.e., a folded structure that can be animated). In order to achieve richer ways of interacting with physical forms made with the LTM materials, the performative aspect of the material structures becomes a key topic in both discussing the potentials and designing them.

Across the projects, we observed only a few instances, particularly in the early stages of ideation and design explorations, that material structures were made to represent the performative qualities of the LTM materials (Figure 6). In other instances, the students used ready-made force sensors (Figure 7, right) to make simple representations of the pressure-sensing composite. Later in the design process, in order to achieve a higher fidelity prototype of the smart Yoga mat concept (Figure 3, left), the students constructed their own flexible pressure sensors using velostat (Figure 7, left). Relying on these interactive prototyping techniques suggests that the LTM materials were dominantly understood as a pressure-sensing flat surface. We suspect that the urge to come up with product concepts with the LTM materials early on, and the gaps between physical materials and the off-the-shelf sensing technologies used for interactive prototyping, left little room for creative explorations of the performative qualities. Even though the concepts made use of thinness and flexibility of the composite, the possibilities for unique expressions emerging from an under-specified structure remained largely underexplored.

Figure 6. A multi-material 3D printed piece combining soft and hard materials made in exploring the possibilities of a performable structure.

Figure 7. Two prototyping approaches during the design process focusing on the pressure-sensing function of the LTM materials: using an off-the-shelf force sensor in early explorations (right) and a velostat-based pressure sensing matrix, constructed by the students in prototyping the Yoga mat concept (left).

Prototyping Dynamic Light

The interaction between time and light can also bring about different experiential qualities. Examples range from dynamics and rhythm of an expressive LED-optical fiber composition (Jansen, 2015) to the dynamics of daylight in buildings (Köster, 2004). The design students used a variety of tools and techniques to explore and represent the experiential qualities of dynamic light, including projection and LEDs (Figure 8). Early on, they used Wizard of Oz techniques, such as a simple setting with a light bulb and a dimmer where they could manually control the intensity of the light and create dynamic light behavior (see Barati et al., 2015). They also used adjectives, such as pulsating and flashing, as well as simple graphs of light intensity as a factor of time (see Jansen, 2015) to qualify the intended light output.

Figure 8. Two techniques used for experiential prototyping the Yoga mat concept:
projection (left) and LEDs connected to an Arduino board (right).

In the later stages of concept specification, the students produced high fidelity representations of dynamic light, using animated and interactive illustrations on a computer screen and light projection on physical surfaces (Figure 9). Programming skills played a crucial role in prototyping dynamic light behavior, enabling the designers to diversify and expand the expressions, as shown in Figure 9. The more sophisticated light behavior was prototyped after the decisions concerning the application and function were already made.

Figure 9. Software developed for simulating sophisticated light behavior:
interface of the HTML file (left) and screen-shots of the graphical simulations.

The explorations not only confirmed that interaction between the elements was a concern in experience prototyping the LTM materials, but also pointed to the complexity of representing the dynamic and performative qualities of the LTM materials simultaneously. The Arduino-based approach using off-the-shelf LEDs and pressure sensors, even though was straightforward when it came to handling and programming, was not optimized for representing unified physical and temporal forms.

Proposed Approach and Tool for Prototyping the LTM Materials

The results of the observational study complemented the interview results in specifying when and how in the design process designers explored the experiential qualities of the LTM materials. It indicated that except for the early explorations concerning the LTM materials’ performable structure (Figure 6), most of the prototyping activities intricately corresponded to the evolution of the application design ideas. The interview results highlighted the role of material samples and prototypes in facilitating the early discussions of the potentials and boundaries of the materials. By externalizing designers’ ideas of the materials and their qualities in some kind of physical manifestation, the prototypes allow the world to speak back (Schön, 1983) in helping designers verify their assumptions. On the other hand, when substitute materials and technologies are used to simulate an underdeveloped composite, further discussions between the designers and the materials expert are needed to clarify how the underdeveloped composite may be processed or behave differently from the designers’ initial assumptions.

In order to support the LTM team in an early exploration and discussion of the LTM materials’ experiential qualities, our first step was to make them aware of the richness of the forth (overlapping) space at the intersection of the luminescent tangible, performable structure and dynamic light (see Figure 2) through a material demonstrator. This section first presents our material-driven experimentations with electroluminescent (EL) materials, aiming at fabricating and engineering this material demonstrator (Step 1). The demonstrator was however not intended to support early prototyping of the dynamic and performative qualities. To that end, the second step was to develop a hybrid sketching tool aiming to enable designers to further explore the design space beyond the limits of a specific design exemplar (Step 2). The tool is proposed as a practical way to facilitate projections of the material’s dynamic and performative qualities across various applications and situations.

Step 1: Electroluminescent Material Demonstrator to Represent the LTM Materials

Instead of using ready-made EL devices and manipulating their forms, we used unprocessed inks and made EL material samples in our university lab. Our approach to understanding the luminescent matter through direct engagement with its chemical and physical properties was similar to that proposed by Olberding, Wessely, and Steimle (2014) and Franinović and Franzke (2015). According to Franinović and Franzke, such a shift from electronics-oriented hardware to physical experiments with smart materials sheds new light on the inherent interactive properties of matter and thus can lead to novel design ideas and creation processes. To guide our experimentation, we followed the materials understanding step of the Material-Driven Design method (Karana et al., 2015).

Electroluminescent (EL) materials in the form of thin-film displays have been used in various commercial products, such as watches. Many artists and designers have pushed the expression and application of these materials by creating exclusive lighting furniture and architectural pieces (e.g., Butterfly Nightlights by Soner Ozenc and John Wischhusen; Blumen Wallpaper by Loop.pH) and interactive on-off installations (e.g., Sonumbra by Loop.pH; Lumibolic by Steve Lee and Meredith Sattler). Recently, researchers have proposed that EL materials are suitable for prototyping thin-film custom-printed displays (Olberding et al., 2014). Unlike OLED’s complex fabrication which requires high-end lab equipment, the chemical inks for making EL materials can be easily processed using a screen-printing method. The in-house processing facilities offer more 2D and 3D design freedom to design custom-made demonstrators, in comparison to standard EL displays.

To get a better understanding of the experiential aspects of the LTM materials in a single smart material demonstrator (Barati, Karana, Jansen, & Hekkert, 2016), we printed the EL materials on a transparent, thin and flexible polymer substrate. The screen-printing process followed a standard method of sequentially depositing and curing multiple layers of materials, including phosphor, dialectic, and conductive materials (see Franinović & Franzke, 2015; Olberding et al., 2014). This fabrication technique results in a high-fidelity representation of the surface light and tactile feel of the flexible OLED component. We then applied Kirigami, which involves cutting techniques to obtain 3D shapes from 2D sheets. The technique enabled not only changes in shape, from cylinder to sphere, but also a broader, more sophisticated range of actions (e.g., rotating hands in opposite directions) and expressions. Finally, to relate the structural performances to the dynamic behavior of light, a light sensor was incorporated in the demonstrator. The deformation of the structure from fully closed to open changes the amount of light received by a light sensor, and hacking a standard DC to AC driver used for EL wires enabled mapping the intensity of light to an analogue input from the light sensor (Figure 10).

Figure 10. The demonstrator features a dimming effect corresponding to structural deformation.

The smart material demonstrator allowed designers to experience the dynamics and performative qualities of a thin luminescent material as it actively elicits certain performances, influenced by the concurrent changes in light behavior and structural deformation. The prototyping process, from screen-printing and shaping to connecting, incorporating the light sensor and hacking the EL driver, provided us with a practical understanding of the boundaries in designing with the EL materials. The final EL demonstrator was designed to represent an instance of the overlooked space in experiential prototyping the LTM materials and to let the development team experience, discuss, and explicate the subtle differences of the LTM materials with the existing materials and technologies. Nevertheless, such a demonstrator was not meant to actively enable further explorations of the overlooked design space. To support such explorations beyond the definitive form of this specific demonstrator, we suggest a supplementary generative tool. This step was motivated by the interview results, specifically the challenge of communicating the dynamic and performative qualities of the application concepts through design sketches. The generative tool aims to facilitate the assimilation of the outcome of the experimental design step into the design and development process.

Step 2: Hybrid Tool to Sketch the LTM Materials’ Physical-Temporal Form

Our criterion in choosing a technique to experience prototyping the LTM materials was to achieve high-fidelity representations of the physical-temporal forms through simple, quick, and efficient ways by drawing on the economical principle of prototyping proposed by Lim et al. (2008). In line with the merits of this principle, prototyping tools and techniques that require too much investment in time and resources and are unnecessarily complex could become a hindrance in the early stages of design, where ideas need to be quickly sketched, prototyped, and perhaps discarded (Buxton, 2007).

Many interaction design researchers have addressed a similar challenge since the 1990s in connection to user-interfaces (UI) (e.g., Landay & Myers, 2001). In order to represent dynamic behavior in early sketches, an early solution was to scan the paper-based sketches into a computer and then script the behavior using multimedia tools (Wong,1992). Electronic sketching systems, such as SILK (Landay & Myers, 2001), allow UI designers to produce free-hand sketches that could be automatically recognized as standard widgets, such as buttons, with attributed behaviors. Currently, there are numerous design tools to create Wireframes and high-end user interfaces, such as Balsamiq and Sketch. However, despite the interactive and animated features of 2D sketches, these tools do not seem to offer much when it comes to experiencing prototyping the experiential qualities of smart material composites at the performative level.

Various technologies exist that allow for fusion of physical structure and digital surface augmentations, including augmented reality. Applications of these technologies range from product customization (e.g., VizeraLabs. (2014, August 13). Vizera [Video File]. Retrieved from https://www.youtube.com/watch?time_continue=4&v=5YV8lo-OxZQ), collaborative development (e.g., Sparks platform, www.spark-project.net), and education (e.g., Shaping Watersheds by Reed et al., 2014). In some of these applications, sophisticated algorithms and high-end tracking devices are required to contour the physical object and track the changes. Commercial companies, such as TWNKLS, provide augmented reality prototyping services, including 3D model tracking, using sensor information from mobile devices. Chroma-key offers a much simpler solution for representing dynamic changes of color, lightness, and even texture on specific areas of kinetic and moving objects that eliminates a need for object tracking to perfectly align the digital layer.

Chroma-key is a special effect for layering two images or video streams together based on color hues. The technique has been used heavily in news-casting, movie-making, and videogame industries to create a simulated world for the user. It allows for creating a realistic illusion of alternative conditions, without actually being engaged in them. Application of Chroma-key in simulating the behavior and operation of real-world systems over time has been mainly used to extract a physical object from its color-marked environments and place it within a virtual environment (e.g., Coles, John, Gould, & Caldwell, 2011). The same technique can be used to digitally augment physical samples or mock-ups that are being moved or deformed. Existing software programs, such as Adobe After Effect, enable designers to manipulate recorded video footages and create color-changing effects (Foole, 2016). In post-processing Chroma-key, the physical deformation and dynamic surface effects are co-located (on the screen) but are not concurrent. Our sketching tool benefits from real-time Chroma-key to approximate concurrent experiences of the LTM materials at sensorial and performative levels.

V.01: Chroma-Key Station

The Chroma-key station supports the prototyping activities through generating dynamic light behavior, replacing it on the screen instead of the color-marked surfaces, and letting the team members modify the light behavior real-time. Figure 11 shows the main components of a real-time simulation setting, namely a webcam, a screen, and an input device in addition to the Chroma-keyed objects. The Chroma-keyed objects are physical mockups that are color-coded at the parts that the designer intends to show dynamic surface lighting. This can be done by simply adding vinyl or velvety stickers on the surface of the physical mockups (Figure 12, left). To produce the Chroma-key effect (Figure 12, right) and the dynamic light behavior we used, respectively, a default and a custom-made function in MAX/MSP/Jitter, which is a visual programming language and environment for music and multimedia development. As the hands of users are interacting with the object, we decided to let the users control the light behavior using their feet (Figure 11, right). Given that the deformation of the objects and the modification of the light behavior are both manual, making the coupling between the dynamic behavior and the deformation is the matter of a live, performative exploration.

Figure 11. The Chroma-key setup and components: the setup consists of four main components, a screen, a webcam, an input device and physical samples of mockups (left), and features of the input device to control the dynamic behavior of light (right).

Figure 12. Preparation and augmentation of the physical mockup:
adding velvety stickers (left) and the appearance of the real-time Chroma-key effect on the screen (right).

Figure 13. The real-time Chroma-key app: a snapshot of the interface (left), the user can select the color by touching it on the screen and use the slider to fine-tune the threshold (right).

The specific design of the Chroma-key tool requires the members of a design/development team to stand in a dedicated prototyping station, so that they would be more active in exploring a wider range of hand-gestures to full-body interactions. The control over the light behavior is made available to the users through a physical input device. The feet-controlling input device, as shown in Figure 11, right, included eight touch-sensors connected to an Arduino Uno microcontroller board. For this early version, we considered a limited number of dynamic light controlling features, namely, color, three fade-in/fade-out patterns (one symmetric and two asymmetric intensity variations), and speed to generate repetitive light rhythms. A randomizer button allows for random combinations of the features (color, patterns, speed). In addition to simple sheet materials, a collection of performable Origami and Kirigami probes was made with polymer and paper sheets to showcase a variety of structural movements.

V.02: Chroma-Key iPad App

The idea of the app was to complement the Chroma-key station and give the designers a dedicated platform to further develop the coupling of light behavior and deformation in relation to specific interaction scenarios. Using the app, the deformation videos and the dynamic behavior parameters, namely intensity-change pattern, color, and speed, could be recorded and linked to a specific interaction scenario (e.g., storage of the mat). In addition, the Chroma-key app allows for specifying the coupling type (e.g., feed-forward, feedback), tagging, and documenting the Chroma-key videos (e.g., activating the mat). Having the option to launch the real-time Chroma-key, the app lets its user quickly perform the sketched coupling between the action and the simulated dynamic light output and record the augmented videos. These features were mainly meant for effective documentation and communication of the temporal form. Other existing frameworks such as the Interaction Frogger framework (Wensveen, Djajadiningrat, & Overbeeke, 2004) can benefit the exploration process, by identifying more expanded characteristics of the coupling (e.g., location, direction).

Feedback of the LTM Project Members on the Proposed Tools

We asked the opinions of the designers and the materials scientist involved in the first interview about the Chroma-key tool and the demonstrator. Even though they had seen and tried the earlier version of the tool (V.01) in the closing exhibition of the LTM project, they were provided with a video of the EL demonstrator, the Chroma-key tool, and its features to refresh their memory. Their direct involvement in a collaborative materials development project provided us with valuable insights regarding the applicability of the tool and the possible limitations in such projects.

Our four participants agreed that the electroluminescent (EL) demonstrator was clear in communicating its core idea.

D2: … you twist and the structure is completely different from cylinder to sphere and then you have the interactive light started […] not only dynamic but also shape changing […] you have the direct understanding of what is happening […]

They collectively acknowledged that the Chroma-key station is a sketching tool with clear advantages for communicating and discussing the boundaries early on in a project.

D1: It is a very quick and easy way of showing the potential concepts. It is like paper prototyping without actually committing too much or prototyping beautiful coding very quickly.

D2: I think it is very nice tool and the whole idea of a tool is fruitful […] it is a kind of tool that demonstrates what is possible.

D3: I actually do think it is useful, you saw my reaction when I tried it. What I liked about it is that is as simple as painting things with black... it is not a really kind of like glossy thing... it is not aiming to completely replace ...it is not like a CG kind of thing that you could probably render very realistic effects […] for me your demonstrator is a sketching tool and I think as such there is clearly benefit.

MS: It [the video] is very instructive and it shows nicely how such a tool could have enhanced the experience and the search for new ideas... and it is a bit of a pity obviously that it was presented within the LTM so late, I mean there was no way to do it another way, but I think it is very useful to have such things from the very beginning […] you can simulate lots of things that are either technically not yet there, or in real-life costs a lot of money and effort that can be spent better otherwise.

D3 identified three moments when the tool might be used in a collaborative setting: (1) when designers are briefed, (2) when presenting initial ideas, and (3) when developing the application concepts.

D3: The first is when we [designers] are briefed. So we get the power points of this materials and the properties, then we have your demonstrator there and we could just have some sheets of black and we can start playing with this. One is the kind of exploration of the properties and understanding of the time-based design and design for behaviors. So this is like basic understanding, understanding the framework of the time-based design. The second is to present our initial ideas. So if I could show my building concept, this little pieces...scale model with the actual behavior this could have been much more powerful than trying to explain the concept with sketches with two pieces of this thing. People do not get it! You need to be quite an imaginative person to imagine the effect, and talking about the behavior is a lot about the effect. The third would be in the communication between the development team and the designer […] in a way very practical way to describe interaction, and have this scripted. So you could translate and add a layer to your demonstrator.

The designers and the materials scientist warned that considering the tool as a simulator could raise false expectations. They touched upon the limitations of the tool in terms of not offering new information about the material and not including its limitations.

MS: You also need to be very careful if you use it for communication of the properties that you need to make sure that it also includes the limitations.

D1: I think as simulator of the product that you are designing it could be a powerful tool but I don’t think that I can learn a lot from it.

The designers also mentioned the necessity of looking at a screen as a limitation that would narrow down the interaction scenarios for which the tool could be used.

D3: A scenario that might not happen in front of the camera, or is difficult to do it in a quite direct positioning, also in scenarios that involve multiple, longer, and more complicated interactions.

D1: I think you still look at a screen to show what it is doing, so if there is something on the person, you are still looking at screen to see how it works.

Discussion

In this paper, we focus on the experience of prototyping underdeveloped smart material composites in collaborative projects, aiming at further developing these novel materials. We have described our approach to identifying the challenges in understanding and communicating the experiential qualities of a specific smart material composite, namely the LTM materials. Through analyzing five student design processes and interviewing three professional designers and a materials scientist involved in early development of these materials, we could relate most of these challenges to the LTM materials’ dynamic and performative qualities.

In order to support the design process, we propose (1) an approach, which advocates the understanding of the luminescent matter through direct engagement with its chemical and physical substances to create a material demonstrator; (2) a tool, which supports early sketching of the physical-temporal form so that the potentials and boundaries of the underdeveloped composites can be collaboratively explored and discussed. The expert feedback, presented in the previous section verified the relevance of such a simple sketching tool in the instances where the designers and the materials scientist exchanged material information and initial ideas. The findings seem to be in favor of the proposed tool development direction and its appropriateness. Both the professional designers and the materials scientist made positive remarks about the scope and the means of support. As elaborated before, the scope was the dynamic and performative qualities of the material and applications, and the means was the direct and tangible interactions with the underdeveloped composite, particularly using a combination of material experimentation, physical and digital prototyping.

Reflection on the Experience Prototyping Tools and Approach

Our experience prototyping approach involved the identification of an initial set of design aspects in relation to the LTM’s distinctive physical-temporal forms, which involved the specification of four spaces emerging as the three elements of material, light, and time interact. To support the development team in understanding and exploring the overlapping design space between these three elements, our approach combined a smart material demonstrator (high-fidelity) and Chroma-key sketching tool (low-fidelity). This mixed-fidelity prototyping approach (McCurdy et al., 2006) aimed to bring the development team’s attentions to the distinctive properties of the LTM materials, namely their performable structure and dynamic surface light, by providing a direct understanding of the experiential qualities. Neither the demonstrator nor the Chroma-key tool was intended to inform about the accurate behavior of the smart materials composite, assuming that such knowledge might not yet be available in early stages of the development. Instead, the resulting prototypes, as noted by the designers, were deliberately incomplete (Lim et al., 2008), to encourage further discussions among the development team concerning the limitations of the real composites and other boundaries of the collaboration (e.g., time and resource constraints). and further discussions among the development team were necessary as they did not inform on the limitations of the real composites and other boundaries of the collaboration (e.g., time and resource constraints).

The appropriateness of the Chroma-key sketching tool in terms of being simple to use and low-cost is indeed bound to the specified needs for exploring and communicating the experiential qualities and boundaries in the early stages of collaborative materials development. The tool, accordingly, enabled quick and direct experiential prototypes of the design ideas, which need to be further discussed within the team for their appropriateness, creative contribution, and so forth. As pointed out by the interviewed designers, the generative stages of design thinking (Sundström et al., 2011) can be leveraged as an opportunity to collaboratively explore the composite’s unique potentials and boundaries. The proposed tool is found useful as a communication means in the discussion and definition of the temporal and performative qualities of a yet-underdeveloped smart material composites. Nevertheless, both the designers and the materials scientist believe that the tool is most useful to complement rather than replace other means of communication, such as 3D rendering, live or video tutorials of the fabrication process, and engineering rules of thumb. Further research would be necessary to assess usability and effectiveness of the proposed tool in context, or to compare it in those terms with other interaction design sketching tools and low-fidelity prototyping techniques.

Limitations and Implications for Collaborative Materials Development

But how do the presented design research work and findings contribute to the overall discourse between design and materials science, and to what extent can the findings be generalized to other smart material composites?

An important implication of our approach for a collaborative development team is to reserve dedicated times within the project to build material demonstrators, which will support the team to explore the materials’ potential and communicate this to all members of the team. We suggest creating material demonstrators to be a collective process where individuals and groups within the development team retain skills and knowledge around particular aspects of the underdeveloped composite. In the making of our demonstrator, direct engagement with ELs chemical and physical substances and the hacking of the EL driver helped us understand the EL’s working principle and its design boundaries. In other words, by getting outside of our disciplinary boundaries and comfort zone, we could reveal novel possibilities that were not initially imagined in designing [with] the EL materials (Barati, Giaccardi, & Karana, 2018). Further, the prototypes created through such a material-driven design process could satisfy the informational requirements of both designers and material scientists (Lee, 2007; Wilke et al., 2016). The demonstrator thus may act as effective ‘boundary objects’ (Lee, 2007; Star & Griesemer, 1989) in collaborative discussions of the LTM materials’ potentials and boundaries. Boundary objects are objects that can coordinate the perspectives of different communities, by being plastic enough to adapt to their needs, yet robust enough to maintain a common identity across sites (Lee, 2007). As mentioned by one of the designers, the 2D sketches of the interactions created by the designers were too abstract and ambiguous for a material scientist, making them ineffective boundary objects.

Nevertheless, the actual demonstrator and tool are to a large extent bound to the specific features of the technology they represent. Such links to the LTM materials were, in fact, the key motive for investing on and making the electroluminescent (EL) demonstrator in the first place. Other smart material composites most likely involve different compositions and ways of interactions, compared to what is presented in this paper. It is important to note that the demonstrator is not a neutral representation of the smart material composite, rather provides a frame of reference, a specific way of looking at its unique aspects and qualities. For instance, a similar kinetic structure might be used in representing a shape-changing smart material composite (Qamar, Groh, Holman, & Roudaut, 2018). Our demonstrator, however, brought forth the performable structure as input, to reach beyond traditional controls, which was largely determined by the LTM materials. The application of the Chroma-key technique can be generalized to other smart material composites that feature surface changes, including color, light, and texture. The technique is, however, sensitive to surrounding illumination and surface glossiness of the physical mockups. This might limit the type of materials that can be used in physical prototyping, particularly if they cannot be effectively color coded and prepared for Chroma-keying. Further, as pointed out by one of the designers, the tool requires its users to look at the screen, which limits the type of application ideas and use scenarios that can be effectively prototyped using the tool. We aim to explore the usability and effectiveness of the demonstrator and the sketching tool in actual multidisciplinary workshops between the designers and materials scientists in a next study.

Conclusion

The main objective of this paper was to provide an approach and a tool to enable specification and discussion of the boundaries and potentials of underdeveloped smart material composites in collaborative material development. We particularly focused on the dynamic and performative qualities of these composites, which were found challenging to explore and communicate in the design process. The paper reported the interview results with design professionals and a materials scientist, and a variety of tools, techniques, and design representations design students developed in five design cases to represent the dynamic and performative qualities of the LTM materials. According to the interview results, the application concepts and prototypes were a turning point in staging discussions between the designers and the materials scientist to communicate the material boundaries. The findings from our analysis of the design cases support the interview results.

Following the identified gap in relation to the early representations of their performative qualities, we conducted a number of material-driven experimentations with electroluminescent materials to develop a material demonstrator to enable experience of dynamic light on performable structures. This way, we could expose a particular design space through the material demonstrator that could clearly represent the core idea of the LTM materials: the dynamic light is tied to the performable structure. The second step of our approach focused on the development of a Chroma-key sketching tool to enable the designers to explore the design space beyond the limits of a specific design exemplar and possibly across various applications and situations. In the discussion, we elaborated some aspects of our approach that might be generalized to other smart material composites and collaborative development projects. These include an identification of the design aspects linking to the physical-temporal form of a specific smart material composite, and a mix-fidelity solution of combining material demonstrators and hybrid sketching tools for an early exploration of the experiential qualities boundaries, collaboratively. It was discussed that sensitivity of Chroma-key to the surrounding illumination and surface glossiness of the physical mockups and viewing the interactions on the screen can respectively impact the tool’s usability and limit the type of application ideas and use scenarios that can be effectively prototyped. The issues related to usability of the proposed demonstrator and tool in actual workshops between designers and material scientists should be further investigated.

Acknowledgments

This work was part of the European Union (EU) project Light Touch Matters, and received funding from the EU’s Seventh Framework Programme (FP7/2007-2013) under grant agreement n°310311. The authors would like to thank the design students and the partners in the EU project for participating in this research. We are grateful to Aadjan van der Helm and Kaspar M.B. Jansen for providing the technical support and prototyping facilities.

References

Addington, M., & Schodek, D. (2005). Smart materials and technologies in architecture. London, UK: Architectural Press.
Ashby, M., & Johnson, K. (2003). The art of materials selection. Materials Today, 6(12), 24-35.
Barati, B., Giaccardi, E., & Karana, E. (2018). The making of performativity in designing [with] smart material composites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (No. 5). New York, NY: ACM. DOI: https://doi.org/10.1145/3173574.3173579
Barati, B., Karana, E., & Hekkert, P. (2015). From way finding in the dark to interactive CPR trainer: Designing with computational composites. In Proceedings of the International Conference on Design and Semantics of Form and Movement (pp. 129-137). Milano, Italy: Politecnico di Milano.
Barati, B., Karana, E., Hekkert, P., & Jönsthövel, I. (2015). Designing with an underdeveloped computational composite for materials experience. In Proceedings of the SIG Conference on Experiential Knowledge (pp. 118-132). Kolding, Denmark: University Southem Denmark.
Barati, B., Karana, E., & Hekkert, P. (in press). Understanding the experiential qualities of Light Touch Matters: Toward a toolkit. Journal of Design Thinking.
Barati, B., Karana, E., Jansen, K. M. B., & Hekkert, P. (2016). Functional demonstrators to support understanding of smart materials. In Proceedings of the 10th International Conference on Tangible, Embedded, and Embodied Interaction (pp. 386-391). New York, NY: ACM.
Barbero, E. J. (2011). Introduction to composite materials design. Oxfordshire, UK: CRC Press.
Bergström, J., Clark, B., Frigo, A., Mazé, R., Redström, J., & Vallgårda, A. (2010). Becoming materials: Material forms and forms of practice. Digital Creativity, 21(3), 155-172.
Buchenau, M., & Suri, J. F. (2000). Experience prototyping. In Proceedings of the 3rd Conference on Designing Interactive Systems (pp. 424-433). New York, NY: ACM.
Burns, C., Dishman, E., Verplank, B., & Lassiter, B. (1994). Actors, hair-dos and videotape–Informance design. In Proceedings of the Conference Companion on Human Factors in Computing Systems (pp. 119-120). New York, NY: ACM. DOI: https://doi.org/10.1145/259963.260102
Buxton, B. (2007). Sketching user experiences: Getting the design right and the right design. San Francisco, CA: Morgan Kaufmann.
Coelho, M., & Zigelbaum, J. (2011). Shape-changing interfaces. Personal and Ubiquitous Computing, 15(2), 161-173.
Coles, T. R., John, N. W., Gould, D., & Caldwell, D. G. (2011). Integrating haptics with augmented reality in a femoral palpation and needle insertion training simulation. IEEE Transactions on Haptics, 4(3), 199-209.
Davis, R., Shrobe, H., & Szolovits, P. (1993). What is a knowledge representation? AI Magazine, 14(1), 17.
Denzin, N. K. (2009). The research act: A theoretical introduction to sociological methods. Piscataway Township, NJ: Aldine Transaction.
Dourish, P. (2001). Where the action is: The foundations of embodied interaction. Cambridge, MA: MIT Press.
Foole, M. (2016). Filming future: Designing the interactivity of products (Unpublished master’s thesis). Delft University of Technology, Delft, the Netherlands.
Franinović, K., & Franzke, L. (2015). Luminous matter-electroluminescent paper as an active material. In Proceedings of the International Conference on Design and Semantics of Form and Movement (pp. 37-47). Milano, Italy: Politecnico di Milano.
Giaccardi, E., & Karana, E. (2015). Foundations of materials experience: An approach for HCI. In Proceedings of the 33rd Conference on Human Factors in Computing Systems (pp. 2447-2456). New York, NY: ACM.
Greenberg, S., & Boyle, M. (2002). Customizable physical interfaces for inter-acting with conventional applications. In Proceedings of the 15th Symposium on User Interface Software and Technology (pp. 31-40). New York, NY: ACM.
Hazlewood, W. R., Dalton, N., Marshall, P., Rogers, Y., & Hertrich, S. (2010). Bricolage and consultation: addressing new design challenges when building large-scale installations. In Proceedings of the 8th ACM Conference on Designing Interactive Systems (pp. 380-389). New York, NY: ACM.
Ingold, T. (2009). The textility of making. Cambridge Journal of Economics, 34(1), 91-102.
Iacucci, G., Iacucci, C., & Kuutti, K. (2002). Imagining and experiencing in design, the role of performances. In Proceedings of the 2nd Nordic Conference on Human-computer Interaction (pp. 167-176). New York, NY: ACM.
Ishii, H., Lakatos, D., Bonanni, L., & Labrune, J.-B. (2012). Radical atoms: Beyond tangible bits, toward transformable materials. Interactions, 19(1), 38-51.
Jansen, B. (2015). Temporal patterns new forms of material thinking in textile design. In Proceedings of the International Conference on Experiential Knowledge and Emerging Materials (pp. 34-47). Delft, the Netherlands: Delft University of Technology.
Jones, W., Spool, J., Grudin, J., Bellotti, V., Czerwinski, M., & Czerwinski, M. (2007). “get real!”: What’s wrong with HCI prototyping and how can we fix it? In Proceedings of the Conference on Human Factors in Computing Systems (Extended Abstracts, pp. 1913-1916). New York, NY: ACM.
Jung, H., & Stolterman, E. (2012). Digital form and materiality: Propositions for a new approach to interaction design research. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction (pp. 645-654). New York, NY: ACM.
Kamila, S. (2013). Introduction, classification and applications of smart materials: An overview. American Journal of Applied Sciences, 10(8), 876.
Karana, E. (2009). Meanings of materials (Unpublished doctoral dissertation). Delft University of Technology, Delft, the Netherlands.
Karana, E., Barati, B., Rognoli, V., & Zeeuw van der Laan, A. (2015). Material Driven Design (MDD): A method to design for material experiences. International Journal of Design, 19(2), 35-54.
Köster, H. (2004). Dynamic daylighting architecture: Basics, systems, projects. Berlin, Germany: Springer Science & Business Media.
Landay, J. A., & Myers, B. A. (2001). Sketching interfaces: Toward more human interface design. Computer, 34(3), 56-64.
Lee, C. P. (2007). Boundary negotiating artifacts: Unbinding the routine of boundary objects and embracing chaos in collaborative work. Computer Supported Cooperative Work, 16(3), 307-339.
Lichter, H., Schneider-Hufschmidt, M., & Zullighoven, H. (1993). Prototyping in industrial software projects-bridging the gap between theory and practice. In Proceedings of 15th International Conference on Software Engineering (pp. 221-229). Piscataway Township, NJ: IEEE.
Lim, Y.-K., Stolterman, E., & Tenenberg, J. (2008, July). The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas. ACM Trans. Comput.-Hum. Interact., 15 (2), 7:1-7:27.
Löwgren, J. (2007). Inspirational patterns for embodied interaction. Knowledge, Technology & Policy, 20(3), 165-177.
Löwgren, J. (2009). Toward an articulation of interaction esthetics. New Review of Hypermedia and Multimedia, 15(2), 129-146.
Löwgren, J. (2015). Beyond conversation: Palpating the hybrid materials. In Proceedings of the 2nd Conference on Research through Design (No. 25). Cambridge, UK: Microsoft Research.
Lundgren, S. (2013). Toying with time: Considering temporal themes in interactive artifacts. In Proceedings of the Conference on Human Factors in Computing Systems (pp. 1639-1648). New York, NY: ACM.
Manzini, E. (1989). The material of invention: Materials and design. Cambridge, MA: MIT Press.
Mazé, R., & Redström, J. (2005). Form and the computational object. Digital Creativity, 16(1), 7-18.
McCurdy, M., Connors, C., Pyrzak, G., Kanefsky, B., & Vera, A. (2006). Breaking the fidelity barrier: An examination of our current characterization of prototypes and an example of a mixed-fidelity success. In Proceedings of the Conference on Human Factors in Computing Systems (pp. 1233-1242). New York, NY: ACM.
McEvoy, M. A., & Correll, N. (2015). Materials that couple sensing, actuation, computation, and communication. Science, 347(6228), 1261689.
Miodownik, M. A. (2007). Toward designing new sensoaesthetic materials. Pure and Applied Chemistry, 79(10), 1635-1641.
Nam, T.-J., & Lee, W. (2003). Integrating hardware and software: Augmented reality based prototyping method for digital products. In Proceedings of the Conference on Human Factors in Computing Systems (Extended Abstracts, pp. 956-957). New York, NY: ACM.
Nathan, A., Ahnood, A., Cole, M. T., Lee, S., Suzuki, Y., Hiralal, P., ... Dyadyusha, A. (2012). Flexible electronics: The next ubiquitous platform. In Proceedings of the IEEE, 100 (Special Centennial Issue), 1486-1517.
Niedderer, K. (2012). Exploring elastic movement as a medium for complex emotional expression in silver design. International Journal of Design, 6(3), 57-69.
Nijholt, A., Giusti, L., Minuto, A., & Marti, P. (2012). Smart material interfaces: A material step to the future. In Proceedings of the 14th International Conference on Multimodal Interaction (pp. 615-616). New York, NY: ACM.
Nilsson, L., Satomi, M., Vallgårda, A., & Worbin, L. (2011). Understanding the complexity of designing dynamic textile patterns. In Proceedings of the Ambience Conference on, Where Art, Technology and Design Meet. Retrieved from http://www.akav.dk/publications/complexity_of_designing_dynamic.pdf
Nimkulrat, N. (2012). Hands-on intellect: Integrating craft practice into design research. International Journal of Design, 6(3), 1-14.
Olberding, S., Wessely, M., & Steimle, J. (2014). Printscreen: Fabricating highly customizable thin-film touch-displays. In Proceedings of the 27th Symposium on User Interface Software and Technology (pp. 281-290). New York, NY: ACM.
Oulasvirta, A., Kurvinen, E., & Kankainen, T. (2003). Understanding contexts by being there: Case studies in bodystorming. Personal and Ubiquitous Computing, 7(2), 125-134.
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533-544.
Parkes, A., & Ishii, H. (2009). Kinetic sketchup: Motion prototyping in the tangible design process. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (pp. 367-372). New York, NY: ACM.
Petreca, B., Baurley, S., & Bianchi-Berthouze, N. (2015). How do designers feel textiles? In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (p. 982-987). Piscataway Township, NJ: IEEE.
Qamar, I. P. S., Groh, R., Holman, D., & Roudaut, A. (2018). HCI meets material science: A literature review of morphing materials for the design of shape-changing interfaces. In Proceedings of the Conference on Human Factors in Computing Systems (No. 374). New York, NY: ACM.
Rasmussen, M. K., Pedersen, E. W., Petersen, M. G., & Hornbæk, K. (2012). Shape-changing interfaces: A review of the design space and open research questions. In Proceedings of the Conference on Human Factors in Computing Systems (pp. 735-744). New York, NY: ACM.
Reed, S., Kreylos, O., Hsi, S., Kellogg, L., Schladow, G., Yikilmaz, M., ... Sato, E. (2014). Shaping watersheds exhibit: An interactive, augmented reality sandbox for advancing earth science education. In Proceedings of the American Geophysical Union Fall Meeting Abstracts (ED34A-01). Washington, DC: AGU.
Saakes, D. P. (2010). Shape does matter: Designing materials in products (Unpublished doctoral dissertation). Delft University of Technology, Delft, the Netherlands.
Schön, D. A. (1983). The reflective practitioner. London, UK: Temple Smith.
Schwandt, T. A. (1994). Constructivist, interpretivist approaches to human inquiry. In N. K. Denzin, & Y. S. Lincoln (eds.), Handbook of qualitative research (pp. 118-137). Thousand Oaks: Sage.
Sefelin, R., Tscheligi, M., & Giller, V. (2003). Paper prototyping–What is it good for?: A comparison of paper- and computer-based low-fidelity prototyping. In Proceedings of the Conference on Human Factors in Computing Systems (Extended Abstracts, pp. 778-779). New York, NY: ACM.
Sennett, R. (2008). The craftsman. New York, NY: Yale University Press.
Solsona Belenguer, J. (2015). Engineering through designerly conversations with the digital material: The approach, the tools and the design space (Unpublished doctoral dissertation). KTH Royal Institute of Technology, Stockholm, Sweden.
Snyder, C. (2003). Paper prototyping: The fast and easy way to define and refine user interfaces. San Francisco, CA: Morgan Kaufmann.
Spillman, W. B., Sirkis, J. S., & Gardiner, P. T. (1996). Smart materials and structures: What are they? Smart Materials and Structures, 5(3), 247.
Star, S. L., & Griesemer, J. R. (1989). Institutional ecology, translations’ and boundary objects: Amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907-39. Social Studies of Science, 19(3), 387-420.
Sundström, P., Taylor, A., Grufberg, K., Wirström, N., Solsona Belenguer, J., & Lundén, M. (2011). Inspirational bits: Towards a shared understanding of the digital material. In Proceedings of the Conference on Human Factors in Computing Systems (pp. 1561-1570). New York, NY: ACM.
Vallgårda, A., & Sokoler, T. (2010). A material strategy: Exploring the material properties of computers. International Journal of Design, 4(3), 1-14.
Vallgårda, A., Winther, M., Mørch, N., & Vizer, E. E. (2015). Temporal form in interaction design. International Journal of Design, 9(3), 1-15.
Van Kesteren, I. (2008). Selecting materials in product design (Unpublished doctoral dissertation). Delft University of Technology, Delft, the Netherlands.
Wakita, A., Shibutani, M., & Tsuji, K. (2009). Emotional smart materials. In J. A. Jacko (Ed.), Human-computer interaction. Ambient, ubiquitous and intelligent interaction (pp. 802-805). Berlin, Germany: Springer.
Wensveen, S. A., Djajadiningrat, J. P., & Overbeeke, C. (2004). Interaction frogger: A design framework to couple action and function through feedback and feedforward. In Proceedings of the 5th Conference on Designing Interactive Systems (pp. 177-184). New York, NY: ACM.
Wilkes, S., Wongsriruksa, S., Howes, P., Gamester, R., Witchel, H., Conreen, M., … Miodownik, M. (2016). Design tools for interdisciplinary translation of material experiences. Materials & Design, 90, 1228-1237.
Wong, Y. Y. (1992). Rough and ready prototypes: Lessons from graphic design. In Proceedings of the Conference on Human Factors in Computing Systems (pp. 83-84). New York, NY: ACM.

Appendix

Structured and semi-structured questions used in the interviews with the designers:

Table 1. Questions used in the interviews with the designers.

IJDesign
Vol 13, No 2 (2019)

Prototyping Mater...

Reading Tools

Navigation Menu

Navigation Menu

IJDesignVol 13, No 2 (2019)

Prototyping Mater...

Reading Tools

Navigation Menu

Navigation Menu

IJDesign
Vol 13, No 2 (2019)