1_3281

Designing for Systems of Smart Things

Mathias Funk 1,*, Berry Eggen 1, and Jane Yung-Jen Hsu 2

1 Eindhoven University of Technology, Eindhoven, the Netherlands
2 NTU IoX Center, National Taiwan University, Taipei, Taiwan

Citation: Funk, M., Eggen, B., & Hsu, Y.-J. J. (2018). Designing for systems of smart things. International Journal of Design, 12(1), 1-5.

Copyright: © 2018 Funk, Eggen, & Hsu. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

*Corresponding Author: m.funk@tue.nl

Mathias Funk is assistant professor in the Future Everyday group in the Department of Industrial Design, Eindhoven University of Technology, the Netherlands. He has a background in Computer Science and a PhD in Electrical Engineering. His research interests include complex systems design, distributed IoT applications, remote data collection, systems for musical expression, and design tools. In the past he has worked at the Advanced Telecommunications Laboratory (ATR) in Japan, at RWTH Aachen, at Philips Consumer Lifestyle, and at Intel Labs, Santa Clara, CA. He is also a co-founder of UXsuite, a high-tech spin-off from Eindhoven University of Technology.

Prof. Dr. Berry Eggen is a full professor and director of research of the Industrial Design Department at Eindhoven University of Technology (TU/e), where he chairs the Future Everyday research group. He is also an adjunct professor in the Faculty of Engineering and Information Technology at the University of Technology, Sydney (UTS), Australia. Over the years he has led design research groups in the areas of information ergonomics, multimodal interaction (including lighting and sound) and intelligent interfaces. His current research activities include interaction design research for playful interactions and the exploration of decentralized systems approaches to ambient intelligence. Within the field of multimodal research his interests include lighting and sound innovations in the living room and public spaces, and the exploration of light and sound as calm environmental information displays.

Jane Hsu is a Professor of the Department of Computer Science and Information Engineering at National Taiwan University, where she served as the Department Chair from 2011 to 2014. As the Director of the NTU IoX Center, Prof. Hsu is leading the global research collaboration on Augmented Collective Beings and Internet of Things. With more than 30 years of experience in AI, her research interests include multiagent planning/learning, crowdsourcing, knowledge mining, commonsense computing, and context-aware smart IoT. Prof. Hsu served on the editorial board of Journal of Information Science and Engineering (2010-), International Journal of Service Oriented Computing and Applications (Springer, 2007-2009) and Intelligent Data Analysis (Elsevier/IOS Press, 1997-2002). She has been actively involved in international conferences of AAAI, IEEE, and ACM, and served as an executive member of the IEEE Technical Committee on E-Commerce (2000), the Taiwanese Association for Artificial Intelligence (2004-), and as TAAI President in 2013 and 2014. Prof. Hsu received the MSRA Collaborative Research Award, and Intel Labs Distinguished Collaborator Award in 2016.

Introduction

The Internet of Things has been a success for enterprises and several industrial sectors, but not yet a success for end users in home environments at the fringe of the cloud. Many studies reveal that beyond the initial amazement, connected devices add little value over their unconnected siblings, focus on too narrow use cases in mostly hypothetical scenarios, often fail to provide a good out-of-the-box experience, and are not able to learn and adapt in a meaningful way to our complex contextualized realities. Indeed, what is available commercially often feels like reiterating technological innovation over actually progressing human capabilities and addressing important needs.

This special issue started from the observation that after years of struggling to gain a foothold, designing for systems has finally found its place and area of application in design for the Internet of Things (IoT). Designing for systems and design for devices connecting into the Internet of Things have been explored not only in recent years, but already some years ago, when ubiquitous computing and later ambient intelligence emerged. While core insights remain valid, technologies have matured and spread significantly over recent years, bringing connected devices and services into the homes and direct proximity of end users.

Design needs to make the step from interaction design for a single device to designing systems beyond a single device (Eggen & Kyffin, 2006; Frens & Overbeeke, 2009; Ryan, 2014). This step changes the context of design from a single device or product towards a heterogeneous system of actors, which can be devices, humans, and the environment. Unlike the design of a single product or service, this is largely uncharted design territory, ridden with complexity, diversity, opaqueness, and intangibility. Technically, such systems are composed of distributed sensors, distributed actuators, and largely invisible logic. This changes how value is added through designed qualities of things and systems of things, how the human in the system is seen, addressed, and engaged at the meta-level of everyday interaction, how we design for changing needs and long-term everyday interaction, and finally, how contextual information, and data in general, are utilized in design.

Shifting design from products to systems is an incremental process and consists of making steps in a particular direction: towards more complex interactivity (Janlert & Stolterman, 2017) in designs on the one hand, and more systemic designs with distributed interfaces and intelligence on the other hand. Together, this demarks the stage for complex, systemic designs that will eventually emerge and form ecologies of things (Jung, Stolterman, Ryan, Thompson, & Siegel, 2008). In the context of this special issue, we were especially interested in research on designing intelligent, connected products, from expectations to design approaches to applications in major IoT contexts. Designing things that are accessible to algorithmic intervention means that we have to look at locality and situatedness not only in the physical realm, but also in data and computation. We need to strive for new qualities, potentially emerging from multiple products coming together, and this will also change how we approach user experience in a connected world. Finally, it means that design researchers need more than ever to understand how to design connected interactive products that might evolve throughout their lifecycle and that are ridden with multilayered complexity.

From Submissions to Papers

This special issue is aimed at addressing one important question: how to design for systems of smart things. Naturally, this question invites diverse answers, from different angles, at different levels, concerning different design processes, subjects of design, and application areas. Due in part to this broad scope, the special issue received 38 submissions from authors across many disciplines. The difference in framing of systems, design, and IoT was especially striking: quite a few papers approached the special issue topic from a rather technical IoT perspective, looking at building management, emergency services, sensor infrastructures, digital manufacturing, sustainable designs, and healthcare provision. Other submissions approached the topic from a user-centered perspective, aiming to understand the experience of IoT systems, for instance in entertainment, wellness, and sports. We received submissions concerned with the design process for IoT, aiming for toolkits, canvases, embedding data in design, requirements elicitation, and structuring the cross-device experience. Finally, we received submissions on philosophical approaches to connected products and systems, shedding light on ethical problems of connected and pervasive devices.

From all the submissions, we made a first selection based on relevance to design theory and practice, the quality of the submissions, and the general fit with the special issue topic. Fifteen papers were not of sufficient quality to match the journal’s standards. Another 12 papers did not fit the topic of the special issue closely enough. It was important for us that each paper could arguably contribute to design research and design practice, and that the overall balance of the selection represented a good match with the core interests of the journal and the general domain of design. In particular, we received surprisingly technical manuscripts or descriptions of design cases in very narrow domains, which would not have fit the journal’s audience. At the same time, we received manuscripts that focused on reiterating the state of the art in designing for IoT without a new viewpoint or angle. Consequently, 11 manuscripts went into the review process, in which over 30 reviewers were selected matching the manuscript contents. Based on the review outcomes, six manuscripts were chosen to go through a second round of revision. The authors further improved the manuscripts and in the end five papers could be accepted for publication.

Towards Designing Systems of Smart Things

This special issue and the finally selected contributions largely target the design of novel and future IoT products—a broad topic that spans from understanding requirements, to new perspectives on designing connected, smart products, to larger systems of connected things in two separate domains: the smart home, and manufacturing industry. In the following, we introduce the five papers of this special issue and attempt to highlight their relevance and contribution to the field of design.

When attempting to design novel IoT products, it is paramount to understand the needs of end users. Although the market for IoT smart-home products is by far not fully developed to maturity, many consumers have developed their own understanding of what these new products can achieve and provide in terms of services and convenience. This can become a problem for design approaches that build on empirical insights to drive innovation: participants are biased towards existing technologies and service offerings and find it difficult to think of future products outside the “box” of the current technological paradigm. The paper “Is Smart Home a Necessity or a Fantasy for the Mainstream User? A Study on Users’ Expectations of Smart Household Appliances” (Coskun, Kaner, & Bostan, 2018) approaches the special issue topic, designing systems, from the perspective of the necessity to understand the needs of broader populations in order to design commercially successful IoT products. After summarizing existing studies with smart-home inhabitants, the authors present findings from a study with early adopters and early majority participants who were recruited from a slightly more diverse pool than in many other academic studies (due to a different recruitment strategy). The participants were recruited to pinpoint the “average” user group’s expectations of future products using a card-based toolkit developed by the authors. The paper provides high-level design recommendations for connected household appliances derived from an analysis of prospective users’ expectations of smart appliances, focusing on what is desirable for mainstream users.

Two of the papers focus on the more intricate practice of designing connected, interactive products while also presenting more nuanced theoretical perspectives on design and designing.

In addition to the existing artifact-based (Jenkins, 2015; Jung et al., 2008; Ryan et al., 2009) and system-based approaches, a relational approach to the analysis and design of IoT artifacts and systems can allow us to understand IoT systems as systems of relations. The paper “Designing IoT Systems that Support Reflective Thinking: A Relational Approach” (Ghajargar, Wiberg, & Stolterman, 2018) aims at design for reflection (Hallnäs & Redström, 2001), or reflective thinking as a reference task of interaction (Whittaker, Terveen, & Nardi, 2000). Adopting a relational approach could improve understanding of what constitutes successful reflection support. This is of increasing interest in HCI and relating this to the domain of IoT can be seen as a novel and relevant contribution to advancing the development of such systems. The authors present a framework that allows designers to analyze and apply this modeling technique to the design of novel IoT artifacts and systems. The framework is based on four relational perspectives: augment me, comply with me, engage me, and, finally, make me think. The paper describes the first three perspectives with examples of implementations, and elaborates the fourth perspective through the presentation of an IoT lamp that has been designed and developed by the researchers. This illustrates the make me think relation as an addition to three other relations that can exist between user and object. This approach can examine the situated interactions that come along with a task or activity, which the user and the system perform collaboratively, and can contribute to design for reflection at a theoretical level. In fact, the authors indicate that these models are useful for analyzing as well as designing. Moreover, this approach can help designers understand how an IoT artifact interacts with its user and surroundings during the task and can activate nuances in the design of the form, texture, and functions between different types of artifacts and systems having different relationships with people.

The paper “Framing Smart Consumer Technology: Mediation, Materiality, and Material for Design” (Pandey, 2018) articulates a new, material-centric perspective on everyday experience with concrete technological artifacts in smart homes. Such artifacts are “always on” in terms of their sensing and connectedness, they have an “absent presence” connecting them to remote services that feature additional means of control, and they involve represented interpretations of data as contextualized data embedding and communication. The author weaves multiple distinct threads of design research and philosophical conversations into a new way to approach designing for domestic life, aiming to provide a clearer critical, reflective, and exploratory engagement with its design space. The paper underlines that a better understanding of the practical use and value of post-phenomenological mediation theory (PPMT) (Verbeek, 2015, 2016) for interaction design provides a contribution to design research. This is illustrated through two design explorations to better understand home technology from a post-phenomenological perspective. The paper presents a speculative design, Hearsay, that has been designed with the aim of exploring alternate non-utilitarian design possibilities and oppositions. This paper embraces intelligence and smartness of devices as material that is open and accessible for design—even apart from its relation to users and context.

Both papers focus more on interacting with and leveraging systems composed of increasingly large numbers of smart things, which is one core aspect of this special issue.

The fourth paper, “Addressing the Need to Capture Scenarios, Intentions and Preferences: Interactive Intentional Programming in the Smart Home” (Funk, Chen, Yang, & Chen, 2018) addresses the emerging problem that smart systems often cannot realize meaningful behavior because they lack contextual information from end users. This is exemplified in the observation that existing rule-based systems for programming the smart home are limited, as the complexities and messiness of everyday life in the home cannot be easily mapped to simple low-level rules. Especially when smart-home installations are used for a longer period, the number of devices and their programmed connections and relations grows. Yet individuals have different intentions and preferences in similar situations, and the same person’s intentions, values, and preferences might also change as circumstances change. A first-person account of such a real-life context is described in several iterations, demonstrating that there is a need firstly to investigate systems of things in different, primarily longitudinal settings, and secondly to let such systems access information about end-user behavior and needs differently. The authors argue that instead of crafting growing numbers of (fragile) rules, we need to enable smart-home systems to identify users’ intentions and to capture various usage scenarios. The paper introduces a domain model of scenarios, intentions, and preferences as part of Interactive Intentional Programming (IIP). IIP is a new approach to designing systems that captures semantically rich, high-level usage scenarios, intentions, and preferences in a novel domain model that can enable adaptive, smart behavior through machine intelligence. The authors explain that IIP is more than a conceptual design framework—it is a programming approach that is ripe for implementation in real-life technologies and integration in smart homes.

The fifth and last paper, “The Antecedents of Successful IoT Service and System Design: Cases from the Manufacturing Industry (Sayar & Er, 2018) turns the focus from the domain of smart homes and domestic personal spaces to manufacturing industry, where manufacturers have historically focused on at-scale product design (Roos, 2016). Nowadays they have to compete with continuous-delivery IoT services (Manzini & Vezzoli, 2003), and are often unequipped to do so. The authors characterize this sector through two “polar type” business cases in manufacturing (aeronautics and trucking) to demonstrate the need for IoT businesses to anticipate these two conditions for successful IoT system and service design. They argue that successful Internet of Things implementations require more than solid product design; they also require well executed system and service design (Baines & Lightfoot, 2013) to facilitate information and knowledge sharing with customers and partners. The authors highlight the importance of six factors—antecedents—when designing for IoT services and systems in the manufacturing industry context, ultimately enabling new ways for customers to participate. While design practitioners can translate these identified antecedents into impactful strategies for designing effective IoT systems, it is important to understand that by focusing too much on the IoT system design itself, they might miss the conditions by which businesses can succeed in IoT service delivery when applying insights for their own application context.

Reflection and Conclusion

This special issue features two important domains: the smart home, with connected intelligent products, and manufacturing industry, with its centrally planned, highly optimized business models (Sayar & Er, 2018). In the latter industrial context, smart things often appear as physical manifestations and extensions of a central cloud, as crystallized subscription-based business models and shallow interfaces that remotely feed input into a central processor holding and aggregating data and intelligence. The technology is designed and engineered according to industrial needs, logistics, and economies of scale. There is a strong dichotomy between customers and end users who, individually, have less agency and autonomy. Consequently, issues of end-user expectations (Coskun et al., 2018), designing user experience as a relational concept (Ghajargar et al., 2018), embodied cognition and intelligence (Pandey, 2018), personalization, end-user programming (Funk et al., 2018), and data privacy are of lesser concern for designers in this context. Large industrial parties pursue a transformation towards embedding technology into large-scale services and service offerings, that is, making all aspects of a business process available and malleable to algorithms. Algorithmic intervention at large scale is what drives innovation, and the mission of design is to facilitate this with low friction.

While this industrial context is of great importance economically and benefits from the established, stable paradigm of the Internet of Things, the context of the smart home can be seen as a more volatile breeding ground. Although it is a rather conventional context, it is more complex through patterns of growth, technological diversity, and inherent variety in end-user needs and behavior. Given the tendency to explore and experiment with new technologies, this context enables new designs that question and redefine earlier notions of connected products and systems. Indeed, in this special issue, we aim to move beyond the mainstream of IoT product design by reframing things and articulating design challenges in things anew. We appreciate that we found resonance in the journey to move beyond this in design, as the five papers in this special issue highlight different aspects of designing connected, intelligent products.

The challenges that we set out for this special issue still stand: design in the context of local IoT systems means designing systems of connected things that are primarily local and act locally for local interests. As a step beyond Interaction Design, this involves emergence of system-level capabilities and qualities. Designers will need to go beyond taking products at face value and address dynamic structural and behavioral complexities in design. New and newly grown IoT systems consist of networked artifacts that form their own ecologies and behave as populations rather than individuals. Apart from the systems perspective, local IoT systems are close to us, situated in our personal spaces, and yet offer highly distributed interfaces to sense and touch. Design needs to account for dealing with meaningful situated knowledge located in peoples’ everyday life; this has important implications for data sharing and semantics, and thereby for privacy and ethics.

Designing new qualities and experiences becomes even more important when considering possible intelligence and higher cognitive abilities emerging from an IoT system. Understanding how a (local) IoT might influence our subjective relationships with the world and our active relationships with other people needs to be balanced with the shape of a local techno-social culture in which connected things act with agency and perhaps even have a stance of their own.

Finally, the design of systems of smart things needs a notion of time; future IoT systems might be predefined only to a certain degree, allowing users and contexts to “fill out” the rest, resulting in a more meaningful local IoT that grows into its own niche within the surrounding ecosystem. Designers need to think about mappings of parameters, interfaces, and data that are individually, socially, and culturally dynamic. Future connected things will no longer be confined to a static existence in a product lifetime, but can become part of an evolutionary progression that takes turns with users, the thing ecosystem, and the environment.

We need to leave increasingly aged conceptualizations of Internet-connected things in the past. With this special issue we argue for changes in how we design for the Internet of Things and for systems, and how we appreciate things and systems of things as a worthwhile topic and direction of design research. We value that the authors and reviewers support our perspective, and hope that the contributions of this special issue also provide for setting and strengthening a research agenda towards richer, more systemic designs that embody intelligence, embrace locality and local data, and help us understand what the experience and aesthetics of future systems of smart things might be and might become.

References

  1. Baines, T., & Lightfoot, H. (2013). Made to serve: How manufacturers can compete through servitization and product-service systems. Chichester, UK: Wiley.
  2. Coskun, A., Kaner, G., & Bostan, İ. (2018). Is smart home a necessity or a fantasy for the mainstream user? A study on users’ expectations of smart household appliances. International Journal of Design, 12(1), 7-20.
  3. Eggen, B., & Kyffin, S. (2006). Experience design. In E. Aarts & J. Encarnação (Eds.), True visions: The emergence of ambient intelligence (pp. 359-376). Berlin, Germany: Springer. doi:10.1007/978-3-540-28974-6_19
  4. Frens, J. J. W., & Overbeeke, K. C. J. (2009). Setting the stage for the design of highly interactive systems. Retrieved from http://www.idemployee.id.tue.nl/e.i.barakova/Papers/Joep09.pdf
  5. Funk, M., Chen, L.-L., Yang, S.-W., & Chen, Y.-K. (2018). Addressing the need to capture scenarios, intentions and preferences: Interactive intentional programming in the smart home. International Journal of Design, 12(1), 53-66.
  6. Ghajargar, M., Wiberg, M., & Stolterman, E. (2018). Designing IoT systems that support reflective thinking: A relational approach. International Journal of Design, 12(1), 21-35.
  7. Hallnäs, L., & Redström, J. (2001). Slow technology: Designing for reflection. Personal and Ubiquitous Computing, 5(3), 201-212. doi:10.1007/PL00000019
  8. Janlert, L.-E., & Stolterman, E. (2017). Things that keep us busy: The elements of interaction. Cambridge, MA: MIT Press.
  9. Jenkins, T. (2015). Designing the “things” of the IoT. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 449-452). New York, NY: ACM. doi:10.1145/2677199.2691608
  10. Jung, H., Stolterman, E., Ryan, W., Thompson, T., & Siegel, M. (2008). Toward a framework for ecologies of artifacts: How are digital artifacts interconnected within a personal life? In Proceedings of the 5th Nordic Conference on Human-computer Interaction: Building Bridges (pp. 201-210). New York, NY: ACM. doi:10.1145/1463160.1463182
  11. Manzini, E., & Vezzoli, C. (2003). A strategic design approach to develop sustainable product service systems: Examples taken from the “environmentally friendly innovation” Italian prize. Journal of Cleaner Production, 11(8), 851-857. doi:10.1016/S0959-6526(02)00153-1
  12. Pandey, S. (2018). Framing smart consumer technology: Mediation, materiality, and material for design. International Journal of Design, 12(1), 37-51.
  13. Roos, G. (2016). Design-based innovation for manufacturing firm success in high-cost operating environments. She Ji: The Journal of Design, Economics, and Innovation, 2(1), 5-28. doi:10.1016/j.sheji.2016.03.001
  14. Ryan, A. (2014). A framework for systemic design. doi:10.7577/formakademisk.787
  15. Ryan, W., Stolterman, E., Jung, H., Siegel, M., Thompson, T., & Hazlewood, W. R. (2009). Device ecology mapper: A tool for studying users’ ecosystems of interactive artifacts. In Extended Abstracts on Human Factors in Computing Systems (pp. 4327-4332). New York, NY: ACM. doi:10.1145/1520340.1520661
  16. Sayar, D., & Er, Ö. (2018). The antecedents of successful IoT service and system design: Cases from the manufacturing industry. International Journal of Design, 12(1), 67-78
  17. Verbeek, P.-P. (2015). Beyond interaction: A short introduction to mediation theory. Interactions (ACM), XXII(3), 26-31. doi:10.1145/2751314
  18. Verbeek, P.-P. (2016). Toward a theory of technological mediation: A program for postphenomenological research. In J. K. B. O. Friis, & R. P. Crease (Eds.), Technoscience and postphenomenology: The Manhattan papers (pp. 189-204). Lanham, MD: Lexington Books.
  19. Whittaker, S., Terveen, L., & Nardi, B. A. (2000). Let’s stop pushing the envelope and start addressing it: A reference task agenda for HCI. Human-Computer Interaction, 15(2/3), 75-106.