The Aesthetics of Coupling: An Impossible Marriage
Lukas Van Campenhout 1,*, Joep Frens 2, Kristof Vaes 1, and Caroline Hummels 2
1 Product Development, Faculty of Design Sciences, University of Antwerp, Antwerp, Belgium
2 Industrial Design, Eindhoven University of Technology, Eindhoven, The Netherlands
This paper is about embodied interaction with digital products and systems, and more specifically, about its aesthetics. We establish a particular form of aesthetics: the aesthetics of coupling. We start with illustrating them by presenting two design research projects, the rich actions camera and the Experimental Payment Terminal. We discuss the characteristics of the aesthetics and coin their corner stone concepts, the persistent and the temporal. After that, we present the design process of two conceptual products: a night lamp and a medication dispenser. Both projects demonstrate how to design for the aesthetics of coupling. In a reflection, we position the aesthetics of coupling among several existing perspectives on interaction aesthetics and elaborate on their significance for the design community. We conclude this paper by highlighting the broad lines of our framework and by indicating directions for future research.
Keywords - Aesthetics of Interaction, Design for Interaction, Embodied Interaction.
Relevance to Design Practice - This research approaches the balancing of the physical and the digital in a pragmatic way. By means of four design projects we directly illustrate the main themes of our stance in this matter. These projects are easily accessible and can inspire design students, design researchers and professional designers.
Citation: Van Campenhout, L. D. E., Frens, J. W., Vaes, K. R. V., & Hummels, C. C. M. (2020). The aesthetics of coupling: An impossible marriage. International Journal of Design, 14(2), 1-16.
Received January 3, 2020; Accepted June 27, 2020; Published August 31, 2020.
Copyright: © 2020 Van Campenhout, Frens, Vaes & Hummels. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content is open-accessed and allowed to be shared and adapted in accordance with the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.
*Corresponding Author: firstname.lastname@example.org
Lukas Van Campenhout is tenure track professor at the University of Antwerp, Faculty of Design Sciences. His research and teaching centers on ‘the enriching physical’, or the role and meaning of the physical world in human interaction with digital products and systems. In addition, he has been working for two decades as an industrial designer of a wide range of products at Achilles Design (www.achilles.be). This work received numerous international design awards (IF 2005, IF 2009, red dot 2012, IF 2013). In 2017, Lukas founded Ground Eight (www.ground-eight.com), the research center of Achilles Design.
Joep Frens is assistant professor at Eindhoven University of Technology. His research focuses on the question of ‘how to design for open and growing systems’. He teaches courses on (interaction) design on all academic levels and advises a number of PhD students. In the academic year of 2014-2015 he held the Nierenberg Chair of Design at the Carnegie Mellon University School of Design. When he sees a sheet of cardboard he makes a model out of it.
Kristof Vaes obtained his Master’s degree in Product Development (Antwerp) in 1996. He worked 15 years for several design teams and offices in Flanders. After developing the necessary expertise in concept development and engineering, his interest evolved towards the human side of product design. In 2014, he received his doctoral diploma in industrial design engineering from Delft University of Technology and University of Antwerp (Product Development) on a thesis titled ‘Product Stigmaticity: understanding, measuring, and managing product-related stigma’. Presently Kristof is a full-time tenure track professor at the Faculty of Design Sciences, University of Antwerp. His academic expertise focuses on human-product interactions, product semiotics, design for wellbeing and inclusive design. His research focuses on the physical and semiotic aspects of the ‘augmented human’. The aim is to go beyond stigma-free design or design for social acceptance to design for empowerment and augmentation.
Caroline Hummels is professor Design and Theory for Transformative Qualities at the department of Industrial Design at the Eindhoven University of Technology (TU/e). Her activities concentrate on designing and researching transformative practices with a focus on being-in-the-world theories, embodied interactions, technology-in-becoming, participatory sensemaking, aesthetics, and social resilience. Her recent quest aims at developing with her team a design-philosophy correspondence, in which not only postphenomenology informs design practice, but also design practice is used to philosophise and to explore, inform and build postphenomenological concepts (i.e., design-informed postphenomenology). Caroline is founder and member of the steering committee of the Tangible Embedded, and Embodied Interaction (TEI) Conference, editorial board member of the International Journal of Design, and member of the Dutch Design Week sounding board. Moreover, she has given a large number of keynote speeches, invited lectures and workshops at conferences, international universities and for industry and governmental institutes worldwide.
This publication is dedicated to the late Kees Overbeeke and was written in loving memory of Herman Van Campenhout.
The aesthetics of coupling are situated in the area of embodied interaction with digital products and systems, more specifically in its aesthetics. According to the framework we establish, the interaction with these products and systems contains events of two kinds. The first kind of events “feel physical”. An example of such an event is the interaction with a physical push button. When the button is pushed in, one feels its movement and resistance. The second kind of events “feel digital”. As an example, we refer to the alternating appearance of different images on a display. When the pushing of the button causes the alternation of the images, both events are considered coupled. In this paper we advocate that, from this coupling, a specific form of beauty can emerge, which “feels physical as well as digital”. It is this intrinsically dual nature that makes the aesthetics of coupling unique. It is also this dual nature that situates the aesthetics of coupling in the very core of embodied interaction itself, which centers on the balance between the physical and the digital. The aesthetics of coupling can only be established when this balance is closely monitored and exploited in the design process of the product or system. Therefore designers and design researchers should be well aware of their nature. This paper aims to stimulate such an awareness.
Before proceeding, we indicate our own position within the research domain of embodied interaction, explain what we mean by coupling and provide some background on aesthetics of interaction.
Embodied interaction aims for a deep integration of digital phenomena in the physical world (Dourish, 2001; Ishii & Ullmer, 1997; Ishii, 2008). The idea is that, when digital phenomena are spread out over the physical world, people can interact with them in a way that is more natural and intuitive “than the abstract and symbolic style of interaction” (Dourish, 2001, p. 206) that is proposed by traditional HCI (Hornecker & Buur, 2006). In a previous publication (Van Campenhout et al., 2019), we nuanced the generally accepted connection of embodied interaction with pragmatic usability, or naturalness and intuitiveness. We claimed that the argument to implement an embodied interaction approach can be an emotional one. This argument relates to the intrinsic characteristics of the physical world: its persistence, slowness and fragility (Van Campenhout et al., 2013; Van Campenhout et al., 2016). These characteristics are at odds with the temporality (Vallgårda & Sokoler, 2010), speed and reproducibility of the digital world. By employing the physical world as a medium in human interaction with digital systems, the digital is constrained and channeled. We argued that this channeling makes human interaction with the digital more meaningful and valuable. The aesthetics of coupling build further on this reasoning.
The application scope of the aesthetics of coupling includes all interactive products and systems. With coupling, we don’t refer to the definition that Ishii and Ullmer (1997) give, namely the coupling between digital information and physical objects. Our definition of coupling starts from the Interaction Frogger framework (Wensveen et al., 2004; Djajadiningrat et al., 2007), and extends it. Wensveen et al. establish coupling as the connection between a user’s action and a product’s function. In this framework, coupling contains six aspects: time, location, direction, dynamics, modality and expression. According to Wensveen et al., when action and reaction are unified on each of these six aspects, the emerging interaction feels natural and intuitive. In our discourse, coupling is not limited to the connection between a user’s action and a product’s reaction. We consider coupling more generic as a connection between events in a user-product interaction routine. Whether these events are caused by the user or the product, is not important. Moreover, we distance ourselves from Wensveen et al.’s strive to maximize naturalness and intuitiveness in interaction. The aesthetics of coupling violate the rules of Wensveen et al.’s framework and question the ideal image of naturalness. They pave the way for emotional values in interaction, rather than pragmatic usability.
Aesthetics of Interaction
As to what aesthetics of interaction are and what they are not, a lot has been written and discussed. For an exhaustive overview of different perspectives on aesthetics of interaction, we refer to Carroll (2008), Hummels and Overbeeke (2010b), and Lenz et al. (2014). In this introduction, we want to provide the reader with a simple, grounded definition that fits the context of this paper. Traditional industrial design focuses on products, their form and their function (Heskett, 1980). It depends upon the “modelling of dimensions in space” (Jones, 1992, p. XXXII). Traditional design aesthetics then concerns basic aesthetic principles of static form, or aesthetics of appearance (Djajadiningrat et al., 2004; Locher et al, 2010): 2D and 3D form and composition, color, visual balance and coherence, visual and tactile properties of materials and textures etc.. Interaction design focuses on the interaction of humans with digital products and systems. This interaction supposes physical movements and as such happens in time (Hallnäs et al., 2002; Jones, 1992). Interaction design aesthetics, or aesthetics of interaction, then offers designers aesthetic principles that play out over time. These principles are related to bodily actions, physical and virtual movements and physical form changes (Djajadiningrat et al., 2007). We position the aesthetics of coupling as one such principle.
Structure of This Paper
This paper contains five sections. First, we pinpoint the aesthetics of coupling by indicating them on the rich actions camera and the Experimental Payment Terminal, two conceptual designs by respectively the second and the first author. Next, we provide a definition for the aesthetics of coupling and discuss their characteristics. We coin the persistent and the temporal as two possible manifestations of digital processes in the physical world and define the aesthetics between them. In the third section, we illustrate how to design for the aesthetics of coupling by presenting two design projects: a night lamp and a medication dispenser. In the fourth section, we reflect on our framework and discuss its uniqueness and importance. Finally, we make the connection with our past research and reveal where our future work is going.
The Aesthetics of Coupling: Two Illustrations
A Precedent: The Rich Actions Camera
The aesthetics of coupling were made explicit throughout our recent work. There was a precedent though: the rich actions camera of Joep Frens.
Frens (2006) defined rich interaction as a design approach for digital products and systems, which pays equal attention to their three main properties: form, function and interaction. This way, rich interaction opposes itself to traditional industrial design thinking, which only puts forward form and function as main design drivers (Heskett, 1980). In practice, this implies that for an interactive product, its interaction can and should be designed, in dialogue with the design of its form and function. Frens concludes his dissertation with the open question of what aesthetic interaction exactly is. He says: “The way how action and feedback are coupled contributes to aesthetic interaction” (p. 180). He gives the example of his rich actions camera, and more specifically, how the user takes a picture with it and saves this picture. We describe it here. A video of this interaction routine can be found at https://vimeo.com/51049251.
The user directs the camera at the object or scene that he wants to capture (Figure 1, left). The camera’s display, which is aligned with its lens, shows in a steady stream what the camera registers. The user takes a picture by pushing the trigger that holds the display. When the trigger is pushed, the display flips away from the lens, and is aligned with the memory card (Figure 1, right). On the display, the picture is frozen. When the user wants to save the picture, he pushes the display, with the picture, towards the memory card (Figure 2, left). When the display touches the memory card, the picture seems to slide from the display in the memory card (Figure 2, right). This specific action routine, and later on the idea of aesthetics residing in coupling, formed the start of the research process that lead to the aesthetics of coupling.
The Experimental Payment Terminal and Its Aesthetics
The Experimental Payment Terminal (EPT) was designed by the first author (Figure 3). We present its interaction routine and highlight five events that reveal the aesthetics of coupling.
The EPT comes with a token (Figure 4), which is the property of the customer, and which can be used to execute a payment interaction on the terminal. The payment terminal is integrated in a rectangular wooden case, which acts as shop counter. It consists of three modules: the vendor module, the customer module and the traveler (Figure 4). We now describe the EPT’s interaction routine. A video of this routine can be found at https://vimeo.com/382383217.
When not active, the traveler rests against the vendor module and both modules form one visual unit with a similar shape to the customer module. Both entities, the vendor module and the traveler versus the customer module, are positioned opposite to each other. The whole setting is visually symmetrical and well balanced (Figure 4).
Event 1: The Movement of the Traveler (Figure 5, left)
The vendor manually enters the price on his keypad. The vendor then grabs the traveler with the price displayed on it and moves it towards the customer. As the traveler moves, the price rotates on the display. The speed and direction of this rotation are coupled to the speed and direction of the traveler’s movement. At the end of this movement, the price has rotated 180° and is oriented towards the customer. For both users, it feels as if the on-screen image is coupled with a gear that is driven by the movement of the traveler. Yet, it is obviously clear that the rotating entity is just a representation and not a real mechanical component.
Event 2: The Placement of the Customer’s Token (Figure 5, right)
The customer fetches his token and places it in the cradle of the customer module. While he does this, the green liquid, representing the money on the customer’s bank account, seems to rise up out of the token. The placing of the token and the on-screen movement of the liquid are naturally coupled, but there clearly is no liquid in the token and the liquid on the display is virtual.
The customer now sees two circular displays in front of him, one on top of the other. The lower one shows the money on his account and the upper one displays the price.
Event 3: The Movement of the Slider (Figure 6, left)
To execute the payment transaction, the customer pushes the slider with the token in the direction of the vendor. During this action, the slider is shown on the customer display, pushing the liquid forward. An illusion is created of watching the slider through the display, as if the latter were a transparent window. As the slider moves forward, it transforms and adopts an on-screen shape. The movement of the slider, its speed and direction, is naturally coupled with the movement of this shape. Yet the user knows he does not see the real slider, but an on-screen representation of it.
Event 4: The Injection of the Money in the Traveler Display (Figure 6, right)
When the customer pushes the slider further, the green liquid is injected in the traveler display. It flows from one display to another, while both displays belong to physically separated entities. The impossibility of this movement is reinforced when the traveler display is filled up, and separates itself from the customer module to move towards the vendor (Figure 7, left). The whole event seems natural, but at the same time, it is clear that no real liquid is injected.
Event 5: The Automatic Movement of the Traveler (Figure 7)
When the traveler reaches the vendor module, it abruptly stops its automatic movement. This stopping generates a natural collision sound, after which the green liquid immediately flows in the vendor module, as if the inertia of this liquid causes the continuation of its movement. Again, the on-screen movement seems to obey to the natural laws of gravity and mass, but no real liquid is moved.
At the end of the transaction, the traveler display shows a “Payment successful!” message, while the customer display shows a lower liquid level, and a new account status. Finally, the customer takes back his token.
All these events contain a physical movement, either executed by the user (events 1, 2, 3 and 4), or automatically actuated (event 5). In each event, these physical movements go together with on-screen movements. Both movements are coupled, and the coupling between them has a natural character to some extent, but feels artificial at the same time. Let us explain this in detail.
The physical and on-screen movements in the EPT are perceived by the user as clearly distinctive. The physical movements feel physical, since they cause a real relocation of separate physical entities: the token, the slider and the traveler. The on-screen movements feel digital, since they are only representations of moving things and are able to appear and disappear, without leaving traces. This inherent contrast prohibits physical and on-screen movements to be perceived as one and the same phenomenon. The user feels this contradiction between them and the impossibility of unifying them. Yet, it is also clear for the user that both physical and on-screen movements are designed to harmonize with each other in such a way that they are perceived very close to one another. They are naturally coupled on several aspects: time, location, direction etc. (Wensveen et al., 2004). This belonging to each other and at the same time being intrinsically different make up a fascinating and engaging tension field. This tension field plays uniquely on the border of the physical and the digital world, and thus must be conceived as immanently present in this border. The purposeful steering of this tension field by the designer is what causes the user’s aesthetic experience, which we call the aesthetics of coupling.
The Aesthetics of Coupling: Definition
Aesthetics by Coupling
The beauty and engagement that resides in the physical world, has a certain character. Watching a butterfly crawling out of its chrysalis, walking bare-foot on wet grass on a spring morning or enjoying good food can be aesthetic experiences. They feel physical, as they are inherently connected to the persistence and tangibility of the physical world.
In contrast, the engagement that comes from the digital world has another character. A gracious animation or a consecutive series of intensely colored images on a display can be strikingly beautiful. This beauty feels digital, since it has the fluent, temporal character of the digital world.
A digital product can offer both kind of aesthetics, physical and digital, at the same time. Feeling the weight of your tablet computer on your lap and touching the cold, fine texture of its aluminium housing causes a pleasing sensation of quality. Watching pictures on it from a summer holiday in a warm country, with an overwhelming nature and rich colors, gives an aesthetic experience as well. The aesthetics of coupling though, belong to neither of these categories. They reside only in products where the physical and the digital are combined in a specific way. We want to make the comparison with a pop song. What makes out the appeal of “Drive” by REM? It certainly is not only the music, which, for any classic music lover will be rather basic. Nor is it only the lyrics, since they seem quite ordinary when they are printed out on paper and read without the accompanying music. The real power of this song lies in the combination of words cited/sung against the background of music. The music and the rhythm lend the words a specific, unique character and vice versa. It is this combination that strikes so many people in such a profound way that they have a hard time explaining what they are struck by.
The Persistent versus The Temporal
The physical world is tangible, static and persistent, while the digital world is intangible, dynamic and transient. Both worlds’ characters are each other’s opposites, and contrast in more than one way. As Vallgårda and Sokoler (2010) put it, digital processes in their original form are not perceivable for us. In order to become perceivable, they need to be translated to a perceivable form. We state that how this translation is realized, is up to the designer. He/she has two possibilities:
- The designer can choose to emphasize the intrinsic digital character of these processes, their fluency and transience, by choosing a medium that fits this character and is able to fully capture it: light, sound, on-screen and projected stills and animations. The two displays on the Experimental Payment Terminal are examples of this approach.
- The designer can also choose to re-materialize, or to translate these processes to a graspable, physical form that does not capture all the aspects of their digital character, but instead offers other assets, such as expressiveness or physical affordances. An example of this is the traveler in the Experimental Payment Terminal, which embodies the transfer of money from the customer to the vendor.
Both approaches result in physical events that are manifestations of digital processes in the physical world. However, the character of these events, i.e. the way we perceive them, differs according to the approach. Events that result from the first approach, feel temporal and fluid. We refer to them as temporal events. Events that result from the second approach, feel permanent and static. We call them persistent. In the same line, we indicate the whole of temporal events as the temporal and the whole of persistent events as the persistent. From an industrial design point of view, we consider the discord persistent–temporal as more accurate than the discord physical–digital, and bring it into use henceforth. We further discuss our reasoning that lead to these terms in the discussion at the end of this paper.
An event can never be persistent and temporal at the same time. This tension between both concepts is exactly what makes out the aesthetics of coupling. The designer couples both persistent and temporal events in such a way that the resulting event appears to be persistent as well as temporal. In that way, the aesthetics of coupling lie in the deliberately naïve strive towards an impossible marriage, with the product acting as if this marriage is possible. This coupling of both types of events is nothing more than intuitively bringing the persistent and the temporal very close to each other, so that the intrinsic contrast that exists between them can surface in the interaction. Once both realms are close enough, they seem to touch each other, as the persistent flows in the temporal and the other way round. At this point, the border between them is not a solid wall, but a thin membrane that is permeable at certain spots. The clearest example of this phenomenon in the EPT is the moment where the slider is pushed forward and transforms into its on-screen incarnation (Figure 6, right). It feels like pushing the drawer right through the membrane, from the persistent into the temporal.
Why is this ostensible union of the persistent and the temporal so engaging? Its inherent impossibility, gives it a magic feel. We explain this further. The concept of familiarity is common in embodied interaction research circles. When composing interaction routines with digital products and systems, designers use or simulate properties of the physical world in order to make these routines natural and intuitive (Dourish, 2001). The emerging interaction then draws upon “our common sense knowledge about the physical world.” (Jacob et al., 2008, p. 202). The aesthetics of coupling now too seek to exploit this common sense knowledge, but at the same time they violate it, by deliberately exposing the temporal. The resulting event thus has an irrational, impossible flavor, since the feeling of familiarity is not only reinforced, but also questioned. The enchantment that comes with the aesthetics of coupling, comprises feelings of magic, surprise and emotion (Stienstra et al., 2012; Wensveen et al., 2004) but also of commotion, insecurity and alienation. We believe beauty comes in different forms and agree with Overbeeke et al. (2005) when they say that emotions may not be “narrowed down to fun” (p. 9).
While designing a digital product, it is the task of the designer to decide which digital processes will be assigned to the persistent and which ones to the temporal. In other words, the designer continuously has to make judgements and decisions on where the border between the persistent and the temporal is to be situated. An awareness of the aesthetics of coupling may be helpful in making these decisions.
An Immanent Presence
The persistent and the temporal are coupled to each other through design, and in this coupling reside the aesthetics. The aesthetics of coupling are autonomous and immanently present in the product or system. They are always already there, since they start from the product’s most fundamental building stones: the persistent, the temporal and the tension between them. In that sense, the aesthetics of coupling are not an expression of a certain feeling or emotion. They are a composition of persistent and temporal characteristics. The visceral, pre-reflective enchantment that stems from them is a consequence of this composition, and does not need further explanation (Hummels & Overbeeke, 2010a; Van Campenhout, 1999). It is the task of the designer to engage in creating this composition and make the aesthetics visible. In the next section of this paper, we show how to design for the aesthetics of coupling.
Designing for the Aesthetics of Coupling
We now present two conceptual design projects in which the aesthetics of coupling were a design driver: a night lamp and a medication dispenser.
The night lamp was designed by the first author together with Floor Van Schayik, a 2nd year master student Human Technology Interaction at Eindhoven University of Technology. A video of the night lamp’s interaction routine can be found at https://vimeo.com/382383674. We describe the routine here.
The lamp is situated on the night table, next to the bed in a children’s room. At the beginning of the interaction routine, the lamp is deactivated (Figure 8, left).
To switch on the light, the user (child or parent) twists the small sphere on the top of the lamp counter clockwise over 30° (Figure 8, middle). The light inside the lamp’s body switches on. The lamp is now in reading mode (Figure 8, right). The child can read a book, or his parent can read a bedtime story to the child.
When the child goes to sleep, he or his parent grabs the sphere and moves it upwards (Figure 9, left). The light inside the lamp’s body moves along, and, following the lamp’s shape, becomes smaller and dims (Figure 9, middle). The sphere’s upward movement reaches a stop and the light jumps from the lamp’s body into the sphere (Figure 9, right). At this moment, the lamp has become a night light.
When the child needs to visit the toilet during the night, he easily finds the lit sphere in the dark and he pushes it down (Figure 10), pushing the light back in the lamp and thus bringing the lamp to reading mode (Figure 8, right).
In the night lamp, the aesthetics of coupling manifest themselves in the union of the sphere’s physical movement and the movement of the light inside the lamp. Both movements occur at the same time and place and share the same direction. In the perception of the user, they are mechanically coupled and act as one. At the same time, the user clearly perceives that the physical lamp and its moveable parts belong to the persistent, while the light and its movement are temporal. The intrinsic contrast between both and the impossibility of their natural coupling is highlighted when the light reaches the top of the lamp’s body and jumps in the sphere.
Our basic idea was to combine a children’s reading lamp and a night light in one product. In the physical shape change from one state to the other, the aesthetics of coupling would become visible. The lamp’s hardware and its shape change was to be the persistent and the light within the lamp was to be the temporal.
One of the early concepts consisted of two spheres, a large and a small one. When the lamp is in reading mode, the large sphere gives light (Figure 11, left). When the child moves the small sphere upwards, he “pulls” the light out of the large sphere, into the small one (Figure 11, right). At that moment, the lamp has become a night light. We quickly materialized the idea in a foam model to have a first impression of the overall dimension of the lamp, and the upwards stroke of the small sphere.
At that time, we decided to explore the temporal, or the light inside the lamp. We made an experience prototype by moving a light bulb up and down in a glass bottle. It was striking that, when the light moved upwards, it reached the narrow part of the bottle and its shape reduced (Figure 12). This way, the temporal underwent a shape change as well, just like the persistent.
With this narrowing shape in mind, we did some form studies in CAD of what the lamp could be like (Figure 13, left). It became clear that we would leave the earlier idea of the two separate spheres. Instead, we would go to a more coherent, unified overall shape. We wanted to, as much as possible, keep this idea of formal coherence through all stages of the handle’s upward movement. We made a dynamic cardboard model (Figure 13, middle and right), based on the CAD geometry, in order to get a grip on the shape of the lamp and the movement of the handle.
At this point, we decided to unify the persistent and the temporal in one model. With the cardboard model as a guide, we made a hollow lamp body in translucent polymer clay, which was hardened in an oven. At the same time, we made an Arduino prototype with a slide potentiometer and a series of white LEDs. We mounted the Arduino assembly in the lamp model and built an experience prototype that materialized the lamp and its interaction (Figure 14). For the first time in the development process of the lamp, we experienced the appeal of the aesthetics of coupling.
We tuned the lamp’s shape further in CAD, split it up in functional parts and developed the technical concept. The lamp’s body parts were 3D printed and the base with the Arduino components was made in wood.
We want to draw the attention to a specific design approach that emerged in the development process of the night lamp: the persistent and the temporal were approached as separate entities and were allowed to compose each other. The result of this design approach is the fact that the very shape of the lamp is determined by the movement and shape change of the light. In retrospect, this dualistic design approach could be reflected in the conceptual sketching and the technical detailing of the lamp. The sketches could contain persistent and temporal content, for example, represented in different colours (Figure 15, left). The technical detailing could contain persistent dimensions, describing the physical geometry of the lamp, and temporal dimensions, describing the shape and movement of the light. This can be seen in Figure 15, right , where 80mm is a persistent dimension, being the stroke of the sphere. The 215mm dimension is a temporal one, being the stroke of the light. Both movements happen in the same time span, so the mean velocity of the light is 2.7 times higher than the mean velocity of the sphere. In reality, this ratio can be dynamic, since it can be digitally controlled. The optimal relation between both movements can only be determined by experience, on the prototype itself (Stienstra et al., 2015).
An-Sofie Rombouts, Emiel Tormans and Lucas Van Dorpe designed Piriwit, a medication dispenser for older people who live independently in a housing. Piriwit was the result of a design project that the first and the third author organised for the 1st year master students Product Development at the University of Antwerp. The end result of the project can be seen at https://vimeo.com/382383894. We briefly describe Piriwit and its interaction routine.
The Piriwit system contains a wall clock, a pill box and a smartphone application. The wall clock features a large round display, and a bird-like object in its center. The pill box contains five drawers for five different types of pills. The wall clock and the pill box are installed in the housing of an older person (figure 16), The wall clock is hung against the wall and the pill box is placed on a shelf underneath it.
Once a week, the older person’s care giver visits her housing and fills up her pill box (Figure 17, left). After that, with the Piriwit application on his smartphone (Figure 17, middle), the care giver uploads her weekly medication data into the wall clock. He does this by literally pouring her data, visualized by a white fluid, from his smartphone into the clock (Figure 17, right).
The clock reacts by taking over the white fluid and rotating it clockwise. The bird at the center of the clock is actuated along three rotations and follows the rotating movement of the white fluid (Figure 18, left). At the end of the rotation, the drop seems to fall in the bird, which makes a swallowing movement (Figure 18, right).
Next, the bird throws four different medication moments, on the clock, each of them visualized as a white pill. Each medication moment is now situated at a specific hour, and the display shows clock arrows (Figure 19).
When it is time to take medication, the white pill in question rotates around the bird, while the bird follows its movement (Figure 20). After a full rotation, the white pill seems to fall from the display into the pill box beneath the clock. It falls through the pill box, lighting up every drawer that it passes, until it reaches the right drawer (Figure 21). The older person can now take the pill out of the pill box (Figure 22).
In Piriwit, the persistent contains the physical wall clock with the moving bird in the middle, the pill box with the five movable drawers, and the smartphone. The temporal contains the on-screen visualization on the smartphone and on the wall clock, and the lights in the drawers of the pill box. Throughout the interaction routine, persistent movements (user movements and the bird’s actuated movements) and temporal (on-screen and light) movements are continuously intertwined in a directed choreography. Within this range of movements, the aesthetics of coupling come to the surface.
The design process of Piriwit was driven by the orchestration of the persistent and temporal, and more specifically, by their respective movements. Especially the movements of the bird were a challenge. The bird rotates around three axes, and its movements needed to look natural and be synchronized with the temporal movements on the smartphone and the wall clock (Klooster & Overbeeke, 2005; Hummels et al., 2007).
Initially, the students planned to numerically code the movements of the bird. They soon realized that this coding would take a large amount of time and would most likely result in an artificial looking motion.
Therefore, they introduced a joystick (Figure 23). The joystick is sized appropriately to match the proportions of the bird, and features three rotational potentiometers on the same location of the bird’s servo motors. This joystick allowed the students to puppeteer the bird in real-time, by copying the joystick’s movement to that of the bird. With this technique, the students were able to easily program complex movements and have the bird move in a natural way.
In Piriwit, the aesthetics of coupling led to a routine of actuated movements, belonging to the persistent: the movements of the bird. These movements are complex, since they are designed to follow and go along with the temporal movements that appear on the round display and on the smartphone. At a certain moment in the process, the students realized that they had to develop a new way to design these movements. This insight was a crucial element in their design process, that defined the further course of it.
In the following, we present three established viewpoints on aesthetics of interaction that influenced our work and position the aesthetics of coupling with respect to them. Additionally, we discuss the uniqueness and importance of the aesthetics of coupling, and reflect on the terms temporal and persistent.
Three Viewpoints on Aesthetics of Interaction
User Experience (UX)
Hassenzahl (2010) established user experience as a way of broadening the common understanding of usability in the context of interactive products and systems. He states that products and systems should not merely fulfil a pragmatic task. Their task should be to realize the psychological well-being of their users. By advancing experience design (Hassenzahl et al., 2013), Hassenzahl provides handles on how to design for user experience. In the experience design approach, the designer consciously aims to create a product or system that fulfils universal human needs like the need for autonomy, competence and relatedness.
The bridge between user experience and aesthetics of interaction is made as Lenz et al. (2013) argue that “beauty in interaction can only emerge, when the interaction is in line with the overarching user experience“ (p. 133), in other words, aesthetics of interaction and user experience are fundamentally linked. Their relation is further explored. Hassenzahl distinguishes three different levels of interaction (Hassenzahl, 2010; Lenz et al., 2013; Lenz et al., 2014):
- The what-level focuses on the goal of the interaction routine, and the functionality that is needed to accomplish it. For example, the main goal of a common digital hand camera is to shoot home-movies. Therefore, it is a mobile device, has substantial storage capacity, is compatible with a standard PC, etc..
- The how-level addresses the physical interaction itself. It considers the sequence of actions needed to obtain the above-mentioned do-goal. In the example of the hand camera, this approach studies which controls the user would manipulate to make a home movie, in which order, what feedback the device offers etc..
- The why-level studies the experiences and emotions that emerge from the interaction and that make up the deeper meaning of the digital device. The owner of the hand camera is able to produce a fluent and engaging home movie, which gives him a feeling of confidence, pride and autonomy. Aesthetic experiences occur on this level.
Lenz et al. (2013) suggest a connection between the how and the why, or between the perceptual and action attributes of the interaction itself and the emerging experience. In other words, high-level experiences, and amongst them, aesthetic experiences, are believed to be dependent on low-level interaction attributes. The aesthetics of coupling illustrate this stance. The persistent and temporal events are low-level interaction attributes and the resulting aesthetics are high-level experiences. The aesthetics depend on the coupling between the events. Lenz et al. ask for more design-oriented knowledge of this relationship. The section “Designing for the aesthetics of coupling” conveys such knowledge, as it clearly illustrates how the aesthetics of coupling act as a driver in a design process.
Computational Technology as a Design Material
Hallnäs (2011) discusses the aesthetics of interaction at a fundamental level. He nuances the experience design perspective, claiming that user experience measurement can never entirely define the conception of an interactive product. He explains that, inside each interactive product, there is an element that determines the product’s interaction and that makes user experience possible. This element is the computational technology itself (Landin, 2009). This computational technology, or computational material (Vallgårda & Sokoler, 2010), depends on the execution of programs, and therefore has its own characteristic: it changes over time, as people interact with it, and as such forms temporal structures. Hallnäs et al. (2002) call this phenomenon temporal gestalt, and the computational material is a temporal material (Hallnäs, 2011). The task for design research now is to investigate how this temporal material builds interactions.
Hallnäs’s theory is taken further by Vallgårda, who establishes her material strategy (Vallgårda & Sokoler, 2010). She refines Hallnäs’s and Redström’s ideas by bringing forward the computational composite. The above defined computational material generates digital processes, continuously locating and relocating electrical charges on a PCB. Since these processes are not perceivable for the human sensorial apparatus, designers have to couple the computational material to another material, which people actually can perceive. This alloy of two materials makes up the computational composite. Vallgårda (2014) developed numerous projects, in which this concept is illustrated. As the second material with which the computational material is connected, she typically chooses traditional, artisanal materials like wood (e.g. PLANKS), copper (e.g. Copper Tiles) and textile (e.g. Bonad, the Dress Room). The results are surprising and beautiful installations, that look and feel very physical and earthy, but at the same time embody temporal gestalt.
From this framework, we adopted the term temporal, and expanded its application scope. Also, we retain the idea that aesthetics of interaction are constituted by an immanent aspect of digital technology: its fluency, or tendency to change over time. The aesthetics of coupling connect the temporal, fluent nature of digital technology itself with the static, invariable character of the physical world. They are fuelled by the contrast between these extremes.
Ethics and Aesthetics of Interaction
Ross and Wensveen (2010) approach aesthetics of interaction from a particular theoretical perspective: Shustermans pragmatist aesthetics (Petersen et al., 2004; Shusterman, 2000), which distances itself from the traditional viewpoint of analytic aesthetics. Central in the viewpoint of analytic aesthetics, is the concept of contemplation. One contemplates a product, which means that one considers its aesthetic appeal as something that autonomously resides in the product itself, isolated from its physical context. The analytic aesthetic experience gives immediate satisfaction (Petersen et al., 2004) and is disinterested, since it is valued for its own sake, instead of serving a purpose outside itself. In industrial design practice, analytic aesthetics may be reduced to visual appearance and may ultimately lead to formalism (Ross, 2008). On the contrary, pragmatist aesthetics, as Shusterman defines them, do not reside within products or artifacts, but rather emerge in human interaction with them (Overbeeke, 2007). Aesthetics are not an abstract, autonomous notion, provided by a single artifact. People experience them while interacting with their everyday surroundings. At the same time aesthetics have, next to intrinsic value, a significance in daily practice (Ross & Wensveen, 2010): an aesthetic experience engages people and stimulates them to reach the goal they pursue. Based on Shusterman’s insights, Ross and Wensveen define aesthetics of interaction as firmly rooted in the socio-cultural context of a product. Since aesthetics of interaction emerge in practice, they have practical consequences: they influence people’s interaction with the products that surround them, and their relation with other people. Therefore, they bear an ethical dimension. If the design of a product can steer its user’s behavior, it should be carefully considered from an ethical point of view. The connection between ethics and aesthetics of interaction forms the very backbone of Ross’s dissertation (2008).
The importance of this framework for our thought development is twofold. First, it advocates a kind of aesthetics that resides in human interaction with products, and not in products themselves. The aesthetics of coupling are of this type, since they happen over time in the coupling between different events. Second, pragmatist aesthetics have a functional connotation, as they elicit certain actions and prohibit others, and as such, prescribe human behavior. This functional connotation is clearly present in the aesthetics of coupling. As discussed before in this paper, the coupling with which they are inherently connected is natural to some extent. Since the aesthetics are the realization of this coupling, they have to contain this partly natural character. That means that they are not completely arbitrary and should be approached as such by the designer. They are, up to a certain degree, determined by the functionality of the product, as they contribute to its functional expressiveness and inherent meaningfulness.
Uniqueness and Importance
The uniqueness of the aesthetics of coupling lies in the fact that they do not stem from the persistent, nor from the temporal. They do not feel uniquely “physical”, nor “digital”. They only exist in the tension field between both event types. This means that designing for the aesthetics of coupling means striving towards a balance between the temporal and the persistent. Designing a digital product to feel as “persistent as possible” is not a good way to reach the aesthetics of coupling. The opposite approach, emphasizing temporal events in order to make the product more versatile, is equally unfit.
In a world where on-screen information becomes increasingly present and where smart glasses are on the verge of a breakthrough, the importance of the temporal in digital systems increases. We believe that the number of couplings between temporal and persistent events will multiply and the aesthetics of these couplings will gain relevance. As such, the temporal as a design driver for the persistent possibly might be a commonly accepted phenomenon within ten years. What would this mean for the professional designer of digital products and systems? In order to allow the temporal and the persistent to compose each other, we advocate a simultaneous development of hardware and software. Both departments would grow towards each other, resulting in overlapping activities. Product designers would make physical prototypes in which on-screen or virtual graphics play an important role and software designers would be involved in physical product design and mechanical development.
A Designer’s Vocabulary
Diefenbach et al. (2013) established a set of adjectives that describe aesthetics of interaction. These adjectives are deliberately descriptive, non-judgemental and non-technology bound. We adhere to this rationale and seek to contribute to a vocabulary that designers can use in research, practice and education.
Designers of interactive products and systems need to articulate the everyday lifeworld in which people interact. In that respect, the terms physical and digital are not helpful, as their meaning is too generic and does not match the ecological nature of the phenomena they intend to describe (Gibson, 1979). On one hand, the term physical has different meanings. It refers to the habitat in which people live, the physical environment, but also to the world of physics, a world of abstract, theoretical constructions. On the other hand, the term digital is confusing as well. It refers to digital processes, which consist of a continuous distribution of electrical charges over a range of electronic components (Vallgårda & Sokoler, 2010). These charges are not attainable for the human perceptual-motor system: people can’t directly perceive or manipulate them. In order to be available for people, these electrical charges have to be translated into a more physical form. Stated otherwise: digital processes themselves do not belong to the everyday lifeworld.
Designers need words that describe how people experience the everyday lifeworld. The persistent and the temporal are such words. They describe physical and digital phenomena in terms of how people perceive them, instead of how they are technologically generated. The term persistent was coined by Gibson (1979) to describe the overall nature of the physical environment in which the animal lives. This physical environment is “mainly rigid but partly non-rigid, mainly motionless but partly movable” (Gibson, 1979, p.14). The persistent as we define it, contains physical objects and artefacts and their movements. The term temporal was defined by Hallnäs et al. (2002) and describes those phenomena that oppose the persistent, because they are temporary, dynamic and transient. In most design projects, the temporal contains on-screen images, light and sound. At large, all phenomena with a temporary character belong to the temporal: fire, lightning, gasses, fluids etc..
The aesthetics of coupling play in the field of interaction design, more specifically in embodied interaction. We defined them by starting from Wensveen et al.’s (2004) definition of coupling, and adapted this definition. In our perspective, coupling concerns the link between different events in a user-product interaction routine, and more specifically, how this link is perceived and experienced by the user.
A designer of interactive products and systems seeks to translate digital phenomena in a form that is accessible for the human body. We stated that the designer has two ways of doing this. He can embody the digital with static or moving physical artefacts, or he can represent it by on-screen information, sound or light. The first strategy results in events that the user experiences as physical. These events belong to what we call the persistent. The second strategy results in events that feel digital or computeresque. These events make up what we call the temporal. The persistent and the temporal oppose each other in the sense that they are experienced as belonging to two different worlds.
The designer sketches an interaction routine, consisting of persistent and temporal events. Within this interaction routine, he tries to couple both types of events with each other. This means that the designer deliberately brings the persistent and the temporal very close to one another, creating the effect that they are joined and form one singular event. This event feels natural and intuitive, but at the same time contains intrinsically opposite characteristics, since its components are vastly different from each other. This contradictory experience forms the essence of the aesthetics of coupling.
In a previous publication (Van Campenhout et al., 2019), we claimed that naturalness and intuitiveness should not be the only reasons for designers to choose an embodied interaction approach. Such an approach may as well be employed in order to create an emotional sensation for the user. The aesthetics of coupling strengthen this stance by violating the familiarity concept and by pursuing an attraction that is based on impossibility.
In further research, we will continue to investigate the aesthetics of coupling. We see possibilities in the field of augmented reality (AR). Typically, in an AR-based interaction style, for example where the user wears a head-mounted display, the persistent and the temporal are deeply integrated and possibly difficult to distinguish. We believe that in that context, the aesthetics of coupling can form a guide towards new ways of balancing the persistent and the temporal.
- Carroll, J. M. (Ed.). (2008). Special double issue on the aesthetics of interaction [Special issue]. ACM Transactions on Computer-Human Interaction, 15(3).
- Diefenbach, S., Lenz, E., & Hassenzahl, M. (2013). An interaction vocabulary. Describing the how of interaction. In Extended Abstracts of the SIGCHI Conference on Human Factors in Computing Systems (pp. 607-612). New York: ACM Press.
- Djajadiningrat, J. P., Matthews, B., & Stienstra, M. (2007). Easy doesn’t do it: Skill and expression in tangible aesthetics. Personal and Ubiquitous Computing, 11(8), 657-676.
- Djajadiningrat, J. P., Wensveen, S, Frens, J, & Overbeeke, K. (2004). Tangible products: Redressing the balance between appearance and action. Personal and Ubiquitous Computing, 8(5), 294-309.
- Dourish, P. (2001). Getting in touch. In P. Dourish (Ed.), Where the action is - The foundations of embodied interaction (pp. 25-53). Cambridge, MA: MIT Press.
- Frens, J. W. (2006). Designing for rich interaction – Integrating form, interaction and function (Doctoral dissertation). Eindhoven, the Netherlands: Eindhoven University of Technology.
- Gibson, J. J. (1979). The ecological approach to visual perception. London, UK: Lawrence Erlbaum Associates.
- Hallnäs, L. (2011). On the foundations of interaction design aesthetics: Revisiting the notions of form and expression. International Journal of design, 5(1), 73-84.
- Hallnäs, L., Melin, L., & Redström, J. (2002). Textile displays: Using textiles to investigate computational technology as design material. In Proceedings of the 2nd Nordic Conference on Human-computer Interaction (pp. 157-166). New York, NY: ACM Press.
- Hassenzahl, M. (2010). Experience design: Technology for all the right reasons. San Rafael, CA: Morgan & Claypool.
- Hassenzahl, M., Eckoldt, K., Diefenbach, S., Laschke, M., Lenz, E., & Kim, J. (2013). Designing moments of meaning and pleasure. Experience design and happiness. International Journal of Design, 7(3), 21-31.
- Heskett, J. (1980). Industrial design. New York, NY: Oxford University Press.
- Hornecker, E., & Buur, J. (2006). Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 437-446). New York, NY: ACM Press.
- Hummels, C. C. M., & Overbeeke, C. J. (2010a). Special issue editorial: Aesthetics of interaction. International Journal of Design, 4(2), 1-2.
- Hummels, C. C. M., & Overbeeke, K. (Eds.). (2010b). Special issue on designing for aesthetics of interaction [Special issue]. International Journal of Design, 4(2).
- Hummels, C., Overbeeke, C. J., & Klooster, S. (2007). Move to get moved: A search for methods, tools and knowledge to design for expressive and rich movement-based interaction. Personal and Ubiquitous Computing, 11(8), 677-690.
- Ishii, H. (2008). Tangible bits: Beyond pixels. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (pp. 15-25). New York, NY: ACM Press.
- Ishii, H., & Ullmer, B. (1997). Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp 234-241). New York, NY: ACM Press.
- Jacob, R. J. K., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., & Zigelbaum, J. (2008). Reality-based interaction: A framework for post-WIMP interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 201-210). New York, NY: ACM press.
- Jones, J. C. (1992). Design methods. New York, NY: Van Nostrand Reinhold.
- Koskinen, I., Zimmerman, J., Binder, T., Redström, J., & Wensveen, S. (2011). Design research through practice: From the lab, field, and showroom. Philadelphia, PA: Elsevier.
- Klooster, S., & Overbeeke, C. J. (2005). Designing products as an integral part of choreography of interaction: The product’s form as an integral part of movement. In L. M. G. Feijs, S. Kyffin, & B. Young (Eds.), Proceedings of the 1st European Workshop on Design and Semantics of Form and Movement (pp. 23-35). Newcastle, UK: Northumbria University.
- Landin, H. (2009). Anxiety and trust and other expressions of interaction [Doctoral dissertation]. Göteborg, Sweden: Chalmers University of Technology.
- Lenz, E., Diefenbach, S., & Hassenzahl, M. (2013). Exploring relationships between interaction attributes and experience. In Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces (pp. 126-135). New York: ACM Press.
- Lenz, E., Diefenbach, S., & Hassenzahl, M. (2014). Aesthetics of interaction: A literature synthesis. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction (pp. 628-637). New York: ACM Press.
- Locher, P., Overbeeke, K., & Wensveen, S. (2010). Aesthetic interaction: A framework. Design Issues, 26(2), 70-79.
- Overbeeke, C. J. (2007). The aesthetics of the impossible [Inaugural lecture]. Eindhoven, the Netherlands: Eindhoven University of Technology.
- Overbeeke, C. J., Djajadiningrat, J. P., Hummels, C. C. M., Wensveen, S. A. G., & Frens, J. W. (2005). Let’s make things engaging. In M. Blythe, K. Overbeeke, A. Monk, & P. Wright (Eds.), Funology (pp. 7-17). Dordrecht, the Netherlands: Kluwer Academic Publishers.
- Petersen, M. G., Iversen, O. S., Krogh, P. G., & Ludvigsen, M. (2004). Aesthetic interaction - A pragmatist’s aesthetics of interactive systems. In Proceedings of the 5th Conference on Designing Interactive Systems (pp. 269-276). New York, NY: ACM Press.
- Ross, P. R. (2008). Ethics and aesthetics in intelligent product and system design [Doctoral dissertation]. Eindhoven, the Netherlands: Eindhoven University of Technology.
- Ross, P. R., & Wensveen, S. A. G. (2010). Designing aesthetics of behavior in interaction: Using aesthetic experience as a mechanism for design. International Journal of Design, 4(2), 3-13.
- Shusterman, R. (2000). Pragmatist aesthetics: Living beauty, rethinking art. Lanham, MD: Rowman & Littlefield Publishers.
- Stienstra, J. T., Alonso, M. B., Wensveen, S. A. G., & Kuenen, S. (2012). How to design for transformation of behavior through interactive materiality. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction (pp. 21-30). New York, NY: ACM Press.
- Stienstra, J. T., Bogers, S. J. A., & Frens, J. W. (2015). Designerly handles: Dynamic and contextualized enablers for interaction designers. In Proceedings of the Conference on Design and Semantics of Form and Movement (pp. 86-94). Amsterdam, the Netherlands: Koninklijke Philips.
- Vallgårda, A. (2014). Giving form to computational things - Developing a practice of interaction design. Personal and Ubiquitous Computing, 18(3), 577-592.
- Vallgårda, A., & Sokoler, T. (2010). A material strategy: Exploring material properties of computers. International Journal of Design, 4(3), 1-14.
- Van Campenhout, H. F. G. (1999). Metamorfose. Eindhoven, the Netherlands: Uitgeverij Damon.
- Van Campenhout, L. V. C., Frens, J. W., Hummels, C. C. M., Standaert, A., & Peremans, H. (2019). The enriching limitations of the physical world. Personal and Ubiquitous Computing, 23(1), 81-98.
- Van Campenhout, L. D. E., Frens, J. W., Overbeeke, C. J., Standaert, A., & Peremans, H. (2013). Physical interaction in a dematerialized world. International Journal of Design, 7(1), 1-18.
- Van Campenhout, L. D. E., Frens, J. W., Hummels, C. C. M., Standaert, A., & Peremans, H. (2016). Touching the dematerialized. Personal and Ubiquitous Computing, 20(1), 147-164.
- Wensveen, S. A. G., Djajadiningrat, J. P., & Overbeeke, C. J. (2004). Interaction frogger: A design framework to couple action and function through feedback and feedforward. In Proceedings of the 5th Conference on Designing Interactive Systems (pp. 177-184). New York, NY: ACM Press.