On the Foundations of Interaction Design Aesthetics: Revisiting the Notions of Form and Expression
The Swedish School of Textiles, University of Borås, Borås, Sweden
Department of Applied Information Technology, Chalmers University of Technology, Gothenburg, Sweden
Form and expression are basic notions in design aesthetics and design aesthetics education. This is something firmly rooted in architecture, product design, industrial design, fashion design and so forth, but how should we understand these notions in interaction design? There is a need here to fill a gap in the foundations of interaction design. This paper revisits “form” and “expression” to discuss the interpretation of these concepts in the context of interaction design aesthetics. The paper provides a general foundational discussion and considers the implications of relating interaction design to design aesthetics at a fundamental level, rather than to notions from behavioral and social science as is usual in the area of Human Computer Interaction.
Keywords – Interaction Design Aesthetics, Design Aesthetics, Interaction Design Form, Interaction Design Expression.
Relevance to Design Practice – The theoretical framework presented in this paper concerns the issue of form thinking in interaction design. It contributes to ongoing discussions on the foundations of interaction design practice.
Citation: Hallnäs, L. (2011). On the foundations of interaction design aesthetics: Revisiting the notions of form and expression. International Journal of Design, 5(1), 73-84.
Received March 1, 2010; Accepted January 23, 2011; Published April 30, 2011.
Copyright: © 2011 Hallnäs. Copyright for this article is retained by the author, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
Corresponding Author: lars.hallnas@hb.se
Introduction
A common view of design is that of defining and shaping the appearance of material and immaterial things, or rather, forming their expression. A more systematic design aesthetic seeks to explain the basic logic of this practice. Central to any such explanation are the notions of form and expression. Studying the “basic laws” of color and form (Itten, 1975) is also what we usually do in the basic course we give at our design schools, just as we study the basic laws of form and material in art and music schools.
When we talk about shaping the actual appearance of a product, we mean all decisions that define this particular product, its outer spatial form, its internal functionality, its interface and so on. What we mean by form is then often related to the old distinction between form and material: form, the way in which material builds things. It doesn’t really matter if we refer to the geometrical form of spatial things (garments or carpets), or the serial form of temporal things (dancing in your new gown or walking back and forth on that old carpet). It is, however, important to note that this notion of form is a very general one where, for example, the outer geometrical shape of objects in Euclidian space is just a special case.
“Expression” is perhaps a notion that is a bit more difficult to get a clear picture of. It is natural to think of it as an idea synonymous with “impression”. i.e. the way things themselves appear in contrast to the way in which we see them. It is a naive distinction we refer to here, as all those epistemological problems involved in such a distinction are really another matter. When we say that design means forming the expression of things, we simply mean that in the process of designing we decide the way in which a thing actually will appear in terms of color (the blue skirt), texture (the soft pillow), sound (the silent motor), behavior (the smart computer interface). Bracketing experience in an attempt to focus on expression, we say that expression is that which displays, shows a thing – and dually that impression is that which sees a thing.
Considering the floor lamp in my room, someone designed it to express that it was a floor lamp. What does that mean? By deciding materials, various treatments, shapes, color of the lampshade and details of construction, somebody defined that which displays this particular thing as a floor lamp to be used as such. The way it lights up the room, Aspects that make it a floor lamp include the way it lights up the room, the way it furnishes the room, the way it defines a reading space, the way it presents itself as a floor lamp, and so forth, ...
Why is this important? What is it that makes the notions of form and expression of fundamental interest with respect to design practice? In industrial design and product design, for example, the answer is somehow obvious. In the process of designing a product, we define the product. We express function as we define the actual form of a product, whether it is a car or a camera. To understand what the design process is all about, we have to understand what it means to express things by defining forms of actual appearance. Similarly, learning to design means learning how to express things by defining forms of actual appearance.
Figure 1. A floor lamp to be used by us as such; something we use when reading.
The task of design, in general terms, is to turn requirements and needs into concrete things. In meeting these requirements and needs, the designer must turn ideas into concrete things, to turn analysis into synthesis, and to define things for use. This means that the notion of a thing, in the most general sense of the word, and the notion of defining things are of fundamental importance in design practice. Form is what defines a thing as such, the way in which material builds the thing. Expression is what displays a thing as such, that which displays the thing. This is also why the notions of form and expression are basic notions for a theoretical foundation of practice in design and art and why explicit or implicit theories of the design process rest on a basic understanding of these notions.
In all design, aesthetics has a dual nature, spanning the expressiveness of the things and systems we design and the expressiveness of their use. We design things, but we also implicitly design acts of use. The expressiveness of use is of focal interest in interaction design, which makes the perspective of action design of basic importance. A basic course in interaction design must, in addition to teaching the laws of spatial form and visual expression, introduce the aesthetics of action design. Interaction design introduces a shift in focus, moving from the things themselves to the action that define them in use. Interaction design aesthetics, as a logical foundation of design practice, also introduces a shift of focus, moving from design by drawing to design by act defining. This involves revisiting the notions of form and expression to understand them from a somewhat different perspective. It is still a matter of understanding the intrinsic relationships between abstract form and concrete expression, but within a different context. To formulate a coherent foundation for an action design methodology is a basic challenge for the aesthetics of interaction design; to revisit the notion of form and expression and to further develop the foundations of interaction design aesthetics.
A systematic design process builds on a series of very basic questions to be answered in one way or another. It can be from the perspective of understanding the brief, from the perspective of understanding the needs and requirements of intended users, or from the perspective of understanding functional requirements and by necessity includes the perspective of forming and expressing a design. This is also what makes it even more important to revisit the notions of form and expression in the context of interaction design in search for precision in the choice of basic formal and expressional variables in the interaction design process. In order to develop expressional working tools we have to build on answers to the question as to what is it that we actually express in interaction design. Interaction design aesthetics has then to focus on the aesthetics of interaction as a foundation of interaction design work (Hallnäs, Jacobs, & Petersen, 2008; Hummels & Overbeeke 2010).
In recent years, the aesthetics of interaction has, in different ways, gained much interest in the design research community. The distinction between experience and expression is of central importance here. In research programs for aesthetic pragmatism (Löwgren, 2009; Petersen, Krogh, Iversen, & Ludvigsen, 2004), experience-centered design and design for emotion (Desmet & Hekkert, 2007; Hekkert, 2006; Jordan, 2000; Norman, 2004; Wright, Wallace, & McCarthy, 2008) performance and impression/experience are in focus, while what we might call aesthetic realism focuses on form and expression (Hallnäs & Redström, 2006; Landin, 2009; Lim, Stolterman, Jung, & Donaldson, 2007). From an aesthetic perspective, aesthetic realism concentrates on the expressional logic of designed things themselves and is consequently closer to a design perspective.
Although the distinctions between the basic rationality of the design process and the empirical analysis of given use is becoming far clearer in the area of interaction design, it still lacks stable foundations in many respects. The notions of interaction design form and interaction design expression are of basic interest here if we aim to maintain that interaction design as design is all about expressing interaction by forming acts of intended use.
What is it that we design in interaction design? What is the role of form and function in the process of designing interaction?
The Logic of Design Expressions
The circle is an abstract, geometric form. It is a way in which material can build circular things. When we draw a circle on a piece of paper, or build some sort of circular thing, we express the abstract form in concreto. That which display the circle in concreto is an expression of the circle as such, be it a computer-generated perfect circle or a hasty sketch on a piece of paper. When we see the computer-generated picture or the sketch, we see that it is a circle.
The general notion of form concerns the logic that holds things together as they are in themselves. In the process of designing, this corresponds to the inherent logic of basic design definitions, a logic that is also reflected in the way we design. Form is what determines the foundation of design work. It is what reflects the basic choices that provide the logical foundation for results as well as for the process itself.
The things we use present themselves to us in various ways. If we search for the intended meaning behind all this we read these presentations as design expressions, that is, as definitional expressions. The logic of design expression is the basic aesthetic guiding force in the design process. What is given in the process of designing is a task to express something, to introduce, to define that which displays a thing. Experience of use is in this respect always secondary.
Design always starts with something given. Designers express that given in the process of designing. It is the definitional logic of resulting expressions that design aesthetics is all about; the way in which we express that given, the rationale behind basic expressional choices in the process of designing.
There is a basic difference between aesthetics as philosophy of art, as critical and empirical studies of the aesthetic experience and aesthetics as a foundation of design work. In what follows I refer to the latter as design aesthetics.
Design aesthetics, as I understand it, concerns the logic of design expressions. The distinction between expression and form refers within the design process to a distinction between the intended gestalt and its definitional logic, that which displays the design and the way in which it is defined. This relation between form and expression introduces a basic axiom of aesthetics: form defines expression. The way in which material builds the thing defines that which displays the thing. In design, there is always a definitional rationale to refer to.
While window shopping, I see a pair of beautiful shoes in a nice green color with smooth lines in contrast to the roughly cut opening of the shoes. What do the shoes display? What displays the shoes? This is what I try to describe by talking about the nice green color, the smooth lines and its contrast to the roughly cut opening of the shoes. But somebody else comes along and sees the same shoes; ugly ones in a horrible green color and boring lines that end with bad finishing at the opening of each shoe. This typical relativisation of value judgments makes it tempting to just bracket the expression and focus instead on impressions and experiences. But that leaves a gap in the design process as it is somehow clear that it is the form of the shoes, the way in which the material builds the shoes, that defines the expression.
Expression is what makes experience possible, which is why concepts and theories of experience can never provide a logical foundation for design aesthetics. The design primarily defines the thing – understood in the broadest sense of the word – itself and not the experience we form of it. Perhaps it is in the process of designing that we come as close as it is possible to the Kantian das Ding an sich.
The constituents of the thought, and a fortiori things themselves, must be distinguished from the images that accompany in some mind the act of grasping the thought – images that each man forms of things. (Frege, 1970, p. 79)
An inherent difficulty here is to make the necessary distinctions between definitional matters of the design process and analytical matters of empirical studies. In a systematic design aesthetics, providing a logical foundation for design practice, form and expression refers back to the definitions that introduce a design. The logic of design expressions concerns the reasons, the why, behind a design and introduce a rationale that relates basic design decisions to each other. Given that I understand the logic behind a certain design object, such as a chair or a computer application, I can explain why it is the way it is. Empirical user studies, on the other hand, focus on how users define a design as some thing to use. There is a fundamental gap between these perspectives, which relates to the mysterious points where we turn analysis into synthesis and vice versa.
Learning to use a thing designed for use is an introduction to the logic of a certain design expression, that is, an act of initiation. When I learn how to play the guitar, I learn the ways of the guitar as a musical instrument, the ways in which a guitar opens up for playing music. Given a deeper understanding, I can teach others, which is to introduce them to the logic of guitar playing.
The notion of design logic should not be confused with the perceptual psychological notion of affordances (Gibson, 1979). The motivation behind basic design decisions can certainly be to introduce action possibilities in the environment, but design rationale is here a logical notion and refers to design form. It refers to the way in which we express a design, not to use possibilities inherent in ready-to-use things. In empirical design studies, of course, it makes sense to trace affordances back to aesthetical considerations.
Interaction Design
Interaction design, with its roots in human-machine interaction, is usually associated with the design of computer-based products and systems with focus on usability issues (Winograd, 1997). But interaction design also relates to forming the way in which we interact with products and systems in general. In this sense, interaction design is a basic component of the design process (Buchanan, 2001). Interaction design is a matter of designing the acts that define the intended use of things and systems. Designing computer interfaces and computational interaction devices is a part of this, but does not wholly define interaction design as a specific area of design.
In this context, aesthetics concerns basic logical principles that relate to the way we define interaction, which entails that notions like “me”, “us”, “users”, “performance” and so forth are logical variables and not given phenomenon open to empirical studies (Hallnäs & Redström, 2006). Within the design process “use”, “users” and the like are logical notions we define, with “intended use” being the focal notion that determines the design; the acts that define intended use then simply refer to a logic of intended use that tells us what the thing or system is by definition. How we define these things and systems through actual use is a different matter, although the inherent logic of design expressions still links them both. When we design a toothbrush for instance, we design a thing to be used in a certain way, giving it a certain meaning through the acts that define its intended use. The way we are supposed to use the toothbrush defines the thing as a toothbrush.
What makes interaction design a bit special in respect of designing computational things is the fact that programming defines acts of use in a sense (Akrich, 1992). In designing a new chair, we consider aspects of the ergonomics of sitting, defining what it means to sit in a chair along with defining the chair itself, but we do not explicitly design acts of sitting in designing the chair.
Interaction design for non-computational things concerns the guiding concepts that define acts of use, that is, the acts of use that define the ready to use design as this or that. For example, certain ways of playing music guide the design of a traditional piano. Playing the piano is implicit in the process of designing a piano as a thing. You strike a key on the traditional piano and we hear a tone. What is the difference between the act of playing a tone on a traditional piano and that of playing the corresponding tone on a digital piano? In the first case, we use a mechanical possibility that the piano as a thing offers. In the second case, we start the execution of a program that defines the playing of a certain sampled tone in a certain way. So the act of playing is partially defined in the process of designing the digital piano. In this sense, interaction design is explicitly part of building the computational thing itself. To program a computational thing means that we build the thing at the same time as we partially define acts of using it, both in terms of what it is as we use it and what it is to use it. Execution of given programs builds the thing in use. Change the program and it will be a different thing to use.
When we design and build a conventional floor lamp, we explicitly introduce components for turning on and off the light, implicitly defining patterns of use when we build it. But there is still a difference in material foundation and interaction design between electrical and mechanical products and computational things and systems.
As the computer disappears in the background, computational technology reappears as a new expressive design material. We build things with a new material when we build computational things, their behavior in use depending on the execution of given programs. This material is just like the material of music, being a time-material that shows itself only when we use things (Hallnäs & Redström, 2006; Löwgren & Stolterman, 2004). To understand the expressiveness of computational technology as material is essential with respect to design practice. It is a new temporal material that is fast becoming just another common industrial design material. It is the computational material that makes this a specific area of design and not some particular “interactive” property of the things and systems we design.
A basic issue of interaction design aesthetics is to investigate how the expressiveness of computational technology builds the expressiveness of use. A challenge for experimental interaction design is thus to develop theoretical foundations through practice-based design research.
Form and Expression
The things we design are present as things to use in our everyday lives. Things make sounds. We feel them. They smell. They function in various ways. They respond and build our lives as we use them to form our living environment.
In the process of designing, we introduce the definitions of these new things, these new systems. In a more general sense, the notion of form strongly relates to the logic of such definitions, that is, to the way in which the design defines the new things, the new systems. It is clear that form in this sense not only refers to the geometrical shape of things as spatial things, that is, not only to the visual and spatial appearance of things as distant objects.
Form, the shape, the general structure of something form the way of doing something. Consider some typical examples:
In respect of geometric form, we say that this or that thing has the form of a square. What does this mean? In plane geometry, a square is a quadrangle where angles and sides are congruent, that is, they have the same form. Geometry is a way to define the spatial configuration of an object.
In respect of the form of an argument, an argument is something that supports a proposition, something showing that a proposition is true. The form of an argument, the logic of the argument definition, is then the way in which an argument shows the truth of a proposition. Take the following argument for example: If it rains, the ground gets wet. It rained, thus the ground is wet. The conclusion is true purely by form of the argument: if A, then B. A, thus B.
In respect of the form of a computer program, stating a complete set of instructions for a computation, the form relates to the way in which the program instructs the computer to compute. The notions of a loop, pattern matching, recursion, iteration are typical formal notions in this sense.
In respect of driving a car, the form of car driving relates to the ways in which we drive a car, that is, to the logic of driving inherent in the way in which the car is designed. There are, for example, formal similarities between a car and a bicycle; their steering makes them different to a tram. We go by bike, by car, by tram, but there is a difference in form that makes them different forms of transportation. If I go by tram, I sit down and let the tram transport me downtown. If I go by bike or by car, it is me that rides the bike or drives the car. I use a tram, a bike or a car to transport myself downtown. In the first case, this means that I sit down and read a book while the tram drives me downtown. There is a big difference in the way in which the tram, the bike and the car allow us to define personal transportation.
These form examples takes us from the geometrical form of three-dimensional things to the way in which we drive a car or go by bike. From the things themselves to the use of them, I drive my car, use my computer or my vacuum cleaner. What is interaction all about in this context?
The notion of interaction is usually defined as reciprocal action, that is, inversely related actions, referring to the interplay between us and other people or interactive things such as computers. The problem with this definition is that it becomes very metaphorical when referring to interaction with a designed thing X. The question is what it means to say that I interact with the washing machine as I use it? When we use a modern washing machine, both we and the machine are washing, but “washing” here means different things. The machine is doing the actual washing, while we handle and monitor the machine. Function may be a quite complicated matter here. Interaction, on the other hand, is perhaps reduced to loading the machine, pressing a button and then sitting in a chair doing nothing, which is the point of a machine doing the job for us. However, when we drive a modern car, driving means handling the interface to a complex mechanical and computational system. Driving is something we do; from a design perspective, it would be misleading to describe this as a closed system of interactions. What we design is a car for people to drive, not a complex system where the user is simply another component of the machine.
Interacting with something keeps it distant and foreign. If you are only interacting with your spouse the relationship may be in trouble. We dwell with nature, and roommates, and anything that we let enter us, and we it. Dwelling with computers means that they have their place, and we ours, and we co-exist comfortably. (Weiser, 1996, p. 3)
As a solution to this dilemma, we could think of interaction simply in terms of what I am doing as I use a design and view this in relation to the dual notion of design functionality. Of course, “What I am doing as I use X” involves both “my” actions and what the machine does from the perspective of me using it. It is a similar case for function with respect to what the thing is doing as I use it. When I use a washing machine I open the hatch and it opens. After loading the machine with clothes and washing powder, I push a button to start the machine and it starts. If the hatch does not open or the machine does not start, I cannot wash and I will most probably phone for repair help.
The basic idea here is to solve the dilemma by simply defining interaction as dual to the standard definition of function in function analysis with reference to me using a design, that is, to introduce the following distinction between function and interaction: function being what things do as we use them and interaction referring to what we do when we use a thing. The aim here is simplicity in theoretical foundations, not to provide a deeper philosophical interpretation of the true meaning of interaction. But to be a meaningful theoretical notion it must of course cover the intuitive notion in some reasonable sense. When I say that I sit in a chair, this involves not only me trying to sit down, but also that I sit down in the chair. Similarly, when I say that I phone my friend using my mobile phone I refer to a use act as a whole.
Based on the duality between these notions of function and interaction we define interaction design form as the way in which the thing or system we design we design relates function and interaction to each other.
To understand what the interaction design process is all about, we have to understand what it means to express acts of use by defining the way in which the design (of the thing/system) relates what we do when using the thing to what the thing itself does when we use it. Similarly, learning interaction design means learning how to express acts of use by defining the way in which the design (of the thing/system) relates what we do when using the thing to what the thing itself does when we use it.
When designing a guitar, I design a certain musical instrument. A focal issue is what it means to play the guitar, particularly what it means to play this guitar I am in the process of designing. The guitar has to sound and be used in a certain way. Interaction design form thus concerns how the design defines guitar playing and in a more general sense defines playing a musical instrument. There is a big difference in interaction design form between a guitar and a violin, and there is an important, but subtler, difference in interaction design form between my favorite guitar and that other guitar that I simply don’t like at all, although in some cases it can be difficult to say exactly what this difference is.
Computational technology gives us a very rich material to express interaction design form. As we build computational things, digital products, we use programs to formalize interaction patterns and to indirectly define acts of intended use (Akrich, 1992). Consider, for instance, the difference in defining photographing in the design of an analogue camera and a digital camera. What a camera does is to take pictures. The digital camera produces digital data; the picture taken is data ready for some program to work on. The picture taken with an analogue camera is film. It is not only the material involved that is different. The logic of defining acts of use is also different. For the digital camera, we use sensor and computational technology to see the picture we take in real time. As we look through the lens of the analogue camera, we see the motive. This is not mainly a difference in function, but a difference in interaction design form and in the logic of the cameras as instruments for performing acts of photography.
Figure 2. As we look through the lens of the analogue camera we see the motive.
Following the definition of interaction design form given above, it seems natural to define interaction design expression as that which displays interaction.
Playing the guitar displays itself in the way in which it sounds and feels and so forth. It might seem difficult to bracket experience here, but the focus in this context is on the ways of sounding and feeling as such, not the experience of the way in which the guitar sounds or feels. Landin (2009) introduces and discusses the notion of expressions of interaction in relation to notions such as trust and anxiety. Here, it is not a matter of the experience of trust or anxiety, but rather that of trust and anxiety as ways in which the use of digital things display themselves.
Taking photos using a digital camera is characterized by the crossroads between analogue and digital techniques, the ways of capturing the image though the lens and the digital image sensor, the ways of relating motives and photo by digital image processing, etc. Through computational and technical complexity, the digital camera introduces digital alienation as an expression of photographing.
Figure 3. For the digital camera, we use sensor and computa-tional technology to see the picture we take in real time.
In the design process, formal variables relate to basic building decisions, to the ways of forming the design. Expressional variables then relate to issues of intended presentation, to what it is that presents the design itself. That form defines expression encompasses the understanding that the interpretation of expressional variables/categories guides the interpretation of formal variables in the process of designing. The main argument for this interpretation is that all basic expressional characteristics of actual use have their foundation in some way in the logic of the definitions that define a given thing as a thing to use.
When you experience the beauty of driving a certain car or using a certain computer application, it is the expression of use that links to what you experience as you apply the basic definitions that define the car as something to drive or the computer application as something to apply. This does not mean that you follow the definitions as rules of use, just that you use the thing as it is defined to be. By contrast, using a fork to comb your hair means that you see the possibility to redefine it as a comb.
The first implication of this is a rather dramatic difference in perspective between the analysis of actual use and design for intended use with respect to expressions of interaction. This is somehow abstract from the perspective of interaction design and momentarily concrete with respect to analysis of actual use. Another main implication is that interaction design aesthetics, in the sense discussed here, is central to all basic design decisions and has little to do with superficial interface styling. Not only does a seemingly purely technical solution have aesthetical implications, to be meaningful with respect to the overall design the solution has to have an aesthetical rationale.
Providing a design rationale for that which displays interaction is an explanation of interaction design form. This is somehow dual to the idea of proving a design with respect to given specifications on the basis of empirical user studies. This is a very important distinction to make in relation to the evaluation of a design. We may explain expressions through derivation by form, but it is a different matter to derive expressions by evaluation of experience.
The notions designers introduce in defining acts of intended use are notions such as driving, browsing, mailing, blogging and so forth. It is then always a matter of a specific notion that says something about the more general notion that it is an instance of; browsing using this or that application is simply browsing, but at the same time something very specific. To view this from the perspective of design aesthetics is then, modulo the given interpretation, really nothing more than searching for the character of these notions in the logic of given definitions, that is, how interaction design form defines them (see the notion of “the character of things” developed in Janlert & Stolterman, 1997).
Interaction Design Dimensions
The strength of notions of form and expression as foundational design aesthetical notions is inherent in the usefulness of the formal and expressional variables we derive. Variables of form and expression are then seen in relation to basic design dimensions such as variables of geometry and color for three dimensional design objects. In reflective analysis, basic design dimensions guide us in understanding the forms that define expressions of given things. They introduce the principal expressional questions designers have to ask in turning analysis into synthesis in the process of designing things.
What natural candidates for the basic dimensions of interaction design space could be proposed here in relation to the notions of interaction design form and expression?
Using things is a matter of relating function and interaction to each other both in space and in time. Designing with a focus on acts of use also refers to the “interface” of things as such, as well as to intended use, and to the processes of learning how to use given things. It therefore seems reasonable to consider timing, spacing, connectivity and methodology as central design dimensions for the classification of interaction design variables:
- Timing – the rhythm and meter of use we introduce.
- Spacing – the space of use we introduce.
- Connectivity – the connections of use we introduce.
- Methodology – the ways of use we introduce.
Timing – Relating function and interaction in use clearly has a temporal dimension; how is what I do to be related with what X does in time? What is the character of interaction with respect to temporal issues? What about the rhythm in timing inherent in interaction? Take any sort of vending machine, an automatic door, a cashier machine or so forth where design decisions with respect to rhythm are essential for the timing character of the interaction design. Take the automatic door as an example:
- Interaction – I open the door and walk into/leave the room/building. Am I just walking, dreaming of other things, or does the seemingly closed door without a handle confuse me?
- Function – the door opens and closes.
- Interaction design form – the very idea with an automatic door is that it automatically relates function and interaction to each other. As I walk through the door it opens and closes itself automatically.
- Interaction design expressions – the ways of walking through the door. Is it the just-walking-through-door, the stop-wait-start-door or the who-knows-what-will-happen-door?
- Timing – as somebody is approaching, when and how fast should the door open? When and how fast should the door close?
- Spacing – what space do we introduce around the door?
- Connectivity – what makes door open and close?
- Methodology – what ways of walking into the building does the door introduce?
Timing is obviously a focal issue in this example. To view this as a form and expression dimension in the process of designing automatic doors changes the perspective from the provision of technical solutions to problems of implementation to challenges in design aesthetics. (Hallnäs, 2006; Ju & Takayama, 2009)
Spacing – Function and interaction in use clearly have a spatial dimension linked to what X does in space. How does the design build space? What about the depth of space inherent in the design? The spacing character of a mobile telephone system is very different from the spacing character of a floor lamp. This is not only a matter of system functionality, but also of interaction design form, which is the way the design of the system relates function and interaction to each other by building the space of use. Take a typical modern mobile phone as an illustration:
- Interaction – I call people and receive calls, send sms and use other applications on the move. I live my mobile life.
- Function – the phone handles I/O through a mobile phone interface and executes programs processing and wirelessly transmitting my voice, text messages etc.
- Interaction design form – central here is that it is a mobile phone, i.e. the wireless way in which it relates phone interaction with phone function. It builds my mobile life.
- Interaction design expressions – the ways of using the phone on the move, the ways of living a mobile life.
- Timing – is there a major difference between the mobile phone and an old fashioned phone with respect to timing? What is rhythm in mobile use?
- Spacing – the very notion of a mobile phone indicates a major difference in relation to an old fashioned phone with respect to spacing. What is space in mobile use?
- Connectivity; there is a clear difference, for example, between the button type mobile phone and the touch screen type mobile phone with respect to connectivity. What is connectivity in mobile use?
- Methodology; there are radical differences between the modern mobile phone and an old fashioned phone with respect to methodology. The ways of use designers introduce are in many cases new ways. This concerns both the logic of the phone as a computational and transmitting device and with the expressions of computational and data communication things in use.
Spacing is a central design dimension for the combination of mobility and communication. For the mobile phone, spacing connects with the Hertzian space that Dunne and Raby (2001) discuss in respect of the invisible space that we build in designing a world of mobile electronic communication.
Connectivity – Relating function and interaction in use is a matter of connecting; how does X relate what I do to with what X does? In what ways do we connect function and interaction in acts of use? For example, is the way to obtain precision in connections a matter of art? In what sense are connections well-defined? Is there ambiguity in the definition of connections?
To illustrate the connectivity dimension we may consider the modern washing machine:
- Interaction – I wash my clothes.
- Function – the machine washes my clothes.
- Interaction design form – the modern washing machine is supposed to do the work for me, which means that interaction design form centers around a simple interface open-load-start-unload-close.
- Interaction design expressions – the potential to do something else as the machine washes my clothes, this being a characteristic of modern life.
- Timing/Spacing – both timing and spacing have much to do with the idea of the washing machine as a machine doing a job for me. I load the machine, start it and then go on with other things elsewhere. The challenge is to design the machine and the use of the machine to create free time and space, so that the expressions of interaction are not just those of anxiety and stress.
- Connectivity – interaction design in this example is very much an issue about connectivity.
- Methodology – whatever ways of use we introduce, relief from work by making washing easier must be guiding principles.
As the washing machine is supposed to do work for me, connectivity variables provide a particular challenge. What does it mean to load and unload the machine? What does it mean to start the machine with respect to timing and spacing?
Methodology – Relating function and interaction in use is a matter method. How do I relate what I do with what X does? In what ways are we supposed to go about building the design in use? Is there room for artistic development in use for instance? The so-called LISP machines, computers built in the 1980, ties specifically for programming and running LISP-programs, were designed for programming as a form of art, very much a matter of interaction design expression.
As examples, consider two basic computer applications:
Mail applications
- Interaction – I write, send and receive mail. Living with my mail.
- Function – the computer and data communication application handles I/O through a computer interface and executes programs processing, sending and receiving my mail.
- Interaction design form – the typical computer interface that defines the notion of mail communication.
- Interaction design expressions – ubiquitous, mobile, speed, anxiety, carelessness, stress etc. are typical examples of expressions (Landin, 2009).
- Timing – the speed in writing, sending, administrating, receiving mail in using a computer and data communication application introduces a radically different rhythm in comparison with surface mail.
- Spacing – is the space we introduce radically different from the space of surface mail?
- Connectivity – writing, reading, sending, receiving.
- Methodology – the ways of “e-mail” we introduce.
Image processing
- Interaction – I process my digital images.
- Function – the application handles I/O through a GUI and executes programs processing and my image files.
- Interaction design form – the digital light room interface.
- Interaction design expressions – the digital light room introduces, among other things, the art of digital image processing.
- Timing – the rhythm of use is as near to the digital image file as the speed of the computer and the application allows. It is a different rhythm from work in the analogue light room.
- Spacing – the space of use we introduce allows for the simulation of a digital light room.
- Connectivity – the connections of use we introduce allow for direct manipulation of images at a low-level pixel representation.
- Methodology – the ways of use we introduce are that of sophisticated methodologies for manipulating digital image files.
The way in which we design the interface of typical computer applications, such as mail and image processing applications, is very much a matter of methodology. Through the interface we define the myths of use (Landin, 2008), the GUI stories about what we do as we use applications.
Timing – Spacing generalizes what many authors have been discussing in relation to the temporal/spatial dimension in interaction design (Löwgren & Stolterman, 2004; Löwgren, 2009; Hallnäs & Redström, 2006; Mazé & Redström, 2005). Connectivity – Methodology generalizes in the same manner central themes in interaction design methodology, that is, interface design and usability methodology.
In the process of designing, we think form, in one way or another. Form thinking means direct involvement with basic, concrete design decisions; how do we define the way in which the design builds this or that, what are the basic variables to consider? Form thinking is the way of synthesis in the design process.
We introduce a fine structure in form thinking through design dimensions that provide foundations for:
- classifying variables in the design process, guiding basic questions,
- systematic design rationale,
- development of design methods,
- development of material for the “basic course” in design.
When designing three-dimensional objects, we have the dimensions of Euclidian space in the back of our minds, together with other expressional dimensions such as surface. Similarly, we may think form in interaction design through the dimensions of timing, spacing, connectivity and methodology or in terms other types of interaction design dimension models.
Assume that we are about to develop ideas for a new automatic vacuum cleaner. The main functionality of the vacuum cleaner is clear, to pull dust and dirt up from the floor. Interaction is also clear on the same level of abstraction. I use the cleaner to clean dust and dirt from the floor. So we have to ask ourselves what it means to use the vacuum cleaner, that is, in what ways it will relate me vacuum cleaning with it vacuum cleaning. The main issue is that the vacuum cleaner is supposed to do the job automatically and liberate me from cleaning. We could, for example, think of interaction design form in the following way:
- Timing – when I am not there,
- Spacing – everywhere in my apartment/house,
- Connectivity – very little to do other than a dustbin to empty,
- Methodology – learning to trust.
This is simultaneously designing (for) freedom and comfort and designing (for) separation and alienation.
Consider the design of a doorbell. Its main function of announcing that someone is at the door is clear, as is interaction. I announce my presence at the door/I live in my apartment/house-having-a-doorbell. But what does it mean to use a doorbell, that is, in what ways will it relate me ringing the doorbell with it ringing.
We may describe ways of relating function and interaction to each other through the following simple schemes of duality:
that FUNCTION, when INTERACTION,
that INTERACTION, when FUNCTION.
This is similar to what we do when we talk, for example, about ways of delivering the morning newspaper in the sense that X throws the newspaper on the lawn when X delivers it.
These formal descriptions suggest basic functional and interactional variables and the way in which they relate to each other, that is, as functional variables in a certain interactional context and interactional variables in a functional context. Here it is not a matter of forms for analysis in relation to a given design, but rather a matter of forms framing design decisions that directly lead to expressional definitions. What we define is the rationale of acts of use, that is, the interaction design that defines expressions of use. I ring the doorbell and inside the house/apartment the bell rings. The bell has rung and I wait. What are the ways of these moments and locales of connection and separation, the way in which the doorbell relates function and interaction to each other with respect to timing and spacing? By the way, what is a doorbell and how to use it? Please see the illustration in Figure 4.
We design (for) control in timing. In this example, the logic of control is that the doorbell rings, when I ring the doorbell. This is the expression we design for. Whether I experience a sense of control as I use the ready-to-use designed doorbell is another matter. If we aim to design for specific emotions, it is not at all clear that there is a conceptual correspondence in the choice of expressions.
Figure 4. Consider the design of a doorbell.
Discussion
Conceptually and with respect to methodology, there is a clear difference in perspective between a design as a solution to a problem – or as something satisfying user needs – and a design as a definition of concepts or things. If we say we solve a problem or satisfy users needs by our design, then there is also something we need to demonstrate. However, if we step back and look at the definition that introduces the solution, naturally we ask what the definition introduces. In the first case, we look for proof in one way or another, be it through empirical user studies or by other methods of evaluation. In the second case, we look for interpretations and examples that explain the definition.
Obviously, the proof of the solution has to build on an interpretation and understanding of what the design is. In this sense, the aesthetical perspective is more fundamental. We can view this relation between Human Computer Interaction studies and interaction design aesthetics in two different directions:
- What does it mean to trace issues of usability back to foundational aesthetical considerations?
- What are the basic expressional categories in interaction design and how do we use them to shape foundations for usability issues?
A user study of an application that is supposed to solve a certain problem X clearly focuses on performance in the sense that it is a matter of studying actual use, that is, the way in which the design relates function and interaction to each other in actual use. Referring questions on use back to design decisions with respect to timing, spacing, connectivity and methodology means that we search for a foundation for performance of use in basic design aesthetical considerations. The mistake here is to reverse the picture and look for the foundation of such design decisions in empirical user studies.
In the other direction, suppose we set out to design an application characterized by ease of use. To proceed, we have to interpret “ease of use” in terms of foundational interaction design concepts. What does it mean to design for ease of use in terms of timing, spacing, connectivity and methodology? The cashier machine could, for instance, be characterized by ease of use in the sense that connectivity is precise and methodology is simple. But is timing that lacks rhythm consistent with ease of use?
The ideas discussed in this paper build on earlier work (Hallnäs & Redström, 2002, 2006) and are closely connected to the work on interaction gestalt in (Lim et al., 2007), Mazé and Redström’s (2005) work on form and the computational object and Landin’s (2009) work on expressions of interaction. Interaction design form, as this notion is defined here, was initially defined in (Hallnäs, 2004).
The notions of interaction gestalt introduced in Lim et al. (2007) and expressions of interaction are clearly related in various ways. Lim et al. then identify interaction gestalt with the shape of interaction:
… it is essential to define and research what the shape of interaction is, which we call interaction gestalt, so that we can help designers articulate and manipulate this unusual type of phenomenon which does not have tangible shapes, and is flexible, ungraspable, and easily changeable … (p. 245)
Where interaction is understood in the following way
… interaction is basically viewed as a phenomenon that emerges in-between people and digital artifacts. It is not inside of the artifact. It is continuously going on and changing over time. (p. 245)
Central to the discussion is also the notion of interaction attributes:
[T]he interaction gestalt is shaped by a set of interaction attributes that must be translated to and manifested in the interactive artifact properties in order to be communicated, perceived, and experienced by users. (p. 246)
Lim et al. (2007) also introduce connectivity as an interaction attribute.
The basic difference in approaches, as we see it, has to do with a difference in theoretical understanding of the notion of interaction, a difference in understanding interaction design dimensions/interaction attributes, but also a difference in approach to foundations. Here, form is the basic foundational issue whereas expression is a derived thing.
The notion of interaction as it is used here is a theoretical concept introduced as a dual notion to function in order to provide simplicity in theoretical foundations. It is not understood as a “phenomenon that emerges in-between people and digital artifacts” (Lim et al., 2007, p. 245).
The discussion here about form and expression in interaction design concerns design aesthetics from the perspective of basic design decisions, not what for, but what it is. In that sense, it means to take one step back from the perspectives of experience design and design for emotion to revisit the elementary notions of form and expression. For example, what does it mean to design a musical instrument? The instrument itself is a thing we have to define, that is, we have to decide the ways in which certain materials are going to build the instrument. But to make the design meaningful as a design of a musical instrument, we also have to define what it means to play the instrument.
From elementary things
that the felt-covered hammers strike the strings when I press the keys on the keyboard of the piano – (elements of very basic connectivity),
to aesthetical visions that I can, with confidence, play softly on the violin, when there is brilliance and depth in tonal quality – (elements of timing).
From the morals of use
that I practice harder, when it is not responding – (elements of methodology),
to the aesthetics of techniques that it provides richness and volume of the tone, when I play in a somewhat rough and edgy manner – (elements of spacing).
The key point is that this also is a matter of form, a matter of basic design decisions defining the thing as a musical instrument. This is also to say that the musical performance displaying me-playing-music has a foundation in interaction design form. Interaction design is in this sense a central component in every design process. What is a bit special in the design of computational things in this sense is that computational technology as design material makes interaction design more explicit; there is a fundamental difference between the old fashioned mechanical instruments and the new digital instruments that relates to issues of interaction design form (see the idea of “rich interaction” in Frens, 2006).
A very basic challenge is then to further explore interaction design aesthetics, both in research and in design education. And also to look for ways to bridge the gap between expression and experience in theory and methodology.
Acknowledgements
The work presented in this paper owes a lot to all those discussions over the years with Johan Redström and Hanna Landin on the foundations of interaction design. Thanks! I am also grateful to the reviewers whose critical comments helped me revise the paper.
References
- Akrich, M. (1992). The de-scription of technical objects. In W. E. Bijker & J. Law (Eds.), Shaping technology / building society (pp. 205-224). Cambridge, MA: The MIT Press.
- Buchanan, D. (2001). Design research and the new learning. Design Issues, 17(4), 3-23.
- Desmet, P., & Hekkert, P. (2007). Framework of product experience. International Journal of Design, 1(1), 57-66.
- Dunne, A., & Raby, F. (2001). Design noir: The secret life of electronic objects. Switzerland, Basel: Birkhäuser.
- Frege, G. (1960). Illustrative extracts from Frege´s Review of Husserl´s philosophie der arithmetik. In P. Geach & M. Black (Eds.), Translations from the philosophical writings of Gottlob Frege (2nd ed., pp. 79-86). Oxford: Blackwell.
- Frens, J. (2006). Designing for rich interaction: Integrating form, interaction, and function [Doctoral dissertation]. Eindhoven, the Netherlands: Technical University Eindhoven.
- Gibson, J. (1979). The ecological approach to visual perception. London: Lawrence Erlbaum Associates.
- Hallnäs, L., & Redström, J. (2002). From use to presence: On the expressions and aesthetics of everyday computational things. ACM Transactions on Computer-Human Interaction, 9(2), 106-124.
- Hallnäs, L. (2004). Interaction design aesthetics - A position paper. In O. W. Bertelsen, M. G. Petersen, & S. Pold (Eds.), Proceedings of the NordiCHI2004 Workshop, Tampere, Finland, October 24, 2004, DAIMI PB-572, Århus, Denmark: Department of Computer Science, University of Århus 2004 (pp. 14-17).
- Hallnäs, L. (2006, February 25). Interaction design aesthetics. Keynote presented at the Conference on Student Interaction Design Research, Göteborg, Sweden. Retrieved December 30, 2010, from http://www.cse.chalmers.se/research/group/idc/sider06/hallnas.pdf
- Hallnäs, L., Jacobs, J. K., & Petersen, M. G. (2008). Introduction to special issue on the aesthetics of interaction. ACM Transactions on Computer-Human Interaction, 15(3), 1-5.
- Hallnäs, L., & Redström, J. (2006). Interaction design: Foundations, experiments. Borås, Sweden: University College of Borås.
- Hekkert, P. (2006). Design aesthetics: Principles of pleasure in product design. Psychology Science, 48(2), 157-172.
- Hummels, C., & Overbeeke, K. (2010). Special issue editorial: Aesthetics of interaction. International Journal of Design, 4(2), 1-2.
- Itten, J. (1975). Design and form - The basic course at the Bauhaus (Revised edition). London: Thames and Hudson.
- Janlert, L-E., & Stolterman, E. (1997). The character of things. Design Studies, 18(3), 297-314.
- Jordan, P. (2000). Designing pleasurable products. London: Taylor and Francis.
- Ju, W., & Takayama, L. (2009). Approachability: How people interpret automatic door movement as gesture. International Journal of Design, 3(2), 1-10.
- Landin, H. (2008). Digital myths and delusions: An approach to investigate interaction aesthetics. Digital Creativity, 19(4), 217-232.
- Landin, H. (2009). Anxiety and trust and other expressions of interaction [Doctoral dissertation]. Göteborg, Sweden: Chalmers University of Technology.
- Lim, Y-K., Stolterman, E., Jung, H, & Donaldson, J. (2007). Interaction gestalt and the design of aesthetic interactions. In Proceedings of the 3rd Conference on Designing Pleasurable Products and Interfaces (pp. 239-254). New York: ACM.
- Löwgren, J. (2009). Toward an articulation of interaction esthetics. New Review of Hypermedia and Multimedia, 15(2), 129-146.
- Löwgren, J., & Stolterman, E. (2004). Thoughtful interaction design. Cambridge, MA: MIT Press.
- Mazé, R., & Redström, J. (2005). Form and the computational object. Digital creativity, 16(1), 7-18.
- Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. New York: Basic Books.
- Petersen, M. G., Iversen, O. S., Krogh, P. G., & Ludvigsen, M. (2004). Aesthetic interaction - A pragmatist aesthetics of interactive systems. In Proceedings of the 5th Conference on Designing Interactive Systems (pp. 269-276). New York: ACM.
- Sullivan, L., (1896, March). The tall office building artistically considered. Lippincott’s monthly magazine, 339, 403-409.
- Weiser, M. (1996, March). Open house. ITP Review 2.0. Retrieved December 30, 2010, from http://www.ubiq.com/hypertext/weiser/wholehouse.doc
- Winograd, T. (1997). From computing machinery to interaction design. In P. Denning & R. Metcalfe (Eds.), Beyond calculation: The next fifty years of computing (pp.149-162). New York: Springer Verlag.
- Wright, P., Wallace, J., & McCarthy J. (2008). Aesthetics and experience-centered design. ACM Transactions on Computer-Human Interaction, 15(4), 1-21.
