Studies of Dancers:
Moving from Experience to Interaction Design
University of Technology, Sydney, Australia
As movement-based, interactive technologies continue to become more embedded in our daily lives, aliveness, vitality and pleasure in our interactions with these technologies are becoming sought-after qualities. Dance is one discipline that works directly with these qualities of aesthetic experience through the moving body. We conducted a series of studies with trained dancers and physical performers to explore ways of working with the moving body in interaction design. The first study was of falling by skilled movers. The aim was to explore the first-person experience and external representations of the act of falling. The second study explored ways of inventing and choreographing movement for use in the design of motion-sensing technologies. The results of the studies were examined to identify an emerging set of methods and tools to enable designers to work with movement and felt experience in the context of movement-based, interactive technologies. The methods and tools enable designers to move between and integrate the three different perspectives of the mover, the observer and the machine. This act of integration makes it possible for the designer to remain accountable to the different views of the moving body, in an approach to interaction design with the primacy of embodied, lived experience at its centre.
Keywords – Dance, Felt Experience, Interaction Design, Kinaesthetic, Movement.
Relevance to Design Practice – This paper contributes to designers’ understandings of the moving body as input into sensing technologies through studies of movement with trained dancers. It suggests possible ways of describing, representing and experiencing movement for use in the design of video-based, motion-sensing technologies.
Citation: Loke, L., & Robertson, T. (2010). Studies of dancers: Moving from experience to interaction design. International Journal of Design, 4(2), 39-54.
Received March 20, 2010; Accepted June 17, 2010; Published August 31, 2010.
Copyright: © 2010 Loke and Robertson. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
*Corresponding Author: lian.loke@uts.edu.au
Introduction
As movement-based, interactive technologies continue to become more embedded in our daily lives, aliveness, vitality and pleasure in our interactions with these technologies are becoming sought-after qualities. The designers of interactive technologies are seeking to develop approaches with a central focus on movement, bodily awareness and felt experience, which account for such “ineffable” qualities of human experience; qualities which can often escape definition or measure, but are a necessary part of meaningful existence.
Dewey (1934) wrote about the “aesthetics” of experience, not in the sense that we associate aesthetics with a specific quality of some artwork, but as a crucial component of all human experience, namely where our need for a sense of the meaningfulness and wholeness of our action is fulfilled and from which a sense of self emerges, is developed and transformed. Writing on Dewey’s work, McCarthy and Wright (2004) noted:
Although experience always occurs in cultural, historical, and material contexts, meaningful engagement depends on the event or action being felt, known, and valued in unique ways. This is also the kind of meaningful engagement that transforms people and systems. (p. 56)
Our understanding of aesthetics in the context of technology design is an aesthetics of experience deeply anchored in the body. In our research we are foregrounding the kinaesthetic (the felt sense of movement) in an aesthetics of experience applied to interaction design, hence an aesthetics of interaction. This is not to say that other modes of sensory experience and action are not relevant for an aesthetics of interaction, but in our case we are placing a sharp lens on movement and its felt experience in order to understand more deeply the phenomena surrounding designing for movement-based interactions with technology. Our research is strongly interdisciplinary and seeks to inform an approach to design which focuses on the aesthetics of interaction by drawing on other disciplines such as dance and performance, that have a long tradition of embodied knowledge, skills, sensibilities, techniques and training grounded in the sensing and feeling body.
With other researchers exploring similar terrain, we share a commitment to designing interactive systems from experiences and explorations of movement, rather than from a technological starting point (e.g., Buur, Jensen, & Djajadiningrat, 2004; Schiphorst & Andersen, 2004; Klooster & Overbeeke, 2005; Hummels, Overbeeke, & Klooster, 2007; Jensen & Stienstra, 2007; Djajadiningrat, Matthews, & Stienstra, 2007; Larssen, Robertson, & Edwards, 2007; Moen, 2007). The appropriation of methods and attentional strategies from dance, performance and somatics, which prioritise direct experience and first-hand, first-person perspectives of movement and kinaesthesia, is a promising direction for research in this area.
The research work presented in this paper is part of a larger project aimed at investigating ways of experiencing, performing, describing and representing the moving body in the design of movement-based interaction (Loke, Larssen, & Robertson, 2005; Loke & Robertson, 2007, 2008, 2009). The focus of this paper is our work with trained dancers. We conducted two studies with trained dancers and physical performers to explore ways of working with the moving body in interaction design. These studies prioritised the first-person experience of moving and sought to capture and articulate this perspective. Our research has a strong phenomenological commitment reflected in the primacy of the lived and embodied experience of the people for whom we may design and of those who participate in our research.
In the studies the movement was analysed from the three perspectives of the mover, the observer and the machine. These three perspectives provide a range of complementary understandings and representations of the moving body that can be used as resources for designers to systematically map the trajectory from the moving body to machine input. By using the term trajectory we explicitly seek to avoid assumptions that we are defining routine applications of a formula, recipe or any predefined mapping from people’s actions to computer responses. Instead we are referring to a pathway through time, specifically a design process that allows design decisions to be made at different stages along the trajectory. It is at these different stages that particular mappings from action to response might be decided. The point is that by enabling first person experiential accounts to be matched, in systematic and principled ways, with the observations of human observers and with the representations available to computers, we can ensure that the computer representations are grounded in, and accountable to, the immediacy of first person experience.
The results of the studies were examined to identify an emerging set of methods and tools for designers to work with movement and felt experience in the context of movement-based, interactive technologies. The use of these methods and tools develops a designer’s embodied intuitions and repertoire for working with movement and its felt experience in the generation and evaluation of design concepts, prototypes and final systems. As Löwgren (2007) pointed out, “One of the key elements in professional design ability is assessment, i.e., judging how “good” a design idea is. (…) experiential qualities are concepts that professional designers can use to develop their assessment skills and hence design better products” (p. 1). It should be noted that the focus of this paper is not to design and implement a complete system, but to give designers tools and techniques for generating, evaluating and performing movement-related concepts through embodied skills and sensibilities. By acquiring these skills and sensibilities designers can work with movement as input into interaction in a more informed way. The methods and attentional strategies imported from dance and performance have been carefully selected and modified to suit the aesthetics of interaction design context, in that designers can quite quickly begin working with the tools and techniques to develop their design sensibilities for working with movement, without requiring years of formal dance/performance training.
In the following sections, we give an overview of related work. We then describe in detail the two studies we conducted with trained dancers. The emerging set of methods and tools for designing movement-based interactions with technology, extracted from the studies, is presented. The paper concludes with motivations for using these methods and tools and suggestions for future work.
Related Work
We briefly summarise the emergence of an approach to technology design which values aesthetic experience and draw attention to research with a particular focus on designing for movement and felt, kinaesthetic experience. We then highlight the importance of representations in design work and the particular challenges of representing movement and its felt experience.
Designing for Movement and Felt Experience
The recent valuing of felt-life and aesthetic experience in the design of interactive computing technologies (McCarthy & Wright, 2004, 2005; Schiphorst & Andersen, 2004; Fiore, Wright, & Edwards, 2005; Jacucci, Jacucci, Wagner, & Psik, 2005; Larssen, Robertson, & Edwards, 2007; Schiphorst, 2007; Loke & Robertson, 2007, 2008; Boehner, Sengers, & Warner, 2008; Wright, Wallace, & McCarthy, 2008) is underpinned by a resurgence of pragmatist aesthetics, particularly the work of John Dewey (1934) and Richard Shusterman (2000). The pragmatist commitment to valuing and enriching the ordinary everyday experience of life and actively imbuing it with meaning through an integration of the sensual, intellectual and emotional aspects of life is reflected in the “aesthetics of interaction” approach to designing interaction (Petersen, Iversen, Krogh, & Ludvigsen, 2004; Fiore et al., 2005; Lim, Stolterman, Jung, & Donaldson, 2007). Aesthetics of interaction is described by Petersen et al. (2004) as an aesthetics of use, rather than an aesthetics of appearance, with a double focus on experience and expression.
In his proposed discipline of somaesthetics, the pragmatist philosopher and Feldenkrais practitioner, Richard Shusterman (2000) further develops Dewey’s holistic approach to aesthetic experience through an incorporation and cultivation of the body. He makes the case that the co-development and intertwining of bodily and mental awareness enriches our capacity for aesthetic experience, thus enabling us to participate more fully in life. In interactive art, Schiphorst (2007) proposes a framework from the field of somatics (the body as experienced) for understanding and accessing the body in everyday life through attentional strategies that can then be applied in design.
The kinaesthetic is an important aspect of aesthetic experience, as it provides us with our self-perception of movement, the felt sense of our bodies in motion (Fraleigh, 1987; Sheets-Johnstone, 1999). The ability to work with the nuances of our kinaesthetic experience is an active area of design research. Researchers are inquiring into how to develop a design sensibility for working with movement, by drawing on first-person methodologies from dance and somatics (Schiphorst & Andersen, 2004; Klooster & Overbeeke, 2005; Djajadiningrat et al., 2007; Hummels et al., 2007; Jensen, 2007; Larssen et al., 2007; Moen, 2007; Schiphorst, 2007). Bodily experience and awareness are taken as a starting point for design. Dance practices marked by exploration, improvisation and creative agency lend themselves to this approach, as opposed to those that are more concerned with imposing a style on the body (for example, ballet). Movement improvisation practices play with the relationship between everyday and abstracted movements, enabling the development of a more creative relationship with our moving bodies (Blom & Chaplin, 1988). Drawing on methods from dance, designers can creatively inquire into new movement possibilities and associated forms of felt experience that can be mediated by technology. Whisper (Schiphorst & Andersen, 2004) and BodyBug (Moen, 2007) are examples of movement-based interactive systems that explicitly exploit and develop a person’s sensing and feeling capacities of movement. In these works the aim is to reflect aspects of the kinaesthetic experience of movement in the actual interaction afforded by the interactive system.
The moving body is in one sense a visual medium—it can be seen by others. We can use our bodies to convey or represent ideas, qualities, forms and other meaningful aspects of the design situation (e.g., Buur et al., 2004; Klooster & Overbeeke, 2005; Djajadiningrat et al., 2007; Hummels et al., 2007). The design strategy of ‘actions before product’ has an emphasis on understanding and exploring physical actions prior to designing “interface mechanisms that afford such actions” (Buur et al., 2004, p.186). In the design of interactive products, approaches are emerging which incorporate movement qualities into product form and interaction (e.g., Klooster & Overbeeke, 2005; Jensen & Stienstra, 2007).
In the context of interaction design, an awareness of the interplay and intertwining between body and mind, action and perception broadens the significance of movements selected for interaction and the forms of bodily engagement enabled by particular technology design decisions. The types of movement and the forms of bodily engagement that are encouraged or allowed for interaction have a corresponding impact on the kinds of experiences we might have in computer-mediated environments.
Representing Movement
The representation of the moving body presents challenges to designers, as the dynamic, temporal nature of the body-in-motion and the felt experience of movement are not easily transferrable to traditional, static forms of representation. Video documentation attempts to alleviate the first challenge of representing the flow of movement in time, but not the second of representing the felt experience of movement.
Visual representations organize perception and are part of visual communication practices. In Henderson’s (1991) study of design engineers and their visual communication practices, sketches and drawings helped coordinate distributed cognition as they allowed for the manipulation of tacit knowledge between individuals. Images and video are commonly used in analyzing movement for use in design (e.g., Buur et al., 2004; Høysniemi & Hämäläinen, 2004). The video action wall technique involves construction of a mosaic of looped video clips of user actions that can be rearranged to highlight, group and compare qualities of movement (Buur et al., 2004).
The fixity of static images, however, allows designers to reason about, critique and hold onto movement-related design concepts and understandings. In designing movement-based interactions, we also need representations of movement that enable us to think about, and link, movement and machine input, processing and output. Movement notations are a potential tool for analyzing and representing movement that can act as a bridge between the performance of movements and the interpretation of movement by the machine. We applied Laban’s (1971) system of movement analysis and notation in the two studies presented in this paper, as it is one of the few notation systems that analyses the dynamics of movement. See also our earlier work on Labanotation (Loke et al., 2005).
Studies With Trained Dancers
We conducted two studies to explore how movement could be understood and utilised in designing interactions with technology. We worked with dancers and performers, trained in a range of practices including acrobatics, butoh, contemporary dance, Feldenkrais, movement improvisation, physical theatre and stilt-walking. Most of the participants were trained in more than one of these areas. The first study was of the falling body by skilled movers. The second study explored ways of inventing and choreographing movement.
The aim of the first study was to explore the act of falling from a first-person, experiential perspective and from an external, observational perspective. These different perspectives are defined in detail in the later section on An Emerging Set of Methods and Tools. In this study we viewed skilled movers as the ethnographic exotic (Geertz, 1973), in order to defamiliarise everyday movements. Falling is a specific form of movement that is on the periphery of our everyday realm of movement. It is a movement we all performed as children that recedes from the everyday action of most adults, only to return in our old age in a more vulnerable form. Another motivation for studying the act of falling is that it is not part of the established movement lexicon in digital praxis. This makes it open for investigation, unlike gestural actions such as reaching and pointing, which are well known and researched in human-computer interaction and virtual reality.
In the second study methods that dancers, trained in movement improvisation and performance making, used to generate, choreograph and document movement were examined as sources of potential methods for designers. An inspirational resource kit was trialled as a tool for inspiring and documenting the movement ideas and choreography. The representations of the moving body from the external view, initially trialled in the first study, were applied to this study and further extended.
Study I. A Study of Falling
Interviews and physical demonstrations were conducted with trained dancers and physical performers to examine the process and experience of falling. These sessions were filmed on digital videotape and also recorded with a digital audio recorder for transcription purposes. The video footage and audio recordings were used as records of the session for later iterative analysis. Analysis was performed on the raw data from two perspectives—an experiential perspective and an external, observational perspective. The analysis generated a range of different descriptions and representations of the falling body. The experiential perspective produced first-person accounts of the process and experience of falling and three characteristic components of movement for describing the act of falling. The external perspective produced visual movement sequences of the moving body, silhouettes of changing spatial shapes of the moving body and Laban Effort/Shape descriptions of the qualitative, dynamic character of the movement. These activities are described in more detail below.
Interviews and Physical Demonstrations
Eight participants were recruited—six female and two male. Each participant took part in a half-hour session where they were asked to physically demonstrate acts of falling using the bodily techniques in which they were trained. During this session, they were interviewed about the act of falling, specifically to determine the techniques for falling, the felt experience or sensation of falling, and how it fits into their practice, both in training and in performance.
Participants undertook their own warm-up prior to the interview/demonstration. Each session began with the participants improvising their own movement and initiating acts of falling to the ground. After a few minutes, the interview began with the researcher asking questions and prompting clarification. Participants would answer verbally and quite often begin to move again to demonstrate aspects of the action/process of falling. In some cases, the researcher asked the participant to teach the technique to the researcher, so that more information could be gained through the learning process.
Analysis from the Experiential Perspective
The raw data was analysed to gain an understanding of the act of falling as experienced by the people performing the movement. Written transcriptions of the interviews were taken from the audio/video records. The video footage was edited to produce a summary of each participant’s demonstration and explanation of falling. These video summaries were useful for returning to a dynamic, visual replay of each participant’s way of falling, and function like selected transcripts as described by Jordan and Henderson (1995) in the use of video data for interaction analysis. The written transcriptions were edited into a more compact form, which we called a first-person experiential account. The first-person experiential accounts were reviewed with participants in a follow-up session to ensure that they were a faithful record and representation of their understanding of falling. Figure 1 contains a sample of excerpts from verbatim transcripts.
Figure 1. First-person experiential accounts of falling.
Both the original footage and the video summaries were viewed multiple times to identify the salient aspects of the act of falling as experienced by the participants. The participant’s verbal articulation of their own movement processes was extracted and these phrases reflected each participant’s individual ways of articulating their understanding of their own movement processes. The number of viewing times for each clip was dependent upon whether any new information or insights were gained into the first-person experience of falling. These phrases were then grouped into three characteristic components of movement, as defined in Table 1. Each of these characteristic components will be elucidated with examples from the participants.
Table 1. Three characteristic components of movement for describing the action of falling.
The movement can be analysed as a process of the body changing relationally in space and time. This movement process can be broken down into a series of distinct stages, which are dependent on the particular movement being analysed. For falling, there are three distinct stages in the process of falling—initiating the fall, descending, and contacting the ground.
The technique for informing or directing the movement process is an intrinsic part of the performance of the movement. There is a range of techniques peculiar to the act of falling for initiating the fall, controlling the descent and contacting the ground safely. Figure 2 summarises the range of techniques demonstrated and described by participants. These techniques can be broadly categorised as being mechanically-based or image-based. In mechanically-based techniques, the focus and emphasis is on the order, organisation and sequencing of body parts in relation to each other and the environment as the movement unfolds. However the conscious focus on the detailed mechanics of the movement lessens as the technique is mastered. As participant 8 explains, you “give over the rational; technique goes into automatic pilot.”
Figure 2. First-person techniques for falling, as described by participants.
Here are some examples of this category of technique for the three stages of falling. Participant 8 goes off-centre to initiate the fall, whereas participant 1 drops her weight vertically down to the ground. Participant 4 uses an internal muscular lift to slow down the descent, and participant 1 controls the slide out to the side by extending the other arm away from the direction she is moving in. Participant 2 contacts the ground safely by making the contact with the ground take the greatest amount of time and cover the greatest surface area of the body, whereas participant 3 releases any tension and softens into the floor. In general, all participants worked with softening into the ground as they landed.
In image-based techniques, the focus and emphasis is on working strongly with the image to direct and inform the movement process. If you surrender fully to the image, the body follows. There is less attention given to specific body parts moving in a certain order. Participants 6 and 7 use the image of a string being cut from the crown of the head to initiate the fall and the image of the body as a bag of bones to descend and contact the ground. A different example is participant 5’s use of the image of being pushed purely to initiate the falling.
The characteristic component of sensing and awareness—internal and external refers to the active sensing and awareness of one’s body in relation to itself and to the external environment. We include the senses of the visual, aural, tactile, and proprioceptive/kinaesthetic (taste and smell are not included as they were not mentioned by any of the participants in this study). In regards to falling, it was interesting to tease out the relationship between the visual and kinaesthetic senses, and how they were utilised in the act of falling.
Protecting the head is crucial when falling. Looking at participant 4’s first-person experiential account (see Figure 1), we can see that she is constantly mapping where her body is in internal relationship to itself. She first ascertains where her head is in relation to the rest of her body, and then maps where her pelvis and feet are in relation to her head. In the act of falling, we draw continuously on our kinaesthetic sensing to know what the body is doing and how it is aligned at any particular moment.
The visual sense is predominantly used to check where one is in the space and in relation to others. The awareness of the external environment is reliant primarily on the visual sense. Participant 6 explains that “Visual sensing keeps me aware of the outside, otherwise can become too internal.” Participant 8 makes the observation that, “You need that visual to know where you are in the space, to remember what plane you are on, especially when you’ve thrown yourself off-centre.” The two senses work together to provide an ongoing awareness of the internal relations and state of one’s body in relation to the external environment.
The felt quality of the movement refers to the sensation or feeling in the body. It is an inextricable part of any movement. A selection of participants’ descriptions of the felt quality of their movements in the act of falling is given in Figure 3. Some participants separated out the felt quality of the descent from the felt quality of the landing. This analysis reveals a diverse range of understandings of the process and experience of falling for these eight participants.
Figure 3. Participants’ description of the felt quality of falling.
Analysis from the External Perspective
When considering the action of falling as input to a video-based motion sensor, it can be modelled and analysed from an external perspective in many ways. We present a range of visual representations of the falling body to which more abstract and complex transformations can be applied. Movement sequences were extracted from the video footage of each participant performing a particular instance of falling. These movement sequences allowed a closer analysis of the moving body in its trajectory through space/time. The movement sequence for participant 4 is shown in Figure 4. The time between each image frame in the sequence is 0.4 seconds.
Figure 4. Movement sequence for participant 4.
A series of silhouettes was made from these movement sequences to draw out the spatial shaping of the body for different types of falls. As for the movement sequences, each snapshot in time is spread out spatially in the representation so the body and any overlaid data can be clearly seen at that instant. The shade of grey deepens over time to indicate the progression of the movement. From these two representations a range of different parameters can be derived. These parameters include the trajectory of the body, the changing position and relation of body parts along the trajectory, distinct types of falls, the dynamically changing pattern of spatial shaping, timing, rhythm and the qualitative, dynamic character of the movement. The dynamically changing pattern of spatial shaping for participant 4 is illustrated in Figure 5. The dots highlight the changing positions of the head, centre of torso and feet—one way of indicating the mapping or tracking of body parts over the trajectory of the fall.
The qualitative, dynamic character of the movement can be described in evocative terms or using the Effort/Shape component of Laban’s system of movement analysis (Laban, 1971; Hutchinson, 1977). Effort is the energy expended in performing the movement or the external expression of the inner attitude of the mover. The analysis of Effort is given in four motion factors of Space, Time, Weight and Flow. The spatial shaping of the body can be analysed in terms of what forms the body makes and the relation of the body to itself and its environment. Shape analysis provides a set of descriptors for dynamic, fluctuating shape characteristics, classified into categories of Shape Form and Shape Quality (other categories exist but have not been used in this research). Shape Form describes the static shapes that the body takes, for example, wall-like, pin-like or ball-like. Shape Quality describes the way the body is changing toward some point in space, for example, opening or closing, indicating the degree of extension or contraction in the body. More specific terms include Rising and Sinking (along the vertical axis of the body), Spreading and Enclosing (along the horizontal axis), and Advancing and Retreating (along the sagittal axis).
The qualitative, dynamic character of participant 4’s fall is suspended and buoyant within a controlled, circular descent. With reference to Figure 5, the corresponding Effort is indirect in Space, sustained in Time, light in Weight and bound in Flow. The Shape begins pin-like, then become arc-like in form as the body arcs backward in spinal extension through a curved trajectory towards the ground. It then becomes more screw-like as contact is made with the ground. The Shape Quality is sinking, retreating and spreading in the descent to the ground. As the hands and front of the body contact the ground, the Shape Quality changes to enclosing, then to rising as the fall is resolved.
Figure 5. Changing spatial shaping of body and mapping of head, centre of torso and feet over time for participant 4, with Effort/Shape descriptions.
The analysis of the first study generated a range of different descriptions and representations of the falling body. From an experiential perspective, these included first-person accounts of the process and experience of falling and three characteristic components of movement for describing the act of falling. The descriptions of the movement process and the felt experience of movement in the act of falling varied considerably across participants. This rich variation suggests unexplored opportunities for accessing the creative potential of the moving body in design work—preliminary exploration of this proposal was then conducted in the second study. From an external, observational perspective, these included movement sequences of the moving body, silhouettes of changing spatial shapes of the moving body and Laban Effort-Shape descriptions of the qualitative, dynamic character of the movement. These external representations act as a bridge between the movements of people and the formulation of recognition algorithms for movement as input. Designers can easily manipulate and annotate these representations to explore different options for interpreting and responding to movements used as input to the machine.
Study II. Choreographing Movement
The primary aim of the second study was to explore ways of inventing and choreographing movement for use in the design of movement-based interactions, by drawing on the practices of dancers and physical performers trained in movement improvisation and performance-making. A secondary aim of the study was to explore forms of representing the choreographed movements and the corresponding interactive treatments of the movements.
Two workshops were conducted with dancers who had previously participated in the study of falling. The workshops were led by the first author, with the assistance of another researcher. The first workshop was a pilot for figuring out how to work directly with dancers and how to get the required data. The workshop was organised with a participatory approach to the design of an interactive, motion-sensing space that detected the act of falling. We were particularly interested in the act of falling and how it could be utilised both choreographically and interactively in these kinds of spaces. The motivation for studying the act of falling in the second study is the same as was expressed in the first study.
Two dancers, Esther and Michael (pseudonyms) took part in the first workshop. The workshop was planned as a semi-structured set of activities, but with the intention of working flexibly with the resources and the skills of the participants. The core activities included a review of the first study of falling, the exploration, development and presentation of choreographic ideas for use in an interactive space, and finally an examination of possible interactive treatments of the choreography. The choreographic ideas were presented in the form of short, 2-5 minute choreographies, consisting of at least three scenes, with each scene employing a different way of falling.
Two key findings from the first workshop will be briefly discussed before presenting in detail the second workshop. The first finding from the first workshop was that techniques for generating improvised movement, such as scoring, could be useful in design exploration and in the enactment of movements for use in interactive systems. Scoring provides a structure for generating and devising movement based on a set of elements or parameters that can be varied as desired. For example, a simple score used in the workshop consisted of three elements of walking, standing still and moving in place. Other parameters of speed, duration, timing, scale, focus, use of space, and so on, can be added to the score.
The second finding from the first workshop proved critical to the success of the study. The session failed to produce adequate data for exploring interactive treatments of choreographed movements and the corresponding representations of both the movements and the machine interpretations of the movements. On reflection, the session was set up with too few contextual constraints for the dancers to work within, resulting in the production of dislocated fragments of choreographed movements that lacked coherency and significance. This highlighted the need for a specific and well-defined context or domain within which to generate meaningful movements, with regards to the framing of the activity of inventing and choreographing movement. This reinforced an earlier finding from a study of movements performed in interaction with Sony Playstation2 Eyetoy™ games, where the context of the games enabled people to perform meaningful movements (Loke et al., 2005). Others have emphasised this need for specific and well-defined restrictions to successfully produce meaningful work. An example is Brandt and Grunnet’s (2000) use of Keith Johnstone’s theatre improvisation techniques and role-playing in future product design.
The research was restructured to utilise this finding. A second workshop was held with two dancers, Esther and Gloria (pseudonyms), to explore choreographing movement within a specific context. In order to focus specifically on the relation between choreographed movements and machine interpretations of those movements, a constructed design situation was set up involving the initial development of a choreographic work sited within a video-based, motion-sensing interactive space. The default physical and technical configuration for the space was a four-screen projection system and an overhead video camera for sensing the activity in the space. The system would need to accommodate a range of human movement, from the ordinary, everyday movements of people to the skilled, choreographed movements of dancers. The proposed system was informed to some degree by our previous work on Bystander, an immersive, multi-user, interactive artwork, built on video-based motion-sensing technologies (Loke & Robertson, 2009).
The Divine and Bodily Experience
A theme was chosen for the work of The Divine and Bodily Experience. It was selected for being an abundant resource of bodily and movement experiences engendered by existing religious and cultural practices, considered to be culturally shared and familiar to people. The meanings and motivations for bodily actions, movements and postures could be informed by such practices. It also provided a richer, recognised context for acts of falling. It was hoped that this theme would provide sufficient inspiration for choreography of movement and imaginings of what the interactive, immersive space might look, sound and feel like. In a choreographic sense, there was room for artistic interpretation of the theme, rather than a stereotypical representation or reproduction of ritual movements. The point here is that the particular theme chosen is not significant in itself, but for its ability to generate meaningful movements and for its accessibility to performers.
A series of four acts was conceived that would address different kinds of movement, different combinations of audience and performers and different models of interaction between people and the system. Two of the four acts are described in Figure 6. Act 2 was assigned to Gloria and Act 3 to Esther.
Figure 6. Description of Act 2 and Act 3.
An inspirational resource kit was given to the two dancers in advance of the workshop to assist with briefing, guiding and inspiring the choreographic work they were to bring to the workshop. The kit provided a set of thematic constraints and various resources for inspiring and documenting the choreographic work. However the dancers were free to interpret the thematic content and bring in their own interests and training. The specific religious practice of Buddhism was chosen as the thematic content for the kit, as much for the ready availability of images and texts, as for well-established traditions of cultivating transcendence through the body. The kit contained image tiles, evocative texts, movement description cards, floor plan of space (A3 size), CD of music/sound samples and written description of the acts and scenarios. There are some similarities to cultural probes (Gaver, Dunne, & Pacenti, 1999) in the format and underlying artistic exchange, although the kit is used here in a more participatory fashion.
A set of initial scenarios was provided to seed the design work and to give some indication of the possible behaviour of the system in response to the activity of people in the space (a selection is presented in Figure 7). These fragmentary scenarios would be reworked into a coherent account of the activity of the performers, audience and system after the workshop to reflect the ideas and decisions made during the workshop.
Figure 7. Initial scenario fragments.
This use of scenarios continues and extends earlier research which explored the production and use of movement-oriented scenarios as a means of investigating interactivity in interactive, immersive spaces based on motion-sensing technologies (Loke & Robertson, 2009). These movement-oriented scenarios focus on the activity and movement of people (typically users as audience) in the space, described in terms appropriate to the kind of space under design. Here we turn the focus from audience to performers. This enables us to examine more complex and choreographed kinds of movement, compared to the everyday.
In the workshop, the dancers presented and explained how they used the kit. Each dancer then demonstrated and led the other workshop participants in performing the choreographed movements. The workshop concluded with the group discussing and documenting possible interactive treatments of the choreographed movements. The activities of the workshop were filmed for later analysis. The analysis of the data generated from the workshop is now described in the following sections.
Inspiring and Choreographing Movement
In this section the development of each dancer’s movement ideas and choreography is described, including their use of the inspirational resource kit for inspiration and documentation. Their different ways of choreographing movement are then discussed.
Using the Kit
Esther worked with most of the resources from the kit—the evocative texts, the image tiles, the movement description cards, but not the music (she preferred to add music later, as she felt that music can dictate the choreography too strongly). She began by selecting a few lines of text that triggered some kind of movement idea. She documented her movement choreography ideas on the large sheet of cardboard by using the image tiles and textual descriptions to build a thread or sequence. Interestingly she placed all of it around the outside of the square representing the physical space of the system, and did not use this space. A photograph of her use of the kit is in Figure 8 (a).
She describes her way of working with the kit as,
It was selecting one of those (evocative texts) that I felt – something that gave me a movement impulse. Even the word ‘suspense’ … I started to think about suspension, and that was something from the last session that I was working with, that sense of teetering and suspension. We live in a constant state of suspense and ambiguity. So from suspense, I thought of acts of suspension, and trying to relate that to my act, which is Swooning in Ecstasy. I started to think about how something would build into some kind of swooning, or ecstatic state. I then started to look at the tiles, and I guess the images – I didn’t choose them aesthetically, it was more about whether I had some kind of feeling state from them.
Looking at her documentation, we can see that it is organised around the images and concepts. The focus is on the body movement arising from these images and concepts. A series of stick figures depicts the sequence of movements in a choreographic phrase, noting the temporal development in terms of pace and repetition.
Gloria did not use the kit for choreographing after the initial inspection of the contents. Instead she chose to develop her movement/choreographic ideas from the movement practice of Qigong. She made brief notes in a small book. At the beginning of the session she transferred these ideas onto the large sheet of cardboard and found some image tiles that resonated with her ideas. A photograph of her use of the kit is given in Figure 8 (b). She describes her way of working with the kit as,
Well that was sort of interesting. I did it the way I usually do things. Having seen this initially and liked it, not knowing how I was going to use it, and did it my usual way. And then, it actually fits. It’s good, all these pictures which I wasn’t actually thinking about when I constructed it, well actually that one fits that, and that one fits that. So it sort of worked in retrospect.
Looking at Gloria’s documentation using the kit in Figure 8 (b), we can see the ideas she had for structuring space and generating movement. There are strong spatial shapes and directions for moving in trajectories through space, for example, circle, figure-eight and radiating out from centre to corners. The body/movement ideas are predominantly expressed as a combination of gestures, energy (chi) flow and imagery. The image of a woman bathing with her hands held up to her face in a prayer position is annotated with the text “blow through palms to clean”. The image of a star-studded sky is annotated with the text of “open arms—petals, stars”. These two examples indicate the type of gesture and imagery to be used in performing these movements. As yet there is no specific sequencing or development of these movement ideas into a definitive choreography.
Figure 8. Documentation of choreographic ideas using the kit by: (a) Esther and (b) Gloria.
The dancers’ quite different ways of working with the kit brought out the multiple functions of an artefact in design—in this case, the set of image tiles played a dual role: an inspirational role in terms of provoking and generating ideas, and a documentary role in terms of providing an alternative form of articulating or presenting an idea. The movement/choreographic ideas are expressed or articulated through a combination of text, sketching and images. This documentation using the kit can then act as a resource for returning to the original ideas as conceived by the dancers.
Inventing and Choreographing Movement
A closer examination of Esther’s process of using the kit reveals a method of inventing and choreographing movement that begins from multiple entry points: a piece of text or a word can invoke a movement impulse or inspire thinking on related or associated concepts; images can evoke a feeling state. A concept can give rise to a movement or kinaesthetic sensation that can then be developed choreographically. A specific example illustrated in Figure 9 begins with an image of a woman lying prostrate on the ground.
Figure 9. Close-up of Esther’s documentation using the kit.
The annotated text by Esther reads “slaking of the demon of ego, whipping the head”. The next reference to the head is “wrestling with the logos”, which then leads to the stick figures depicting a sequence of positions, where the movement transitions are initiated with the head. In this example, there is a clear connection between the original image, the concepts and the movement choreography. This is exactly the kind of connection we wanted to make visible, as a resource for designers.
An examination of Gloria’s process of choreographing movement (from observations and video footage) reveals another method for choreographing movement. She begins with a movement phrase taken from a traditional movement form. She then experiments with variations of the movement phrase through actual movement improvisation. She uses imagery and energy qualities to inform the character and shape of the movement. She uses her intuitive sense of feeling things in the body to decide what works for this choreography. For example, with the “blow through palms to clean” gesture, she begins by breathing into the hands in prayer position, focusing on the rhythm of the breath. She plays with the hands expressing the expansion and contraction of the lungs, varying the scale and speed of the hand movements, until a certain arrangement of gestures and body movements is reached. The crystallisation of the choreography is not explicitly explained by her, other than as a confirmation felt in-the-body through the integrated acts of sensing, feeling and moving.
Gloria’s method of devising movement highlights a crucial aspect of working with the moving body, which is to have an understanding of movement ‘in-the-body’. Movement possibilities are experienced and developed through the sensing, feeling and moving body. Rather than simply observing the movements of another, the imperative here is for designers to explore and perform the movement ideas, so that they can acquire an understanding of movement that is rooted in their own felt, bodily knowing.
Documenting Movement
This section describes ways of documenting choreographed movement and the accompanying machine interpretations of that movement. Descriptions and representations of the movement choreography and interactive treatment were produced both during and after the workshop. A sample of these descriptions and representations is presented here to illustrate the range of textual descriptions and visual diagrams that can be used to capture the salient aspects of the moving body, for use in the design of movement-based interaction. The representations were crafted using two criteria. The first was to facilitate the re-generation of the performance and the experience of the movement. The second was to capture critical aspects of the movement for consideration when it came to deciding what should be included for input and interaction with various interactive technologies. Each of these descriptions and representations is discussed in terms of the role they can play in the design of movement-based interaction.
Directions for Choreographed Movement
Descriptions of the inspiration and direction for choreographed movement, resulting from the workshop, were documented. This set of written descriptions of the choreographed movement for the performers was refined into a more coherent account from the original wordings and ideas contributed by the dancers. The refined accounts were reviewed by the dancers to ensure that the rewording was faithful to their original accounts and that nothing crucial had been omitted or misinterpreted. These descriptions are a written record of the choreographed movement, that details the specifics of how the body moves, the motivation for the movement and the kind of act in which the movement is contained. They provide directions for performing the movement. They can be thought of as a form of movement-oriented scenario; scenarios being scripts for action (Ehn & Sjögren, 1992) and a flexible means of representing current and future uses of systems (Kyng, 1995). An example from Act 2 is presented in Figure 10.
Figure 10. Directions for choreographed movement, Act 2 Section 2.
Visual Movement Sequences
The movement sequences are a visual representation of the key movement phrases and postures of the body in the form of a series of images. They serve as visual reminders of what the movement looks like: the organisation of the body and its parts, the shape of the body and the relationship of the body to its environment. They correspond to the directions for choreographed movement of the performers. They were extracted from the video footage and photo documentation taken in the workshop. An example of a visual movement sequence with Laban Shape analysis is given in Figure 11. It corresponds to the original choreographic ideas documented in Figure 9.
Figure 11. Visual movement sequence of Act 3, Section 1, with Laban Shape analysis descriptions.
The performer is sequencing through a series of four postures. The head initiates the transitions between the postures. The last transition is rolling back up to a standing position.
The movement sequences may be useful for informing the design of the interactivity in terms of what the system can see or detect and its subsequent response. Comparison of the movement sequences over time can assist in determining the points of transition or differentiation, which in turn can be used for triggering specific system responses. It should be noted that the video footage of the choreography was taken from a side-on view rather than an overhead view due to the lack of technical infrastructure to support the mounting of an overhead video camera. This substitution does not detract from the process advocated here for exploring the mapping from an observer perspective to a machine perspective. For any concrete design situation, the appropriate views of the body required by the chosen sensing technologies would be used.
Shape Analysis
The Shape analysis is a description of the changing forms and spatial qualities of the moving body. The Laban system of movement analysis was used to analyse and describe the spatial shaping of the body in relation to itself and to its environment. An example of Shape analysis for a movement sequence is given in Figure 11. The Shape Form for the key postures varies from ball-like for the first posture to wall-like for the second posture where the performer is in a splayed position on the ground, then back to ball-like as the performer contracts back to a crouching position. The Shape Quality is sinking and enclosing for the first transition from standing to the crouched posture with leg extended in front. It becomes spreading and advancing as the performer dives into the splayed position belly-down. It then becomes enclosing as the performer gathers her limbs in towards her centre and pushes off into a crouch.
Spatial Movement Schemas
The spatial movement schemas are based on Labanotation floor plans (Hutchinson, 1977). They provide an at-a-glance view of the changing configurations and spatial trajectories of people present and moving in the space. An example of a pair of spatial movement schemas for act 2, section 1 is given in Figure 12. The diagram on the left shows the trajectories of four performers entering from the four corners of the space and weaving around the space. The diagram on the right shows the four performers moving along a circular path in a clockwise direction.
Figure 12. Spatial movement schemas for Act 2, Section 1.
Machine Input Schemas
Machine input schemas illustrate the choices of machine inputs, the detection and interpretation of the input and the corresponding system response. An example of a machine input schema for the first section of the second act, Ritualising the Space, is presented in Figure 13. It was decided during the workshop that the incoming video data would be processed to detect the speed and shape of the trajectory of the performers. In response, the system would produce a wall of sound that intensified with increasing speed. Over time, the trace pattern of motion that emerges is circular in shape, indicated by the circle (labelled A). It is one possible interpretation of the performers’ movements that focuses on the speed and shape of the trajectory produced by their locomotion through the space.
Figure 13. Machine input schema for Act 2, Section 1.
The ‘Collecting Qi’ gesture was also nominated for detection, as the spatial shaping of the body was considered to be easily differentiable from the rest of the movements. In the diagram this decision is indicated by the silhouetted figure labelled C. The nomenclature for the Shape analysis descriptions could be referenced here, to provide a language for mapping between the observational perspective and the machine perspective. The gesture itself could be loaded with symbolic meanings, which could be expressed through visual or aural forms of system output.
An Emerging Set of Methods and Tools
The results of the two studies contributed to a set of methods and tools for working with the moving body in interaction design. The methods and tools were selected and crafted to enable designers to move between and integrate the three perspectives of the mover, the observer and the machine (see Figure 14). The first-person, experiential perspective of the mover ensures designers are accountable to the felt, lived experience of the mover and to the potential users of technology. The observer perspective provides the view of the body from the outside as seen by another person, enabling the framing of movement from a range of different and complementary views including, but not limited to, the biomechanical, the social, the cultural and the ecological. It enables the designer to stand in for other people in the environment and to embed the moving body in various domains and contexts of use. The mover can also be in the position of observer of their own movements, for example during review of recorded movements. The machine perspective focuses on the sensing and interpretation of the moving body by the computer, as determined by the choice of input sensors and processing algorithms. It ensures designers are accountable to the machine view of the movements of users and that appropriate mappings are made between user activity and machine interpretation and response. The particular methods and tools derived from the two studies focus on describing, representing, experiencing, generating and structuring movement for use in interaction with motion-sensing technologies.
Figure 14. The three perspectives of the mover, the observer and the machine.
The Mover
The perspective of the mover generates first-hand, first-person experience of the moving body. The primary data is in-the-body, where skills are developed for performing, attending to and articulating movement and its felt experience. Secondary data is the explicit articulation of the primary data in the form of words, text and diagrams. This is a necessary resource for design communication and shared understanding, however as Boehner, Sengers, and Warner (2008) stress, aesthetic experience “cannot be fully understood through rational explanation but must be lived.” From the studies, we identified methods and tools for generating and documenting both primary and secondary data. The next section presents methods/techniques for generating primary data. Tools for documenting the secondary data include i) first-person experiential descriptions such as first-person experiential accounts (Study 1, Figure 1) and characteristic components of movement (Study 1, Figure 2 and 3); and ii) a combination of text, sketching and images such as the inspirational resource kit (Study 2, Figure 8).
A set of techniques for generating, experiencing and structuring movement, which could assist designers of movement-based interactive technologies in working with the moving body from a first-person, experiential perspective is now described. The techniques focus on ways of exploring and improvising with the moving body to cultivate skill and a refined awareness of the sensing, feeling and moving body. The three techniques are i) Playing with everyday movements and gestures; ii) Scoring; and iii) Generating movement from imagery. In the following sections, we describe the principle of each technique, with examples.
Playing With Everyday Movements and Gestures
Everyday movements and gestures can be the starting point for exploration of new movement possibilities and experiences. A movement can be performed with kinetic variations of speed, scale and direction to produce different patterns, dynamics and qualities of movement. Speed varies from slow to fast. Scale varies from small to large. Direction varies relative to the body axis or to the axis of external space. The principle of making strange is activated in this technique, as a habitual movement is defamiliarised through the application of kinetic variations. This in turn creates new imaginings of our movement possibilities.
For example, you can take a simple, everyday gesture such as swinging your arm up and down. You can vary the performance of the gesture by moving your arm to and fro very slowly and smoothly or with a jagged stutter. The focus here is on the relation between the movement and the felt sensation of movement. Experiment doing this with your eyes open or closed and note how this changes your awareness of the process and sensation of the movement. A butoh technique makes strange with the everyday act of walking, by slowing down the pace of your walk to as slow as possible. This speed change brings sharply into focus how our body organizes itself in walking and the precariousness of our balance at this very slow speed. The act of walking becomes a negotiation of weight transfer and balance.
Scoring
The method of scoring used in movement improvisation provides a structure for generating and patterning movement based on a set of parameters or constraints that can be varied as desired. Parameters or constraints related to speed, duration, timing, scale, direction, focus, use of space, relation to others, use of props, and so on, can be added to the score. Multiple people can use the same score or work with different scores simultaneously to generate varying patterns of movement. Imagery or qualities of movement can be incorporated into the score. For example, a simple score dealing with motion, stillness and speed has four elements: i) walking at normal pace, ii) walking very slowly, iii) standing still and iv) moving in place. The timing, the change in pace, the direction, the location, the transitions and so on are left up to the performer.
Scores can be used for improvising movement to explore movement ideas for interaction or for generation and enactment of movement in user testing. They are a simple yet extremely powerful way of generating semi-structured patterns of movement. They can be used for playing with everyday movements and are accessible to unskilled movers. Movement scores can be combined with traditional personas and scenarios for use in user testing when loosely structured patterns of movement are required. The choice of parameters in the score is highly flexible and can be manipulated for each persona in a given situation.
Generating Movement from Imagery
A different kind of technique uses imagery to shape body movements and generate distinct movement qualities, such as ‘like a heavy stone’ or ‘like a floating feather’. The image can be localized to a part of the body, extended beyond the physical body or be outside the body in space. For example, you might move your leg as if it contained a viscous fluid or your bones were brittle twigs. Or you might imagine that a long string was pulling you up by the crown of the head towards the heavens. Imagine moving through mud or the air surrounding you is extremely cold. The focus is on generating felt sensation in the body corresponding to the image, which in turn informs how you move. The more fully you commit to the image, the more convincing the expression. You become the image. The use of imagery for generating movement can be considered a way of making the familiar strange. Embodying images in this manner allows us to experience the felt sensation of our moving bodies in a radically different way to the everyday and may result in novel concepts for design work.
The Observer
Tools for describing, representing and analysing movement, from the perspective of the human observer, were trialled and developed in the two studies. They provide observational data on the sequencing and bodily organization of the body-in-motion, as well as descriptions of the dynamic qualities of the moving body. The tools were crafted with the role of acting as return points to the active, moving body and direct experience of movements by designers and representative users. Examples of each tool were provided in the two studies and references back to the studies are indicated in brackets for each tool.
The range of tools includes i) textual descriptions of observed movements and choreography such as movement scenarios (Study 2, Figure 7) and directions for choreographed movement (Study 2, Figure 10); ii) visual movement sequences and silhouettes (Study 1, Figure 4, Figure 5; Study 2, Figure 11); iii) Laban Effort/Shape movement analysis (Study 1, Figure 5; Study 2, Figure 11); and iv) spatial movement schemas in Labanotation floor plans (Study 2, Figure 12).
Some computerised motion recognition systems process the in-coming video stream using algorithms based on Effort/Shape parameters. These kinds of systems are intended to recognise the more expressive components of human movement. Our use of Laban movement analysis and representations emphasising silhouettes and spatial shaping of the body fits well with this kind of system, as they can potentially act as bridging representations between the movements of people and the formulation of recognition algorithms for detecting the dynamic, qualitative aspects of movement. The tools offered here for the observer perspective focus on an analysis of the moving body in terms of its structure and physical organisation, the qualitative, dynamic and expressive aspects of movement and the changing paths of moving bodies in space and time. They can be further extended to bring in aspects of the social, cultural and ecological context for building a more enriched picture of how people act and interact in particular situations.
The Machine
The important point to understand when designing representations for machine input, processing and output, is that machine interpretation of the input must be programmed specific to the system under design and the sensor technology employed. For video-based sensors, the input is dynamic visual data of moving bodies, which can be broken down frame by frame. Designing the machine interpretation of the input rests on conceptual decisions taken by designers about how the technology might interpret moving bodies in the system under design. Visual representations of the machine input and interpretation can be constructed to assist in the mapping of human activity/movement in relation to machine behaviour.
The machine input schema was developed during these studies for that purpose. It visually diagrams the movements of users and the points or trajectories of input for interaction, based on the selected input mechanism, interpretation of the input and corresponding system response. An example of a machine input schema was given in Study 2, Figure 13. It functions as a bridging representation between the conceptual decisions regarding sensing and interpretation of the movements of people and the algorithms to process the movement-related input data. Just as importantly, it ties design decisions to user research and participation.
Conclusion
Our approach to the design of movement-based interactive technologies gives primacy to the first-person, felt experience of movement, where the body-in-motion and its felt, kinaesthetic experience are the generative source and medium for exploration of dynamic, qualitative concepts for design. The body is regarded as the ultimate test of successful engagement with interactive systems, products and spaces. The methods and tools we presented here can provide resources for exploring, generating and testing design concepts and prototypes, grounded in sensory movement experiences. Movements can be explored and documented from the three perspectives of the mover, the observer and the machine to allow movements to be transformed in a principled way to become input into sensing technologies. Each perspective offers orientation, guidance and tools at each stage of the design process. Our approach to design is a non-prescriptive, open process of inquiry and exploration instead of going straight to product.
The use of these methods and tools for working with movement can bring more aliveness, vitality and pleasure into the design process itself, as well as potential user experiences with interactive technologies. The challenge for designers is the incorporation of aesthetic and experiential qualities into our interactions with future technologies. Approaches drawing on dance and performance, such as ours, offer possibilities for opening up the landscape of the experiential, moving body in the design of new forms of movement-based interactive technologies.
Future work will continue the investigation into ways of designing for movement and felt experience in interaction design. New avenues of inquiry into other movement domains include somatics (e.g., Feldenkrais, Body-Mind Centering) and Eastern mind-body practices. These movement practices have a different set of values and conceptions of the body and aesthetic experience that may yield interesting and relevant insights and challenges for the design of interactive technologies.
References
- Blom, L. A., & Chaplin, L. T. (1988). The moment of movement: Dance improvisation. Pittsburgh, PA: University of Pittsburgh Press.
- Boehner, K., Sengers, P., & Warner, S. (2008). Interfaces with the ineffable: Meeting aesthetic experience on its own terms. ACM Transactions on Computer-Human Interaction, 15(3), 35-63.
- Brand, E., & Grunnet, C. (2000). Evoking the future: Drama and props in user centered deign. In Proceedings of the Conference on Participatory Design (pp. 11-20). New York: CPSR.
- Buur, J., Jensen, M. V., & Djajadiningrat, T. (2004). Hands-only scenarios and video action walls: Novel methods for tangible user interaction design. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 185-192). New York: ACM.
- Dewey, J. (1934). Art as experience. New York: Capricorn Books.
- Djajadiningrat, J. P., Matthews, B., & Stienstra, M. (2007). Easy doesn’t do it: Skill and expression in tangible aesthetics. Journal of Personal and Ubiquitous Computing, 11(8), 657-676.
- Ehn, P., & Sjögren, D. (1992). From system descriptions to scripts for action. In Design at work: Cooperative design of computer systems (pp. 241-268). Hillsdale, NJ: Lawrence Erlbaum Associates.
- Fiore, S., Wright, P., & Edwards, A. (2005). A pragmatist aesthetics approach to the design of a technological artefact. In Proceedings of the 4th Decennial Conference on Critical Computing: Between Sense And Sensibility (pp. 129-132). New York: ACM.
- Fraleigh, S. H. (1987). Dance and the lived body: A descriptive aesthetics. Pittsburgh, PA: University of Pittsburgh Press.
- Gaver, B., Dunne, T., & Pacenti, E. (1999). Cultural probes. Interactions, 6(1), 21-29.
- Geertz, C. (1973). The interpretation of cultures. New York: Basic Books.
- Henderson, K. (1991). Flexible sketches and inflexible data-bases: Visual communication, conscription devices, and boundary objects in design engineering. Science, Technology and Human Values, 16(4), 448-473.
- Höysniemi, J., & Hämäläinen, P. (2004). Describing children’s intuitive movements in a perceptive adventure game. In Proceedings of the Workshop on Multimodal Corpora (pp. 21-24). Paris: European Language Resources Association.
- Hummels, C., Overbeeke, C. J., & Klooster, S. (2007). Move to get moved: A search for methods, tools and knowledge to design for expressive and rich movement-based interaction.Personal and Ubiquitous Computing, 11(8), 677-690.
- Hutchinson, A. (1977). Labanotation or kinetography Laban: The system of analyzing and recording movement (3rd ed.). New York: Theatre Arts Books.
- Jacucci, C., Jacucci, G., Wagner, I., & Psik, T. (2005). A manifesto for the performative development of ubiquitous media. In Proceedings of the 4th Decennial Conference on Critical Computing: Between sense and sensibility (pp. 19-28). New York: ACM.
- Jensen, M. V. (2007). A physical approach to tangible interaction design. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (pp. 241-244). New York: ACM.
- Jensen, M. V., & Stienstra, M. (2007). Making sense: Interactive sculptures as tangible design material. In Proceedings of the 2007 International Conference on Designing Pleasurable Products and Interfaces (pp. 255-269). New York: ACM.
- Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. Journal of the Learning Sciences, 4(1), 39-103.
- Klooster, S., & Overbeeke, C. J. (2005). Designing products as an integral part of choreography of interaction: The product’s form as an integral part of movement. In Proceedings of the 1st International Workshop on Design and Semantics of Form and Movement (pp. 23-35). Newcastle, UK: Northumbria University.
- Kyng, M. (1995). Creating contexts for design. In J. M. Carroll (Ed.), Scenario-based design: Envisioning work and technology in system development (pp. 85-107). New York: John Wiley & Sons.
- Laban, R. (1971). The mastery of movement (3rd ed.). Boston: Play.
- Larssen, A. T., Robertson, T., & Edwards, J. (2007). Experiential bodily knowing as a design (sens)-ability in interaction design. In Proceedings of the 3rd European Conference on Design and Semantics of Form and Movement (pp. 117-126). Newcastle, UK: Northumbria University.
- Lim, Y. K., Stolterman, E., Jung, H., & Donaldson, J. (2007). Interaction gestalt and the design of aesthetic interactions. In Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces (pp. 239-254). New York: ACM.
- Löwgren, J. (2007). Fluency as an experiential quality in augmented spaces. International Journal of Design, 1(3), 1-10.
- Loke, L., Larssen, A. T., & Robertson, T. (2005). Labanotation for design of movement-based interaction. In Proceedings of the 2nd Australasian Conference on Interactive Entertainment (pp. 113-120). New York: ACM.
- Loke, L., & Robertson, T. (2007). Making strange with the falling body in interactive technology design. In Proceedings of the 3rd European Conference on Design and Semantics of Form and Movement (pp. 164-175). Newcastle, UK: Northumbria University.
- Loke L., & Robertson T. (2008). Inventing and devising movement in the design of movement-based interactive systems. In Proceedings of 20th Australasian Conference on Computer-Human Interaction (pp. 81-88). New York: ACM.
- Loke, L., & Robertson, T. (2009). Design representations of moving bodies in interactive, immersive spaces. International Journal of Human-Computer Studies, 67(4), 394-410.
- McCarthy, P., & Wright, P. (2004). Technology as experience. London: MIT Press.
- McCarthy, J., & Wright, P. (2005). Putting felt-life at the centre of human-computer interaction. Cognition Technology & Work, 7(4), 262-271.
- Moen, J. (2007). From hand-held to body-worn: Embodied experiences of the design and use of a wearable movement-based interaction concept. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (pp. 251-258). New York: ACM.
- Petersen, M. G., Iversen, O. S., Krogh, P. G., & Ludvigsen, M. (2004). Aesthetic interaction: A pragmatist’s aesthetics of interactive systems. In Proceedings of the 5th Conference on Designing Interactive Systems (pp. 269-276). New York: ACM.
- Schiphorst, T., & Andersen, K. (2004). Between bodies: Using experience modeling to create gestural protocols for physiological data transfer. In Proceedings of the CHI 2004 Fringe. New York: ACM. Retrieved January 10, 2010, from http://www.sfu.ca/~amb4/iat320/Week%202/Between_Bodies.pdf
- Schiphorst, T. (2007). Really, really small: The palpability of the invisible. In Proceedings of the 6th ACM SIGCHI Conference on Creativity and Cognition (pp. 7-16). New York: ACM.
- Sheets-Johnstone, M. (1999). The primacy of movement. Philadelphia, PA: John Benjamins.
- Shusterman, R. (2000). Pragmatist aesthetics: Living beauty, rethinking art. Lanham, MD: Rowman and Littlefield.
- Wright, P., Wallace, J., & McCarthy, J. (2008). Aesthetics and experience-centered design. ACM Transactions on Computer-Human Interaction, 15(4), 1-21.
