10-1543

Bauhaus Legacy in Research through Design:
The Case of Basic Sonic Interaction Design

Stefano Delle Monache and Davide Rocchesso*

Department of Architecture and Arts, Iuav University of Venice, Venice, Italy

Current trends in design research suggest that future interactive artefacts will return to communicating through their intrinsic physical and sensory characteristics, through their shape, inherent kinematics, tactile, and acoustic responses to manipulation. In sonic interaction design (SID), several disciplines are at work to shape the acoustic appearance of objects, to convey information through sound, and to enrich the experience of artefacts through sharable use. Basic research in the foundation of SID is aimed at developing an up-to-date curriculum, effective tools, and grounded practices. Inspired to the post-Bauhaus tradition and pedagogy, basic sonic interaction design traces systematic explorations in formgiving and formthinking issues of sounding objects. A research through design approach leverages the evidence, emerging from the development of exercises and progressive accumulation of physical sketches, to the status of design knowledge and theory.

Keywords – Basic Design, Research through Design, Sonic Interaction.

Relevance to Design Practice – Basic sonic interaction design contributes to ongoing discussion about research on the foundations of interaction design, with a specific focus on sonic interaction. The development and collection of basic exercises are means to distill scientific contributions to a designerly way of knowing.

Citation: Delle Monache, S., & Rocchesso, D. (2014). Bauhaus legacy in research through design: The case of basic sonic interaction design. International Journal of Design, 8(3), 139-154.

Received May 16, 2013; Accepted June 8, 2014; Published December 31, 2014.

Copyright: © 2014 Delle Monache & Rocchesso. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

*Corresponding Author: stefano.dellemonache@iuav.it.

Stefano Delle Monache is assistant professor at the Department of Architecture and Arts, Iuav University of Venice. He graduated in law in 2000, and in electronic music and audio technologies in 2008. In 2012 he earned the Ph.D. in Science of Design. His main research interests span design practices, methods and evaluation of sound in interaction, and interactive sound models for computational artefacts. He is currently involved in the EU project SkAT-VG (Sketching Audio Technologies using Vocalizations and Gestures, 2014-2016), aimed at exploiting and developing the use of voice as a sound sketching and prototyping tool in sound design practices. He has been board member of the Associazione di Informatica Musicale Italiana (AIMI—Italian Association of Music Informatics).

Davide Rocchesso received the Ph.D. degree from the University of Padova, Italy in 1996. He is associate professor at the Iuav University of Venice, Italy. He has been the coordinator of EU project SOb (the Sounding Object) and local coordinator of the EU project CLOSED (Closing the Loop Of Sound Evaluation and Design), and of the Coordination Action S2S^2 (Sound-to-Sense; Sense-to-Sound). He has been chairing the COST Action IC-0601 SID (Sonic Interaction Design), and is now also coordinating the EU project SkAT-VG (Sketching Audio Technologies using Vocalizations and Gestures).

Introduction

Before the advent of electronics the acoustic appearance and behaviour of artefacts were inherently linked to the physical properties and mechanical configuration of things (and their decay). The increasing availability of miniaturised microprocessors, sensors, and actuators makes it possible nowadays not only to shape permanently the sound quality of computational artefacts, but also to customise it according to use situation and personal moods. Sonic interaction design (SID) is positioned at the crossroad of human-computer interaction and interaction design. The object of this discipline is the study and exploitation of sound as one of the principal channels to convey information, meaning, aesthetic, and emotional qualities in interactive contexts. In this respect, the research community is strongly committed to 1) constructing solid foundations for the development of the design discipline, and 2) grounding the research activity in the design practice. Therefore, a major debate pertains to the methodology and practice of a research through sound design (RtD) and its outcomes in terms of theoretical contributions.

Basic research in SID is concerned with the foundational aspects of this novel discipline, that is the crafting and the designerly manipulation of the form and configuration of sounding objects. In this article we reflect on and reassess some basic research practices in which basic design, in the spirit of post-Bauhaus tradition, meshes with research through design of sonic interactions. Especially, we argue that basic design still represents a valuable approach to tackle the complexity of contemporary design research in the context of computational (sounding) artefacts. The peculiar characteristics of basic design (i.e., methodological, epistemological, ecological, interactive, and educational) intertwine with the ongoing, lively debate around the nature of (sonic) interaction design research and its conceptual and methodological standards.

Sonic Interaction Design

In the article, that appeared in the special issue of the Journal of New Music Research on The Future of Sound and Music Computing (SMC), Widmer et al. (2007) drew attention on a new area of research problems on sound-based interactive systems. This whole field of study, not previously addressed within the SMC community, was labeled as sound interaction design. Compared to the field of Auditory Display, which is more broadly concerned with the use of non-speech sound to present information, sound interaction design shifted the focus on the role of sound under the perspective of interaction, especially continuous and multisensory. The article insisted on the effort of “using sound in artificial environments in the same way that we use sound feedback to interact with our everyday environment,” thus emphasising the lack of evaluation methodologies for sound design, and of robust knowledge on everyday sound perception. Indeed, designing sound in interaction is not only concerned with displaying acts of use with an artefact, but also with recovering the synchronic and performative aspect of the design practice itself, whether the designer should produce the sound for a sporty electric car, the scratching sound for supporting stylus-based interaction with tablets, or the gait sonification for video-games or motor rehabilitation systems. In a few words, sound design practice and research were missing appropriate knowledge and theories on and for designing the acoustic behaviour of artefacts, despite of a widespread knowledge on modelling and generating sound and music through computational approaches.

In the scope of SMC, the topic of sound interaction design lately became a new field of research, and as a discipline it was re-labeled as Sonic Interaction Design (SID, 2007–2011, see http://sid.soundobject.org), thus stressing a stronger pertinence to the whole world of the audible and vibrations. From a taxonomical perspective, one could say that Sonic Interaction Design is to Sound and Music Computing and Auditory Display as Interaction Design is to Human-Computer Interaction. SID focuses on exploring new roles of sound as means to mediate the action-perception loop when performing actions on and through artefacts, being them art pieces, products, systems, or environments. The design challenge is shifted on how to exploit the expressive qualities of sound, either as display or input, to extract meaning from and respond to (everyday) physical activities. The purpose is to create meaningful, engaging, and aesthetically pleasing sonic interactions. In this sense, sound computing is not merely modelling nor generating sound, but affecting through design an overall shape aspect of things, that is their appearance, identity, and experience of use (see Brazil, 2009; Franinović & Serafin, 2013, pp. 39-76; Rocchesso & Serafin, 2009; for a comprehensive state of the art on SID, and Hermann, Hunt, & Neuhoff (2011) for an overview on the state of the art on sonification and connections with SID).

In our line of research, the design and assessment of sounding objects are approached from a perceptual perspective, that is sound in the action-perception loop is investigated through design practices. In this respect, the discourse around the form of sounding objects and our approach to basic design are introduced and contextualised in the following subsection.

Sounding Objects, Formgiving and Formthinking Issues

Artefacts whose computational materials (Vallgårda & Sokoler, 2010) are characterized by inherent acoustic features have been defined sounding objects (Rocchesso, 2004). The term originated in the context of human-computer interfaces, to denote appropriate digital sound models provided with dynamic, perceptually-relevant interactive control, as sonic counterparts of visual widgets (Rocchesso, Bresin, & Fernström, 2003). As user interfaces moved from screen-based metaphors to tangible interactions, the concept evolved to signify any kind of design in which the sonic interaction is situated, concrete, performative, and nonrepresentational (e.g., non symbolic), either as a display or as in input medium (Franinović & Salter, 2013, pp. 39-76). An exhibition on Sonic Interaction Design took place in 2011 at the Norwegian Museum of Science, Technology and Medicine, in Oslo(Behrendt & Lossius, 2011, see http://sid.bek.no/). The works exhibited represented an exemplar selection of sounding objects, in which sound is functional to active explorations.

The proper materiality of sounding objects raises general issues of formgiving and formthinking in both meaning of craft practice and design choices. In this respect, manifold lines of research are crystallising a grammar of basic elements and organisational principles around the form of interaction (Lim, Lee, & Kim, 2012), just as decades of explorations in visual form thinking led to the development of a visual literacy (Albers, 2006; Dondis, 1974). Indeed, the physicality emerging from the combination of computational elements and materials, being it wood, air, or liquids, is not only a matter of streamlining or styling. Computers need to be provided with perceptual and expressive capabilities and actuators/displays that manifest computed effects on the environment (Valgårda & Sokoler, 2010; Holman, Girouard, Benko, & Vertegaal, 2013). Starting from the assumption “function resides in the expression of things” (Hallnäs & Redström, 2002), the aesthetics and the expression of interaction are constantly redefined in terms of meaningful, foundational elements, linking form and function (Hallnäs, 2011).

Although form and configuration are extremely volatile concepts, they are part of the tacit knowledge embodied in design activities. Research in the foundations of design showed a renovated interest towards the Bauhaus experience and its legacy. Research efforts are addressed at emphasising the humanistic value of the Bauhaus experience, and re-contextualising it in the digital domain (Anceschi, 2006, pp. 57-67; Binder, Löwgren & Malmborg, 2009; Boucharenc, 2006; Findeli, 2001). Basic design, in particular, is the natural venue wherein teachers and students engage in a systematic investigation of the foundations of design and develop theoretical tools to handle materials that are apparently without qualities (Löwgren & Stolterman, 2004). Within these premises, explorations in basic SID tackle the form and expression of interaction from the sonic standpoint, and are concerned with the fundamentals of auditory perception. Our investigation in basic SID aims at distilling the peculiar contributions of prominent representatives of the Bauhaus tradition, in the light of the ongoing discussions on the foundations of interaction design practice. Since a survey of the Bauhaus’ history is out of the scope of this article (Simonini, 2006), the following section highlights the rich, dynamic contributions of well-known educators such as J. Itten, L. Moholy-Nagy, J. Albers, and T. Maldonado: Expression, improvisation (Itten), multisensory and proto-ecological approaches (Moholy-Nagy), perceptual understanding, rigour and intersubjectivity (Albers), and problem-solving and scientific attitude (Maldonado) are the key elements around which we structure our basic SID exercises (Franinović, 2008; Rocchesso, Polotti, & Delle Monache, 2009).

The Legacy of Basic Design

The original Bauhaus manifesto was aimed at building a design curriculum based on a synthesis of art, science, and technology. The education of future designers was primarily based on the distinction between Formlehre and Werklehre that is the study of form and crafting. The workshop represented the ideal setting to enable this learning. The fulcrum of the Bauhaus curriculum was the preliminary course, also known as Vorkurs, Grundkurs (as taught at school of Ulm), or Basic course (as renamed at the New Bauhaus in Chicago). The basic course was aimed at introducing students to the design problem of form. Students developed their perceptual-motor skills, manual modelling skills in manipulation of materials, and investigated the physical nature of materials and the basic laws of design. Basic design sets up an environment primarily devoted to research rather than creation. The teaching of basic design is condensed in exercises and problems, to be solved within the framework of specific constraints (e.g., economy of time and/or means, reduction of parameters). The main difference between the two categories of assignments consists in their settlement: Problems admit solutions, while exercises do not. Instead, exercises promote an experiential learning through exploration of wicked problems of form. Infinite, yet consistent variations are admitted. Basic SID promotes a holistic view of design. In basic SID exercises, the comprehension of the dynamic interplay between parts and wholes (e.g., sensing and actuating strategies, the gesture-sound loop, how the latter is affected by other senses, and the mediating role of physical objects) is methodologically grounded in aesthetics and practice.

Sensitising to Form: Johannes Itten

Johannes Itten’s approach to basic design was deliberatively expressionistic. Typically, the main goal of basic design is to develop the creative personality of students through sets of controlled exercises. The replica is meant to have a scientific value in making explicit and transmittable a knowledge otherwise secretly kept. Itten’s pedagogy was largely based on sensory stimulation. Breathing and relaxation exercises were instrumental to sensitise the receptiveness of students. Exercises were aimed at making students prepared-for-action. Exercises and design problems around colour, material, texture, and rhythm were largely based on his theory of contrasts (Itten, 1975). Forms and their variables were investigated by exploring the tension between their polar opposites (e.g., light/dark, soft/hard). Experimentation of form was carried out through playful improvisation, thus emphasising the value of a learning by doing approach. Along the same line, sound walks and blindfolded explorations of audiotactile interactions were exploited in SID educational research as means to introduce and sensitise apprentice designers to sonic interaction (Rocchesso, Serafin, & Rinott, 2013). Similarly, the expressionistic attitude à-la Itten was largely exploited in a set of basic exercises conceived to explore the principle of contradiction, in the design of continuous sonic interactions (Rocchesso, Polotti, & Delle Monache, 2009). The resulting sounding objects are parts of the Gamelunch installation (see Figure 1): Continuous interactions with graspable sensor-augmented bottles and cutlery (e.g., cutting, piercing, pouring, stirring) are sonified in order to contradict the gesture or the material being manipulated. The tension created by the sound feedback emphasises the importance of sound in everyday life gestures, and bodily awareness (Delle Monache, Polotti, & Rocchesso, 2013, pp. 225-233).

Figure 1. Visitors performing with sensorised tableware of the Gamelunch. The squeaking fork and knife, the braking jug, the liquid salad bowl, and the sandlike bowl either contradict, through sound, the percept of gesture or of the material manipulated. Continuous interaction and sound are brought to the foreground.

A Phenomenological Approach to Multisensory Interaction: Lázló Moholy-Nagy

Knowledge of materials and mastery of tools are distinctive of Moholy-Nagy’s pedagogy. Basic assignments had a dual purpose: either a specific plastic element (e.g., texture, motion, space, volume, density) was explored through different media and along different sensory channels (e.g., painting, drawing and photography–vision, assembly and sculpting–haptic, music–audition), or, conversely, the expressive potential of the various plastic elements were explored with only one medium at a time. In exercises such as the hand sculptures and the tactile charts1, multisensory qualities of the human experience were brought to the foreground, and space-time relationships were considered in their emotional aspects:

This experience of the visible relationships of position may be checked by movement—alteration of position—and by touch, and it may be verified by other senses. [...] It is possible to distinguish forms and space through hearing, too. (Moholy-Nagy, 1937, p. 25)

Basic explorations on the qualities of touch are aimed at extracting practical implications for the design of product surfaces, such as handles, steering wheels, and packages. Moholy-Nagy’s phenomenological investigation on the nature of design (Findeli, 1990) can be seen as a sort of predecessor of the later designerly contributions of the Gibsonian psychology, on the relevance of the perceptual-motor experience:

Vision in motion is simultaneous grasp - Simultaneous grasp is creative performance—seeing, feeling, and thinking in relationship and not as a series of isolated phenomena. It instantaneously integrates and transmutes single elements into coherent whole. This is valid for physical vision as well for the abstract. […]

Vision in motion also signifies planning, the projective dynamics of our visionary faculties. (Moholy-Nagy, 1969, p. 12)

Biotechnics was the scientific discipline addressed to study organic forms (Kiesler, 1939). Described as a method of creative activity, biotechnics strongly connects with the emerging field of design research on Organic User Interfaces (OUIs). In OUIs, the tight link between form and function is dynamically shaped by the reactive interplay between computational elements, basic forms, and materiality (Bongers, 2013). In current design research and education, basic exercises in continuous interaction (Rocchesso, Polotti, & Delle Monache, 2009) exploit body movement, including touch, as a main tool to investigate multisensory qualities of material properties and physical shapes (Spence & Gallace, 2011), to improve perceptual-motor skills, and design rich interactions (Djajadiningrat, et al., 2004).

Rigorous Design Research: Josef Albers

The work of Josef Albers represents a milestone in the framework of basic design as it is taught today in design schools. His contribution is especially linked to the refinement of the basic practices and to his studies on the inherent deceptive, unstable nature of colour perception, condensed in the well-known Homage to the Square series. His specific focus on interaction of colours is a designerly systematisation of a vast part of Gestalt research on figure-ground phenomena in visual perception (Albers, 2006). His pedagogical motto to open eyes emphasised the need to mitigate and even discard the influence of cognitive heuristics and confirmation biases in design cognition (i.e., the tendency to interpret evidence in order to confirm pre-existing beliefs) (Hallihan, Cheong, & Shu, 2012).

Perceptual understanding is at the centre of Albers’ teaching. The systemic coherence and the increasing complexity of the exercises are remarkable (Kelly, 2000), as in the exercise “1 colour appears as 2—looking like the reversed grounds” (Albers, 2006, p. 18) (and the ascending and descending variants “3 as 4” and “3 as 2”). Albers’ assignments mainly fall in the category of exercises rather than problems (i.e., explorations with no unique solutions, instead with potentially infinite variations). Assignments are introduced with demonstrations which constitute target examples. Exercises are accurately formulated in terms of criteria and objectives (i.e., target perceptual effects), yet without rejecting the use of narrative language. A trial and error approach is exploited to enable decision-making and foster experiential learning. (Self-)evaluation skills are improved through an iterative process of judgement and refinement. At a design stage where implications and hypotheses are hardly verbalised, models and sketches constitute implicit arguments of the designer’s current understanding of the phenomena and relationships under investigation. On this standpoint, objectivity, in a designerly acceptation, is a central aspect of the innovative contribution of Albers to design teaching. His approach represents a synthesis of the scientific instances of generalisability, repeatability, and transmission, though within a designerly way of knowing. Hypotheses and theory are embodied in the text of exercises, and exposed to falsifiability, through the execution of the assignment. In turn, the collections of resulting artefacts represent arguments in support of the thesis and serve as tools of theoretical refinement. A phenomenological approach based on intersubjectivity endorses objective evaluation (Bozzi, 1978; Vicario, 1993).

Finally, Albers’ pedagogy payed a great attention to the role of tools. Ideal tools should not divert students from the core of the learning objective. This means that if the purpose was to learn about colour, then the student should not cope with problems connected to tools (e.g., brushes, pigments), and use colour paper instead. Constrained, yet purpose-oriented tools have the major quality of awakening latent sensitivities.

Problem-solving Embodied: Tomás Maldonado

Tomás Maldonado is the fourth radical representative of the post-Bauhaus schools. His contribution is mainly framed in the experience of the School of Ulm—Hochschule für Gestaltung (HfG Ulm). As professor and chancellor, he introduced wide-ranging changes to the Bauhaus curriculum towards a science-centred, vocational approach. Several new disciplines (e.g., cybernetics, theory of information, systems theory, semiotics, ergonomics, etc.) were introduced, with the aim of bringing a solid methodological foundation to design thinking and action. The basic course was also involved in a profound transformation. The learning by doing based on playful and free improvisation is constrained into disciplined, brief-oriented learning activities. The paradigm of the exercise is replaced with the model of the design problem, with well-defined objectives and constraints. The exploratory approach is replaced by the analytic and synthetic model of problem-solving (Anceschi, 2006, pp. 57-67). The assignments become extremely detailed and are conceived as pure abstractions of real situations. The ultimate goal is to provide students with strong, critical and methodological abilities that they could more easily apply in the “real” design practice.

The Antiprimadonna, literally anti-queen bee, is a famous basic design exercise conceived by Maldonado in the early 1960s: The visual exercise challenges the formal organisation of seven vertical bands of variable width and colour, in such a way that none of them plays the role of the prima donna. The assignment exposes designers to experimenting with perceptual hierarchies in visual pattern design, with the aim of developing compositional skills and mastering the emergence of hierarchies in a controlled way. An analogous basic SID exercise tackles the non-hierarchical arrangements of sonic patterns. This design problem was conceived as an abstraction of auditory displays capable to leverage the attention and create awareness of the surroundings. The acoustic Antiprimadonna envisages the organisation of a soundscape of five elementary sonic interactions (e.g., impacts, frictions, and liquid sounds) where none of them stands out.

This exercise represents an investigation in the auditory phenomena of figure-ground segregation (Winkler, Denham, & Nelken, 2009). Figure 2 shows a GUI of Antiprimadonna: The congruence of synthesised sound events (Leech, Gygi, Aydelott, & Dick, 2009) and the manipulation of the structural and transformational invariants of sonic interactions (Warren & Verbrugge, 1984) were exploited as means to specify expectations in the listeners, that is affecting through design the priming of sound stimuli from periphery to the centre of attention (Bakker, van den Hoven, & Eggen, 2012). This exercise was proposed in several educational contexts, and it is remarkable how students got different yet interesting and balanced soundscapes.

Figure 2. Example of GUI with five sound panels, realised for the acoustic Antiprimadonna. Digital sound models of impact, friction, and population of bubbles are exploited to synthesise a well-balanced soundscape of five interactions.

All these contributions form the landmarks around which we are consolidating a research through design activity that can be renamed as basic sonic interaction design. In basic SID, explorations in the foundation of interaction design mesh with explorations in auditory perception in interaction. The next section is organised in four sub-sections, that: i) stress how explorations in basic interaction design complement with the research and development of ecologically-founded sound synthesis algorithms; ii) illustrate the enactive approach of basic SID, by grounding the arguments in the description of specific exercises; iii) draw attention on the significance of appropriate software environments and raw computational materials as means to experiment sound design solutions; iv) show the methodological implications of basic SID, in terms of research through design outcomes.

Basic Sonic Interaction Design

Basic SID can be defined as a practice focused on understanding through designing the formal, relational properties of sounding objects (Franinović & Visell 2008; Rocchesso, Rinott, & Serafin, 2013, pp.125-150). Basic SID is centred on human perception and action that exploits a logic based on aesthetics, yet provides a well-established approach which combines educational research with theoretical and methodological foundations of design.

Point, Line, Surface Revisited

A comparative reading of Hallnäs (2011), Lim, Stolterman, Jung, and Donaldoson (2007), and Valgårda and Sokoler (2010) provides clues of the basic properties of interaction, its formal elements, and principles of organisation. Furthermore, a reference to the Bauhaus tradition of basic design, as especially taught by Itten, is almost explicit in these studies. In Hallnäs (2011), the timing of using a thing (i.e., the rhythm and the metrics), its space, the connectivity, and the methodology which link function and interaction in acts of use (e.g., an augmented or automated interface, the set of operations which compose the use of a thing) are proposed as basic design dimensions of the interaction form, just as point, line, texture, or colour are basic elements of 2D form. As these visual elements can be organised according to principles of scale, symmetry, movement, and so forth, similarly the interaction form manifests in phenomenal shapes (i.e., interaction gestalts) that can be described and organised according to a set of basic attributes (e.g., speed, continuity, concurrency). Here, the research on interactivity attributes by Lim, Lee, and Kim (2012) is what best approximates the basic design approach: in several workshops, students were asked to design an interactive artefact, some teams by exploiting a given set of attributes describing the spatio-temporal shape of interaction, some others by designing as they normally would. The authors report a significant better quality of the outcomes produced by those teams that were previously sensitised and introduced to interactivity attributes. The basic dimensions of interaction form were approached by Valgårda and Sokoler (2010) from a material properties perspective: the timing-spacing pair manifested in the physical expression of computational materials of exhibiting reversible and accumulative changes, whereas connections between function and interaction were reflected in the property of computed causality. As an example, research on paper computing focuses on the use of shape memory alloys (SMA) to memorise and control dynamically the shape of paper characters and mechanisms (Qi, & Buechley, 2010).

The focus of Basic SID exercises is on auditory perception in object manipulation. Typically, dimensions and attributes of interaction are coupled to the dynamical properties of some (digital) sound models: for example, in one proposed exercise the rhythmic and cyclic shape of slicing vegetables on a chopping board is explored in combination with several rhythmic sound feedbacks and strategies. In another exercise, the continuous action of screwing a Moka coffee machine is investigated along three discrete stages of tightness of connection (i.e., low, ok, too high), coupled to the spectro-temporal evolution of a friction sound model (e.g., glass harmonica–low connection, rubbing sound–tight coupling, squeaking sound–too tight stage). The in-depth discussion of these exercises in continuous and multisensory interaction can be found in our previous work (Rocchesso, Polotti, & Delle Monache, 2009). Systematic perceptual training through hands-on activity is aimed at improving perceptual discrimination and enabling the ability to perceive the physical, relational properties of events. Basic SID exercises delve into those auditory invariants encoded in the environment (i.e., the artefact), in order to achieve varying, yet consistent percepts and facilitate or affect, through design choices, the occurrence of behaviours. The final goal is to develop cognitive abilities and compositional skills in incorporating perceptual factors in works and recognise them in the work of others.

A Procedural Sound Approach to Basic SID

A procedural audio approach to the basic design of sonic interaction is complementary to the ones discussed in the previous subsection: the interactive sound feedback is generated starting from the computed description of the characteristics of the sound producing event (i.e., the sounding object), according to a perceptually-relevant set of rules and control logics applied to live input (Farnell, 2011, pp. 313-339; Hermann, 2011, pp. 399-427). In procedural sound, synthesis parameters ideally coincide with the parameters describing the underlying physical process. The pressure signal is no longer seen only as variations of frequency and amplitude over time, instead it is the acoustic, causal result of specific interactions, or in other words behaviours. That sound is the sound of that action, the sound of rhythmic slicing, the sound of screwing two parts together, and the peculiar sound of touching a physical shape characterised by specific formal features. Therefore, synthesis algorithms of sounding objects potentially embody behaviours prior than a specific sound, that is for example hitting, breaking, bouncing, scraping, walking, or any other combination of sound producing (inter)actions. The synthesised sound embodies audible affordances, and as such provides information about the interaction with the virtual environment. It is not by chance that the resulting digital sound models are often identified by the name of the action which normally produces that sound. This is the inherent meaning of the common quotation sound affords action. In this respect, a procedural approach to sound design aims at favouring a realism-in-depth of the interactive experience, according to the ecological perception of the world (Chemero, 2003; Vicario, 2003, pp.17-31). As a consequence, it is straightforward to cross the sound synthesis parameters with the spatio-temporal shape of interaction and its attributes. This shift in thinking is remarkable and extremely relevant not only for the practice, but also for the development of sound tools tailored to the design activity (see further, subsection A toolkit for exploration in basic SID).

Learning by Doing and Inter-observation in Basic SID Exercises

Franinović, Visell, and Hug (2007) explored the feasibility of a basic approach to SID in several workshop settings. Specific exercises, such as ear-cleaning, acousmatic explorations of everyday contexts, Foley-oriented physical sound synthesis, and design methods, such as speed-dating and body-storming, were repositioned and bent in the spirit of basic design. Recent educational research efforts investigated a structured process of research through sound design, based on incremental functional-aesthetic and phenomenological assessment of sonic sketches, demonstrations and prototypes (Delle Monache, & Rocchesso, 2010).

Usually, preparatory exercises do not make use of any software-based tool, except for audio/video recording and playback for analytical purposes. Analytical skills are developed through assignments that may focus on either simple, immediate descriptions of sounds detached from their source, or on guided descriptions of sound quality in interaction, or more complex analyses of sonic interactions in context. Foley-oriented, sound synthesis assignments may require to create multi-layered sounds and scenes (e.g., producing the sound of fire crackling) or to sonify specific interactions with a given artefact (e.g., producing the sonic display of a cooker). The latter assignments facilitate the enactive discovery of the relationships between sound, interaction with and between materials, and gestures. In addition, bodily awareness and performativity are improved.

A Synthesis-through-analysis Basic Exercise

Synthesis-through-analysis exercises are aimed at opening the ears. Sensitisation to sound in interaction is achieved through playful improvisation. One proposed exercise fosters the investigation and reflection on the informative, yet deceptive nature of sound, at the same time introducing a discourse on the designerly role of procedural sound tools (Farnell, 2010). This exercise is generally carried out as group activity in order to enable shared doing and discussion.

Setup: ordinary objects such as clips, strings, pens, marbles, boxes of various shapes and materials, elastic bands, brushes of various type, tape, pipes and others are arranged according to their interaction characteristics (Gaver, 1993), and presented on a table placed in front of the audience. A collection of intuitive, immediate to catch, synthetic sounds, possibly referable to interactions with the objects available is prepared beforehand (e.g., the sound of a marble rolling in a metal box).

Procedure: In the first half of the exercise, a recognition task of the synthetic sonic processes (e.g., crumpling, rolling bodies, impacts, types of friction) is introduced and paced as group discussion. Before the synthetic quality of the sound samples is revealed, a sound making task is assigned. A quick demonstration is shown in order to encourage participants to reproduce, in a few minutes, the target sounds with the objects available on the table. After some trials, the synthetic nature of the sounds is revealed, and a discourse around procedural approaches to sound design (Farnell, 2010; Delle Monache, Polotti, & Rocchesso, 2010) introduces the exercise to its second part. The next assignment concerns the parametric manipulation of several properties of virtual sounding objects (e.g., shape, size, materials, and types of interactions of the digital sound models) and makes use of the procedural sound tool, previously used to produce the samples in the recognition task. The task concerns the manipulation of the synthetic sounds in order to provide target sensations of the virtual object or process (e.g., lighter/heavier, smaller/bigger, slower/faster, other, and their combination). Synthesis exercises envisage simple manipulation of a few variables as well as complex conditioning of sensors-captured control signals. Sonic sketches are assessed in group discussion.

The exercise stresses the informative potential of sound. The concepts of affordance, structural and transformational invariants, as well as the subjective discrimination abilities are internalised through doing. The sound making assignment reinforces the insights and fosters awareness of one’s perceptual-motor skills. Participants understand the perceptual effect that physical dimensions (e.g., mass, force) and properties (e.g., shape, size) have on sounding objects. They learn the basics of some acoustic phenomena like impacts and frictions and temporally-patterned sound events (e.g., bouncing, breaking, crumpling events), and focus on the tight coupling of the sound-action loop and its expressive potential. This exercise is preparatory to approach digital sound models when designing sonic interactions.

In a similar way, a basic approach was exploited to enable the exploration of paper-driven sonic narratives in a workshop setting (Delle Monache, Rocchesso, Qi, Buechley, De Götzen, & Cestaro, 2012). The introduction to basic paper engineering and paper computing techniques was functional to embed sound synthesis in simple popables, and provide them with expressive sonic interactions. Basic assignments required the coupling of a basic interaction, such as pulling a tab, turning a flap, pushing, and sliding a character, with a specific sound model, in order to facilitate quick sketches of sounding pop-ups (see Figure 3).

Figure 3. Example of sonic interactive pop-up on paper. The continuous action of pulling the tab is coupled to the displacement of the flock of birds from the tree to the clouds, and is augmented with a crackling sound of the tree branches.

The workshop experience generated useful reflections on how procedural sound computing on paper may exploit tangible, movable interfaces as significant tools for human-computer interaction designs2.

A Toolkit for Basic SID Explorations

Tools represent the counterpart of a formgiving and formthinking approach to interaction design. Moholy-Nagy and Albers strongly insisted on keeping the knowledge of material and its formal variables strictly separated from the tools used to operate on it. In Moholy-Nagy, the goal was to put apprentices in the position of better comprehending the aesthetic quality of plastic elements and the technological implications derived by mastery of tools. For Albers, tools had to be as neutral as possible and heavily constrained in order to avoid interferences in matter investigation, and to make the design process as objective as possible (i.e., repeatable through continuous rehearsal). Frustration derived by the how-much-to-how-much problem was found to be an effective way to focus on perceptual understanding and mastery of colour papers.

Current approaches to procedural sound synthesis are split in two main categories (Farnell, 2010; Rocchesso, 2004): a) signal-based models aimed at reproducing specific perceptual effects independently from the source (e.g., rain, fire, walking), and b) physics-based models wherein the generated sound is the resultant of computed interactions between virtual objects (e.g., impacts, frictions, etc.). The parametric control of signal-based models is often non-intuitive due to the complex control layers needed to operate the synthesis engines. On the contrary, in physics-based, or physically informed models, the synthetic sound feedback has an intrinsically natural behaviour, since it is energetically consistent with the action performed. Sound is described in terms of configurations, materials, geometries, and interacting forces. Major bottlenecks are generally represented by relatively high computational costs and little familiarity with exotic physical parameters (e.g., Stribeck velocity, Reynolds number etc.).

The Sound Design Toolkit (SDT) can be framed within these premises. The SDT is a publicly available software package, providing a set of physics-based models for interactive sound synthesis. The palette includes several families of sound models such as contact phenomena between solids (impact, friction, rolling, crumpling, bouncing, breaking), and liquid-related events and processes such as bubbles, dripping, burbling, pouring, and splashing (Delle Monache, Polotti, & Rocchesso, 2010). The SDT is developed as Pure Data and Max/MSP externals and patches3 and leans on the contribution of two major EU projects. In the SOb project (Sounding Object, 2001–2003), the first version of the library of physics-based sound models was developed and demonstrated in tasks of human-object continuous interaction (Rocchesso, Bresin, & Fernström, 2003). In the project CLOSED (Closing the Loop of Sound Evaluation and Design, 2006–2009), further development of the SDT (sound models and GUIs) was instrumental to investigate a structured process of product sound design. The ongoing EU project SkAT-VG (Sketching Audio Technologies using Vocalizations and Gestures, 2014–2016) is using the SDT in the research and development of a tool for supporting the sketching stage of the sound design process. The sound algorithms are developed according to three main points: i) auditory perceptual relevance; ii) cartoonification, i.e., simplification of the underlying physics and exaggeration of its most relevant aspects in order to increase both computational efficiency and perceptual clarity; iii) parametric temporal control ensuring appropriate, natural, and expressive articulations of sonic processes (Rocchesso, Bresin, & Fernström, 2003). The GUI architecture of the SDT, in both PD and Max/MSP environments, is designed in order to support i) a naïve physics approach to sound design (Smith & Casati, 1994); ii) a polyphonic allocation of sound models; and iii) an easy connectivity and interactive control with external devices. As an example, the screenshot in Figure 4 shows the palette with the currently available sound models, and two instances of the splash model (Pure Data version). Each parameter is controllable with external devices (e.g., via MIDI, OSC, etc.). Control maps can be edited, saved, and recalled to rapidly compare a large number of drawn sketches. Configurations of parameters can be stored as presets and written on disk as text file. The SDT provides a designerly environment, computationally affordable for real-time applications on ordinary hardware, to facilitate the coupling of sound models with physical objects.

Figure 4 . Two instances of the splash model.
The upper right window shows an example of a control map that manages the temporal occurrence of splashing events in the upper sound model. The lower right window dynamically manipulates the size of the virtual bubble.

In the SDT, procedural sound design gets potentially closer to early Foley artistry. Sound designers are provided with a palette of virtual sounding objects that can be combined to create dynamic sound events.

Sound Design Implications and Reflections

Synthetic sound models, software, and algorithmic procedures are essential materials to work with when experimenting design solutions through physical, sonic interactive sketches. In the SDT, the procedural approach to sound design is combined with the purpose-oriented characteristics of the tool. First of all, where it is reasonable, parameters are displayed in a conventional range of 0-100 float units, in order to make the exploration more intuitive (e.g., for those parameters whose physical values are normally associated to huge numbers, and whose meaning is difficult to grasp).

Figure 5 shows the GUI of the impact model which implements a modular structure resonator-interactor-resonator, representing the interaction between two vibrating objects described by means of their resonating modes (i.e., their frequency, decay time, and gain) (Adrien, 1991). The red parameters describe the characteristics of the striker, the green parameters describe the quality of the contact, and the lower left box describes the modes of resonance of the struck object.

Figure 5 . SDT, impact model. The GUI depicts a general architecture shared by all the sound models.

Looking back at Albers’ experiences, we notice that colour and sound share the same common how-much-to-how-much problem. Adding how much stiffness to how much hammer mass to how much velocity to how much decay can become extremely frustrating, yet the more one advances the exploration the more the understanding of the perceptual contribution of the single parameters becomes clear. Setting the resonant modes for the spectral content of a glass sound may be straightforward by making a spectral analysis of a sound sample, and then extracting frequency, decay, and gain profiles. Nonetheless, as soon as one approaches the fine tuning of the glass sound quality, major frustrating difficulties may arise. By manipulating the decay time and the contact surface (i.e., the shape of the contact) parameters, a glass sound can easily move from a rounded crystal glass to a glass tumbler. If stiffness is also manipulated, the glass sound easily turns into a metal sound, which may emphasise a stronger pertinence to the striker (a metal spoon?) (Giordano, & McAdams, 2006). Similarly, changes in global frequency affect the perception of materials, for instance turning the glass into a wooden sound. Therefore, the models are extremely malleable, yet strongly constrained. In a complementary way, Foley practices and sound synthesis in the SDT foster the perceptual training of the whole action-sound loop. The whole learning process can be assimilated to Albers’ idea of automatic drawing. The models do not need to be programmed, but only acted on through a continuous rehearsal. The approach to sound modelling in the SDT reflects the same concern of use of colour paper as compared to the use of pigments. Research through basic design practices provides important feedback on the value and effectiveness of both the design process and the digital tools used.

Synthetic sound and physical computing will be key elements in product sound design. Notwithstanding, procedural sound design is hardly able to find the practitioners’ approval. On one hand, the current role of the sound designer is often confined to the role of sound selector, on the other hand, design-oriented tools and methodologies are strongly needed in sound creation practices. New sound tools should internalise proper design thinking, and their development should be grounded in design research and practice. Educational research is potentially ideal in combining the achievement of an up-to-date curriculum with the development of effective tools and approaches to design (Langeveld, van Egmond, Jansen, & Özcan, 2013).

Basic as Research through Design Paradigm

We believe that a renovated approach to basic design can contribute to the lively debate around the so-called research through design practices (RtD). The methodological discussion concerns the relations between science and design research (Koskinen, Zimmerman, Binder, Redström, & Wensveen, 2012; Stolterman, 2008), the ideal role of theory, and the development of conceptual and methodological standards that can produce rigorous design theory (Zimmerman, Stolterman, & Forlizzi, 2010). Forlizzi, Stoltermann, and Zimmerman (2009) contended the need of a different research approach to HCI that would “leverage the design process of repeated problem reframing as a method of scholarly enquiry” (p. 2894). We notice that this research attitude is inherent to the pedagogy of basic design and embodied in the practice of collecting and continuously rehearsing basic exercises.

Designing Hypotheses, Exploring Theories

One major concern shared by basic design and RtD is how emerging knowledge can lead to a theoretical advancement. Höök and Löwgren (2012) recently proposed the notion of strong concept. Strong concepts are abstract design elements, elicited from the specific use situation, and potentially relevant to a whole range of designs. They are generative and influential elements prone to foster theoretical reflection and academic articulation. According to Anceschi (2006, pp. 57-67), production of foundational theory in basic design is axiomatic in the way formal and expressive research meshes with design and teaching/learning activities. In basic SID, theoretical constructs are typically manifested and formalised through basic exercises, while raw models of experimental sonic interactive artefacts (i.e., sounding objects) constitute externalised knowledge. Figure 6 frames the structure process of basic exercises: a loop of reflective design practices aims at collecting data and at formatively evaluating early hypotheses.

Figure 6. The structured loop of basic SID exercises: the initial set of hypotheses is refined through continuous rehearsal of designs. Reflective design research and practices become means to distill theory in objectives and constraints of the basic exercise.

Emerging design elements are added, discarded, refined, and meshed in higher-level conjectures, i.e., basic assignments (Rocchesso, Polotti, & Delle Monache, 2009). Gaver (2012) and Bowers (2012) contended the generative and provisional aspects of RtD theory, communicated through annotated portfolios. Annotations form intermediate-level knowledge relevant to a family of designs (i.e., the portfolio). Like annotations, the texts of basic exercises are rhetorical devices that serve as strategical and tactical purpose of inquiry (Buchanan, 2001). Basic exercises represent synthetic, descriptive hypotheses that, through a process of constant re-assessment, have the potential to generate theoretical insights. Like in portfolios, basic assignments are organised in such a way to communicate the coherence of designs.

From the methodological viewpoint, basic SID and the RtD approaches are unified in the key role played by artefacts, sketches, and prototypes, as means of developing, articulating, and communicating design knowledge. Artefacts are indeed dynamic means of embodied design thinking, intentionally and implicitly set in the design rationale and through crafting (Buxton, 2007; Lim, Stolterman, & Tenenberg, 2008). The well-known Ishii’s glass bottles (Ishii & Ullmer, 1997) are usually mentioned as a historical exemplar of RtD: the technology offers the possibility to reposition the identity of computational materials and interactive products. Figure 7 shows some relevant RtD exemplars which moved the design reflection from the desktop metaphor of GUIs, through direct manipulation of musical information, to the physical, ecological manipulation of sonic information embodied in the interactive artefact4. Often quoted in the literature on SID, these examples contributed to nourish the term sounding object to the status of strong concept.

Figure 7. From left to right: (a) Ballancer; (b) Squeezables; (c) Reactable; (d) Pebblebox; (e) Audioshaker. Well-known exemplars of sonic interactive object that fostered the reflection around the relations between sound and interaction, and contributed to the formalisation of the term “sounding object” as strong concept. (see endnote 2 for descriptions and references).

Stolterman and Wiberg (2010) advocated a concept-driven research tightly coupled with hands-on design as means to challenge existing theories and new ideas. In the field of SID, the EU project CLOSED (Closing the Loop of Sound Evaluation and Design), whose theoretical goals are clearly stated in the acronym, successfully meshed RtD activities with basic design practices. Major outcomes of the Closed project (2006-2009) i) shed light on many issues related to perception and meaning of sound in interaction (Houix, Lemaitre, Misdariis, & Susini, 2007); ii) developed a set of ecologically-coherent synthetic sound models (Delle Monache et al., 2009) accessible via a software application, especially suitable for educational purposes and research through sound design (Delle Monache, Polotti, & Rocchesso, 2010); iii) contributed to a systematisation of the process and activities inherent to the sound design practice, by integrating basic design with situated methods (Visell, Franinović, & Scott, 2008). Among the other basic works, the Spinotron, shown in Figure 8, is one of the concept designs realised in the Closed project: this abstract physical object is a valuable example of experimental research on auditory perception in interaction that brings out human perceptual-motor capabilities by focusing the design on the sonic information embodied in the features of the artefact (Lemaitre et al., 2009). As experimental design, the Spinotron was used to evaluate how different strategies in sound design may affect the performance in simple tasks, such as pumping at a constant rate.

Figure 8 . The Spinotron. Left: the abstract final prototype, the pumping gestalt is coupled with a synthetic model of ratcheted wheel sound. Right: internal configuration of the device. Pumping at a constant rate is facilitated through sound.

Finally, Figure 9 shows a thorough map of design research strategies proposed by Frankel and Racine (2010).

Figure 9. A geography of design research, adaptation from Frankel and Racine (2010, Figure 1).
A basic design approach is concerned with the epistemology of SID research and its communication within a designerly way of knowing. As such, basic SID elements span along the whole arc between research about and through design.

Basic design as a practice is research-oriented, based on phenomenology and aesthetics, and yet naturally intertwined with the epistemology and foundations of design. The sphere of activity of basic SID can be located anywhere between basic, concept-driven research about design and applied research through design. In addition, it has been demonstrated that early and repeated exposure (after the prototyping step) to design examples improves the quality of creative work (e.g., as it happens in traditional basic design pedagogy) (Kulkarni, Dow, & Klemmer, 2012). Early exposure to examples work as a source of inspiration, provide a selection of existing solutions and constructs, and sets the abstract threshold of acceptance for a good quality composition (Bartneck, 2009).

Collecting and sharing basic SID exercises is essential to developing a literacy based on the accumulation of repertoires of sounding paradigmatic exemplars, and contributing to a shared and expressive language of sonic interaction design (Bardzell, Bolter, & Löwgren, 2010; Pauletto et al., 2011, pp. 59-65).

Conclusions

Future product designers will need a specific competence on interactive sound. If properly grounded in the design practice, SID research can strongly contribute to the development of theories on and for sound design and to the foundation of a reliable curriculum. In turn, it is likely that design outcomes generate new questions and topics, thus setting the agenda for future research aimed at advancing scientific knowledge of specific domain disciplines. As an example, the encouraging results of the use of voice as a designerly tool to produce fast and rough sonic sketches (Ekman & Rinott, 2010) raised several questions about how sound events are identified by humans and which may be the salient sonic characteristics involved in the identification. Recent studies in experimental psychology investigated the potential of vocal imitations as means to convey the basic acoustic characteristics of sound events (Lemaitre, Dessein, Susini, & Aura, 2011). In the larger scope of future sound design tools, computers could be trained to identify real time, voice-produced sound events, and synthesise them accordingly, with further possibility to sculpt and refine them through vocalisations and gestures. Within this framework, the basic design legacy represents a valuable, readily operational attitude towards research through design, focusing an approach that can be renamed as research through basic design. The exemplar value of basic design is found in the philological recovery of the art/science/technology unity, claimed in the original Bauhaus manifesto and in various ways carried out in the several experiences of its major representatives, especially Moholy-Nagy and Albers. The several pedagogical approaches based on economy of time and/or means, experimentation, and reduction of parameters represent effective strategies to tackle the complexity of sound design for interactive contexts. In addition, exploiting a basic design attitude prevents the proliferation of the umpteenth framework.

In sonic interaction, a research through basic design is concerned with i) the development of a reliable corpus of basic SID exercises; ii) the investigation of the basic design process, that is understanding and making progressively explicit the tacit knowledge involved in the structuring of concepts, i.e., theoretical outcomes. The entangled era of disappearing yet ubiquitous computers requires a strong aesthetic and technological understanding of computational materials and purpose-oriented tools, and Moholy-Nagy’s approach to experimentation still remains a guiding reference. Given the irreplaceable value of sketches and models as means to develop perceptual-motor skills, the palette of raw materials and tools to deal with in interactive contexts should necessarily include microprocessors, components, and appropriate software environments.

Basic SID exercises synthesise a range of explorations on auditory perception in interaction. In basic SID, the traditional Bauhaus distinction between Formlehre and Werklehre is mitigated by the use of ready-made artefacts, in order to exploit the rich information coming from everyday life situations. The objective knowledge, emerging from the exploratory activities on the form and expression of sonic interaction, nurtures the theoretical foundation of practice in sonic interaction design. Sonic interaction design leverages a design culture on the world of the audible and vibrations, and contributes to the global advancement of the science of design.

Acknowledgments

The authors are pursuing this research as part of the project SkAT-VG and acknowledge the financial support of the Future and Emerging Technologies (FET) programme within the Seventh Framework Programme for Research of the European Commission, under FET-Open grant number: 618067.

Endnotes

  1. 1. See for instance http://bauhaus-online.de/en/atlas/werke/tactile-board, a well-know example of tactile board, by Otti Berger, from the preliminary course under L. Moholy-Nagy in 1928.
  2. 2. An advanced prototype of sonically augmented popable can be watched at http://vimeo.com/36679365.
  3. 3. http://soundobject.org/SDT/. For a thorough description of the sound models and the GUI’s architecture we refer to our previous publications [Delle Monache, et al., 2009; Delle Monache, Polotti, & Rocchesso, 2010], and a video tutorial that can be watched at: http://vimeo.com/album/2105400.
  4. 4. A) The Ballancer, an experimental tangible interface which exploits the metaphor of balancing a ball along a tiltable track to perform a variety of continuous control tasks, (Rath, & Rocchesso, 2005) B) Squeezables which allow manipulation of musical information based on physical efforts (Weinberg, 2002), C) Reactable, a collaborative tabletop TUI for musical purposes (Jordà, Geiger, Kaltenbrunner, & Alonso, 2007), D) Pebblebox, a grains-based tactile interface for granular sound synthesis (O’Modhrain, & Essl, 2004), E) Audioshaker (Hauenstein, Jenkins, 2004), an ecological tangible interface for direct manipulation of sonic information.

References

  1. Albers, J. (1963). Interaction of color (1st. ed.). New Haven, CT: Yale University Press.
  2. Adrien, J. M. (1991). The missing link: Modal synthesis. In G. De Poli, A. Piccialli, & C. Roads (Eds.), Representations of musical signals (pp. 269-298). Cambridge, MA: MIT Press.
  3. Anceschi, G. (2006). Basic design, fondamenta del design [Basic design, foundations of design]. In G. Anceschi, M. Botta, & M. A. Garito (Eds.), L’ambiente dell’apprendimento – Web design e processi cognitivi [Learning environment – Web design and cognitive processes] (pp. 57-67). Milan, IT: McGraw Hill.
  4. Bakker, S., Hoven, E. A. W. H. van den, & Eggen, J. H. (2012). Knowing by ear: Leveraging human attention abilities in interaction design. Journal on Multimodal User Interfaces, 5(3), 197-209.
  5. Bardzell, J., Bolter, J., & Löwgren, J. (2010). Interaction criticism: Three readings of an interaction design, and what they get us. Interactions, 17(2), 32-37.
  6. Bartneck, C (2009). Notes on design and science in the HCI community. Design Issues, 25(2), 46-61.
  7. Binder, T., Löwgren, J., & Malmborg, L. (Eds.) (2009). (Re)searching the digital Bauhaus. London, UK: Springer.
  8. Bongers, B. (2013). Anthropomorphic resonances: On the relationship between computer interfaces and the human form and motion. Interacting with Computers, 25(2), 117-132.
  9. Boucharenc, C. G. (2006). Research on basic design education: An international survey. International Journal of Technology & Design Education, 16(1), 1-30.
  10. Bowers, J. (2012). The logic of annotated portfolios: Communicating the value of ‘research through design’. In Proceedings of the 9th Conference On Designing Interactive Systems (pp. 68-77). New York, NY: ACM.
  11. Bozzi, P. (1978). L’interosservazione come metodo per la fenomenologia sperimentale [Inter-observation as a method for experimental phenomenology]. Giornale Italiano di Psicologia [Italian Journal of Psychology], No.5, 229-239.
  12. Brazil, E. (2009). A review of methods and frameworks for sonic interaction design: Exploring existing approaches. In S. Ystad, M. Aramaki, R. Kronland-Martinet, & K. Jensen (Eds.), Auditory display - Lecture notes in computer science (No. 5954, pp. 41-67). Berlin, Germany: Springer.
  13. Buchanan, R. (2001). Design research and the new learning. Design Issues, 17(4), 3-23.
  14. Buxton, B. (2007). Sketching user experiences: Getting the design right and the right design. Amserdam, NL: Morgan Kaufmann.
  15. Chemero, A. (2003). An outline of a theory of affordances. Ecological Psychology, 15(2), 181-195.
  16. Delle Monache, S., Rocchesso, D., Qi, L., Buechley, L., De Götzen, A., & Cestaro, D. (2012). Paper mechanisms for sonic interaction. In S. N. Spenced (Ed.), Proceedings of the 6th International Conference on Tangible, Embedded, and Embodied Interaction (pp. 61-68). New York, NY: ACM.
  17. Delle Monache, S., & Rocchesso, D. (2010). Experiencing sonic interaction design: Product design activities at the SID summer school 2010. In A. Valle & S. Bassanese (Eds.), Proceedings of 18th Colloquio di Informatica Musicale [Colloquium on Music Informatics] (pp. 87-91). Venezia, IT: IUAV University of Venice.
  18. Delle Monache, S., Polotti, P., & Rocchesso, D. (2010). A toolkit for explorations in sonic interaction design. In K. Delsing & M. Liljedahl (Eds.), Proceedings of the 5th Audio Mostly Conference on Interaction with Sound. (pp. 1-7). New York, NY: ACM.
  19. Delle Monache, S., Polotti, P., & Rocchesso, D. (2013). The Gamelunch: Basic SID explorations of a dining scenario. In S. Serafin & K. Franinović (Eds.), Sonic interaction design: Fresh perspectives (pp. 225-233). Cambridge, MA: MIT press.
  20. Delle Monache, S., Devallez, D., Drioli, C., Fontana, F., Papetti, S., Polotti, P., & Rocchesso, D. (2009). Algorithms for ecologically-founded sound synthesis (deliverable 2.3 of project CLOSED). Retrieved July 30, 2014, from http://closed.ircam.fr/uploads/media/CLOSED_D2.3.pdf
  21. Dondis, D. A. (1974). A primer of visual literacy. Cambridge, MA: MIT press.
  22. Djajadiningrat, T., Wensveen, S., Frens, J., & Overbeeke, K. (2004). Tangible products: Redressing the balance between appearance and action. Personal and Ubiquitous Computing, 8(5), 294-309.
  23. Ekman, I., & Rinott, M. (2010). Using vocal sketching for designing sonic interactions. In K. Halskov & M. G. Petersen (Eds.), Proceedings of the 8th Conference on Designing Interactive Systems (pp. 123-131). New York, NY: ACM.
  24. Farnell, A. (2010). Designing sounds. Cambridge, MA: MIT Press.
  25. Farnell, A. (2011). Behaviour, structure and causality in procedural audio. In M. Grimshaw (Ed.), Game sound technology and player interaction concepts and developments (pp. 313-329). New York, NY: Information Science Reference.
  26. Findeli, A. (1990). Moholy-Nagy’s design pedagogy in Chicago (1937-46). Design Issues, 7(1), 4-19.
  27. Findeli, A. (2001). Rethinking design education for the 21st century: Theoretical, methodological, and ethical discussion. Design Issues, 17(1), 5-17.
  28. Forlizzi, J., Stolterman, E., & Zimmerman, J. (2009). From design research to theory: Evidence of a maturing field. In Proceedings of the 3rd IASDR Conference on Design Research (pp. 2889-2898). Seoul, Korea: Korean Society of Design Science.
  29. Franinović, K. (2008). Basic interaction design for sonic artefacts in everyday contexts. In Proceedings of the Swiss Design Network Symposium (pp. 95-112). Berne, Switzerland: Bern University of Applied Science.
  30. Franinović, K., & Serafin, S. (Eds.) (2012). Sonic interaction design: Fresh perspectives. Cambridge, Mass: MIT Press.
  31. Franinović, K., & Salter, C. (2013). Experience of sonic interaction. In S. Serafin & K. Franinovíc (Eds.), Sonic interaction design: Fresh perspectives (pp. 39-76). Cambridge, MA: MIT Press.
  32. Franinović, K., Visell, Y., & Hug, D. (2007). Sound embodied: A report on sonic interaction design for everyday objects in a workshop setting. In G. P. Scavone (Ed.), Proceedings of the 13th International Conference on Auditory Display (pp. 334-341). Montreal, Canada: Mc Gill University.
  33. Franinović, K., & Visell, Y. (2008). Strategies for sonic interaction design: From context to basic design. In P. Susini & O. Warusfel (Eds.), Proceedings of the 14th International Conference on Auditory Display. Paris, France: Institute de Recherche et Coordination Acoustique/Musique. Retrieved October 27, 2014, from http://icad08.ircam.fr/static/
  34. Frankel, L., & Racine, M. (2010). The complex field of research: For design, through design, and about design. In D. Durling, R. Bousbaci, L. -L. Chen, P. Gauthier, T. Poldma, S. Roworth-Strokes, & E. Stolterman (Eds), Proceedings of the Design Research Society (DRS) International Conference (No. 043). Montreal, Canada: Université de Montréal. Retrieved October 27, 2014, from http://www.designresearchsociety.org/joomla/proceedings/drs-2010.html
  35. Gaver, W. W. (1993). What in the world do we hear? An ecological approach to auditory event perception. Ecological Psychology, 5(1), 1-29.
  36. Gaver, W. (2012). What should we expect from research through design? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 937-946). New York, NY: ACM.
  37. Giordano, B. L., & McAdams, S. (2006). Material identification of real impact sounds: Effects of size variation in steel, glass, wood, and plexiglass plates. The Journal of the Acoustical Society of America, 119(2), 1171-1181.
  38. Hallihan, G., Cheong, H., & Shu, L. H. (2012) Confirmation and cognitive bias in design cognition. In Z. Siddique & R. Nagel (Eds.), Proceedings of 9th ASME Conference on Design Education (pp. 913-924). Chicago, IL: American Society of Mechanical Engineers.
  39. Hallnäs, L. (2011). On the foundations of interaction design aesthetics: Revisiting the notions of form and expression. International Journal of Design, 5(1), 73-84.
  40. Hallnäs, L., & Redström, J. (2002). From use to presence: On the expressions and aesthetics of everyday computational things. ACM Transactions on Computer-Human Interaction, 9(2), 106-124.
  41. Hauenstein M, & Jenkins, T. (2004). Audio shaker. Retrieved July 30, 2014, from http://nurons.net/audioshaker/about.htm
  42. Hermann, T. (2011). Model-based sonication. In T. Hermann, A. Hunt, & J. G. Neuhoff (Eds.), The sonication handbook. Berlin, Germany: Logos Publishing House.
  43. Hermann, T., Hunt, A., Neuhoff, J. G. (Eds.) (2011). The sonication handbook. Berlin, Germany: Logos Publishing House.
  44. Holman, D., Girouard, A., Benko, H., & Vertegaal, R. (2013). The design of organic user interfaces: Shape, sketching and hypercontext. Interacting with Computers, 25(2), 133-142.
  45. Höök, K., & Löwgren, J. (2012). Strong concepts: Intermediate-level knowledge in interaction design research. ACM Transactions on Computer-Human Interaction, 19(3), 23-41.
  46. Houix, O., Lemaitre, G., Misdariis, N., & Susini, P. (2007). Experimental classification of everyday sounds (deliverable 4.1 of project CLOSED). Retrieved July, 23, 2014, from http://closed.ircam.fr/uploads/media/closed_deliverable_4.1_part2_public.pdf
  47. Ishii, H., & Ullmer, B. (1997). Tangible bits: Towards seamless interfaces between people, bits and atoms. In S. Pemberton (Ed.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 234-241). New York, NY: ACM.
  48. Itten, J (1975). Design and form: The basic course at the Bauhaus. London, UK: John Wiley & Sons.
  49. Jordà, S., Geiger, G., Kaltenbrunner, M., & Alonso, M. (2007). The reacTable: Exploring the synergy between live music performance and tabletop tangible interfaces. In B. Ullmer & A. Schmidt (Eds.), Proceedings of the 1st International Conference on Tangible and Embedded Interaction (pp.139-146). New York, NY: ACM.
  50. Kelly, R. R. (2000). Recollections of Josef Albers. Design Issues, 2(16), 3-24.
  51. Kiesler, F. (1939). On correlation and biotechnique: A definition and test of a new approach to building design. The Architectural Record, 9, 60-75.
  52. Koskinen, I., Zimmerman, J., Binder, T., Redström, J., & Wensveen, S. (2011). Design research through practice: From the lab, field, and showroom. Waltham, MA: Morgan Kaufmann.
  53. Kulkarni, C., Dow, S. P., & Klemmer, S. R. (2012). Early and repeated exposure to examples improves creative work. In N. Miyake, D. Peebles, & R. P. Cooper (Eds.), In Proceedings of the 34th Conference of the Cognitive Science Society (pp. 635-640). Sapporo, Japan: Cognitive Science Society.
  54. Langeveld, L., van Egmond, R., Jansen, R., & Özcan, E. (2013). Product sound design: Intentional and consequential sounds. In D. A. Coelho (Ed.), Advances in Industrial Design Engineering. Rijeka, Croatia: InTech. Retrieved July 29, 2014, from http://www.intechopen.com/books/advances-in-industrial-design-engineering/product-sound-design-intentional-and-consequential-sounds
  55. Leech, R., Gygi, B., Aydelott, J., & Dick, F. (2009). Informational factors in identifying environmental sounds in natural auditory scenes. Journal of the Acoustical Society of America, 126(6), 3147-3155.
  56. Lemaitre, G., Dessein, A., Susini, P., & Aura, K (2011). Vocal imitations and the identification of sound events. Ecological Psychology, 23(4), 267-307.
  57. Lemaitre, G., Houix, O., Visell, Y., Franinović, K., Misdariis, N., & Susini, P. (2009). Toward the design and evaluation of continuous sound in tangible interfaces: The Spinotron. International Journal on Human-Computer Studies, 67(11), 976-993.
  58. Lim, Y. K., Stolterman, E., Jung, H., & Donaldson, J. (2007). Interaction gestalt and the design of aesthetic interactions. In I. Koskinen & T. Keinone (Eds.), Proceedings of the Conference on Designing Pleasurable Products and Interfaces (pp. 239-254). New York, NY: ACM.
  59. Lim, Y., Lee, S., & Kim, D. (2012). Interactivity attributes for expression-oriented interaction design. International Journal of Design, 5(3), 113-128.
  60. Lim, Y. K., Stolterman, E., & Tenenberg, J. (2008). The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas. ACM Transactions on Computer-Human Interaction, 15(2), 7-34.
  61. Löwgren, J., & Stolterman, E. A. (2004). Thoughtful interaction design: A design perspective on information technology. Cambridge, MA: MIT Press.
  62. Moholy-Nagy, L. (1937 December). The new Bauhaus and space relationship. American Architects and Architecture, 26, 23-28.
  63. Moholy-Nagy, L. (1947). Vision in motion. Chicago, IL: Paul Theobald.
  64. O’Modhrain, S., & Essl, G. (2004). PebbleBox and CrumbleBag: Tactile interfaces for granular synthesis. In M. J. Lyons (Ed.), Proceedings of the 4th International Conference on New Interfaces for Musical Expression (pp. 74-79). Singapore: National University of Singapore.
  65. Pauletto, S., Rinott, M., López, M., Kessous, L., Franinović, K., Drori, T., Costanza, E., Delle Monache, S., & Hug, D. (2011). Sketching and prototyping. In D. Rocchesso (Ed.), Exploration in sonic interaction design (pp. 59-65). Berlin, Germany: Logos Publishing House.
  66. Qi, J., & Buechley, L. (2010). Electronic popables: Exploring paper-based computing through an interactive pop-up book. In M. Coelho & J. Zigelbaum (Eds.), Proceedings of the 4th International Conference on Tangible, Embedded, and Embodied Interaction (pp. 121-128). New York, NY: ACM.
  67. Rath, M., & Rocchesso, D. (2005). Continuous sonic feedback from a rolling ball. IEEE MultiMedia, 12(2), 60-69.
  68. Rocchesso, D., Bresin, R., & Fernström, M. (2003). Sounding objects. IEEE MultiMedia, 10(2), 42-52.
  69. Rocchesso, D. (2004). Physically-based sounding objects, as we develop them today. Journal of New Music Research, 33(3), 305-313.
  70. Rocchesso, D., & Serafin, S. (2009). Sonic interaction design. International Journal of Human-Computer Studies, 67(11), 905-906.
  71. Rocchesso, D., Serafin, S., & Rinott, M. (2013). Pedagogical approaches and methods. In S. Serafin & K. Franinović (Eds.), Sonic interaction design (pp.125-150). Cambridge, MA: MIT Press.
  72. Rocchesso, D., Polotti, P., & Delle Monache, S. (2009). Designing continuous sonic interaction. International Journal of Design, 3(3), 13-25.
  73. Simonini, I. (2006). Storia del basic design [History of basic design]. In G. Anceschi, M. Botta, & M. A. Garito (Eds.), L’ambiente dell’apprendimento – Web design e processi cognitivi [Learning environment – Web design and cognitive processes] (pp. 69-88). Milan, IT: McGraw Hill.
  74. Smith, B., & Casati, R. (1994). Naïve physics: An essay in ontology. Philosophical Psychology, 7(2), 225-244.
  75. Spence, C., & Gallace, A. (2011). Multisensory design: Reaching out to touch the consumer. Psychology & Marketing, 28(3), 267-308.
  76. Stolterman, E. (2008). The nature of design practice and implications for interaction design research. International Journal of Design, 2(1), 55-65.
  77. Stolterman, E., & Wiberg, M. (2010). Concept-driven interaction design research. Human-Computer Interaction, 25(2), 95-118.
  78. Vallgårda, A., & Sokoler, T. A. (2010). Material strategy: Exploring material properties of computers. International Journal of Design, 4(3), 1-14.
  79. Vicario, G. B. (1993). On experimental phenomenology. In S. C. Masin (Ed.), Foundation of perceptual theory (pp.197-219). Amsterdam, the Netherlands: Elsevier Science.
  80. Vicario, G. B. (2003). Prolegomena to the perceptual study of sounds. In D. Rocchesso & F. Fontana (Eds.), The sounding object (pp.17-31). Firenze, Italy: Mondo Estremo.
  81. Behrendt, F., & Lossius, T. (Eds.) (2011). Sonic interaction design. Catalogue of an exhibition at Norwegian museum of science, technology and medicine. Bergen, Norway: Bergen Center for Electronic Arts.
  82. Visell, Y., Franinović, K., & Scott, J. (2008). Experimental sonic objects: Concepts, development and prototypes (deliverable of project CLOSED). Retrieved 30 July 2014, from http://closed.ircam.fr/uploads/media/closed_deliverable_3.2_public.pdf
  83. Warren, W. H., & Verbrugge, R. R. (1984). Auditory perception of breaking and bouncing events: A case study in ecological acoustics. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 704-712.
  84. Weinberg, G. (2002). Playpens, fireflies and squeezables: New musical instruments for bridging the thoughtful and the joyful. Leonardo Music Journal, 12, 43-51.
  85. Widmer, G., Rocchesso, D., Valimaki, V., Erkut, C., Gouyon, F., Pressnitzer, D., Penttinen, H., Polotti, P., & Volpe, G. (2007). Sound and music computing: Research trends and some key issues. Journal of New Music Research, 36(3), 169-184.
  86. Winkler, I., Denham, S. L., & Nelken, I. (2009). Modeling the auditory scene: Predictive regularity representations and perceptual objects. Trends in Cognitive Sciences, 13(12), 532-540.
  87. Zimmerman, J., Stolterman, E., & Forlizzi, J. (2010). An analysis and critique of research through design: Towards a formalization of a research approach. In O. W. Bertelsen, P. Krogh, K. Halkoy, & M. G. Petersen (Eds.), Proceedings of the 8th Conference on Designing Interactive Systems (pp. 310-319). New York, NY: ACM.