3_1515.html

Understanding Design for Dynamic and Diverse Use Situations

Mieke van der Bijl-Brouwer 1,* and Mascha van der Voort 2

1 Design Innovation Research Centre, University of Technology, Sydney, Sydney, Australia
2 Faculty of Engineering Technology, University of Twente, Enschede, the Netherlands

The design research community acknowledges that usability and user experience are largely influenced by user characteristics, goals and contexts of use. For industrially manufactured products, these use situations are often dynamic and diverse. However, little guidance can be found in literature on the incorporation of dynamic and diverse use situations (DDUS) into the design process. This paper explores this issue by means of an analysis of user centred design literature and an empirical study of design for DDUS in design practice. We retrospectively studied three projects in which a product with DDUS was designed. Based on this study, we identified different effective strategies executed by design practitioners to analyse DDUS. We also observed difficulties in designing for DDUS. These difficulties include the identification of relevant aspects within this broad spectrum of use situations and sharing knowledge of product use between team members. We conclude that there is a need for guidance in the creation of flexible frames of reference of product use that evolve with solutions in the design process, particularly for design projects that cannot rely on the reuse of such a frame of reference from similar previous projects.

Keywords – Context of Use, Design Process, Dynamic and Diverse Use Situations, Usability, User Experience, User-Centred Design.

Relevance to Design Practice – We provide a starting point to close the gap between prevailing user centred design methodology, which recommends consideration of the specification of users, goals and contexts without explaining how to do this, and design practice in which designers have to deal with the dynamics and diversity of situations in which products are used.

Citation: van der Bijl-Brouwer, M., & van der Voort, M. (2014). Understanding design for dynamic and diverse use situations. International Journal of Design, 8(2), 29-42.

Received April 15, 2013; Accepted March 14, 2014; Published August 31, 2014.

© 2014 van der Bijl-Brouwer & van der Voort. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

*Corresponding Author: mieke.vanderbijl-brouwer@uts.edu.au

Mieke van der Bijl-Brouwer is Senior Research Fellow at the Design Innovation Research Centre at the University of Technology, Sydney. Her research focuses on human-centred design processes and frame creation/design thinking in the context of non-traditional design domains such as public services and education. She gained an MSc degree in Industrial Design Engineering from Delft University of Technology and completed a PhD at the University of Twente in 2012 (both cum laude). Her doctoral research sought to understand the process of designing for dynamic and diverse use situations in the design industry, including the development and evaluation of guidelines to support designers in this design process.

Mascha van der Voort is Associate Professor at the Laboratory of Design, Production and Management of the University of Twente. She leads the research group on Use Anticipation in Product Design. This group aims at supporting designers in anticipating use within product design processes in order to improve user-product interaction. This aim is pursued by means of the development of new design approaches and tools. The developed design supports particularly focus on eliciting and sharing knowledge of product use by multi-disciplinary design teams as well as the exchange of this knowledge between designers and stakeholders, notably end-users. To this purpose, the developed approaches and tools are derived from methodologies such as scenario-based product design, user involvement and participatory design. Techniques including workshops, gaming and virtual reality are frequently part of these design supports. Concurrently, Mascha teaches subjects related to human factors, usability and research methods.

Introduction

When designing products for many different use situations, designers need insight into the variations between these situations and the differing requirements each situation imposes on a design. For example, when designing a compact photo camera, designers could observe cameras being used by students while taking pictures of themselves at a party or by skiers taking pictures of mountains in the cold. In the first situation, the camera needs to provide support in getting everyone in the picture. In the second situation, the camera needs to be controllable while wearing gloves and to include a display that allows reading in bright sunlight. A compact camera will be used in many different situations, which makes it very difficult to predict and analyse all these situations and integrate this knowledge into the design process.

This research explores the relationship between varying use situations and usability, and how designers deal or could deal with this relationship in their design process. Usability has long been recognized as an important design consideration. Great frustration arises when basic products such as doors, taps and light switches turn out not to be simple at all (Norman, 1998). Apart from preventing user frustration, usability has become established as an important issue with respect to the marketing and sale of products and is therefore of commercial value. For example, den Ouden (2006) showed that product returns are half of the time caused by non-technical failures that occur when a product does not satisfy customers’ expectations, often as a result of usability problems. As usability is about customer satisfaction, in the long run it affects repurchase intent and cross purchasing, product returns, demand on customer support and brand perception (van Kuijk, 2010).

The ISO 9241-11 (1998) standard provides guidance on the specification and evaluation of the usability of a product. It defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (p. 2). The word ‘specified’ in this definition can lead to the interpretation that usability can only be defined for a fixed use situation, for example, the usability of a compact camera for skiers wanting to take a picture of the mountains. Since industrially manufactured products are never used in one specific use situation, the usability of products will necessarily vary. Many researchers acknowledge the dependence of usability on its use situation and the variety of these situations. For example, Nielsen (1993) states that a designer needs to consider the entire spectrum of intended users and ensure that an interface is usable for as many users as possible. Maguire (2001) argues that it is incorrect to describe a product as ergonomic or usable without also describing the context of use, in other words, for whom the product is designed, what it will be used for and where it will be used. The dependency of usability on varying use situations also counts for ‘user experience’. For example, Buchenau and Suri (2000) mention that:

The experience of even simple artefacts does not exist in a vacuum but, rather, in dynamic relationship with other people, places and objects. Additionally, the quality of people’s experience changes over time as it is influenced by variations in these multiple contextual factors. (p. 424)

From the above, it can be concluded that users, goals and contexts often vary, and these varying use situations lead to variable levels of usability as well as user experience for certain products.

Dealing with the myriad of situations in which products are used is a difficult aspect of the design process. Norman (1986) explains this issue in commenting:

Designing a system that matches the user’s needs confronts the designer with a large number of issues. Not only do users differ in their knowledge, skills, and needs, but for even a single user the requirements for one stage of activity can conflict with the requirements for another. (p. 43)

We conclude that the difficulty of dealing with the relationship between usability and user experience and the dynamics and diversity of use situations is acknowledged in the field of design research. We find it surprising that despite this widespread recognition, guidance in dealing with dynamic and diverse use situations (DDUS) in the design process is hard to find. This research thus aims to further explore the process of design for DDUS, leading to the development of a support that could provide this guidance.

Before guidance can be developed, it is necessary to gain a better understanding of the nature of the problem of designing for DDUS in design practice. Enhancing understanding of the design activity itself before developing design support is increasingly promoted in the design research field (Blessing & Chakrabarti, 2009; Dorst, 2007; Stolterman, 2008). In this paper, we present both a literature study on designing for DDUS and an empirical study of how designers currently deal with DDUS in design practice. In the following, we firstly present a theoretical framework for the different activities of designing for DDUS. Then we present the results of the empirical study of these activities in design practice, alongside the results of a literature study of these same activities. We conclude with the identification of directions for the development of design guidance based on these insights.

Theoretical Framework of Design for DDUS

The main question this study answers is how designers currently deal with DDUS. Dealing with DDUS comprises two aspects: the kinds of solutions designers propose to accommodate products to DDUS and the design processes that underlie the generation of these solutions. In an earlier publication (van der Bijl-Brouwer & van der Voort, 2009), we investigated the first aspect and indicated different solution types proposed by designers for specific design problems, which included varying use situations. These solution types ranged from ‘one-size-fits-all’ solutions and adjustable features and accessories to ultimately deciding to design different versions of products for different target use situations (segmenting). Generating these solutions did not lead to any observable problems when designers had a clear insight into the dynamics and diversity of the use situation. We decided not to focus further on the generation of solutions in the development of a support for dealing with DDUS. This paper focuses instead on the second aspect, the design processes aimed at DDUS. Before defining research questions for the study on how designers attend to DDUS in the design process, we firstly present a theoretical framework that defines DDUS and outlines the different design activities within the design process and how they relate to DDUS.

Dynamic and Diverse Use Situations

Dynamic and diverse use situations are defined as the use situations of products that are used by varying users, with varying goals and/or in varying contexts of use. Dynamic use situations refer to the change of situations over time for one product. For example, one day you might use your car to drive to work to be on time for a meeting. The next day you might use it to transport groceries home from the supermarket. Diverse use situations refer to the change of situations through time and space for different copies of the same product. For example, someone else might possess the same type of car, but only use it for recreational purposes such as going on vacation. Use situations are defined as the user, the goal of the user and the context of use of a product (see Figure 1).

Figure 1. A use situation is defined as a user with a certain goal in a certain context of use.

When considering the use situation in relation to usability and user experience, the question is which characteristics or aspects of this use situation influence these interaction qualities. From a comparison of different studies and definitions, it can be concluded that influencing user characteristics include physical, cognitive and sensory characteristics, personality, knowledge, experience and bodily skills (Desmet & Hekkert, 2007; ISO 9241-11, 1998; Jordan, 1998; Kim & Christiaans, 2011; Nielsen, 2002; Shneiderman, 2000). Goals could be personal goals or practical goals (Cooper, 1999). Personal goals concern issues like having fun or not making mistakes; practical goals are those directly related to product use. Influencing aspects of the context of use include physical aspects such as light and noise, objects, social aspects, for example societal attitudes, the technical environment, for example network connectivity and information, and events (Maguire, 2001; Rosson & Carroll, 2002; Schilit, Adams, & Want, 1994; Shackel, 1984).

To be able to take a use situation into account in a user centred design process, the designer needs to know which aspects of targeted use situations influence the quality of the user-product interaction, in which way and which use issues are important. Use issues are here defined as specific qualities of usability or user experience resulting from a certain user-product interaction. This can be illustrated by the example of a bicycle. If a design project is aimed at designing a touring bicycle, the issues ‘comfort of body position’ and ‘visibility in traffic’ will probably be most important. If this design project is targeted at designing a racing bicycle, ‘efficiency with regard to speed’ will be more important. The first type of bike requires a solution that offers users their preferred comfortable body position; the use issue for the second type of bike requires a solution that offers the most aerodynamic body position. In the first case, the designer needs to know something about variation in preferences of users. In the second case, the designer needs knowledge about variations in expected air conditions and wind resistance of the user’s equipment. The relevance of use situation aspects thus depends on the use issues that define the success of the product.

One of the difficulties in identifying relevant use situation aspects is that their relevance with regard to use issues depends on the type of solution. This is consistent with the general notion that designers explore and define problem and solution together (Cross, 2007; Dorst & Cross, 2001; Lawson, 2006). As an example, consider the issues of the comfort and safety of the touring bicycle and using the brakes of a bicycle. If the designer chooses a solution in which the brakes are controlled by hand, the user aspects ‘hand power’ and some anthropometric hand data are relevant. However, if the designer chooses a solution in which the brakes are controlled by the feet, the hand aspects are no longer relevant. Instead, the aspects ‘experience with pedal brakes’ and ‘available learning time’ become relevant. Dourish (2004) discusses such interdependency, adding that contextuality is a relational property that holds between objects and activities, suggesting that use situation aspects cannot be investigated independently of solutions. For each product and interaction, other use situation aspects can be relevant. The interdependency of use situation aspects, use issues and solutions has a large influence on the required design activities for DDUS. This is discussed further in the results of our empirical study.

Activities in Designing for DDUS

This section further explores the activities of designing for DDUS by comparing the characteristics of designing for DDUS to design methodology. Traditional design models distinguish activities aimed at analysing the design problem, creating a frame of reference based on this analysis, creating solutions and evaluating those solutions with regard to the frame of reference. This is reflected in, for example, the basic design cycle of Roozenburg and Eekels (1995, see Figure 2) in which the frame of reference is represented as a number of criteria to which solutions are compared in evaluations. When designing for DDUS, we therefore distinguish similar activities. Firstly, ‘use situation analysis’ includes activities aimed at predicting the situations in which products will be used. To be able to use the results of this analysis, it is necessary to prioritize the most important aspects within these use situations. This corresponds to the ISO recommendation to give particular attention in the selection of contexts for usability evaluations to those attributes judged to have significant impact on the usability of a product (ISO 9241-11, 1998, p. 8). Secondly, we distinguish activities that make the results of this analysis explicit in a frame of reference. Thirdly, in solution generation designers create solutions for conflicting requirements from different use situations. Finally, evaluation includes activities aimed at anticipating use issues based on evaluating the interaction between solutions and the variety of use situations as represented in the frame of reference. Figure 3 indicates these elements. Based on this theoretical framework, we decided to firstly gain insight into if and how these activities are executed in design practice and what guidance exists in the literature in relation to these activities.

Figure 2. Modified representation of the basic design cycle of Roozenburg and Eekels (1995).
The frame of reference for evaluations is presented as a number of criteria.

Figure 3. The basic design activities in design for DDUS.
Use situation analysis can generate an explicit frame of reference for use evaluations.

Studying Design for DDUS in Design Practice

Objective

Based on the aforementioned theoretical framework, we set up an empirical study to analyse how designers currently deal with DDUS in the process of designing a particular product for a targeted set of use situations, such as the process of designing a touring bicycle. The study focused on usability rather than user experience, although the ‘satisfaction’ element of the usability definition is strongly related to user experience. The sub questions are:

  1. How are DDUS aspects analysed in design practice?
  2. How are DDUS aspects prioritized in design practice?
  3. How is the usability of solutions evaluated with regard to DDUS?
  4. How are use situations and use issues made explicit and communicated?

The question of how designers create solutions for DDUS was not part of this study, it being assumed that dealing with DDUS in solution generation does not essentially differ from other types of designing for broad and ‘ill-defined’ problems. We acknowledge that solution generation is an essential part of designing for DDUS. However, since the creation of solutions for DDUS does not differ from general approaches to creating design solutions, we decided that further analysis of this activity was outside the scope of this study.

Method

Since the research questions are open-ended, they require an exploratory research approach. A case study approach was found appropriate, because it allows the study of real-world contemporary events that do not require control over behavioural events (Yin, 2009, p. 8). The objects of these case studies are design projects. Since the studied design projects had a time span from one to five years, we decided to study the projects retrospectively.

Cases

We selected three cases for the retrospective study based on the following requirements:

  • They should concern design projects in which usability was considered an important issue.
  • The projects should be finished recently so interviewees could easily recall project issues.

Case A concerned the design of a wide-format printer developed by a multinational company that provides digital document management technology and services. Case B concerned the design and evaluation of the installation features of a home health monitoring system for the elderly developed by a multinational company. Case C concerned the design of a bicycle carrier by a product design and consultancy agency. A bicycle carrier is a product that can be placed on a car’s tow bar to transport one or two bicycles.

Data Gathering

For each project, two or three actors played an important role in the user research and design of the project. The respondents had considerable knowledge of the design decisions that were made with regard to product use. The following roles were distinguished: designers, usability experts and project managers. Designers could make decisions with regard to the product design and in all cases were more or less involved in testing the design. Usability experts conducted user research and evaluated the usability of the product, but only had an advisory role with regard to design changes. Project managers coordinated the project and set priorities.

The respondents were firstly introduced to the concept of DDUS. Then, in a group interview, they were asked to discuss and write down on a flip chart the use situation aspects that had played a role in their design project. Based on an analysis of this discussion, we created ‘use situation aspect cards’, each describing an aspect of the use situation, the information source for the aspect, how it influenced product use and the way it was dealt with in the design solution (see Figure 4). These use situation aspect cards were used in subsequent individual interviews with the participants to confirm the interpretation of the results. Furthermore, general questions were asked in the individual interviews with regard to the design process. The main topics discussed in the individual interview were the role of the respondent in the project and the role of usability, techniques that were applied to improve usability in this project and the use situation aspects, their information source and their variation.

Figure 4. Example of a use situation aspect card.

For each use situation aspect card, the relevance of the listed use situation aspect with regard to usability was briefly discussed. The respondents were then asked to select the most relevant cards with regard to usability (see Figure 5). These aspects were discussed in more detail including a confirmation of the information source, the variation of the aspect and the solution that was implemented to accommodate the product to the use situation aspect.

Figure 5. A participant in the retrospective study analysing and prioritising the use situation aspect cards.

Data Analysis

The group interviews were recorded on a digital voice recorder. The interview transcripts and the overview of use situation aspects created by respondents during the group interview were used to create the use situation aspect cards. The recordings of the individual interviews were also transcribed. Relevant sections of the transcripts were identified and assigned to a specific use situation aspect or a general process issue. For each use situation aspect, sheets were created in which all quotes relevant to that aspect from the different actors came together. In an iterative process, a general strategy with regard to use situation analysis, evaluation and communication for each use situation aspect was assigned based on these sheets. Similar strategies for the three projects were then clustered. From this clustering, more general principles and strategies with regard to designing for DDUS were formulated.

Results

The printer (case A) was developed for colour printing wide formats such as posters in a professional setting. Although it was a high-tech project with a focus on engineering, usability was considered important from the start of the project. The main usability issues were the learnability of the graphical user interface (GUI) and paper handling and cartridge handling. Physical effort and accessibility were important issues for the latter goals. The user-centred approach of this five-year project (> 100 man-years) included work place visits and iterative user testing from lab tests to full-scale 3-month in situ beta-tests.

The home health care portal for elderly people (case B) is a system that consists of one or more devices that measure a physiological parameter such as blood pressure connected to the user’s own television, which displays the graphical user interface. The project was a subproject of one year (1.5 man-years) in which the installation process was optimised with regard to usability. This included hardware, software and a manual. The main usability issue was effectiveness: Are people able to install the product by themselves? Other issues were acceptability (do people begin the installation process) and subjective efficiency (how long do people feel it takes to complete the process). The approach consisted of expert tests, internal tests with colleagues and in situ user testing with nine persons/households within the target market.

The bicycle carrier (case C) was developed for transporting bicycles for recreational use. The case was originally only a re-styling assignment. However, the first evaluations by the project team, in which design proposals were shown to bicycle dealers, revealed that usability was indeed a crucial issue. The design studio had developed a similar product years before and is regularly involved in design projects that concern products related to bicycles. The main usability issues were effectiveness and prevention of errors that cause damage and comfort. The approach included internal testing (self-tests and tests with colleagues) and expert testing. The complete project took about one year (3 man-years).

For cases A, B, and C, we identified respectively 21, 25 and 17 use situation aspects based on the initial group interviews. After the individual interviews in which these aspects were discussed by means of the use situation aspect cards, we removed irrelevant or redundant cards. This led to a final selection of respectively 9, 15 and 8 relevant varying use situation aspects. Figure 6 shows some examples of use situation aspects for the three cases, categorized into types of aspects.

Figure 6. Examples of relevant varying use situation aspects for the three studied projects,
categorized with respect to the user, their goal and the context of use.

Analysing DDUS

Based on the literature and empirical results for research question 1: ‘How are use situations analyzed in design practice?’, we identified the following use situation analysis strategies:

  • Studying users and use situations of current solutions directly
  • Retrieving self-reports
  • Consulting personal domain knowledge
  • Consulting domain knowledge within the organization
  • Consulting experts

Direct study of users and use situations of current solutions

The literature describes many methods that give insight into current use situations and relevant usability issues in the analysis stage of the design process. For specific use situation aspects, data may already be available because others have previously researched it. For example, anthropometric information is largely available through the work of Dreyfuss (1968) and later researchers such as the TU Delft ergonomics group (Daanen, Krul, & Molenbroek, 2004). For other aspects, product developers should conduct user research themselves to collect this information. This type of user research includes user observation, interviews and focus groups (Sharp, Rogers, & Preece, 2007), ethnography (Blomberg, Burrell, & Guest, 2003) and contextual inquiry (Beyer & Holtzblatt, 1998). These types of real-world studies and usability evaluations of current solutions were reflected in all the analysed projects. In all cases, end-users and contexts of use were studied or involved directly to obtain insight in several use situation aspects. The techniques applied to collect this information included observations during in situ user testing, interviews and field visits.

Self-reports of Users & After Sales Feed Back

Approaches that have a larger chance of giving insight into the broadness of use situations and issues are those in which users report their use experiences for themselves. Examples of these kind of self-reporting techniques are retrospective interviews (Rosson & Carroll, 2002) and probing (Gaver, Dunne, & Pacenti, 1999). These types of self-reports were also found in design practice, including cultural probes and on-line information sources.

A specific type of self-report is after-sales feedback (den Ouden, Yuan, Sonnemans, & Brombacher, 2006), which only provides feedback after the product has been introduced to the market. Since the future use situation can never completely be predicted—we cannot design the user experience (Redström, 2006)—after sales feedback gives new insight into unforeseen use situation aspects. A study by van Kuijk, Kanis, Christiaans, and van Eijk (2007) showed that this feedback can be a very valuable source of information in the development process. This was also reflected in the studies in design practice. A designer from company A, for example, claimed that they know that they do not know everything when the product is introduced to the market, but that they can use after sales information to improve the next version of the product.

Designer (case A): “You continuously learn about [the preferences of our users]. What we know now is different from what we knew four years ago. But we won’t accommodate [these preferences] anymore in this product. Instead, we will accommodate them in this product’s brothers and sisters.”

Personal Domain Knowledge

As opposed to the aforementioned studies with actual end-users in actual contexts, this study revealed that other sources were also used to obtain insight in DDUS. The first source is personal domain knowledge. Most participants indicated they often make use of their own knowledge about the use situation, either from personal experience with the domain or from previous projects they had worked on. For example, one of the respondents in case B indicated that he knew that the variety of levels of fitness influenced the concentration level and consequently the learnability of the system because he had personally experienced this when using other products.

Organisational Domain Knowledge

Another information source is the knowledge available in an organization. This was reflected in company A, which had been developing the same kind of products for years and which kept records of relevant use situation aspects, for example, the sizes of environments and the types of documents printed. Case C identified an intermediate information source that sits somewhere between personal and organisation domain knowledge. Because this case concerned a product used in a common environment, the design team could make use of information sources close to the company. For example, they used their own cars and bicycles in usability evaluations of their own and of competitor products.

Experts

Although end-users and environments are the most direct source of use situation information, designers indicated that it was sometimes more efficient to consult experts to gain insight into the variations between use situations. For example, the design team of the bicycle carrier gathered many insights from consultations with bicycle dealers. Not only did the bicycle dealers have a wide knowledge of types of bicycles that should fit the bicycle carrier, they also knew much about the different needs of users with regard to using the bicycle carrier.

Influence of Design Context and Nature of the Project on Use Situation Analysis

The sources of information that can be used to gather information about use situation aspects seem to depend to a large extent on the context in which the project takes place and the nature of the project. When the product is developed in a context in which the project is preceded with projects for similar use situations, a designer can rely on organizational knowledge as described in case A. When the project concerns common use situations as in the case of many consumer products, a designer can rely to a certain extent on their personal domain knowledge.

Prioritising DDUS

The second research question was how DDUS aspects are prioritized in design practice. When asked to order situation aspects with regard to importance, participants from the same team often used different categorizations. This suggests that prioritizing use situation aspects was not an explicit activity in the studied design projects. The relevance of use situation aspects depends on the extent to which it influences usability, based on severity (criticality) and the frequency in which the use situations will occur, see e.g., Rosson & Carroll (2002, p. 230). Most critical aspects that were mentioned in this study were those that resulted in mistakes, made use impossible or made users lose their confidence in the product. As mentioned in the theoretical framework, gaining insight into the extent to which a use situation aspect influences usability requires an exploration of the relation between use situations and usability. A study of the use situation aspects that were considered in the three design projects gave more insight into this aspect. Although some connections between use situations and usability issues can be predicted, most influences are uncertain.

Preconditions: Predicting What Will Not Be Effective

For some usability issues, it was possible for the teams to predict their occurrence for certain use situation aspects. For example, for the user characteristic ‘knowledge of a certain language’, the printer team could predict if people would be able to read the instructions on the product or in the manual. Not knowing the right language would make it impossible for the user to achieve that goal. These aspects can be considered boundary conditions; if they are not taken into account it will definitely deliver unsatisfying results.

Difficult to Predict Relations between Use Situation Aspects and Usability Issues

For some use situation aspects, it was less obvious what their consequences were in relation to a product. For example, in the health monitoring system it appeared that the self-esteem of users influenced their acceptance of the system, but it was not obvious which solution was needed to overcome this problem. The fact that many use situation aspects and usability issues were interrelated further complicated the activity of analysing the relations between these aspects and issues. For example, in the bicycle carrier case, the comfort of positioning the bike on the carrier relates to the weight of the bike, but also to the strength of the user. In the printer case, the role of the user relates to the frequency of use and the experience with printing. Both aspects have influence on the learnability of the system. In some cases, it was not at all clear which use situation aspect caused differences in usability issues. For example, in the case of the health monitoring system, users would get into trouble at different points in the installation procedure, but this seemed not to be related to a specific user or context characteristic.

These difficulties in anticipating use are also demonstrated in a study of Kanis (1998) who showed that user characteristics can set boundary conditions by indicating what users will not do, but they do not give insight into what users will do. Therefore the relevancy of use situation aspects cannot be defined completely at the start of a design project; it will become clearer after the first usability evaluations. Usability testing is thus not only useful for providing insight into usability issues, but also to identify relevant use situation aspects. In the following section, we describe how usability was evaluated in the projects studied.

Evaluating Use

The third research question was: ‘How is the usability of solutions evaluated with regard to DDUS?’ All participants in this study acknowledged the importance of early usability evaluation. Each project involved one or more cycles of formative usability evaluations. Different means to get insight into future usability issues were discussed, ranging from self-evaluations to beta-tests with end-users. We classified these evaluation types as either internal usability evaluations or external evaluations with experts or end-users. Apart from these evaluation types, the results showed how the dynamics and diversity of use situations were reflected in the test conditions of these evaluations.

Internal Usability Evaluation

In all cases, internal evaluations without actual end-users were executed. With internal usability evaluations, we mean that prototypes or mock-ups are tested inside the company with colleagues who are outside the project, or by self-tests of the designer in which they build models to quickly test the design themself. Obviously, testing yourself does not give insight into variations in use and can only be applied to domains that are personally well known to the designer. Furthermore, it seemed to be limited to testing physical actions. In early design phases, more insights into variations in actions can be gathered by testing early prototypes with users inside the company. This happened in all the studied cases. For example, the usability expert of case B mentioned: “Before going outside we did some early tests with secretaries to be able to remove the basic problems. So you usually firstly test inside.”

External Usability Evaluation with Experts

In cases B and C, the developers deliberately chose different types of users to test the product to obtain insight into variations of use. In tests with experts, these variations of use could be revealed as well; apart from their personal preference of performing an action, the experts also had a broad view on types of use. This happened in case C, when the design team tested the bicycle carrier with bicycle dealers.

External Usability Evaluation with Intended End-users

Finally, tests with different types of end-users can reveal variations of use. In case A and B, tests with end-users were applied in later design phases when working prototypes were available and the first problems with the design had been removed based on internal tests. In case A, extensive user tests were conducted in situ with five representative clients who used the product for three months. In case B, tests with nine elderly persons and couples were conducted in their homes. In case C, no extra tests were conducted with end-users because the team members indicated they had gathered enough information from the evaluations with colleagues, the experts (bicycle dealers) and the client.

Test Conditions

To gather results that are externally valid, test conditions of usability evaluations should reflect the context of use (Bevan & Macleod, 1994; Cushman & Rosenberg, 1991, p. 53). When designing for DDUS, it is not only necessary that test conditions represent the use situations, it is also essential to evaluate in multiple use situations. We expected that field tests would offer more opportunities for evaluating prototypes with regard to multiple contexts of use than lab tests. This was also reflected in the cases. Both case A and B, and to a lesser extent case C, included extensive tests in the field.

For the different formal usability evaluations, one or more use situation aspects were consciously varied. For both internal tests and the beta-tests with the printer in case A, user characteristics were consciously varied. For example, participants of different heights were invited to observe how this would influence paper handling. In the health monitoring system project in case B, random situations within the chosen target group (elderly in their own home) were used to test the product, because the design team had no insight yet into what relevant varying use situation aspects would be. Since the health monitoring system was a completely new product with no comparable competitors, it seems that the variation of certain use situation aspects in test conditions depends on the extent to which the designers are familiar with the use of comparable products.

Usability tests can have a more or less formal character depending on the conscious choice for an evaluation approach. The informal tests, such as the self-tests or undocumented tests with colleagues, have a larger chance of not revealing issues for variable use situation aspects because of their opportunistic character and limited availability of varying test persons and test environments. However, as mentioned by the usability expert of case B, they are useful for revealing ‘the basic problems’. These are usability issues that seem independent of use situation aspects. Another benefit is that since informal tests do not need much time, they can be easily applied to proceed in an iterative design process. However, to obtain reliable insights in the usability of a solution, formal tests with end-users would need to be conducted as part of the design process as well.

Communicating Knowledge of DDUS

The last question of this study was how knowledge of DDUS and usability was made explicit and communicated. In this section, the results with regard to communication are divided into communication about usability insights and communication about use situations. The results show the influence of the design context on the communication of these issues.

Communicating Usability Insights

All teams made reports of the analyses and usability evaluations and additionally presented the results verbally in PowerPoint presentations, although respondents of case B and C indicated that their documentation process could be improved. The designer in case C indicated there was not enough time for good documentation during the project. He mentioned one instance where he made the report only weeks after a certain observation, where he had communicated the results verbally right away.

The use of the reports as a means of communication is low. Some designers indicated that they didn’t like to read the reports or that they would only read the conclusions, while usability experts questioned if the designers had read the reports. To make sure results of analyses and usability evaluations would give input to the design process, results were communicated verbally in team meetings or face to face, and/or visually by means of video. Most respondents indicated that it is better to do the observations and evaluations yourself, although in none of the cases were the designers involved in all the observations and evaluations. From this and the earlier insight that team members often relied on personal knowledge of product use, we conclude that there is still a need to communicate usability issues and related use situation aspects otherwise.

Communicating Use Situation Aspects

Some specific use situation aspects were well documented. For example, in case A, a complete reference set of possible types of prints was documented; in case C, different bicycle sizes were documented. These aspects consider the earlier mentioned ‘boundary conditions’. In case A, important user characteristics were documented in personas (see e.g., Cooper, 1999). In case B and C, some issues were made explicit in a start document or in critical scenarios, but these were not kept up to date. We could not observe to what extent the reports and presentations included information about the use situation aspects. However, it was clear that some usability issues and related use situation aspects were not made explicit at all, as can be concluded from the following comments:

Usability expert (case B): “Experience with technology is a user aspect that I know is important from previous projects.”

Designer (case B): “That is also knowledge you gain in your personal life. If I look at my parents and see how much problems they have with relatively simple things, then you know that will occur frequently. The funny thing is, we hardly communicated this aspect.”

Influence of Design Context on Communicating Use Situations and Use Issues

The documentation of knowledge of product use seemed to depend on the time and budget available for the project and the extent to which the project could build on knowledge of previous projects. The company in case A develops products that are very similar to each other. They also had the largest budget with regard to usability. They kept the most extensive records of use situations and usability issues. For unfamiliar use situations, as in case B and C, knowledge of use situations and related usability issues needs to be built from scratch, which means it can only be made explicit in the course of the design process. However, in both cases, knowledge of use situations was only made explicit and communicated at the start of the project and not regularly updated. Communication during the design process relied completely on verbal communication; the reports generated were not read or were created too late. The following section discusses the possible consequences of this lack of shared knowledge of product use further.

Discussion

The study of how designers deal with DDUS in design practice resulted in insights that either led to recommendations for design for DDUS directly or to indications for the further development of design support.

Use Situation Analysis As an Iterative Process

The study showed that various effective sources of information were used to insights into varying relevant use situations, including direct studies of end-users and use situations, self-reports, personal domain knowledge, organisational domain knowledge and experts. Particularly, experts were an efficient source of insights into the dynamics and diversity of use situations and related usability issues. Recommendations for design for DDUS should stimulate designers to efficiently employ aforementioned sources of information to gather insight into relevant varying use situation aspects.

The designers identified that the difficulty in designing for DDUS is the inability for relevant use situation aspects to be defined in advance. Although above mentioned sources can give insight into the broadness of use situations, they mostly do not reveal which aspects of this broad spectrum of situations are most relevant when it comes to designing solutions. Use situation aspects that are not ‘boundary conditions’ only become clear in evaluations of prototypes or of comparable products. Some relevant use situation aspects become clear only after the product has been introduced to the market. At this point, after-sales feedback can be used to gain a better understanding of the relation between use situation aspects and use issues. However, to be able to take these varying situation aspects into account during the design process, it is necessary to conduct usability evaluations early on and throughout the design process. Recommendations for design for DDUS should therefore also encourage these types of evaluations.

More valid results can be retrieved in usability evaluations when test conditions reflect the most important use situations. When the results of usability evaluations would not be representative of the variety of real world intended use situations, they would not lead to useful recommendations to adjust the evaluated solution to better accommodate those use situations. To set up the evaluation, we recommend the need for a frame of reference that reflects these most important use situation aspects. As mentioned before, these use situation aspects can often only be retrieved from these same usability evaluations. Building such a frame of reference can therefore only be done iteratively, as illustrated in Figure 7. Before there is a prototype in a design project, insight into the most important use situation aspects can be gathered through use situation analyses and/or knowledge of comparable products (personal or organization knowledge, or additional research). During the design process, evaluations of prototypes can be used to further update this frame of reference. A design support to guide designers in dealing with DDUS should include guidance in the development of such an evolving frame of reference of product use.

Figure 7. Building a frame of reference of most important use situation aspects and usability issues is an iterative process.

Sharing Knowledge of Product Use

This study revealed that knowledge about use situations and related usability issues often remained unshared in the design teams. The need for shared knowledge of product use in design is discussed by Buchenau and Suri (2000, p. 425), who state in their work on user experiences that to work effectively as a design team, it is important to develop a common vision of what the team is trying to bring into being. According to Badke-Schaub, Neumann, Lauche, and Mohammed (2007), very few empirical studies have been conducted on the implications of a lack of a common vision or ‘team mental model’ in design related areas. The latter authors assume that mental models for design can relate to knowledge about the task, the process, the group, the competence and the context. Knowledge of the variety of use situations and usability issues would fall into the category of task models, which relates to a person’s stored knowledge regarding the particular task, including product knowledge such as information about the problem, the goal and the solution. If team members are to perform well and make successful collective decisions, they need to share task mental models up to a certain level (Badke-Schaub et al., 2007). This decision-making involves the choice of solution proposals and the development of criteria to use in the activities of evaluating and choosing solutions (Visser, 2006). When considering product use, these evaluation criteria would include the intended use situations and desired level of usability and user experience. It is important to pay attention to collective decision-making because usability problems can often be traced back to decisions made in the development process (den Ouden, 2006; Harkema, 2012). We therefore assume that knowledge of use situations and potential use issues are types of knowledge that need to be shared and which will influence design team performance by influencing the collective decisions designers make. The development of design guidance should therefore include a support that is aimed at sharing knowledge of product use in design teams.

An Explicit and Flexible Frame of Reference of Product Use

We assume that generating an explicit representation of this knowledge would be a first step to better sharing knowledge of product use. Knowledge of product use should be made visible so “it can be seen, talked about, and potentially manipulated” (Suchman, 1995, p. 63). We thus propose to generate an explicit frame of reference of product use that reflects the dynamics and diversity of use situations and how these situations relate to use issues.

To allow the evolvement and record of information on use situation aspects and related usability issues throughout the design process, such a frame of reference needs a flexible character. A means to make information about varying use situations and related use issues explicit is the use of models that represent commonalities in information about specific categories of use situation aspects. Several such models have been developed within the design research community. The collection of attributes for a typical user is called a user profile (Rubin, 1994; Sharp et al., 2007). Personas (Cooper, 1999) are a familiar means to represent user profiles. They represent the target users as a hypothetical person with specific characteristics, a name and an image. Beyer and Holtzblatt (1998) developed five types of consolidated work models for computer-supported cooperative work (CSCW) in their contextual inquiry approach. These models show the common structure in the work different people do. For example, the flow model represents the communication and coordination necessary to do the work and the physical model shows the physical structure of the work environment as it affects the work. Thus, each category of use situation aspects can be captured in a specific type of model. The models are particularly useful for CSCW because they deal with the complexities of collaborative work, although they might also be applicable to other types of products. Both personas and contextual inquiry models are based on large quantities of verified data about users and other use situation aspects, these being translated into models by a thorough interpretation step to reveal the similarities (see e.g., Pruitt & Grudin, 2003). Although both personas and the contextual models seem easy to adjust, doing so without an additional interpretation step will decrease the strong value of the models for giving an accurate view on commonalities in actual information on use situations. This makes them less flexible to adjust later in the design process.

In the studied design projects, the use situation aspects that were made explicit in a frame of reference mostly considered constraints or boundary conditions and were reflected in requirements. Less predictable use situation aspects were only made explicit in the case in which the company continuously designed similar products and could invest in building personas, for example. In other cases, these use situation aspects were sometimes made explicit at the beginning of the design project, such as the critical scenarios that were created in case C, but not adjusted during the design project. We therefore conclude that there is a need for a support that stimulates building and updating such an explicit and flexible frame of reference of the most important use situation aspects and usability issues, particularly for cases where this frame of reference cannot be reused from previous projects.

Conclusions

In this paper, we have examined how designers deal with the fact that products are used in a myriad of use situations. The design research community acknowledges this issue, but guidance on how to deal with it in the design process can not be found. Our study of the means by which designers analyse and prioritize use situation aspects, evaluate solutions with regard to these use situations and communicate knowledge of use situations showed that (1) designers employ different effective strategies in gathering knowledge of varying use situations and use issues, (2) identifying the most relevant aspects within these use situations requires an iterative approach, and (3) individual knowledge of use situations often remains unshared within design teams. Insights 1 and 2 lead directly to the following recommendations for design for DDUS:

  • Employ research methods that show the variety in use situations, including direct studies of use situations, acquiring user self-reports and consulting experts.
  • Conduct usability evaluations early and throughout the product development process (including in between projects) and use them to articulate both use issues and related use situation aspects.

Our future research will include an exploration of the means by which those recommendations should be further introduced to design practice. A third recommendation based on insight 2 would be to build a frame of reference of product use in an iterative process by prioritizing use situation aspects based on usability evaluations. Since this is less straightforward than recommendation 1 and 2, additional support is needed to guide designers in this process. This design support should then allow for the creation of explicit, flexible frames of reference, particularly for products with use situations that are unfamiliar to the design team. In future research, we will explore means to generate and visualise such frames of reference.

With regard to the lack of shared knowledge of product use (result 3), we conclude that when there is no clear shared ‘frame of reference’ of use situations and issues, communication about the quality of proposed solutions and appropriate intended use situations in design decisions could be difficult, which in turn influences the usability and user experience of the final product. We assume that the creation of an explicit frame of reference can contribute to the sharedness of knowledge of product use. Our future research will include an evaluation of the effect of such an explicit frame of reference on sharing knowledge of product use and an exploration of other means to share this knowledge.

Acknowledgments

We would like to express our gratitude to the participants in the retrospective case studies for offering their time to this research and their valuable feedback. This study was executed as part of the doctoral research of the first author at the University of Twente, the Netherlands. This research was part of the ‘Design for Usability’ research project, financed by the Innovation-Oriented Research Programme ‘Integral Product Creation and Realization (IOP IPCR)’ of the Netherlands Ministry of Economic Affairs, Agriculture and Innovation.

References

  1. Badke-Schaub, P., Neumann, A., Lauche, K., & Mohammed, S. (2007). Mental models in design teams: A valid approach to performance in design collaboration?, CoDesign, 3(1), 5-20.
  2. Bevan, N., & Macleod, M. (1994). Usability measurement in context. Behaviour and Information Technology, 13(1-2), 132-145.
  3. Beyer, H., & Holtzblatt, K. (1998). Contextual design: Defining customer-centered systems. San Francisco, CA: Morgan Kaufmann.
  4. Blessing, L. T. M., & Chakrabarti, A. (2009). DRM, a design research methodology. London, UK: Springer.
  5. Blomberg, J., Burrell, M., & Guest, G. (2003). An ethnographic approach to design. In J. A. Jacko & A. Sears (Eds.), The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications (pp. 964-986). Hillsdale, NJ: Lawrence Erlbaum Associates.
  6. Buchenau, M., & Suri, J. F. (2000). Experience prototyping. In D. Boyarski & W. A. Kellogg (Eds.), Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 424-433). New York, NY: ACM.
  7. Cooper, A. (1999). The inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity. Indianapolis, IN: Sams.
  8. Cross, N. (2007). Designerly ways of knowing. Basel, Switzerland: Birkhauser.
  9. Cushman, W. M., & Rosenberg, D. J. (1991). Human factors in product design. Amsterdam, the Netherlands: Elsevier.
  10. Daanen, H. A. M., Krul, A. J., & Molenbroek, J. F. M. (2004). DINED anthropometic database. Retrieved March 24, 2014, from TU Delft http://www.dined.nl
  11. Den Ouden, E. (2006). Development of a design analysis model for consumer complaints: Revealing a new class of quality failures. (Unpublished doctoral dissertation). Eindhoven University of Technology, Eindhoven, the Netherlands.
  12. Den Ouden, E., Yuan, L., Sonnemans, P. J. M., & Brombacher, A. C. (2006). Quality and reliability problems from a consumer’s perspective: An increasing problem overlooked by businesses? Quality and Reliability Engineering International, 22(7), 821-838.
  13. Desmet, P. M. A., & Hekkert, P. (2007). Framework of product experience. International Journal of Design, 1(1), 57-66.
  14. Dorst, K. (2007). Design research: A revolution-waiting-to-happen. Design Studies, 29(1), 4-11.
  15. Dorst, K., & Cross, N. (2001). Creativity in the design process: Co-evolution of problem-solution. Design Studies, 22(5), 425-437.
  16. Dourish, P. (2004). What we talk abouth when we talk about context. Personal and Ubiquitous Computing, 8(1), 19-30.
  17. Dreyfuss, H. (1968). The measure of man: Human factors in design. New York, NY: Whitney Library of Design.
  18. Gaver, B., Dunne, T., & Pacenti, E. (1999). Design: Cultural probes. Interactions, 6(1), 21-29.
  19. Harkema, C. (2012). Revealing unawareness in usability related decision-making. (Unpublished doctoral dissertation). Eindhoven University of Technology, Eindhoven, the Netherlands.
  20. ISO. (1998). ISO 9241-11 Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability. Geneva, Switzerland: International Organization for Standardization.
  21. Jordan, P. W. (1998). An introduction to usability. London, UK: Taylor & Francis.
  22. Kanis, H. (1998). Usage centred research for everyday product design. Applied Ergonomics, 29(1), 75-82.
  23. Kim, C., & Christiaans, H. (2011). Usability problems: The influence of user diversity. In N. Noozenburg, L. Chen, & P. J. Stappers (Eds.), Proceedings of IASDR2011, the 4th World Conference on Design Research. Delft, the Netherlands: TU Delft.
  24. Lawson, B. (2006). How designers think: The design process demystified (4th ed.). Oxford UK: Elsevier.
  25. Maguire, M. (2001). Context of use within usability activities. International Journal of Human-Computer Studies, 55(4), 453-483.
  26. Nielsen, J. (1993). Usability engineering. Boston, MA: Academic Press.
  27. Nielsen, L. (2002). From user to character: An investigation into user-descriptions in scenarios. In Proceedings of the 4th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 99-104). New York, NY: ACM.
  28. Norman, D. A. (1986). Cognitive engineering. In D. A. Norman & S. W. Draper (Eds.), User centered system design (pp. 31-59). Hillsdale, NJ: Lawrence Erlbaum Associates.
  29. Norman, D. A. (1998). The design of everyday things. London, UK: MIT Press.
  30. Pruitt, J., & Grudin, J. (2003). Personas: Practice and theory. In Proceedings of the 2003 Conference on Designing for User Experiences (pp. 1-15). New York, NY: ACM.
  31. Redström, J. (2006). Towards user design? On the shift from object to user as the subject of design. Design Studies, 27(2), 123-139.
  32. Roozenburg, N., & Eekels, J. (1995). Product design: Fundamentals and methods. Chichester, UK: Wiley.
  33. Rosson, M. B., & Carroll, J. M. (2002). Usability engineering: Scenario-based development of human-computer interaction. San Francisco, CA: Morgan Kaufmann.
  34. Rubin, J. (1994). Handbook of usability testing. New York, NY: John Wiley & Sons.
  35. Schilit, B., Adams, N., & Want, R. (1994). Context-aware computing applications.In Proceeding of the 1st Workshop on Mobile Computing Systems and Applications (pp. 85-90). Los Alamitos, CA: IEEE.
  36. Shackel, B. (1984). The concept of usability. In J. L. Bennett, D. Case, J. Sandelin, & M. Smith (Eds.), Visual display terminals: Usability issues and health concerns (pp. 45-85). Englewood Cliffs, NJ: Prentice-Hall.
  37. Sharp, H., Rogers, Y., & Preece, J. (2007). Interaction design: Beyond human-computer interaction (2nd ed.). Chichester, UK: John Wiley & Sons.
  38. Shneidrman, B. (2000). Universal usability. Communications of the ACM, 43(5), 84-91.
  39. Stolterman, E. (2008). The nature of design practice and implications for interaction design research. International Journal of Design, 2(1), 55-65.
  40. Suchman, L. (1995). Making work visible. Communications of the ACM, 38(9), 56-64.
  41. Van der Bijl-Brouwer, M., & van der Voort, M. C. (2009). Strategies to design for dynamic usability. In Proceedings of IASDR Conference on Design Research. Seoul, Korea: Korean Society of Design Science.
  42. Van Kuijk, J. I. (2010). Managing product usability; How companies deal with usability in the development of electronic consumer products. (Unpublished doctoral dissertation). Delft University of Technology, Delft, the Netherlands.
  43. Van Kuijk, J. I., Kanis, H., Christiaans, H. H. C. M., & van Eijk, D. J. (2007). Usability in product development practice: After sales information as feedback. Proceedings of IASDR Conference on Design Research. Hong Kong: Polytechnic University.
  44. Visser, W. (2006). Designing as construction of representations: A dynamic viewpoint in cognitive design research. Human-Computer Interaction, 21(1), 103-152.
  45. Wilson, J. R. (2000). Fundamentals of ergonomics in theory and practice. Applied Ergonomics, 31(6), 557-567.
  46. Yin, R. K. (2009). Case study research, design and methods (4th ed.). Thousand Oaks, CA: SAGE.