Design and Computer Simulated User Scenarios: Exploring Real-time 3D Game Engines and Simulation in the Maritime Sector

5-1887

Snorre Hjelseth*, Andrew Morrison, and Kjetil Nordby

The Oslo School of Architecture and Design, Oslo, Norway

Designers often use scenarios to approach complex design problems holistically. Efficient use of scenarios can be a challenge in complex dynamic user contexts due to mediation and tool limitations that use traditional scenario techniques in design practice. We investigate the use of real-time 3D game engines on the part of interaction/product designer-researchers as a design tool to conceptualise and simulate possible future shared user scenarios in the maritime domain. Three exploratory and qualitative case studies are presented, drawing on collaborative and participatory design, action research, and research through design. Results reveal that real-time 3D game engines can simulate multiple complex behaviours that otherwise would have been impossible to materialize with traditional visual scenario or storytelling methods. This opens up new possibilities for how designers can handle complex user relations and related risk factors when designing. For the maritime sector this has further potential for the handling of safety critical operations.

Keywords – Design Tools, User Centred Design, Simulated Scenarios, Game-engines, Real-time.

Relevance to Design Practice – This research shows how designers can handle complex and UCD maritime design challenges where game engines offer potential for materialising future simulated scenarios.

Citation: Hjelseth, S.,Morrison, A., & Nordby, K. (2015). Design and computer simulated user scenarios: Exploring real-time 3D game engines and simulation in the maritime sector. International Journal of Design, 9(3), 63-75.

Received April 25, 2015; Accepted November 21, 2015; Published December 31, 2015.

Copyright: © 2015 Hjelseth, Morrison, & Nordby. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

*Corresponding Author: sh@aho.no.

Snorre Hjelseth in as industrial designer with special interest for design thinking in the maritime and offshore innovation sector. His PhD research is focusing on visualization and simulation as a means in interdisciplinary design processes. Snorre received his Master’s degree in industrial design from the Oslo School of Architecture and Design (AHO), and the Indian Institute of Technology Bombay (IIT). He is now working as a design researcher at AHO in the ONSITE research project.

Kjetil Nordby is associate professor at the Oslo School of Architecture and Design, and holds a Master in Interaction Design from Umeå University and a PhD from Oslo School of Architecture and Design. Kjetil is the manager of Ocean Industries Concept Lab, and has extensive experience leading large industry driven research projects in Norway.

Andrew Morrison is Director of the Centre for Design Research at the Oslo School of Architecture and Design (AHO) in Norway and Professor of Interdisciplinary Design at the Institute of Design (IDE). As co-ordinator of research, Andrew takes part in and leads a range of design research projects that cover Communication Design, dynamic interfaces and social media; RFID, mediation and activity; Service Design and innovation in leadership; electronic arts installation; narrative and mobile media; practice-based research/research by design; online research mediation; and design research methods. Andrew also focuses on design writing, fiction, and criticism. He has been central to the ongoing redesign and teaching in the PhD program at AHO. He has supervised a dozen PhD students at AHO and others at the University of Oslo in design, media, and education. Andrew is a member of the Research Committee and the Board of AHO. He was paper co-chair for Nordes 09, Engaging Artifacts, and 3rd Nordic Design Research Conference (www.nordes.org), and has been a board member of the Design Research Society. He has published widely in journals, books, and online, and has a special research interest in online research mediation. He has edited and co-edited several collections of papers and chapters related to design and new media.

Introduction

Setting the Stage

Human failure is by far the largest contributor to maritime accidents (e.g., Bjørneseth, Dunlop, & Strand, 2008). In the maritime industry there is a move towards including User Centred Design (UCD) processes (Norman & Draper, 1986) in an effort to ensure safer operations at sea (Mills, 2006). The maritime sector deals with safety critical operations where a range of systems, equipment, vessels, and personnel collaborate in complex operations. In the early phases of designing for such operations, designers need to deal with multiple levels of complexity in contextually related systems.

A major challenge for performing UCD in maritime settings is to holistically understand the needs and actions of users and use practices in terms of both contexts and systems. Computer simulated scenarios realised by game engines may help us meet this challenge through simulation and visualization of several systems and behaviours simultaneously (Winsberg, 2010). A scenario can be described as series of hypothetical actions and events (Bødker, 2000). Broadly, simulation refers to the modelling of real world situations via representational, meditational and, increasingly, computational design techniques. A game engine is a software framework used in the process of creating and running computer games. The game engine renders that which enables the game experience in the game world through modules of technical infrastructures (Nideffer, 2003). Simulation software is part of the game engine’s running of game functions or the mimicking of real world behaviours based on mathematical models.

We take up the use of real-time 3D game engines as a design tool by interaction/product designer-researchers in order to simulate possible future shared user scenarios in the maritime domain. This innovation perspective places attention on conceptualisation and simulation in design practice and inquiry. However, applying computer simulated scenarios to UCD is not straightforward. When deploying computer simulation in UCD, there is a need to develop rich and pragmatic approaches that connect human actions with technologically mediated renderings.

Below we examine how computer simulated scenarios can be used as a means to facilitate a maritime design process related to risk, safety, and operations by posing two key questions: 1) How may computer simulated user scenarios be used as a means to facilitate a design process to explore and reveal possible design solutions and problems in the maritime sector? and 2) How may a game engine be used as a design tool in user scenario development to facilitate design for complex systems and contexts? We present three cases to explore the questions. These cover: 1) crisis management in a busy navigation channel (shipping), 2) dynamic positioning in the polar areas (offshore), and 3) helicopter deck design (offshore and shipping).

On Scenarios

Scenarios are used in several different contexts and have various meanings in the fields in which they are taken up. In Human-Computer Interaction (HCI), Carroll (2000) referred to scenarios as the “stories about people and their activities” (p. 46). Visser, Stappers, van der Lugt, and Sanders (2005) claimed that “When important decisions have to be made, a clear and convincing argument can be made using a scenario of the interaction based on the design and the knowledge about its context” (p. 135). In the maritime sector, scenarios have traditionally been used in training (Barnett, Gatfield, & Pekcan, 2003), such as emergency and crisis management for situation awareness or task analysis in complex engineering operations (Maslin, 2013).1

On Simulations

Banks (2011) described simulation as a methodology where mathematical or symbolic models are used to describe the behaviour of a real world process or system. Simulation is widely used in many scientific fields, and Winsberg (2010) argued that computer simulation has had impact “in almost every scientific study—from quantum chemistry to the study of traffic—flow patterns” (p. 4).

In design, computer simulation has enabled designers to deal with the complexity of design decisions in which theory and design can be experienced immediately. Simulations with the ‘human-in-the-loop’ are often used to analyse systems operated by a human or for training purposes (Narayanan & Kidambi, 2011). Human-in-the-loop simulations are often used in human factors testing where a real human being is needed in the simulation (McKneely, Wallace, Perry, & Winters, 2001). Examples abound on using computer simulation for usability testing in product development. Among the topics covered are virtual prototypes in usability testing (Kuutti et al., 2001), Virtual Reality (VR) simulation (Manninen, 2000; Thalen & Voort, 2012; Tideman, Voort, & Houten, 2008), augmented reality (Woohun & Jun, 2005), and experience based virtual prototyping simulator (Kumar, Hedrick, Wiacek, & Messner, 2011).

Other work takes up the challenges of using simulations in design practice. Following on from ground breaking earlier studies within architecture on humans and virtuality, Turkle (2010) turned her attention to pitfalls in simulation. She further observed that “the virtual makes something seem more real,” and that “computer-aided design made theory become more alive” (p. 13), and Winsberg (2010) made the same observations. In our design cases we found that it is very important to inform the design session participants that the scenarios are mediated representations of “reality.”

Previous research in the maritime sector (Grech, Horberry, & Smith, 2002; Kristiansen & Nordby, 2013) has shown the positive implications of carrying out user surveys of complex user-related operations at sea. However, related simulations have not been implemented as part of the front-end conceptualisation phase of design, but in later stages in development. One exception is Kristiansen and Nordby (2013), who used simulators in the early phase development for testing interface ideas. However, there is little literature on how simulation has influenced design practice in relation to user scenarios on UCD in the front-end of innovation.

Game Engines

In our research we focus on real-time simulation visualized in real-time rendering (Möller, Haines, & Hoffman, 2008 p.18) by way of game engines. Game engines are a generation of tools that emerged from entertainment rather than industrial and scientific needs, and are used to create games for platforms like PlayStation and personal computers. A core attribute of game engines is aesthetic presentation and efficient production of content. We chose to use game engines since they provide a graphical user interface and the framework to model and simulate computer game functions useful for rendering scenarios without a need for expert knowledge of computer coding. This is in line with other research such as ‘Serious gaming’ that takes up the combination of game engines and simulation for the purpose of developing skill and knowledge about contexts that are unavailable in a normal learning environment (Susi, Johannesson, & Backlund, 2007).

Game engines like Unity, CryEngine, and Unreal Engine have been adopted as tools in shaping simulations (Kumar et al., 2011). Typical functions in game engines are a 2D or 3D graphics-rendering engine, object collision detection, a physics engine, animation integration, artificial intelligence (AI), sound integration, scripting, and network extensions. A game engine like Cryengine has an editor that has two modes that can be employed when designing scenarios to be simulated. First, the modelling mode uses physics and AI in running the simulation. Second, there is the mode of ‘human-in-the-loop’ integration or gaming mode in which a user may be involved in the scenario. In our research we mix these two simulation modes.

Research Methods and Design Techniques

Design Research in Action

In our inquiry we followed two main intersecting qualitative methodological strands: action research and research by design. Action Research is a method that allows immediate research on problems and solutions that are reflected on in action (Avison, 1997; Hollingsworth, 1997; Miller, 1994). The processes of action research deal with interactive inquiries where problems are shared collectively and knowledge is produced from action. This makes it possible to merge practice and research into the same research setting. Research by design (Morrison & Sevaldson, 2010) is a design practice based inquiry that takes up relations between practice and theory.2

The design cases were organised as commercial design projects where design briefs and project goals were defined at the outset for the design of scenarios to be used in support of concept development. The research on each case was then planned in relation to the project’s design need and how it might benefit from using simulated scenarios. A course of design action and collaboration was then planned together with industrial partners.3 These partners were medium and small sized companies, LysTech, Norwegian Maritime Education (NMU), a large state owned concern called Norwegian Coastal Administration, and a leading commercial company called Kongsberg Maritime.

During these processes we made design briefs, drew up project plans, organised pre-design meetings, constructed 3D objects, drew 3D context environments, made notes on how to model scenario settings, planned scenarios using drawing on paper, made physical models of ships, used different types of maps, worked with live data from ship traffic, sketched ideas, framed design problems, and wrote minutes from design workshops. The research data consisted of the design artefacts we made, audio and video recordings, screen captures, and research notes documenting and reflecting on the processes. We also gathered feedback from users and industrial partners on how they experienced the processes. We jointly analysed the material by reflecting on the processes of creating the scenarios, how simulation was to be used, and how it changed and elaborated the discursive process of products, systems and users in context for concept development.

Participatory Design

We used co-design and participatory design methods on innovation (Dalsgaard & Halskov, 2010), computer systems research (Büscher, Eriksen, Kristensen, & Mogensen, 2004), materiality (Jacucci & Wagner, 2007), tangible user interface development (Kim & Maher, 2008), embedding expert users (Humphreys, Leung, & Weakley, 2008), and collaborative sketching (Johansson, 2006).

Collaborative design sessions with stakeholders and actors from the companies and a specialist simulation designer (the lead author) were adopted. We found that there was a need for more interactive communication in small group design meetings (G. Olson, J. Olson, Carter, & Storrøsten, 1992) or collaborative design sessions (Gül & Maher, 2009). This also applied to communication about and uses of 3D simulated representations through which we were able to create more immersed experiences in user scenarios employing game engines. In such sessions designers imagine how products and systems are being applied in ‘use situations.’ This anticipation is a huge challenge because when designing new products and systems we ‘design use before use’ rather than ‘design in use time’ (Ehn, 2008). Designing in ‘use time’ is about being situated in context and the situations of use. When this is supported by simulations, we call this simulated use time design.

Collaborative design sessions offer potentially productive spaces for making use of multiple competencies in designing and in wider iterative and participatory processes of developing scenarios for subsequent use (e.g., Buur & Larsen, 2010). Through several such sessions we worked together with NMU, Kongsberg Maritime, and LysTech, and discussed ideas and how to shape innovation strategies. Through co-creative design sessions we created views on how the maritime industry might deal with early phase innovation (Jenssen & Randoy, 2002), and how it may address matters of design complexity and UCD. By using scenarios we were able to connect design places to user and actor participation in collaborative design sessions that drive UCD through tools facilitated by a designer.

We applied different mediation techniques to shape these narrative scenarios (Rosson & Carroll, 2003), such as scale maps and models (Bratteteig & Wagner, 2010), the drawing of navigation patterns, videos, and storytelling (Beckman & Barry, 2009). We experienced that these methods were effective in setting the stage for discussion. We also used problem re-framing techniques (e.g., Lawson & Dorst, 2009) in collaborative design sessions to create new thinking patterns concerning design processes, conceptualisation and innovation strategies.

Cases

To research the potential of the game engine to model scenarios and to simulate actions and behaviours, we drew up three different conceptual design process cases. Case studies are an exploratory approach to social science research concerning contemporary phenomenon within real-life contexts (e.g., Yin, 2009). They are applied in maritime research and inquiry into simulation and user-generated innovation (Buur & Larsen, 2010). We applied the game engine to create scenarios based on the design problematic in each case, and looked at different ways of simulating actions and behaviours. The game engine was then used and explored in collaborative design sessions to create collective and shared understanding as well as collective creativity (Sanders & Rim, 2001) through reflection in action and the ability to mediate iterations from co-participants and users in case based inquiry.4

Crisis Management, Dynamic Positioning and Locational Safety

Case Study 1: Crisis Management in the Oslo Fjord

This case deals with crisis management in the Oslo fjord based on the likely future increase of vessel traffic based on the new ferry concession for 2015. A team of actors representing different parts of the industry collaborated with the research team to find new areas for research and development based on present and future risk factors in the fjord. At the moment the area is the most hazardous coastline in Norway, and an increase of traffic might add considerably to existing risks and challenges.

There was a need to create shared understanding of existing navigation and traffic challenges in the fjord among the project team which involved researchers, designers, system engineers, captains, and product management. The team needed to be able to apply shared knowledge and their own expertise in creating new scenarios that would include increased traffic patterns.5

The case design brief was based on finding possible risk scenarios in the ship traffic patterns and behaviours. Further, we were to study the navigation challenges based on identified scenarios before, during, and after a critical situation had occurred. Here, there is a need to mimic the real world situation of the users. In this case study it was important to use actual scale 3D models and program the ship movement according to speed in order to simulate real world conditions. This helped users recognize the scenario in order relate it to their experience in real world situations.

Facilitating the design of an intended system for ships and simultaneously relating it to traffic behaviour for several ships required a simulation model that allowed us to chart each of the ship behaviours, such as ship speed for 8 ferries and their crossing of the paths of a cruise ship and a container ship. This represents a typical situation in the area. To observe the ensuing situations, a dynamic view of the situation was needed that could allow usto follow the scenario in motion. It was also necessary that the situation could be seen from individual ship bridges so as to be able to explore the navigation challenges on each separate vessel.

The game engine allowed ship behaviour to be simulated through pre-programmed path and speed. The ships could be placed at desired locations in the virtual environment and the whole scenario visualized in 3D. The terrain was modelled based on real world GIS data and was textured using aerial photos from the area. In order to simulate ships crashing into one other, we applied collision detection to the 3D objects which detect interaction when 3D objects come in contact with other 3D objects.

Because the scenario mimics real world conditions, it can be difficult to see objects that are in the distance. To tackle this problem we used the game engines ability to shift between multiple points of view, for instance, between first person user perspectives and third person overview perspectives.

The Process: Physical Modelling and Scenarios

In the first collaborative design session we invited professional participants working in the Oslo fjord, including former captains of vessels in the area, vessel traffic operators, pilots, and the Norwegian Coastal Administration. To facilitate the discussion we used physical boat models on top of digital maps and AIS information on a Microsoft SUR40 touch table. The participants accessed the models when explaining existing and possible future events by adding and moving models representing the ships (Figure 1). More ship traffic was added in the scenarios and the discussion continued on regulations and the concern of traffic density in the area.

Figure 1. Frame from a video showing the pilot moving the ship models.

Prior to the second session, a simulation and visualisation of the scenarios from the first session, using a game engine, were developed by the designer who acted as a facilitator. The simulation and visualization used the actual scales between the ship models and landscape; the ships had hydrodynamic properties allowing them to be simulated with individual navigation paths. The scenarios evolved in an iterative process where participant feedback was implemented in the simulation and visualisation. At first we simulated eight ferries and the crossing cruise ship and bulk transportation. The traffic pattern triggered discussion about collision scenarios (Figure 2).

Figure 2. Frame from a ship navigation simulation.
This part of the simulated scenario shows the ferry and concentration of other ship traffic.

A scenario was devised where one of the ferries had a blackout and a cruise ship had to avoid collision, and where there was the potential of hitting a passing bulk carrier. Distance between vessels is deceptive on the ocean but also in apparent range. This is because speed, currents, and weather are more forceful than they seem. The scenario was discussed in the group as to what type of factors might lead to such a situation (Figure 3). By changing the surrounding ships’ movements we added new factors in the scenario that improved or degraded the situation. The free movement camera option in the 3D tool was used when the simulation was running, and, at the same time, new ships were added or moved to implicate the simulation. In the group, one captain argued that ferries are more likely to end up in dangerous situations or breach regulations because they know they are more manoeuvrable than big cruise ships or bulk ships.

Figure 3. Frame from a collaborative design session video that shows the crash scenario
being facilitated by the designer closest to the screen.

In the final iteration we created an accident scenario based on the previous collision between the cruise ship and the bulk ship already visualised. At this point we added a rescue helicopter, leisure boats, passengers, and crew to materialise how the situation could evolve. This scenario was not iterated in the way previous ones regarding navigation were. However, this situation stimulated discussion about how the simulation itself could be used to coordinate possible SAR events in a collaborative setting with live data.

A number of different possible risk and accident scenarios were discussed in the collaborative design sessions. New ideas were developed that included the development of automatic docking systems, new collision alarm systems, new pilot training programmes, and the need for research about crisis management.

Case Findings

There were a number of findings from this case concerning relations between use of tools, collaboration, and reflection in and on action:

The tangible interface had limitations related to scale and visualizing the scenarios from the user perspectives on the ships.
In using the game engine to facilitate the discussion we were able to create an accurate scale between objects and ‘landscape.’
Simulating ship physics and applying AI behaviour made it possible to approximately mimic speeds and movement.
Being able to change perspectives made it possible to facilitate a proposed scenario which often triggered more questions based on the visualization.
The ability to add or change the scenario during the simulated activity led to the actors providing more precise input on risk factors in the scenarios.

Case Study 2: Offshore Dynamic Positioning in Icy Conditions

In the second case study we focused on design and development in maritime research projects at the Oslo School of Architecture and Design and at the University College of Southeast Norway. Both research projects investigated decision-making support systems for ship navigation. When oil drilling is moving north to latitude 75, challenges arise concerning technology, operational solutions, safety, logistics, personal equipment, accessibility, equipment quality, rig collaboration, winter preparations, and transport (Andersen, 2014), as well as key environmental concerns. The case covered a small collaborative design session with a Dynamic Positioning Operator (DPO), on a supply vessel operation in the polar areas. We explored possible scenarios and reflected on a new ship bridge concept. The research projects required specific feedback on interface and ship design in relation to different operations where ice might be an issue. To do so there was a need to mimic the icy conditions in the north with realistic and accurate models.

The design challenge was to facilitate the design space between the specialist maritime user and the designer. We wanted realistic scenarios to be at the centre of the discussion where the same model could be used to simulate many different scenarios if ideas would arise in the discussion. The simulation allowed us to create scenarios that were not pre-planned but developed and facilitated during discussions with the expert user. To do so, the 3D models needed to be very detailed and expansive so as to relate everything from the contextual situation to interface details in order mimic real world conditions.

In the case study, the game engine had to be able to import 3D objects with over a million polygons and simulate buoyancy on water to mimic real world behaviours so it appeared realistic when used with avatars that the user could use to move in the virtual environment. The detail was required to enable discussions on specifics in the design concept in relation to contextual information, such as on the drill rig. The 3D models needed the same manoeuvring and propulsion functions as a real supply ship, which was added to the ship model using an existing action script from the game engine’s pre-made assets.

Collaborative Design Session Process: Revealing Ice Related Issues

We started the discussion with the DPO on the basics on how he situated the supply vessel in relation to the offshore rig. Our plan was to go through the standard procedures to see where the ship design and ice conditions were potentially critical. The pilot then started to explain how the boat’s new radar system worked. This changed the situated knowledge and direction of the imagined development as the new system forced us to engage with the core area of navigation, tools, and readability. We moved the editor view to the radar system displayed in the Ulstein Bridge Vision concept (see Figure 4), and the pilot showed how their new radar covered 4 sectors, as opposed to one radar with blind zones, and how that was beneficial because it was possible to get a better overview in areas with ice.

Figure 4. Collaborative design session with an offshore supply ship pilot.
The pilot explains the radar system concerning positioning of a vessel related to a rig and ice.

The pilot then noticed the Head-Up Display (HUD) display on the glass of the Ulstein concept bridge. He said that he was curious how that would work in direct sunlight (Figure 5), and reflected that, “This can be solved with the sun protection film that we use anyway when we have direct sunlight.” He then asked if we could make the environment conditions foggier and add night conditions.

Figure 5. Frame from a session video showing how the HUD reacted when changing the sunlight positions.

When adding the fog conditions, the DPO argued that there is a problem with GPS positioning above latitude 75, and that this would be a problem when approaching rigs in conditions with limited vision. He said that they use the radar sometimes within a 500m limit in such foggy conditions, until securing a visual check of the rig. However, some of these procedures might need to be changed in the icy conditions of the far north. The discussion continued about how the DPO would approach the rig with the vessel; the designer changed the scenario according to the DPO’s instructions. When moving the camera past the rig rescue boats, a question arose about their relevance. The DPO argued that common procedure is to call the rig to let it know if one is in front of the lifeboats, which is an area one tries not to be in.

The DPO argued that the wind is a critical element that is always given close attention. We then added a 10 m/s wind to the scenario, and we could see the vessel starting to move toward the rig. The DPO argued that he always tries to have the same heading as the rig and stay on the off drift areas to prevent collision should they lose control of the boat. It is also important to keep an eye on the anchor chains (Figure 6). These various needs and conditions contain complex dynamic components, intersecting technical and knowledge systems, and potentially changing seas, and what are often evolving scenarios that may entail partly unpredictable human actions.

Figure 6. Frame from a video showing the DPO describing the anchor chain risk.

When the vessel was then positioned 10 meters from the rig, the DPO asked if we could see the situation from the DP console position. The view was changed accordingly, and the DPO argued that the ship chimney was placed in such a position that he would have problems seeing if the rig vent was open if a bulk transfer (water, mud, gas, fuel) was to be effected. We also simulated the crew position when connecting the transfer tubes.

The discussion then continued to the helicopter landing procedures and helicopter ditching rescue operations. A scenario was created with a helicopter in the water, and we added a Man Overboard Boat (MOB) as a rescue boat. We then simulated the MOB approach delivering wounded workers back to the supply ship. The DP operator explained how he would protect the MOB from wind and waves on the leeward side of his vessel, and that he would use the DP to keep stable and stationary.

We then talked about icebergs as a risk factor, and placed an iceberg close to the rig. The DPO argued that icebergs could be plotted in the radar to see if they are on a collision course with the rig. If so, we would need to try and change the course of it or try to break it up. If an iceberg was less than one metre thick, he said they would try to break it up with the aft section of the ship. We then re-enacted this situation. A new question about icing problems then arose and along with it a need to de-ice; we continued the discussion on how to carry out de-icing of the supply vessel and rig. This may be a new service needed in the polar areas much like airplane de-icing.

At one point the DPO was testing the avatar mode of the game engine and accidentally jumped over board. This led to discussion about ‘man overboard’ procedures, and what the DPO should do if there is a possibility of someone being sucked into the propellers. In these events, the scenarios and user co-designed simulation based responses, and each level of work and related scenarios together presented additional needs and perceptions, and also indicated considerable scope for further development.

Case Findings

Through this case we found that:

The DPO was able to use the visualization to explain operation and the risks and challenges.
The designer was able to visualize most of the discussion during the design session. However not all details could be added in real-time, such as the bulk tubes or cargo loading.
A new scenario was created based on the discussion, namely the helicopter crash.
Some scenarios were simulated using vessel moment (physics), vessel lights, and use of MOB in emergency scenarios (‘human-in-the-loop’).
New challenges and questions came up, based on the visualization and simulation, such as the helicopter rescue scenario.
New discussions arose when the simulation did not go as planned, such as when the DPO jumped in the water with the avatar.
Real-time manipulations of direct sunlight could be applied to discuss the HUD display.
No pre-made calculations had been done on fog density input to view distance; this was a problem when the DPO requested a specific view distance.

Overall, this case found that the game engine provided means to visualize and model scenarios for offshore operations that expert users could relate to and where they could apply their experience. This method provided insights at a basic operational level, moreover, the DPO thought that this was a good way to focus not only on how they steer the ship but also on issues and factors that create the basis of human decisions. Simulating factors like wind, fog, and ice, in combination with small action events provided more in-depth knowledge on special situations. Usually, new conversations started after finding interest points in existing discussions describing equipment or situations of use in detail, such as when the DPO described how the mariners look for steam from the vent tubes under the ring when starting bulk transfer. If the vent tubes had not been visualised in the 3D model, the topic would probably not have been discussed.

Case Study 3: Light Systems on Offshore Helidecks

Critical Visuals

The third case dealt with a new light system for helidecks on semi-submersible drill rigs. The design goal was to simulate the light system in action in a helicopter landing scenario. The scenario was developed to explore and reveal how users might interact with the new systems. The contextual co-design and user-informed design challenge was to review the proposed light concept with helicopter pilots during the landing and boarding process. Factors like candela values, fog density, and view distance were important issues. We found no mathematical simulation system that could model all the factors in such a simulation, and that the users’ subjective impression of light will change with age. The light system is currently tested in a lab to meet the required candela specification, however such tests do not give an impression as to how the light will behave offshore.

For the system functionality of the helideck light system to be simulated, the 3D models had to be connected to a system that triggered functionality by turning a light on or off. Behaviours from the simulated users (AI) and helicopter landings were used as triggering mechanisms. The game engine allows for such programming through the game editor interface. This made it possible to explore and reveal time-based factors related to the product interaction. The scenario and product could be changed during the design session making it possible to explore interaction challenges and possibilities.

The Process: Helicopter Landing

The process started with a meeting with LysTech, where we agreed on carrying out a small test that visualized and simulated a helicopter approaching the rig at night using the rig’s light system (Figure 7). A video of the simulation was developed (and subsequently used in a meeting with a shipyard in Korea) as the first visualization of the system in use. It was then decided to use this simulation and visualization to facilitate a product meeting between the helicopter service and the LysTech company in Norway.

Figure 7. Frame from a simulation of a helicopter approaching a rig helideck at night.
The LysTech light system is shown by the yellow circle and green “H”.

In addition to the view of pilots and to extend our insights prior to talking more with pilots, and to understand passengers’ experience and needs, we held a small collaborative design session with an experienced seaman who takes helicopters to work out on the rig. The idea was to obtain extended insight into the experience of landing on the helideck as an expert passenger who has been on numerous visits. Using the game engine as means to facilitate discussion, we went through the different steps of the landing process. This participant argued that it is possible to be confused by all the noise, notably wind, and the blinding lights of the rig. This is especially so for new workers with little experience of the landing and working conditions.

In response, a new gate and living quarter lead light system was visualised and introduced; the user thought this was a good idea. We discussed how the light should look and where it should be placed on railings that are critical to navigating between the helicopter, deck spaces, and entry points to interiors of the rig. Our real-time light visualization and simulation presented direct representation on how the system would look; different solutions were developed and discussed with the expert traveller (Figure 8). These solutions included variations of light placement and colours, and how this system could be part of a safety video before helicopter take-off prior to going offshore.

Figure 8. Frame from a video. The user explains where the light tubes should be placed on the helideck gate
so that traveller-workers can see the steps.

Based on this design session, we developed a scenario implementing different elements of the product. This was to be presented at an existing product meeting in Norway with helicopter pilots (users), suppliers, and customers from CHC Helicopter Service, Statoil, Marine Aluminium, Bayards, and Frictape. The scenario we demonstrated included a series of different events starting with the helicopter landing, a passenger walking to living quarters, and an emergency evacuation event. The same scenario simulation was run in different weather conditions. This is crucial in this professional work setting where climate variation is at times considerable, and when weather conditions are extreme.

This intervention via simulated scenarios started discussions on how the light can help a pilot to see his or her altitude in relation to the helideck when approaching. This included deck texture, better visibility concerning fog, ice issues, and matters of fuel draining on the helideck in case of a fuel spill. One of the four helicopter pilots to this session argued that it would have been useful to use the lights to see whether or not the helideck was ready for landing. Our real-time engine allowed us to respond to this and to visualise this suggestion immediately by using red lights around the already demarcated circle shape of the helipad (Figure 9). During the meeting, CHC Helicopter Service decided to test the light system on a test-helideck at Sola airport and to submit it to wider user testing.

Figure 9. Screen capture from a game engine.
The circle of lights was changed during the user meeting to visualize the helicopter pilot’s idea about light colour codes.

Case Findings

Through this case we found that:

By using the game engine to visualize and simulate the light system in user scenarios, participants in the meeting and design sessions were able to understand the system and its function in different situations.
It was more difficult to create new scenarios during the meeting with 16 participants rather than the small collaborative design session.
Using AI behaviours allowed us to trigger systems that motivated new events. This enabled us to study detailed product functions, such as if the lead lights should turn on when the helideck gate is opening or after it has opened (this was not identified as an issue before the simulation though).
Real time visualization and simulation made it possible for the designer and actors to reflect in action and modify the scenarios to continue the iteration.

Through this case we were able to visualize and simulate the behaviour of a system being designed. When using the game engine in a collaborative design session with specialists, we were able to discover critical factors and create solutions to meet them.

Analysis and Discussion

Abstraction, Actualisation, Projection

Traditional design practice argues that abstraction should be kept to a level that maintains attention on specific design issues. However, there is little research on trying to make design models incorporate the complexity of contexts of intersecting systems and simultaneous and emergent user needs. In our cases, attention to complexity enabled more exploration that allowed us to address new questions.6 One example was the lifeboat scenario from Case Two. If we had not integrated the lifeboats into the model, the questions about their relation to the vessel operation would probably not have been discussed. The real-time simulation and visualization capabilities of the game engine were the momentum that allowed this type of interaction.

The use of real-time simulation of physics and AI brings life and time into scenarios. Modelling scenarios, with and without a ‘human-in-the-loop,’ and where new events were shaped based on the unknown simulation outputs, drew on the notion of reflection-in-action as part of collaboration and design thinking. However, we do not see these simulations as a test of physics or usability, but as an explorative approach that reveals new factors for investigating and tackling ill-defined and wicked problems. In Case Three, placing the light cable on the railing was thought to be free of serious issues; however, the user informant argued that on ships these rails are often used to fasten ropes! In the simulation space, by changing the position of the light tubes, we discovered that we could light up the deck as well as steps, thereby providing an added safety feature.

All types of simulations deal with issues of accuracy and verisimilitude. Through our cases we found that we can approach these issues from two angles: a) by mathematics and b) via subjective situated user feedback. Using mathematics to describe a ship’s speed, we can measure how fast a ship will complete one nautical mile. When doing the helideck light simulation, it was difficult to use a mathematical model because the light experiences are affected by several intersecting conditions, and humans’ perception of light is different according to age and changing weather properties. A key element when using simulation via subjective situated user participation is that unexpected events happen. When the DPO was controlling the avatar in the Case Two, he suddenly jumped into the water. This was clearly not something that was planned; however, the avatar itself had built in swimming capabilities that allowed it to be controlled when in the water. Such scenarios may not merely help us to look into actual work and safety matters, or indeed experience unexpected ones, they may also suggest ways to look into a wider, complex, and merging set of conditions, people, and actions that we cannot always appreciate or see holistically while immersed in safety critical work.

Reflecting on the Game Engine

Our focus on the game engine has been to investigate what support it allows in tackling design challenges that designers experience in the front-end of innovation. What is needed in this design space is something that allows for reflection-in-action on both an individual and a collective level where designers, actors, and users can participate. However, this reflection is also based on modifying the scenarios in an iterative process of reflection-in-action. Each case study showed that participants were able to apply their knowledge and experiences to the related scenarios in order to modify new iterations of events. This alone does not guarantee collective understanding between designers, actors and users. A discursive process around the simulated environment and operations is therefore needed to support the facilitation in a design group involved in exploratory designing.

The main difference between the use of game engines and traditional scenario development techniques—staged plays sessions (Simsarian, 2003), storytelling (Lerdahl, 2001), exploratory design games (Brandt, 2006, p. 59), experience prototyping (Buchenau & Suri, 2000), and visual storytelling (Buxton, 2007, p. 277)—is the ability to combine several types of media on the same platform, and to apply behaviours to objects and systems that can be simulated real time in context. However, there are elements of existing scenario methods that have advantages over game engines, for example, the time used to construct the scenario, and fine-tuned use situations where it is not possible to ask the AI character itself how it experienced a situation.

The interface of the CryEngine tool used in the cases has come a long way in relation to being suitable for adoption by designers with basic knowledge about 3D software and programming. However, many designers are most likely not going to become expert programmers at the level that is needed in order to design code for custom entities. A team including both a designer and programmer might be preferable, but again this may make the workflow much more complicated and the tool may lose its qualities as a design tool. Further work is needed into these relations.

Another question is what a simulation constitutes in the sense of a scenario. It is possible to view the simulation on two levels. One is the traditional mathematical system approach that is calculated based on formulas. For example, this is the physics system and the light system in the game engine. The other level of simulation is the behaviour of AI characters and objects. There is a thin border between visualization and simulation here. In a scenario simulation the relation between behaviour, action, and time is critical.

Interesting aspects that arise with scenario-centred computational simulation include the possibility to investigate, experiment, reveal, and explore very complex situations and activities. Two examples show how issues of perceived complexity outweighing situated knowledge may be countered via design, centred on and realised through exploration, with game engines and simulation. First, when we used the physical artifacts in our first collaborative design session with the Norwegian Coastal Administration when discussing crisis management in the Oslo fjord, at one point in the design session, one of the pilots said, “There is a collision alarm on the cruise bridge that is automatically turned off when approaching this area.” The event described was of huge interest for further investigation, but the physical models did not allow us to go inside the cruise bridge to investigate the alarm system. This gave us the idea of bringing in full-scale bridge simulators into these collaborative design sessions to look at specific interface and navigation issues.

Second, when using the game engine to facilitate the discussion of supply ships in Case Two, the ability to instantly change the perspective view based on discussion input took the discursive process into a new dimension where relativity of scale could be presented and achieved. However, it is critical that a designer who has expert knowledge of the tool is the one facilitating the tool-mediational process, while responding to insights and direction from the expert user/s. It is a considerable challenge to use the game engine interface in interacting with and modifying the scenarios. These case based examples should not be seen to suggest that this is a simple process or activity, however, they reveal considerable potential for further inquiry.

Conclusions

We see that simulated scenarios provide a very powerful means to approach maritime design complexity because they provide a systematic connection of interaction between users, system, and operational factors. Traditional UCD scenario techniques lack the ability to handle complexity issues typical in maritime contexts and to efficiently visualise them. Examples of this occur in micro and macro operational levels of user interaction and overall operational implications. Several systems might need to be engaged to see how they influence a situation in a wider perspective. This requires time-based design tools that can materialise a situation and make it accessible to be modelled and analysed by designers. Different simulation techniques offer the possibility of mimicking real world physics conditions. By simulating trigger mechanisms and behaviours in time-base scenarios it is possible to construct user situations that interact with the contextual conditions. The result of this is that design processes may become more immersed into ‘design in use time’ related to context.

Game engines as design tools can be used to model user scenarios in complex systems; they also facilitate the possibility for multiple behaviours to be simulated at the same time. This method enables the designer to shape scenarios with great complexity. These scenarios can be used to foster reflection-in-action on the part of designers, actors, and users. By using real-time visualization technology in combination with pre-made AI models and physics scripts applied to designed 3D objects, it is possible to create simulation of instances of users interacting with systems and adjusting them ‘en route.’ These approaches offer designers a new role in development where scenarios are critical in order to understand relations between users, systems, and operation in the maritime sector. Consequently, the designer (or design team) has a central and important role in design sessions in linking the scenario, tool, and methods. It is the designer who connects the design elements of systems, operations, and technology user interactions, and combines them in relation to context and situations. This applies not only for UCD related design issues, but also technical engineering related problems. We believe that this orientation of the designer and team may also likely result in a more user-focused design processes than generally appears in maritime design. This might help to reduce human failure as a main cause of accidents.

Using game engines and simulation has allowed us to explore situation and settings in front end stages of design processes, where we could investigate ideation and mediate them through a tool with users and in relation to complex use for the maritime sector.7 We see that in the conceptual phases of design there is a need to create a more convergent perspective on use situation in relation to design of products, interactions, and systems, and that simulations using game engines can assist in this. The users in Case Two expressed this during their design session with us. The dynamic process operator user has hundreds of hours of simulator experience. It was because we used a designer to facilitate the model and simulation interaction that the user did not need to focus on the navigation and handling of the offshore supply-ship; instead, he could focus on the overall operation.

Similarly, the scenario mediated collaborative design sessions developed into a form of revelation in and through the processes of intersection between the content and expert knowledge, the capacities of the tool, and the dialogue between participants. When we explored and tested scenarios and new events, or needs emerged that needed attention, we again found that these could be modelled and simulated. Based on our observations, we argue that in the conceptual phases of design there is a need to create a more convergent perspective on use situation in relation to design of products, interactions, and systems, and that simulations using game engines can help achieve this. Concerning the design tool properties of the game engine, the free camera views enabled unhindered navigation through the scenarios by allowing a person and a team of persons to look at different aspects or options, and enabled them to focus on micro and macro levels of an activity or event.8 We see advantages of using game engines as design tools and materials beyond the maritime sector in contexts of similar complexity. In this respect, there is further room for design-driven innovation to add insights to the current body of research and contribute to design through simulation.

Acknowledgements

The authors would like to thank the University College of Southeast Norway and the Oslo School of Architecture and Design for financing the research. We would also like to thank the companies involved in the case studies: Kongsberg Maritime, Kongsberg Norcontrol, NMU, The Norwegian Coastal Administration, LysTech, and CHC Helicopter Service. In addition, we would also like to give a special thanks to the offshore ship officer Robin Hjertvik that works as a dynamic positioning operator in the North Sea.

Endnotes

1. Different types of user scenario methods are often used in UCD processes when designing for user experience, such as staged play sessions (Simsarian, 2003), storytelling (Lerdahl, 2001), exploratory design games (Brandt, 2006, p. 59) and experience prototyping (Buchenau & Suri, 2000). Through visual storytelling (Buxton, 2007, p. 277), designers can visualize sequences or animations of the user, use, objects, behaviours, events and interaction, in context and over time. This enables the designer to explore the different relations between factors and help frame and re-frame problems.

2. Central to this is the concepts of reflection on and in action (Schön, 1983, 1987). Research by design, offers a way to combine research on reflection in action with what is being designed and to simultaneously relate this reflexively to design theory and analysis, in and through participation. These approaches allowed us to explore several aspects of the simulation tool and its relations in design practice, and the role and potential of real-time adjustment.

3.Through collaborative design sessions with these various partners we generated ideas and concepts based on the use of simulated scenarios and game engines. The technical process of constructing the scenarios dealt with modelling the context, objects and their behaviours. Scenario events of existing or future situations were then constructed in collaboration with users or the industry partners. This gave us insight into how our approach functioned as a design tool in a commercial setting, knowledge that is needed if we are to understand how to tackle the design challenges described above. The collaborative design sessions were documented using video-recording (Iversen & Buur, 2003). The first author of this article was actively involved modelling scenarios, simulations and facilitated the collaborative design sessions using participatory action research methods (Cahill, 2007).

4.Each of the cases addresses a specific need in exploring an individual design space in relation to use and situation. While differing, these cases together provided us with a spread of experience in design and contextual responses from a range of participants and users that would allow us to heuristically discuss the application and development of game engines and simulation in early concept phase development.

5.Accidents are often triggered by a series of events and it is important to have an overview of the situation before, during and after it being played out. Accordingly, we needed to investigate factors before an accident, the accident moment, subsequent search and rescue operation (SAR), as well as considering factors such as how to minimise pollution from oil spills and wider safety processes. The design challenge was how to create a platform to share knowledge and experience, and generate new scenarios and ideas. To do so, there was a need to implement expert knowledge and get expert parties to share their tacit knowledge and experiences.

6.The three cases show how the game engine has been used in different design, scenario and use settings. They also indicate the application of game engines to augment what is typically on offer in looking forwards to meet and anticipate some of the safety and critical needs in the maritime sector. Each case had the basic need to understand processes and saw the value of spaces for collaborative discursive engagement and resulting modifications and new designs and achieved this through co-design activity. We found that the game engine has the capacity to be applied to model user scenarios that occur at sea and that can be simulated in a temporal sequence with object behaviour and interaction.

7.Earlier research has found that tools like game engines require expert knowledge, typically from HCI. However, recent trends from other domains of interaction design and industry contexts of use show that designers are adopting game engines and coding to design interactive games, interfaces and systems. We have shown how a designer is able to use knowledge and skills from CAD applications when approaching the game engine as a design tool. The results indicate that user scenarios can be modelled and simulated without first hand expert knowledge about computer coding. It is the intuitive editor interfaces in the game engine that allow for an effective and communicative workflow to be achieved and conveyed to others in dynamic and dialogical settings of work and need. Naturally there is room for further access to expert coders and for teams that include a mix of designers and programmers. In our cases, scenario input was modelled by way of a discursive process together with specialists and users. The real-time technology allowed us to change the simulation model when the simulation was running. The immersive capabilities of the real-time technology created a fast workflow in an iterative process where the game engine could be used as part of a design session setting. When decisions on concepts had to be made, the simulated scenarios were used to create shared understanding between the designer, actors and users.

8.Being able to change mode view had huge advantages when facilitating the design sessions. It gave the scenario a better flow between the factors involved in the details in the design and how the design related to the overall operation scenario.

References

Andersen, I. (2014, February 21). 23. Konsesjonsrunde. Dette er de største utfordringene med oljeutvinning i Barentshavet [23th Concession-round. These are the biggest challenges with oil extraction in the Barents Sea]. Teknisk Ukeblad. Retrieved April 4, 2014, from, http://www.tu.no/petroleum/2014/02/21/dette-er-de-storste-utfordringene-med-oljeutvinning-i-barentshavet.
Avison, D. (1997). Action research in information systems. In G. McKenzie, J. Powell, & R. Usher (Eds.), Understanding Social Research: Perspectives on Methodology and Practice (pp. 196-209). London, UK: The Falmer Press.
Banks, C. (2011). What is modeling and simulation? In J. Sokolowski & C. Banks (Eds.), Principles of modeling and simulation: A multidisciplinary approach (pp. 3-24). Hoboken, NJ: John Wiley & Sons.
Barnett, M., Gatfield, D., & Pekcan, C. (2003). A research agenda in maritime crew resource management. In Proceedings of the International Conference on Team Resource Management in the 21st Century. Daytona Beach, FL: Embry-Riddle Aeronautical University.
Beckman, S., & Barry, M. (2009). Design and innovation through storytelling. International Journal of Innovation Science, 1(4), 151-160.
Bjørneseth, F., Dunlop, M., & Strand, J. (2008). Dynamic positioning systems: Usability and interaction styles. In K. Tollmar & B. Jönsson (Eds.), Proceedings of the 5th Nordic Conference on Human-Computer Interaction. (pp. 43-52). New York, NY: ACM Press.
Brandt, E. (2006). Designing exploratory design games: A framework for participation in participatory design? In G. Jacucci & F. Kensing (Eds.), Proceedings of the 9th Conference on Participatory Design (pp. 57-66). New York, NY: ACM Press.
Bratteteig, T., & Wagner, I. (2010). Spaces for participatory creativity. In T. Robertson, K. Bødker, T. Bratteteig, & D. Loi (Eds.), Proceedings of the 11th Conference on Participatory Design (pp. 51-60). New York, NY: ACM Press.
Büscher, M., Eriksen, M. A., Kristensen, J., & Mogensen, P. (2004). Ways of grounding imagination. In A. Clement & P. van den Besselaar (Eds.), Proceedings of the 8th Conference on Participatory Design (pp. 193-203). New York, NY: ACM Press.
Buchenau, M., & Suri, J. (2000). Experience prototyping. In D. Bouyarski & W. Kellogg (Eds.), Proceedings of the 3rd Conference on Designing Interactive Systems (pp. 424-433). New York, NY: ACM Press.
Buur, J., & Larsen, H. (2010). The quality of conversations in participatory innovation.CoDesign, 6(3), 121-138.
Buxton, B. (2007). Sketching user experiences: Ggetting the design right and the right design. Amsterdam, the Netherlands: Elsevier.
Bødker, S. (2000). Scenarios in user-centred design–Setting the stage for reflection and action. Interacting with Computers, 13(1), 61-75.
Cahill, C. (2007). Including excluded perspectives in participatory action research. Design Studies, 28(3), 325-340.
Carroll, J. (2000). Making use: Scenario-based design of human-computer interactions. Cambridge, MA: The MIT Press.
Dalsgaard, P., & Halskov, K. (2010). Innovation in participatory design. In T. Robertson, K. Bødker, T. Bratteteig, & D. Loi (Eds.), Proceedings of the 11th Conference on Participatory Design (pp. 281-282). New York, NY: ACM Press.
Ehn, P. (2008). Participation in design things. In D. Hakken, J. Simonsen & T. Robertson (Eds.), Proceedings of the 10th Conference on Participatory Design (pp. 92-101). New York, NY: ACM Press.
Grech, M., Horberry, T., & Smith, A. (2002). Human error in maritime operations: Analyses of accident reports using the Leximancer tool. In Proceedings of the 46th Human Factors and Ergonomics Society Annual Meeting (pp. 1718-1721). Thousand Oaks, CA: Sage.
Gül, L., & Maher, M. (2009). Co-creating external design representations: Comparing face-to-face sketching to designing in virtual environments. CoDesign, 5(2), 117-138.
Hollingsworth, S. (Ed.). (1997). International action research. London, UK: The Falmer Press.
Humphreys, T., Leung, L., & Weakley, A. (2008). Embedding expert users in the interaction design process: A case study. Design Studies, 29(6), 603-622.
Iversen, O., & Buur, J. (2003). User centred design through the keyhole: Video design case. In M. Rauterberg, M. Menozzi, & J. Wesso (Eds.), Proceedings of IFIP TC 13th International Conference on Human-Computer Interaction (pp. 431-438). Amsterdam, the Netherlands: IOS Press.
Jacucci, G., & Wagner, I. (2007). Performative roles of materiality for collective creativity. In B. Shneiderman, G. Fischer, E. Giaccardi, & M. Eisenberg (Eds.), Proceedings of the 6th SIGCHI Conference on Creativity and Cognition (pp. 73-82). New York, NY: ACM Press.
Jenssen, J., & Randoy, T. (2002). Factors that promote innovation in shipping companies. Maritime Policy & Management, 29(2), 119-133.
Johansson, M. (2006). Collaborative sketching–Co-authoring future scenarios with bits and pieces of ethnography. CoDesign, 2(3), 179-189.
Kim, M., & Maher, M. (2008). The impact of tangible user interfaces on spatial cognition during collaborative design. Design Studies, 29(3), 222-253.
Kristiansen, H., & Nordby, K. (2013). Towards a design simulator for offshore ship bridges. W. Rekdalsbakken, R. Bye, & H. Zhang (Eds.), In Proceedings 27th European Conference on Modelling and Simulation (pp. 212-218). Ålesund, Norway: European Council for Modeling and Simulation.
Kumar, S., Hedrick, M., Wiacek, C., & Messner, J. (2011). Developing an experienced-based design review application for healthcare facilities using a 3D game engine. Journal of Information Technology in Construction, 16, 85-104. Retrieved April, 7, 2014 from http://www.itcon.org/2011/6
Kuutti, K., Battarbee, K., Sade, S., Mattelmaki, T., Keinonen, T., Teirikko, T., & Tornberg, A. (2001). Virtual prototypes in usability testing. In Proceedings of the 34th Hawaii International Conference on System Sciences (pp. 5029-5036). Washington, DC: IEEE Computer Society.
Lawson, B., & Dorst, K. (2009). Design expertise. Oxford, UK: Elsevier Architectural Press.
Lerdahl, E. (2001). Staging for creative collaboration in design teams: Models, tools and methods. (Doctoral dissertation). Norwegian University of Science and Technology, Trondheim, Norway.
Manninen, T. (2000). Multimedia game engine as distributed conceptualisation and prototyping tool–Contextual virtual prototyping. In Proceedings of the Conference on Internet, Multimedia Systems and Applications (pp. 99-104). Las Vegas, NV: IASTED/ACTA Press.
Maslin, E. (2013, November 13). Gaming technology goes upstream. Offshore Engineer, 38, 22-24.
McKneely, J., Wallace, D., Perry, A., & Winters, J. (2001). Human systems engineering process and methods. In W. Karwowski (Ed.), International encyclopedia of ergonomics and human factors (pp. 1846-1849). London, UK: Taylor & Francis.
Mills, S. (2006). Integrated marine electronic systems–Some user associated issues for the designer. The Journal of Navigation, 59(03), 423-433.
Miller, N. (1994). Participatory action research: Principles, politics, and possibilities. New Directions for Adult and Continuing Education, 63, 69-80.
Morrison, A., & Sevaldson, B. (2010). ‘Getting going’ – Research by design. FORMakademisk, 3(1). Retrieved April 7, 2014, from https://journals.hioa.no/index.php/formakademisk/article/view/136/132
Möller, T., Haines, E., & Hoffman, N. (2008). Real-time rendering (3rd ed.). Wallesley, MA: A K Peters.
Narayanan, S., & Kidambi, P. (2011). Interactive simulations: History, features, and trends. In L. Rothrock & S. Narayanan (Eds.), Human-in-the-Loop Simulations (pp. 1-13). London, UK: Springer.
Nideffer, R. (2003). Game engines as embedded systems. In V. Vesna (Ed.), Database aesthetics: Art in the age of information overflow (pp. 211-232). Minneapolis, MN: University of Minnesota Press.
Norman, D., & Draper, S. (1986). User centered system design: New perspectives on human-computer interaction. Hillsdale. NJ: Lawrence Erlbaum Associates.
Olson, G., Olson, J., Carter, M., & Storrøsten, M. (1992). Small group design meetings: An analysis of collaboration. Human Computer Interaction, 7(4), 347-374.
Rosson, M., & Carroll, J. (2003). Scenario-based design. In A. Julie & S. Andrew (Eds.), The human-computer interaction handbook (pp. 1032-1050): Mahwah, NJ: Lawrence Erlbaum Associates.
Sanders, L., & Rim, S. (2001). Collective creativity. LOOP: AIGA Journal of Interaction Design Education, 3. 1-6.
Schön, D. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books.
Schön, D. (1987). Educating the reflective practitioner. San Francisco, CA: Jossey-Bass.
Simsarian, K. T. (2003, April). Take it to the next stage: The roles of role playing in the design process. In G. Cockton & P. Korhonen (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Extended Abstracts, pp. 1012-1013). New York, NY: ACM
Susi, T., Johannesson, M., & Backlund, P. (2007). Serious games: An overview (Tech. Rep. No. HS- IKI -TR-07-001). Skövde, Sweden: Skövde Institutionen för kommunikation och information.
Thalen, J., & van der Voort. M. (2012). Facilitating user involvement in product design through virtual reality. In T. Xinxing (Ed.), Virtual reality – Human computer interaction. Rijeka, Croatia: InTech.
Tideman, M., Voort, M., & Houten, F. (2008). A new product design method based on virtual reality, gaming and scenarios. Journal on Interactive Design and Manufacturing, 2(4), 195-205.
Turkle, S. (2009). Simulation and its discontents. Cambridge, MA: The MIT Press.
Visser, F., Stappers, P., van der Lugt, R., & Sanders, E. (2005). Context mapping: Experiences from practice. CoDesign, 1(2), 119-149.
Winsberg, E. (2010). Science in the age of computer simulation. Chicago, IL: University of Chicago Press.
Woohun, L., & Jun, P. (2005). Augmented foam: Aa tangible augmented reality for product design. In B. Werner (Ed.), Proceedings of the 4th International Symposium on Mixed and Augmented Reality (pp. 105-109). Los Alamitos, CA: IEEE Computer Society.
Yin, R. (2009). Case study research: Design and methods. Los Angeles, CA: Sage.