A Tool-supported Development Process for Bringing Touch Interactions into Interactive Cockpits for Controlling Embedded Critical Systems Arnaud Hamon1,2, Philippe Palanque1, Yannick Deleris2, David Navarre1, Eric Barboni1 1 ICS-IRIT, University Toulouse 3, 118, route de Narbonne, 31062 Toulouse Cedex 9, France {lastname}@irit.fr ABSTRACT Since the early days of aviation, aircraft cockpits have been involving more and more automation providing invaluable in improving navigation accuracy, handling unusual situations, performing pre-planned sets of tasks and gathering always increasing volume of information produced by more and more sophisticated aircraft systems. Despite that long lasting unavoidable tide of automation, the concept of human in the loop remains of prime importance playing a complementary role with respect to automation. However, the number of commands to be triggered, the quantity of information (to be perceived and aggregated by the crew) produced by the many aircraft systems require integration of displays and efficient interaction techniques. Interactive Cockpits as promoted by ARINC 661 specification [2] can be considered as a first significant step in that direction. However, a lot of work remains to be done in order to reach the interaction efficiency currently available in many mass market systems. Interactive cockpits belonging to the class of safety critical systems, development processes and methods used in the mass market industry are not suitable as they usually focus on usability and user experience factors upstaging dependability. This paper presents a tool-supported model-based approach suitable for the development of new interactions techniques dealing on an equal basis with usability and dependability. We demonstrate the possibility to describe touch interaction techniques (as in the Apple iPad for instance) and show its integration in a Navigation Display application. Finally, the paper describes how such an approach integrates in the Airbus interactive cockpits development process. Keywords Tactile interactions, development process, model-based approaches, interactive cockpits INTRODUCTION With the advent of new technologies in everyday life, users are frequently confronted with new ways of interacting with LEAVE BLANK THE LAST 2.5 cm (1”) OF THE LEFT COLUMN ON THE FIRST PAGE FOR THE COPYRIGHT NOTICE. 2 AIRBUS Operations 316 route de Bayonne 31060 Toulouse cedex 9 {Firstname.Lastname}@airbus.com computing systems. Beyond the well-known fun effect concurring to the user experience grail [16], these new interaction technologies aim at increasing the bandwidth between the users and the interactive system. Such an increase of bandwidth can be performed by improving the communication from the computer to the operator for instance by offering sophisticated and integrated visualization techniques making it possible to aggregate large sets of data in meaningful presentations as proposed in [35]. This paper focusses on the other communication channel i.e. from the operator towards the computer by exploiting new interaction techniques in order to improve performance. More precisely the target interaction techniques belong to the recent trend of touch interactions [37]. Beyond the bandwagon effect of touch interfaces, the paper presents why such interfaces are relevant for the command and control interactions in cockpits. However, recent research contributions in the area of Human-Computer Interaction demonstrate that touch interactions decrease usability due to both the device itself (the hand of the user is always in the way between the device and the eyes) and to the lack of consistency between different environments [25]. These usability issues (derived from basic principles in HCI) have been confirmed by empirical studies on touch interactions on the Apple iPad [24] leading to the strong statement from Don Norman “Gestural Interfaces: A Step Backwards In Usability” [25]. Beyond these usability issues, dependability ones have also to be considered. Indeed, classical WIMP [36] interfaces have been standardized for more than 20 years [13] and many development platforms, specification techniques [3] and components-of-the-shelf (COTS) are widely available. These components have thus been thoroughly tested and validated over the years. This is not the case for touch-based interfaces for which no standards are available (beyond the always evolving ones provided by major players such as Microsoft [22] or Apple [1] which are of course conflicting), no dedicated programming environments and no long-term experience to build upon. This ends up with even less dependable interfaces where faults are distributed to the hardware, the operating systems, the interaction drivers and finally the application itself. As we are envisioning the introduction of such interfaces in the domain of cockpits for large civil aircrafts these two aspects of usability and dependability have to be addressed in a systematic way. This paper proposes a contribution for engineering such interaction techniques by proposing a model-based approach (presented in section 3) supporting both reliability and usability making such interactions amenable to the constraints imposed by certification authorities. We detail how this approach can be used for the complete and unambiguous description of touch-based interaction techniques and how such interactions can be tuned to support better usability (section 4). This approach combines different techniques including formal analysis of models, simulation and, in particular, analysis of log data in a model-based environment. The applicability of the approach is demonstrated on a real-size case study (section 5) providing interactions on the Navigation Display (ND). Finally, the paper presents how the approach can be integrated in Airbus development process which involves suppliers’ activities before concluding the paper and highlighting future work (section 6). remained mainly un-centralized i.e. distributed throughout the cockpit. This is something that has changed radically with interactive cockpits applications which have started to replace the glass cockpit. The main reason is that it makes it possible to integrate information from several aircraft systems in a single user interface in the cockpit. This integration is nowadays required in order to allow flight crew to handle the always increasing number of instruments which manage more and more complex information. Such integration takes place through interactive applications featuring graphical input and output devices and interaction techniques as in any other interactive contexts (web applications, games, home entertainment, mobile devices …). THE CONTEXT OF INTERACTIVE COCKPITS AND WHY TOUCH INTERACTIONS CAN HELP Since the early days of aviation, each equipment in the cockpit was offering to pilots an integrated management of an aircraft system. Indeed, each aircraft system (if needed) was proposing an information display (such as a lamp or a dial) and an information input mechanism (typically physical buttons or switches). Such integrated cockpit equipment was offering several advantages such as high coherence (the equipment offers in the same location all the command and display mechanisms related to a given aircraft system) and loosely coupling (failure of one equipment had no effect on the other ones). However, in order to perform the tasks required to fly the aircraft, the crew needs usually to integrate information from several displays and to trigger command in several aircraft systems. With the increase of equipment in the cockpit and the stronger constraints in terms of safety imposed by regulatory authorities such information integration has become more and more demanding to the crew. In the late 70’s the aviation industry developed a new kind of display system integrating multiple displays and known as “glass cockpit”. Using integrated displays it was possible to gather within a single screen a lot of information previously distributed in multiple locations. This generation of glass cockpit uses several displays with electronic technology called CRT. Such CRT screens receive information from aircraft system applications, then process and display this information to crew members. In order to send controls to the aircraft systems, the crew members have to use physical buttons usually made available next to the displays. Control and displays are processed independently (different hardware and software) and without integration. In summary, integration of displays was a first step but commands Figure 1. The interactive cockpit of the Airbus A350 However, even though ARINC 661 specification compliant interfaces have brought to aircraft cockpits the WIMP interaction style that prevailed in PC’s for the last 20 years as aforementioned, post-WIMP interactions can provide additional benefit to aeronautics as it did in the area of mobile interfaces or gaming. Next section presents a study that have been carried out with Airbus Operation in order to assess the potential benefits of touch interactions in the cockpit context and in case of positive results to define processes, notations and tools to go from the requirements to the certification. TOUCH INTERFACES IN A COCKPIT CONTEXT Despite previous researches showing that multi-touch interfaces do not always provide with high level of usability [25] the aeronautic context is so specific only mentioning thorough training of the crew and severe selection process that this type of modality might be considered. Several taxonomies of input modalities have been presented in past such as the ones of Buxton [6], Foley [9], Mackinlay [19], and Lipscomb [18]. Among these existing input modalities, only tactile HMI (on touch screens or tablets) fits the criteria required for integration in a cockpit. Indeed, voice recognition’s rate (~80-95%) is too low even compared to a maximum error rate of 10-5 needed for non-critical systems in cockpits. 3D gestures technologies such as Microsoft Kinect are not mature enough to be implemented as well. Parker et al. [31] demonstrated that the use of mouse is more adapted for touch screens than fingers which introduce unwanted cluttering and ergonomic issues among others. On the other hand, [12] developed a detailed comparison of touch versus pen and suggest combining the two modalities. These pen devices, such as ImpAct [38] or MTPen [33], allow a large variety of innovative interactions: a precise selection technique [5] and page switch [33] for example. However, such devices must be physically linked to the cockpit for safety reasons. On one hand, such a link shall be long enough to allow pilot to interact freely with the display; but on the other hand such a length would allow the link to wind around critical cockpit hardware such as side sticks and throttles. This could cause catastrophic events. Therefore, the use of styli is not appropriate inside cockpits. Finally, [6] does not take into account tangible interfaces such as [32]. Due to similar problems these modalities are not envisioned in the frame of new cockpit HMI. Similarly to [10] creating an adapted set of gestures for diagram editions, we intend to design a set of multi-touch interactions for interactive cockpits. This set of interactions intend to balance ([17]) tactile non-precision for clicking with the use of gestures such as [5] for example. However, compared to [10], the target software environment is regulated with strict certifications detailed later on in the paper. Finally, translating physical controls towards software components will increase their modifiability and the evolutions capabilities of the entire cockpit system. As current interactive cockpit HMI are ARINC 661 based, whose design is adapted for WIMP interaction paradigms (Windows, Icons, Menus, Pointing Device), these HMI rely on such WIMP interaction paradigms. The envisioned change of modality is part of the whole HMI redesign. Indeed changing only the interaction modality without adapting the HMI content and structure would not be satisfying in term of usability and user experience. Regarding systems functionalities, multi-touch in cockpit allows navigating in 2D content or 3D and implement sets of gestures similar to [15], multi-touch curve edition as in [34] for example. A recent survey of tactile HMI is available in [7]; however, the considered tactile interactions in this paper do not include gestures concerning touch-screens like [20]. MODELING TOOL AND IMPLEMENTATION SUITE he IC formalism is a formal description techni ue dedicated to the specification of interactive systems [23]. It uses concepts borrowed from the object-oriented approach (dynamic instantiation, classification, encapsulation, inheritance, client/server relationship) to describe the structural or static aspects of systems, and uses high-level Petri nets to describe their dynamic or behavioral aspects. ICOs are dedicated to the modeling and the implementation of event-driven interfaces, using several communicating objects to model the system, where both behavior of objects and communication protocol between objects are described by the Petri net dialect called Cooperative Objects (CO). In the ICO formalism, an object is an entity featuring four components: a cooperative object which describes the behavior of the object, a presentation part (i.e. the graphical interface), and two functions (the activation function and the rendering function) which make the link between the cooperative object and the presentation part. An ICO specification fully describes the potential interactions that users may have with the application. The specification encompasses both the "input" aspects of the interaction (i.e. how user actions impact on the inner state of the application, and which actions are enabled at any given time) and its "output" aspects (i.e. when and how the application displays information relevant to the user). This formal specification technique has already been applied in the field of ir raffic Control interactive applications [23 , space command and control ground systems [28 , or interactive military [4 or civil cockpits [3]. The ICO notation is fully supported by a CASE tool called PetShop [30] . All the models presented in the next section have been edited and simulated using PetShop. DESIGN, SPECIFICATION AND PROTOTYPING TOUCH INTERACTION TECHNIQUES OF Left-hand side of Figure 2 presents a development process dedicated to the design of interfaces involving dedicated interaction techniques. This process is a refinement of the one presented in [26] exhibiting the issue of deployment of application in a critical context. Due to space constraints, we don’t present the entire process (which can be found in [26]) but focus on the interaction technique design phase (box on the top right-hand side of Figure 2). The goal of this phase is to specify and design gesture models from the description formulated earlier during the development cycle. This phase is the result of iterations over three steps: interaction definition, interaction modeling and analysis of non-nominal cases detailed on the right-hand side of Figure 2. The following paragraphs will explain more precisely these parts of the process. The Process Applied for the Design of a or Interaction Techniques Design and Evaluation Interaction Definition This design phase of the process consists in analyzing high level requirements to define the gestures set and their properties (number of fingers, temporal constraints…). s a lot of work has been devoted in the past on touch-based and gesture interactions exploiting the literature provides information useful for determining usable gestures. However, as explained above, the context of aircraft cockpits is very different from classical environments and requires adapting/modifying them to fit that specific context. Here is an example of an information touch-based interaction technique (as could be seen in any manual describing such interaction). Single finger tap-and-hold is defined as follows: “The pilot touches the screen without moving. A graphical feedback displays the remaining time to press before the tapand-hold is detected. Once the tap-and-hold is detected, the corresponding event is triggered when the pilot lifts his/her finger off the screen.” For such high level and informal descriptions, we were not able to use specific tools: frameworks for high level gesture formalization, such as [14] lack to describe, for example, the quantitative temporal aspects of interactions as well as how to ensure their robustness and, for instance, how to assess that all the possible gestures have been accounted for. In order to solve this problem, the interaction modeling phase aims at formalizing these features in the early phases of the design as well as to ensure a continuity of the process and requirement traceability needed for certification. Interaction modeling This second step consists in refining the high-level gesture definition into a behavioral model. We present here two of the various iterations that have to be performed in order to reach a complete description of the interaction technique. Beyond, the standard interactions i.e. with which the operator make no mistake, the final description also has to cover unintended interaction errors that can be triggered by the user (slips) or due to unfriendly environmental issues (vibrations, low visibility, …) which are common in aeronautics. Figure 3 represents the model describing the behavior of the tap-and-hold interaction technique as previously defined. This models built using the ICO formal description technique does not cover every possible manipulation done by the user and can be considered as a nominal tap-and-hold interaction. Figure 2 - Development process for Interaction specification and implementation Indeed, the constraint such as “without moving” in the description above is very strong and not manageable by the user in case of turbulences or other unstable situations. To improve usability, such constraints have to be relaxed usually by adding thresholds (time, distance ...). This is the case on standard mouse interactions (such as double-click) on a PC. The particularity of this phase is also to already involve system designers to provide interaction designers with operational and aircraft systems specificities that might require specific gestures of adaptation of existing ones. Technical aspects such as number of fingers involved for input, touch properties (pressure, fingers’ angle…) are also defined at that stage. It allows gesture designers to define the most adapted gestures taking the entire cockpit environment into account as well as operational aspects. For instance, gestures might require adaptation according to the various flight phases. This joint work also increases the efficiency of the overall process by setting development actors around the same table and enhancing their comprehension of the overall process of gesture definition as well as to forecast interaction techniques evolution in the cockpit. However, remaining at this high-level of abstraction does not provide enough details to the developer which will in the end lead to leave a lot of choices outside of the design process. The developer will have to make decisions without prior discussion with the designers leading to inconsistent and possibly un-adapted interactions. Figure 3 - Initial tap-and-hold model in ICO From the initial state (one token in place Idle) only transition touchEvent_down is available. This means that the user can only put the finger on the screen, triggering that event from the device. When that event occurs, the transition is fired and the token removed from place Idle and set in place Long_Press. If the token remains in that place more than 50 milliseconds, (condition [50] in transition called validated) that transition automatically fires setting a token in place Validated. In that state, when the user removed the finger from the touch screen, the event toucheventf_up is received, the transition called toucheventf_up_1 is fired triggering the event long_press that can then be interpreted as a command by the application. Right after touching the device and before the 50 milliseconds have elapsed, the user car both move the finger on the device or remove it. In both case the interaction comes back to the initial state and no event long_press is triggered (this behavior is modeled by transition toucheventf_up_2 and toucheventf_move). These are the two means offered to the user in order to cancel the command. Non-nominal cases analysis and modelling A non-nominal case is a flow of user’s events that does not lead to the completion of the intended interaction. This third step consists in analyzing possible user input to determine non-nominal case during the interaction, that can occur due to interaction mistakes (as presented above) or due to conflicts within the detection process when several interactions are competing (possible interference with a simple tap presented in Figure 4). Figure 4. Behavioral model of the simple tap As a user interface is likely to offer several gestures it is important to carefully define their behavior so as they don’t interfere. For instance, the interface should allow the coexistence of both the tap and the tap-and-hold, but if the tap-and-hold detection is detected too early, a simple tap will become impossible to perform. The lower part of the figure represent the feedback time. The figure represents the fact that this graphical feedback only starts after a duration t1 (after the user’s finger has been in contact with the touch device). If all non-nominal cases, timing and graphical constraints have not been covered during the analysis phase, the gesture behavioral model is adapted to refine and make the description comprehensive. The resulting model corresponding to our last iteration is presented in Figure 8. Figure 5 and Figure 7 present two different scenarios covered by our model. In the first scenario, the pilot presses on the device and after a few milliseconds the standard feedback appears. As long as the pilot holds his finger on the touchscreen, the inner radius of the ring decreases until the ring becomes a disk. When the finger moves a little, the ring becomes orange. When the tap_and_hold is completed, a green disk appears under the pilot’s finger. When the pilot removes his finger from the device, the tap_and_hold event Figure 5. First scenario (success) refining the behavior and describing the graphical feedback is generated and forwarded to the upper applicative levels. In the second scenario, after the graphical feedback has appeared, the pilot moves the finger far away from the position where the initial press has occurred and the interaction is missed. Figure 6 - Graphical description of a long press Having two independent models as the ones in Figure 3 and in Figure 4 raises additional problems related to the graphical feedback that the interaction technique has to provide in order to inform the user about the progression of its interaction. Two models would solve the problem related to the difficulty to trigger the tap event but this would produce interfering feedback (as both models will have to provide feedback at the same time even though only either tap or tapand-hold will be triggered. To avoid unwanted fugitive feedback in this case, our design proposes to trigger the tapand-hold feedback after a certain delay. The corresponding specification is presented Figure 6. Allowing the use of high-level events such as the one triggered by the tap-and-hold interaction technique requires the use of a transducer to translate low level touchscreen events into higher level events. There are three main low level touchscreen events (down, up and move) produced when the finger starts or stops touching or moves on the screen. These low level events must be converted into a set of five higher level events corresponding to the different steps while using the tap-and-hold interaction technique (succeed when the gesture detection is completed, sliding while the finger moves within the correct range, too early, when the finger stops touching the screen before the correct delay, outside, when the finger is too far from the initial contact point, and missed when the finger stops touching too far from the initial point. Figure 7. Second scenario (failure) refining the behavior and describing the graphical feedback Figure 8 presents the behavior of such a transducer using the ICO formal description technique which is the result of the last iteration within our design process. Additionally to the behavior of the detection of the correct gesture, such interaction requires some graphical feedback to keep the user aware of the gesture detection. As presented on Figure 5 and Figure 7, several feedbacks are provided to make explicit the different phases of the detection. In an ICO specification, these feedbacks correspond to the rendering related to token movements within Petri net places. As explain at the beginning of this section, no rendering is performed until the first step is reached to avoid unwanted feedback, meaning that there is no rendering associated to the places Idle, Delay, Sliding_1 and Out_1. The feedback of the interaction technique only occurs with the rendering associated to the four places Tap_and_hold (beginning of the progress animation), Sliding_2 (orange progress animation), Out_2 (red circle) and Validated (green circle). Interaction tuning The distinction is to be made between the validity of the behavior model and the tuning of this model. In previous phases of the process, the objective is to define precisely the behavior of the model and both to identify and to define a set of parameters in the models that might require further tuning to produce smooth interactions. At that stage we consider that the behavior will not be altered but only some of its parameters (as for instance the 50 milliseconds aforementioned). Based on the gesture modeling and specification, the gesture model can be tuned using the parameters identified during the previous steps. As for the tap-and-hold touch interaction, the set of parameters that have been identified are summarized in Table 1. When the transducer is in its initial state (represented by the place Idle at the center of the Petri net) it may receive the low level event toucheventf_down (handled by the transition toucheventf_down_1). When received a token is put in the place Delay, representing the beginning of the first waiting phase until the first time threshold is reached. While waiting, two low level events may occur, leading to three cases:  The toucheventf_up event (handled by the transition toucheventf_up_1) that leads to abort the tap-and-hold as the touch has been released too early (before the threshold is reached), triggering the high level event TapAndHold_early) and making the Petri net going back to its initial state.  The toucheventf_move event (handled by the transition toucheventf_move_1) that leads to triggering a high level event TapAndHold_sliding meaning a short move (within the correct range) has been detected but the tap-and-hold interaction is still possible (a token is then put in the place Sliding_1). As a recursive definition, when a token is in the place Slinding_1, the Petri net behaves in the same way as when a token is in the place Delay: o While waiting to reach the first time threshold, any move within the correct range does not modify the state of the Petri net (firing of transition toucheventf_move_2, with respect to its precondition “P0.distance(P) < threshold1” and triggering TapAndHold_sliding event). o Within this delay if the finger moves outside the correct range (firing of transition toucheventf_move_3 and triggering TapAndHold_out event) or a premature low level event touchevent_up occurs (firing of the transition toucheventf_up_2 and triggering TapAndHold_early) the tap-and-hold detection is aborted (in the first case a token is put in place Out_1, waiting for the touch to be released, and in the second case the Petri net returns to its initial state). o While a token is in the place Out_1, toucheventf_move or toucheventf_up events may occur. In the first case, the state of the Petri net does not change (firing of toucheventf_move_4) and in the second case (firing of the transition toucheventf_up_3) a high level event TapAndHold_missed is triggered, while the Petri net returns in its initial state.  If no event occurs within the current delay (firing of transitions Timer_1_1 or Timer_1_2), the beginning of the tap-and-top interaction technique has been detected, leading to put a token in the place Tap-and-hold.  When a token is in the place Tap-and-hold, a second step of the detection begins, with the same behavior as the first one (waiting for the second threshold to be reached, while moves within the correct range are allowed). The behavior remains exactly the same and is not described due to space constraints  If the corresponding token stays within the correct delay in place Tap_and_hold and Sliding_2, the tap-and-hold gesture is validated (firing of transition Timer_2_1 or Timer_2_2) and a token is put in place Validated, making possible to trigger the event TapAndHold_succeed, when the next toucheventf_up event occurs (firing of transition toucheventf_up_6), corresponding to the successful detection of the interaction technique. Figure 8 - Behavior of the touchscreen transducer Table 1 – Tap-and-hold model's parameters Element Type Unit Value R_max Distance Pixels tunable Δt Time ms tunable The right side of Figure 9 represents the symbols used for the heading mode on the ND. The circled symbol corresponds to the heading selected by the pilots. In order to change the selected heading value, the Tap_and_Hold is used to select the current heading symbol directly on the touch screen. Δt1 Time ms tunable CONCLUSIONS AND PERSPECTIVES Returned value Position Pixel^2 Lift From Design to Deployment With the model conversion and implementation phase, this process addresses the following issues: compatibility with existing/future hardware/software environments, link with suppliers and certification authorities. We have explicitly taken into account this activity in our process as it is useless to design interaction in the cockpit that will never be implementable in that context and that will never go through certification. Our formal description technique provides additional benefits such as properties verification [27]. This is not detailed further in the current paper due to space constraints. However, as [29] demonstrated, the process is fully compliant with the safety requirements for cockpits HMI. In addition, transformations of ICO models into SCADE Suite descriptions is on the way for which the code generator is DO178B [8] DAL-A certified which allows implementing models in the CDS easily. CASE STUDY The paper has presented the use of formal description techniques for the specification and prototyping of touchbased interfaces. Beyond the design issues that have been presented in this paper more technical ones might have a strong impact on the deployability in the cockpit of future aircrafts. Indeed, while, ARINC 661 [2] describes a clientserver communication protocol which was developed to facilitate and reduce the costs of the WIMP HMI implementation into cockpits, the use of another modality, such as multi-touch, raises the question of the compatibility of the protocol. Work needs to be done to determine whether or not an extension of the actual A661 is enough or if the current standard has to be modified. Indeed, in the actual architecture, the response time between CDS request and UA answer is too long to provide usable multi-touch interactions and the interactions feedback will not be displayed fast enough. In order to palliate this problem, more responsibilities may be included in the user interface server itself. Currently, the conversion from ICO models to SCADE Suite is done manually. Future work will consist in developing an automated conversion tool to support moving from the design/specification level to implementation. ACKNOWLEDGMENTS This work is partly funded by Airbus under the contract CIFRE PBO D08028747-788/2008 and R&T CNES (National Space Studies Center) Tortuga R-S08/BS-0003029. REFERENCES Figure 9 - A tactile Navigation Display The context of the case study is the development of a new HMI for an existing User Application (UA) of the cockpit. We have considered the Navigation Display (ND) application of the A350. On the display configuration Figure 9, the NDs are circled in orange. The NDs display the horizontal situation of the aircraft on a map. The possible information displayed is weather, traffic information… he Flight Management System (FMS) also sends to the NDs information regarding the aircraft flight plan and flying mode. Then NDs then display the various graphical constructions (routes, waypoints…) according to the FMS data. Figure 9 represents a ND of an aircraft flying in HE DING mode (pilots are commanding the aircraft’s heading). The displays in the A350 are not tactile but we based our work on plausible scenarios such as: “ he pilots have just received a message from an air traffic controller, for a heading modification. The flying pilot needs to execute the order and wants to modify the current heading of the aircraft using the ND touchscreen.” 1. Apple Corp. iOS Human Interface Guidelines. http://developer.apple.com/library/ios/#documentation/UserExperience/ Conceptual/MobileHIG. Date of Access: 04/03/2011 2. ARINC 661-4, Prepared by Airlines Electronic Engineering Committee. Cockpit Display System Interfaces to User Systems. ARINC Specification 661-4; (2010) 3. Barboni E., Conversy S., Navarre D. & Palanque P. Model-Based Engineering of Widgets, User Applications and Servers Compliant with ARINC 661 Specification. 13th conf. on Design Specification and Verification of Interactive Systems (DSVIS 2006), LNCS Springer Verlag. p25-38 4. Bastide R., Navarre D., Palanque P., Schyn A. & Dragicevic P. A Model-Based Approach for Real-Time Embedded Multimodal Systems in Military Aircrafts. Sixth International Conference on Multimodal Interfaces (ICMI'04) October 14-15, 2004, USA, ACM Press. 5. Benko H., Wilson A. & Baudisch P. Precise selection techniques for multi-touch screens. In Proceedings of the SIGCHI conference on Human Factors in computing systems (CHI '06), ACM, New York, NY, USA, 1263-1272. 6. Buxton W.. 1983. Lexical and pragmatic considerations of input structures. SIGGRAPH Comput. Graph. 17, 1 (January 1983), 31-37. 7. Chouvardas, V G, Amalia N Miliou, and M K Hatalis. 2005. Tactile display applications: A state of the art survey. Proc. of the 2nd Balkan Conference in Informatics: 290–303. C 8. DO-178B Software Considerations in Airborne Systems and Equipment Certification: Radio Technical Commission for Aeronautics (RTCA) European Organization for, 1992. 9. Foley, James D., Wallace, V. L. and Chan, Peggy. "The Human Factors of Computer Graphics Interpretation Techniques." IEEE Computer Graphics and Applications 4, no. 11 (1984) 13-48. 10. Frisch M., Heydekorn J. & Dachselt R. Investigating multi-touch and pen gestures for diagram editing on interactive surfaces. ACM Int. Conf. on Interactive Tabletops and Surfaces (ITS '09). ACM, New York, NY, USA, 149-156. 11. Hesselmann T., Boll S. & Heuten W. SCIVA: designing applications for surface computers. 3rd ACM symp. on Engineering interactive computing systems (EICS '11). ACM, New York, NY, USA, 191-196. 12. Hinckley K., Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton. 2010. Pen + touch = new tools. 23rd annual ACM symp. on User interface software and technology (UIST '10). ACM, 27-36. 25. Norman, D. A. (2010). Natural User Interfaces Are Not Natural. Interactions, 17, No. 3 (May - June). 26. Palanque P., Barboni E., Martinie C., Navarre D., Winckler M. A Tool Supported Model-based Approach for Engineering Usability Evaluation of Interaction Techniques. ACM SIGCHI conference Engineering Interactive Computing Systems (EICS 2011), Pisa, Italy, ACM DL. 27. Palanque P. & Bastide R. Verification of an Interactive Software by analysis of its formal specification. IFIP Human-Computer Interaction conference (Interact'95) Norway., 27-29 June 1995, p. 191-197. 28. Palanque P., Bernhaupt R., Navarre D., Ould M. & Winckler M. Supporting Usability Evaluation of Multimodal Man-Machine Interfaces for Space Ground Segment Applications Using Petri net Based Formal Specification. Ninth Int. Conference on Space Operations, Italy, June 18-22, 2006. 13. IBM (1989) Common User Access: Advanced Interface Design Guide. IBM, SC26-4582-0 29. Palanque P., Eric Barboni, Célia Martinie, David Navarre, and Marco Winckler. A model-based approach for supporting engineering usability evaluation of interaction techniques. 3rd ACM SIGCHI symp. on Engineering interactive computing systems (EICS '11). ACM, New York, NY, USA, 21-30. 14. Kammer D., Wojdziak J, Keck M., Groh R. & Taranko S. Towards a formalization of multi-touch gestures. ACM Int. Conf. on Interactive Tabletops and Surfaces (ITS '10). ACM, New York, NY, USA, 49-58. 30. Palanque P., Ladry J-F, Navarre D. & Barboni E. High-Fidelity Prototyping of Interactive Systems can be Formal too 13th Int. Conf. on Human-Computer Interaction (HCI International 2009) LNCS, Springer 15. Kin K., Miller T., Bollensdorff B., DeRose T., Hartmann B. & Agrawala M. Eden: a professional multitouch tool for constructing virtual organic environments. Proc. of conf. on Human factors in computing systems (CHI '11). ACM, USA, 1343-1352. 31. Parker J.K., Mandryk R.L. & Inkpen K. (2006). Integrating Point and Touch for Interaction with Digital Tabletop Displays. IEEE Computer Graphics and Applications, 26(5), pg. 28-35. 16. Lai-Chong Law E., Roto V., Hassenzahl M.. Vermeeren A. & Kort J. Understanding, scoping and defining user experience: a survey approach. 27th international conference on Human factors in computing systems (CHI '09). ACM, New York, NY, USA, 719-728. 17. Lee S. & Zhai S. The performance of touch screen soft buttons. In Proceedings of the 27th international conference on Human factors in computing systems (CHI '09). ACM, New York, NY, USA, 309-318. 18. Lipscomb, James S., and Pique, Michael E. "Analog Input Device Physical Characteristics." SIGCHI Bul. 25, no. 3 (1993) 40-45. 19. Mackinlay, Jock D., Card, Stuart K. and Robertson, George G. "A Semantic Analysis of the Design Space of Input Devices." Human Computer Interaction. Vol. 5. Lawrence Erlbaum (1990) 145-90. 20. Marquardt N., Jota R., Greenberg S. &. Jorge J. The continuous interaction space: interaction techniques unifying touch and gesture on and above a digital surface. 13th IFIP TC 13 international conference on Human-computer interaction -. Springer-Verlag, 461-476. 21. McDermid J. & Ripken K. 1983. Life cycle support in the Ada environment. Ada Lett. III, 1 (July 1983), 57-62. 22. Microsoft Corporation. Microsoft Surface User Experience Guidelines. Available on MSDNAA. 2009. 23. Navarre D., Palanque P., Ladry J-F. & Barboni E. ICOs: A model-based user interface description technique dedicated to interactive systems addressing usability, reliability and scalability. ACM Trans. Comput.Hum. Interact., 16(4), 18:1–18:56. 2009. 24. Nielsen, J. (2010): iPad Usability: First Findings From User Testing. Jakob Nielsen's Alertbox, April 26, 2010 32. Pedersen E.W. & Hornbæk K. Tangible bots: interaction with active tangibles in tabletop interfaces. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 2975-2984. 33. Song H., Benko H., Guimbretiere F., Izadi S., Cao X., & Hinckley K. Grips and gestures on a multi-touch pen. 2011conf. on Human factors in computing systems (CHI '11). ACM, USA, 1323-1332. 34. Sun Q., Fu C-W. & He Y. An interactive multi-touch sketching interface for diffusion curves. In Proc. of the 2011 conf. on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 1611-1614. 35. Tesone D. & Goodall J. Balancing Interactive Data Management of Massive Data with Situational Awareness through Smart Aggregation. IEEE Symp. on Visual Analytics Science and Technology (VAST '07). IEEE, pp. 67-74. 36. van Dam A. 1997. Post-WIMP user interfaces. Commun. ACM 40, 2 (February 1997), 63-67 37. Wang F. and Ren X. 2009. Empirical evaluation for finger input properties in multi-touch interaction. In Proceedings of the 27th international conference on Human factors in computing systems (CHI '09). ACM, New York, NY, USA, 1063-1072. 38. Withana A., Makoto Kondo, Gota Kakehi, Yasutoshi Makino, Maki Sugimoto, and Masahiko Inami. ImpAct: enabling direct touch and manipulation for surface computing. 23rd annual ACM symposium on User interface software and technology (UIST '10). ACM, New York, NY, USA, 411-412.