Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges

Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi Place des Doyens, 1 – B-1348 Louvain-la-Neuve (Belgium)

Talk’s outline What is model-based UI design? What is a UI model? What do we need to get a quality UI model? How many models do we need to characterize a UI? From model-based UI design to model-driven UI engineering Three dimensions What are the models? Which language? What is the methodological approach to manipulate them? What is the tool support? Promises Successes vs failures Challenges What can we do today together?

What is the today’s situation regarding user interfaces? Technological aspects of user interfaces progress significantly faster than Software engineering aspects It takes time to develop a user interface with a new device, a new interaction technique It takes more time to develop a toolkit It takes even more time to rely on a model-driven approach Usability engineering aspects New user interfaces are shipped with usability problems because Little or no experience No past, no empirical evidence Empirical experiments require a lot of resources if done carefully [Dragicevic et al.,2004]

What is model-based UI design? Classical approch to developing UIs with a UI builder (IDE) Describe it, prototype it, code it with trial and errors, repeat

What is model-based UI design? UI builders offer only partial solution Desired Interface Table with dynamic data Gantt chart Direct manipulation of widgets UI Builder Solution Window Menu bar Palette (icons) Scrollbars Other widgets: drop-down list, etc.

What is model-based UI design? UI builders cannot produce their own UI (no bootstrapping)

What is model-based UI design? Classical approch to developing UIs Advantages UI is graphical by nature A UI can be Rapidly prototyped Easily modified Visually demonstrated Shortcomings No structured method for layout and programming Selection of widget can be inappropriate Layout of widget can be tedious, repetitive Problem of the spaghetti of callbacks Any UI modification can lead to unstructured design

What is model-based UI design? Model-based UI design consists of Describing a UI in a UI model Relying on this UI model to drive the UI development lifecycle While pursuing the following goals: To provide comprehensive UI development environments To deliver robust development lifecycle support To improve portability To integrate usability with UI development A structure in 3 dimensions Models UsiXML Language Step-wise method Software tools support involves described in Development methodology

What is model-based UI design? What is a model? Webster’s definitions: « One who is employed to display clothes or other merchandise » « A set of plans for a building » (good analogy) « A system of postulates, data, and inferences presented as mathematical description of an entity or state of affairs » Our definition A structured set of plans for developing a UI A system of postulates, data, and inferences presented as a UI description (or specification) that drives the UI development lifecycle

What is model-based UI design? What is a model? A simple example for understanding purposes: a system login Finding the right model abstractions is NOT Replicating code in another way Capturing all physical properties

What is model-based UI design? What is a model? Finding the right model abstractions IS Turning important aspects into UI design options E.g. LabelAlignement instead of absolute coordinates Deriving final properties from these options E.g., compute coordinates from LabelAlignement = {left, right, aligned, centred, justified} LabelAlignement = left ButtonPlacement = TotalEquil

What is model-based UI design? What is a model? Finding the right model abstractions IS Describing UI widgets independently from technology Why? Identification label Input text Push button

What is model-based UI design? What is a model? Because different evolutions time Platform User Environment Language

What is model-based UI design? What is a model? Because different evolutions

What is model-based UI design? What is a model? Because same widget appears in many different technologies Because many different ways to reach the same goal (e.g., selector) Check Box (Windows) BoxArray (Garnet) XmToggleButton (OSF-Motif)

What is model-based UI design? What is a model? In order to derive UIs systematically GrafiXML

What is model-based UI design? What is a model? In order to derive UIs systematically Edit the UsiXML Show attributes A click on the tree highlights the corresponding UsiXML GrafiXML

What is model-based UI design? What do we need to get a quality UI model? Abstractions are always limited, but extensible Plateau effect: Do not introduce too many, otherwise too complex (<> simple) Do not introduce too few, otherwise too simplistic (<> simple) C1: Need to ensure quality properties of a model Completeness Consistency Correction Expressiveness … CUI Model CIO graphicalCIO graphicalContainer graphicalIndividualComponent

What do we have so far? With this first set of abstractions, we go from Final UI (FUI) to Concrete UI (CUI) Consequently, a CUI is independent from Any language: (X)HTML, Java, Visual Basic, C++ Any computing platform: Windows, Linux, MacOS Concrete user Interface S Final user Interface S

And now what? UIs are not only graphical, but could be vocal, tactile, 3D, haptic, an multimodal So, there is a need for more abstraction Abstract Container Abstract Individual Component with Output Abs. Ind. Comp. with Input Abs. Ind. Comp. with Control

What do we have so far? With this second set of abstractions, we go from CUI to Abstract UI (AUI) Consequently, a AUI is independent from Any interaction modality Concrete user Interface S Final user Interface S Abstract user Interface S

And now what? UIs, even if they are abstract, are structured according to the target system as opposed to be user-centered So, there is a need for an abstraction that is computing independent Task and domain models (UML Class Diagram) System Login Enter information Check U+P []>> Enter Username Enter Password ||| Check existence Verify correspondence |||

What do we have so far? With this third set of abstractions, we go from AUI to Task and Domain (T+D) Consequently, a T+D is independent from any implementation Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S

Do we need more models? Users want to determine their path on each platform [Forrester Research,2003]

Do we need more models? Demo FlashiXML

Do we need more models? Multiplicity of contexts of use TV is multi-media family device #1 Family Device Booking notification Everywhere connectivity for simple data exchange Travelling Travel booking site Powerful interface for complex operations Working Multimedia Travel programme Sporting Experience Role Location

Do we need more models? Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest [Grolaux et al.,2002] Demo FlexClock

So what we have now is Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use User S [Calvary et al.,2003] In GrafiXML Platform S Environment S

Multi-platform for Emergency Three platforms Pocket PC Desktop PC Wall Screens

Multi-platform for Emergency Model and method Design the reference screen first Refine the others screens later By applying graceful degradation By applying transformation techniques

Graceful degradation [Florins & Vanderdonckt,2004]

Transformation rules Add all >> Add > << Remove all < Remove >> > << < > < Group box Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 1 Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7

What do we have now Cameleon Reference Framework Final user Interface T Concrete user Interface T Task and Domain T Abstract user Interface T T=Target context of use Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use http://www.plasticity.org UsiXML unsupported model UsiXML supported model User S User T [Calvary et al.,2003] Environment T Reification Abstraction Reflexion Translation Platform S Environment S Platform T

Other challenges raised C2: Need to cover Semantics: metamodel as a UML Class Diagram Syntax: UsiXML language Stylistics: C3: Minimum amount of models C4: Maximum amount of models Root insert search criteria view results aux show item details show list items select item insert criteria submit edit edit criteria submit view results show view details view items show select

Towards model-driven engineering Possible definition MDA is an OMG initiative that proposes to define a set of non-proprietary standards that will specify interoperable technologies with which to realize model-driven development with automated transformations. Not all of these technologies will directly concern the transformation involved in MDA. MDA does not necessarily rely on the UML, but, as a specialized kind of MDD (Model Driven Development), MDA necessarily involves the use of model(s) in development, which entails that at least one modeling language must be used. Any modeling language used in MDA must be described in terms of the MOF language to enable the metadata to be understood in a standard manner, which is a precondition for any activity to perform automated transformation .

Towards model-driven engineering C5: Keep decision decision as SDLC is evolving Support for annotation-based design

Example of preference-based design Goal = to input the temperature of a human body Solution n°1: edit box Solution n°2: list box Solution n°3: drop down list Solution n°4: field with scroll bar Solution n°5: thermometer

How to argue for preference? Use the QOC notation Question? [McLean & Belotti,1998] Criteria 1 Criteria 2 Criteria j Criteria m Option 1 Option 2 Option i Option n = negatively affects = positively affects

Preference example Problem Solution 1 Solution 2 (RE1) (RE2) suggests suggests avoids contradicts respects = for = against (A1) = drop down list requires less space than radio buttons (A2) = some possible values become obsolete when they are infrequently used (A3) = drop down list shows better then current value than the possible values (A4) = radio buttons are faster and easier to manipulate than drop down list (A5) = radio buttons are recommended in Microsoft Windows style guide (A6) = radio buttons do not allow users to change the values drop down list radio buttons (A4) is contradicted by (A2) and (A3) : the widget should be more used for output than input. (A6) is contradicted by (A3) : it is better to present all possible values than only one at a time (A5) is an autority argument , and can fall in front of (A1)+(A2)+(A3) [Vanderdonckt,1997] (A1) (A2) (A3) (A4) (A5) (A6) Standard compatibility (e.g. Windows) Cognitive respect Minimal actions Dialog representativity Prompting Clarity contradicts

Theory of argumentation [Toulmin,1975] [Perelman, 1978] Design problem Design option parameter (parameter, value) Is described by Is solved by Accepted solution Set of admissible solutions Solution 1 Solution 2 Solution n Counter-argument (rebuttal, counter-claim) Argument (ground, typically data) can be rejected because of Can be accepted thanks to Is justified by Design claim Warrant Is reinforced by Qualifier satisfied unsatisfied Is related to corresponds to Is justified by Is qualified by

Towards model-driven engineering C6: Need to support (de)composition Demo FormiXML

Towards Model-Driven Engineering C7: Support for multi-path development Different types of engineering Forward engineering Reverse engineering Lateral engineering Diagonal engineering (with or without shortcuts) Different approaches Top-down Bottom-up Middle-out Different development paths Example: Round-trip engineering = composition of Reification CUI -> FUI Reflexion FUI -> FUI Abstraction FUI -> CUI Reflexion CUI -> CUI

Towards Model-Driven Engineering C8: Support for multi-fidelity Demo SketchiXML

Towards model-driven engineering Other challenges (C13 -> C16) High ceiling Low threshold Wide walls Capabilities Resources (time, experience,…) 100% 50% Ceiling Threshold First generation Second generation MDA CASE tools Interface builders and integrated environments Resource win for applications supported by MDA-compliant tools Resource win for applications supported by first-generation

Towards model-driven engineering Other challenges (C13 -> C16) High ceiling Low threshold Wide walls Capabilities Resources (time, experience,…) 100% 50% Ceiling Threshold First generation Second generation Third generation Integrated Development Environments UI types Walls

Towards model-driven engineering C17: Effort incrementality and C19: rendering engines UsiXML interpreter for Graphical user interfaces: XHTML, XUL, Java, Flash, Tcl-Tk Vocal user interfaces: VoiceXML Multimodal user interfaces: X+V Haptic user interfaces: HaptiXML Support for Distributed user interfaces Mixed-reality user interfaces

Developed by Donatien Grolaux Problem: how to design a UI that takes care of multiple displays? Solution: Migration = DEMIPLAT DistriXML = software architecture for migrating UIs from one platform to another at run-time DistriXML [Grolaux & Vanderdonckt,2005] Pencil Palette Painting Painting tool

Demonstration DistriXML [Grolaux & Vanderdonckt,2005]

Demonstration using two displays from two different computers DistriXML

Example using a Pocket PC DistriXML

UI Migration: DEMIPLAT De tach DistriXML

UI Migration: DEMIPLAT De tach - Mi grate DistriXML

UI Migration: DEMIPLAT De tach - Mi grate - Pl astify DistriXML

UI Migration: DEMIPLAT De tach - Mi grate - Pl astify - At tach DistriXML

This is not a floating toolbar Process DistriXML

This is migration ! Computer B Process Process Computer A DistriXML

Towards model-driven engineering C18: More dynamic aspects Mixed reality Minority report

Conclusion What can be really supported? [Skezely,1996] 0 2000 4000 6000 8000 10000 12000 14000 16000 Word KB KB Parser Query Network ops. UI states Presentation Actions Update Interactions Application logic User interface Generated code Models

Conclusion Successes: Work for very-well defined domains Really fosters collaboration Supports flexibility, maintainability (Belgium) Failures Does not work for very complex, dynamic UIs Takes a lot of resources for Models (some) Method (more) Tool (even more) Support for mnemonics and shortcuts

How to participate? Join the UsiXML Consortium today! Anybody can be a member: observer, participant, developer Send an e-mail to [email_address] and/or « Register » on the web site: for this purpose, provide Your firstname, last name, affiliation, desired level of membership, and motivations Think of your own research in terms of models, e.g., Instead of evaluating a Graphical UI for a certain platform, conduct its evaluation on the corresponding CUI model: it is more general, you will gain more audience Instead of developing a separate tool, make it UsiXML-compliant: you will gain more audience, the results are more largely applicable Output UsiXML session at conferences UsiXML CD-ROM, etc.

How to participate? Develop web services for UsiXML Open positions in UsiXML project (from April 2009) PhD students Internships for MSc students

Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs

Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges

More Related Content

Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges