WAMS: A Flexible API for Visual Workspaces Across Multiple Surfaces
Authors: 
Scott Bateman, Carl Gutwin, Hamid Mansoor, 
Miguel Nacenta, Michael van der Kamp, Mykyta Baliesnyi, Kolton Gagnon, Jesse RollheiserAuthors Info & Claims
Scott Bateman, Carl Gutwin, Hamid Mansoor, Miguel Nacenta, Michael van der Kamp, Mykyta Baliesnyi, Kolton Gagnon, Jesse RollheiserAuthors Info & ClaimsArticle No.: 184, Pages 1 - 40
Abstract
Applications that use multiple devices and surfaces provide new opportunities for innovative interaction -- but despite the wide variety of research that has been carried out on multi-surface systems, building these kinds of applications is still difficult. In particular, multi-surface apps that use interactive visual workspaces are complicated because current tools do not provide low-level access to a connected and interactive graphical canvas that is shown on different devices. This difficulty limits the explorations that designers and developers can carry out within the multi-surface design space. To address this problem, we have developed WAMS -- an open-source web-based toolkit that provides several programming abstractions for building visual-workspace applications across multiple surfaces. WAMS uses three main concepts -- a virtual visual workspace, views onto that workspace, and graphical workspace objects -- and provides support for connecting multiple devices, creating and manipulating objects, managing and laying out views, and handling events from multiple surfaces. WAMS simplifies the development of a wide variety of applications including composite display configurations, shared-workspace groupware, systems that place different UI elements onto different devices, and bring-your-own-device applications. We describe WAMS's main abstractions and concepts, provide several examples that show the breadth of the approach, and assess the toolkit in terms of effectiveness, coverage, extensibility, and integration with existing practices and tools.
Supplementary Material
Supplemental video
- Download
- 164.45 MB
References
[1]
Felix Albertos-Marco, Victor M. Penichet, and Jose A. Gallud. 2016. Virtual Spatially Aware Shared Displays. In ICWE 2016 International Workshops on Current Trends in Web Engineering - Volume 9881. Springer-Verlag, Berlin, Heidelberg, 196--199. https://doi.org/10.1007/978--3--319--46963--8_19
[2]
Dzmitry Aliakseyeu and JBOS Martens. 2006. Sketch Radar: a novel technique for multi-device interaction. In conference; British Computer Society Human Computer Interaction. British HCI group, 45--49.
[3]
Rafael Ballagas, Meredith Ringel, Maureen Stone, and Jan Borchers. 2003. iStuff: a physical user interface toolkit for ubiquitous computing environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). Association for Computing Machinery, New York, NY, USA, 537--544. https://doi.org/10.1145/642611.642705
[4]
Rafael Ballagas, Michael Rohs, and Jennifer G Sheridan. 2005. Sweep and point and shoot: phonecam-based interactions for large public displays. In CHI'05 extended abstracts on Human factors in computing systems. 1200--1203.
[5]
Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment. In ACM International Conference on Interactive Tabletops and Surfaces (Saarbrücken, Germany) (ITS '10). Association for Computing Machinery, New York, NY, USA, 121--130. https://doi.org/10.1145/1936652.1936676
[6]
Jakob Bardram, Sofiane Gueddana, Steven Houben, and Søren Nielsen. 2012. ReticularSpaces: activity-based computing support for physically distributed and collaborative smart spaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2845--2854.
[7]
Patrick Baudisch, Nathaniel Good, Victoria Bellotti, and Pamela Schraedley. 2002. Keeping things in context: a comparative evaluation of focus plus context screens, overviews, and zooming. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). Association for Computing Machinery, New York, NY, USA, 259--266. https://doi.org/10.1145/503376.503423
[8]
Michel Beaudouin-Lafon and Alain Karsenty. 1992. Transparency and awareness in a real-time groupware system. In Proceedings of the 5th annual ACM symposium on User interface software and technology. 171--180.
[9]
Benjamin B Bederson, James D Hollan, Ken Perlin, Jonathan Meyer, David Bacon, and George Furnas. 1996. Pad: A zoomable graphical sketchpad for exploring alternate interface physics. Journal of Visual Languages & Computing, Vol. 7, 1 (1996), 3--32.
[10]
Benjamin B. Bederson, Jon Meyer, and Lance Good. 2000. Jazz: An Extensible Zoomable User Interface Graphics Toolkit in Java. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology (San Diego, California, USA) (UIST '00). Association for Computing Machinery, New York, NY, USA, 171--180. https://doi.org/10.1145/354401.354754
[11]
Andrea Bellucci, Ignacio Aedo, and Paloma Díaz. 2017. ECCE Toolkit: Prototyping Sensor-Based Interaction. Sensors, Vol. 17, 3 (2017). https://doi.org/10.3390/s17030438
[12]
Richard Bentley, Thilo Horstmann, and Jonathan Trevor. 1997. The World Wide Web as enabling technology for CSCW: The case of BSCW. Computer Supported Cooperative Work (CSCW), Vol. 6, 2--3 (1997), 111--134.
[13]
Jacob T Biehl and Brian P Bailey. 2004. ARIS: An Interface for Application Relocation in an Interactive Space. In Graphics Interface, Vol. 2004.
[14]
Jacob T. Biehl, William T. Baker, Brian P. Bailey, Desney S. Tan, Kori M. Inkpen, and Mary Czerwinski. 2008. Impromptu: A New Interaction Framework for Supporting Collaboration in Multiple Display Environments and Its Field Evaluation for Co-Located Software Development. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI '08). Association for Computing Machinery, New York, NY, USA, 939--948. https://doi.org/10.1145/1357054.1357200
[15]
Richard A Bolt. 1980. "Put-that-there" Voice and gesture at the graphics interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques. 262--270.
[16]
Frederik Brudy, Joshua Kevin Budiman, Steven Houben, and Nicolai Marquardt. 2018. Investigating the role of an overview device in multi-device collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--13.
[17]
Frederik Brudy, Christian Holz, Roman Rädle, Chi-Jui Wu, Steven Houben, Clemens Nylandsted Klokmose, and Nicolai Marquardt. 2019. Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--28. https://doi.org/10.1145/3290605.3300792
[18]
Frederik Brudy, David Ledo, Michel Pahud, Nathalie Henry Riche, Christian Holz, Anand Waghmare, Hemant Bhaskar Surale, Marcus Peinado, Xiaokuan Zhang, Shannon Joyner, and others. 2020. SurfaceFleet: Exploring Distributed Interactions Unbounded from Device, Application, User, and Time. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 7--21.
[19]
Pei-Yu (Peggy) Chi, Yang Li, and Björn Hartmann. 2016. Enhancing Cross-Device Interaction Scripting with Interactive Illustrations. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5482--5493. https://doi.org/10.1145/2858036.2858382
[20]
Apoorve Chokshi, Teddy Seyed, Francisco Marinho Rodrigues, and Frank Maurer. 2014. ePlan multi-surface: A multi-surface environment for emergency response planning exercises. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces. 219--228.
[21]
Andy Cockburn, Amy Karlson, and Benjamin B. Bederson. 2009. A review of overviewdetail, zooming, and focuscontext interfaces. Comput. Surveys, Vol. 41, 1 (Jan. 2009), 2:1--2:31. https://doi.org/11.1145/1456650.1456652
[22]
Cooper Davies, Jade White, Alec McAllister, Adam Saroka, Omar Addam, Fatemeh Hendijani Fard, and Frank Maurer. 2016. A Toolkit for Building Collaborative Immersive Multi-Surface Applications. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces (ISS '16). Association for Computing Machinery, New York, NY, USA, 485--488. https://doi.org/10.1145/2992154.2996879
[23]
Alan Dix and Corina Sas. 2010. Mobile personal devices meet situated public displays: Synergies and opportunities. International Journal of Ubiquitous Computing, Vol. 1, 1 (2010), 11--28.
[24]
Douglas Engelbart. 2018. 1968 "Mother of All Demos" by SRI's Doug Engelbart and Team. https://www.youtube.com/watch?v=B6rKUf9DWRI
[25]
Paul Dourish and Victoria Bellotti. 1992. Awareness and coordination in shared workspaces. In Proceedings of the 1992 ACM conference on Computer-supported cooperative work. 107--114.
[26]
Paul Dourish and W Keith Edwards. 2000. A tale of two toolkits: Relating infrastructure and use in flexible CSCW toolkits. Computer Supported Cooperative Work (CSCW), Vol. 9 (2000), 33--51.
[27]
Jeff Dyck, Carl Gutwin, TC Nicholas Graham, and David Pinelle. 2007. Beyond the LAN: Techniques from network games for improving groupware performance. In Proceedings of the 2007 ACM International Conference on Supporting Group Work. 291--300.
[28]
W Keith Edwards. 1994. Session management for collaborative applications. In Proceedings of the 1994 ACM conference on Computer supported cooperative work. 323--330.
[29]
George W Fitzmaurice, Azam Khan, William Buxton, Gordon Kurtenbach, and Ravin Balakrishnan. 2003. Sentient Data Access via a Diverse Society of Devices: Today's ubiquitous computing environment cannot benefit from the traditional understanding of a hierarchical file system. Queue, Vol. 1, 8 (2003), 52--62.
[30]
Luca Frosini, Marco Manca, and Fabio Paternò. 2013. A Framework for the Development of Distributed Interactive Applications. In Proceedings of the 5th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (London, United Kingdom) (EICS '13). Association for Computing Machinery, New York, NY, USA, 249--254. https://doi.org/10.1145/2494603.2480328
[31]
Hans Gellersen, Carl Fischer, Dominique Guinard, Roswitha Gostner, Gerd Kortuem, Christian Kray, Enrico Rukzio, and Sara Streng. 2009. Supporting device discovery and spontaneous interaction with spatial references. Personal and Ubiquitous Computing, Vol. 13 (2009), 255--264. Publisher: Springer.
[32]
TC Nicholas Graham, W Greg Phillips, and Christopher Wolfe. 2006. Quality analysis of distribution architectures for synchronous groupware. In 2006 International Conference on Collaborative Computing: Networking, Applications and Worksharing. IEEE, 1--9.
[33]
Saul Greenberg. 1990. Sharing views and interactions with single-user applications. ACM SIGOIS Bulletin, Vol. 11, 2--3 (1990), 227--237.
[34]
Saul Greenberg. 2007. Toolkits and interface creativity. Multimedia Tools and Applications, Vol. 32 (2007), 139--159.
[35]
Saul Greenberg and Chester Fitchett. 2001. Phidgets: easy development of physical interfaces through physical widgets. In Proceedings of the 14th annual ACM symposium on User interface software and technology. 209--218.
[36]
Saul Greenberg and David Marwood. 1994. Real time groupware as a distributed system: Concurrency control and its effect on the interface. In Proceedings of the 1994 ACM conference on Computer supported cooperative work. 207--217.
[37]
Irene Greif. 1988. Computer-supported cooperative work: A book of readings. Morgan Kaufmann Publishers Inc.
[38]
Jens Grubert and Matthias Kranz. 2017. HeadPhones: Ad Hoc Mobile Multi-Display Environments through Head Tracking. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 3966--3971. https://doi.org/10.1145/3025453.3025533
[39]
Jens Grubert, Matthias Kranz, and Aaron Quigley. 2016. Challenges in mobile multi-device ecosystems. mUX: The Journal of Mobile User Experience, Vol. 5, 1 (Aug. 2016), 5. https://doi.org/10.1186/s13678-016-0007-y
[40]
Jonathan Grudin. 2001. Partitioning digital worlds: focal and peripheral awareness in multiple monitor use. In Proceedings of the SIGCHI conference on Human factors in computing systems. 458--465.
[41]
Carl Gutwin. 2001. The effects of network delays on group work in real-time groupware. In ECSCW 2001: Proceedings of the Seventh European Conference on Computer Supported Cooperative Work 16--20 September 2001, Bonn, Germany. Springer, 299--318.
[42]
Carl A Gutwin, Michael Lippold, and TC Nicholas Graham. 2011. Real-time groupware in the browser: testing the performance of web-based networking. In Proceedings of the ACM 2011 conference on Computer supported cooperative work. 167--176.
[43]
Peter Hamilton and Daniel J. Wigdor. 2014. Conductor: enabling and understanding cross-device interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). Association for Computing Machinery, New York, NY, USA, 2773--2782. https://doi.org/10.1145/2556288.2557170
[44]
Tommi Heikkinen, Jorge Goncalves, Vassilis Kostakos, Ivan Elhart, and Timo Ojala. 2014. Tandem Browsing Toolkit: Distributed Multi-Display Interfaces with Web Technologies. In Proceedings of The International Symposium on Pervasive Displays (Copenhagen, Denmark) (PerDis '14). Association for Computing Machinery, New York, NY, USA, 142--147. https://doi.org/10.1145/2611009.2611026
[45]
Ken Hinckley et al. 2003. Bumping objects together as a semantically rich way of forming connections between ubiquitous devices. Ubicomp2003 (2003).
[46]
Ken Hinckley, Gonzalo Ramos, Francois Guimbretiere, Patrick Baudisch, and Marc Smith. 2004. Stitching: pen gestures that span multiple displays. In Proceedings of the working conference on Advanced visual interfaces. 23--31.
[47]
Steven Houben and Nicolai Marquardt. 2015. WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). Association for Computing Machinery, New York, NY, USA, 1247--1256. https://doi.org/10.1145/2702123.2702215
[48]
Da-Yuan Huang, Chien-Pang Lin, Yi-Ping Hung, Tzu-Wen Chang, Neng-Hao Yu, Min-Lun Tsai, and Mike Y Chen. 2012. MagMobile: enhancing social interactions with rapid view-stitching games of mobile devices. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia. 1--4.
[49]
Zheng Huang and Jun Kong. 2020. A Toolkit for Prototyping Tabletop-Centric Cross-Device Interaction. International Journal of Human--Computer Interaction, Vol. 36, 6 (April 2020), 536--552. https://doi.org/10.1080/10447318.2019.1662667
[50]
Peter Hutterer, Benjamin S Close, and Bruce H Thomas. 2006. TIDL: mixed presence groupware support for legacy and custom applications. In Proceedings of the 7th Australasian User interface conference-Volume 50. 117--124.
[51]
K. Inkpen, K. Hawkey, M. Kellar, R. Mandryk, K. Parker, D. Reilly, S. Scott, and T. Whalen. 2005. Exploring Display Factors That Influence Co-Located Collaboration: Angle, Size, Number, and User Arrangement. In Proceedings of HCI International.
[52]
H. Ishii. 1990. TeamWorkStation: towards a seamless shared workspace. In Proceedings of the 1990 ACM conference on Computer-supported cooperative work (CSCW '90). Association for Computing Machinery, New York, NY, USA, 13--26. https://doi.org/10.1145/99332.99337
[53]
Heidi Pi Jensen, Marius Pallisgaard Olsen, and Mikael B Skov. 2016. PinchPan: Investigating Children's Collaboration in Cross-Device Interaction. In Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology. 1--6.
[54]
Nima Kaviani, Matthias Finke, Sidney Fels, Rodger Lea, and Hua Wang. 2009. What Goes Where? Designing Interactive Large Public Display Applications for Mobile Device Interaction. In Proceedings of the First International Conference on Internet Multimedia Computing and Service (Kunming, Yunnan, China) (ICIMCS '09). Association for Computing Machinery, New York, NY, USA, 129--138. https://doi.org/10.1145/1734605.1734637
[55]
Dawar Khan, Sehat Ullah, Dong-Ming Yan, Ihsan Rabbi, Paul Richard, Thuong Hoang, Mark Billinghurst, and Xiaopeng Zhang. 2018. Robust tracking through the design of high quality fiducial markers: an optimization tool for ARToolKit. IEEE access, Vol. 6 (2018), 22421--22433.
[56]
Seungki Kim, Donghyeon Ko, and Woohun Lee. 2017. Utilizing smartphones as a multi-device single display groupware to design collaborative games. In Proceedings of the 2017 Conference on Designing Interactive Systems. 1341--1352.
[57]
Clemens N. Klokmose, James R. Eagan, Siemen Baader, Wendy Mackay, and Michel Beaudouin-Lafon. 2015. Webstrates: Shareable Dynamic Media. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (Charlotte, NC, USA) (UIST '15). Association for Computing Machinery, New York, NY, USA, 280--290. https://doi.org/10.1145/2807442.2807446
[58]
Naohiko Kohtake, Jun Rekimoto, and Yuichiro Anzai. 1999. InfoStick: an interaction device for Inter-Appliance Computing. In Handheld and Ubiquitous Computing: First International Symposium, HUC'99 Karlsruhe, Germany, September 27--29, 1999 Proceedings 1. Springer, 246--258.
[59]
Ricardo Langner, Tom Horak, and Raimund Dachselt. 2017. VisTiles: Coordinating and Combining Co-located Mobile Devices for Visual Data Exploration. IEEE transactions on visualization and computer graphics, Vol. 24, 1 (2017), 626--636.
[60]
David Ledo, Steven Houben, Jo Vermeulen, Nicolai Marquardt, Lora Oehlberg, and Saul Greenberg. 2018. Evaluation Strategies for HCI Toolkit Research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--17. https://doi.org/10.1145/3173574.3173610
[61]
Chengzhao Li, Carl Gutwin, Kevin Stanley, and Miguel Nacenta. 2016. All across the circle: using auto-ordering to improve object transfer between mobile devices. Proceedings of Graphics Interface 2016 (2016).
[62]
Zhen Li, Michelle Annett, Ken Hinckley, and Daniel Wigdor. 2019. Smac: A simplified model of attention and capture in multi-device desk-centric environments. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, EICS (2019), 1--47. Publisher: ACM New York, NY, USA.
[63]
Carsten Magerkurth, Maral Memisoglu, Timo Engelke, and Norbert Streitz. 2004. Towards the next generation of tabletop gaming experiences. In Proceedings of Graphics interface 2004. 73--80.
[64]
Marco Manca and Fabio Paternò. 2016. Customizable Dynamic User Interface Distribution. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (Brussels, Belgium) (EICS '16). Association for Computing Machinery, New York, NY, USA, 27--37. https://doi.org/10.1145/2933242.2933259
[65]
Nicolai Marquardt, Robert Diaz-Marino, Sebastian Boring, and Saul Greenberg. 2011. The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies. In Proceedings of the 24th annual ACM symposium on User interface software and technology. 315--326.
[66]
Masood Masoodian, Saturnino Luz, and David Kavenga. 2016. Nu-view: A visualization system for collaborative Co-located analysis of geospatial disease data. In Proceedings of the Australasian Computer Science Week Multiconference. 1--10.
[67]
Jérémie Melchior, Donatien Grolaux, Jean Vanderdonckt, and Peter Van Roy. 2009. A Toolkit for Peer-to-Peer Distributed User Interfaces: Concepts, Implementation, and Applications. In Proceedings of the 1st ACM SIGCHI Symposium on Engineering Interactive Computing Systems (Pittsburgh, PA, USA) (EICS '09). Association for Computing Machinery, New York, NY, USA, 69--78. https://doi.org/10.1145/1570433.1570449
[68]
Meredith Ringel Morris, Anqi Huang, Andreas Paepcke, and Terry Winograd. 2006. Cooperative gestures: multi-user gestural interactions for co-located groupware. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 1201--1210.
[69]
Meredith Ringel Morris, Kathy Ryall, Chia Shen, Clifton Forlines, and Frederic Vernier. 2004. Beyond" social protocols" multi-user coordination policies for co-located groupware. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. 262--265.
[70]
Brad A Myers. 2001. Using handhelds and PCs together. Commun. ACM, Vol. 44, 11 (2001), 34--41.
[71]
Brad A Myers, Choon Hong Peck, Jeffrey Nichols, Dave Kong, and Robert Miller. 2001. Interacting at a distance using semantic snarfing. In Ubicomp 2001: Ubiquitous Computing: International Conference Atlanta Georgia, USA, September 30--October 2, 2001 Proceedings 3. Springer, 305--314.
[72]
Miguel A Nacenta, Dzmitry Aliakseyeu, Sriram Subramanian, and Carl Gutwin. 2005. A comparison of techniques for multi-display reaching. In Proceedings of the SIGCHI conference on Human factors in computing systems. 371--380.
[73]
Miguel A Nacenta, Carl Gutwin, Dzmitry Aliakseyeu, and Sriram Subramanian. 2009. There and back again: Cross-display object movement in multi-display environments. Human-Computer Interaction, Vol. 24, 1--2 (2009), 170--229.
[74]
Miguel A Nacenta, Regan L Mandryk, and Carl Gutwin. 2008. Targeting across displayless space. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 777--786.
[75]
Miguel A Nacenta, Satoshi Sakurai, Tokuo Yamaguchi, Yohei Miki, Yuichi Itoh, Yoshifumi Kitamura, Sriram Subramanian, and Carl Gutwin. 2007. E-conic: a perspective-aware interface for multi-display environments. In Proceedings of the 20th annual ACM symposium on User interface software and technology. 279--288.
[76]
Miguel A Nacenta, Samer Sallam, Bernard Champoux, Sriram Subramanian, and Carl Gutwin. 2006. Perspective cursor: perspective-based interaction for multi-display environments. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 289--298.
[77]
Michael Nebeling. 2017a. Playing the Tricky Game of Toolkits Research. In HCI Tools 2017. Technical HCI Toolkits. 4.
[78]
Michael Nebeling. 2017b. XDBrowser 2.0: Semi-Automatic Generation of Cross-Device Interfaces. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 4574--4584. https://doi.org/10.1145/3025453.3025547
[79]
Michael Nebeling and Anind K. Dey. 2016. XDBrowser: User-Defined Cross-Device Web Page Designs. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). Association for Computing Machinery, New York, NY, USA, 5494--5505. https://doi.org/10.1145/2858036.2858048
[80]
Michael Nebeling, Maria Husmann, Christoph Zimmerli, Giulio Valente, and Moira C. Norrie. 2015. XDSession: Integrated Development and Testing of Cross-device Applications. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '15). ACM, New York, NY, USA, 22--27. https://doi.org/10.1145/2774225.2775075
[81]
Michael Nebeling, Theano Mintsi, Maria Husmann, and Moira Norrie. 2014a. Interactive development of cross-device user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). Association for Computing Machinery, New York, NY, USA, 2793--2802. https://doi.org/10.1145/2556288.2556980
[82]
Michael Nebeling and Moira Norrie. 2012. jQMultiTouch: lightweight toolkit and development framework for multi-touch/multi-device web interfaces. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems. 61--70.
[83]
Michael Nebeling, Elena Teunissen, Maria Husmann, and Moira C Norrie. 2014b. XDKinect: development framework for cross-device interaction using kinect. In Proceedings of the 2014 ACM SIGCHI symposium on Engineering interactive computing systems. 65--74.
[84]
Jeffrey Nichols, Brad A Myers, Michael Higgins, Joseph Hughes, Thomas K Harris, Roni Rosenfeld, and Mathilde Pignol. 2002. Generating remote control interfaces for complex appliances. In Proceedings of the 15th annual ACM symposium on User interface software and technology. 161--170.
[85]
Heidi Selmer Nielsen, Marius Pallisgaard Olsen, Mikael B. Skov, and Jesper Kjeldskov. 2014. JuxtaPinch: exploring multi-device interaction in collocated photo sharing. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services (MobileHCI '14). Association for Computing Machinery, New York, NY, USA, 183--192. https://doi.org/10.1145/2628363.2628369
[86]
Takashi Ohta and Jun Tanaka. 2015. MovieTile: Interactively Adjustable Free Shape Multi-Display of Mobile Devices. In SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications (Kobe, Japan) (SA '15). Association for Computing Machinery, New York, NY, USA, Article 18, bibinfonumpages7 pages. https://doi.org/10.1145/2818427.2818436
[87]
Katie O'Leary, Tao Dong, Julia Katherine Haines, Michael Gilbert, Elizabeth F. Churchill, and Jeffrey Nichols. 2017. The Moving Context Kit: Designing for Context Shifts in Multi-Device Experiences. In Proceedings of the 2017 Conference on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS '17). Association for Computing Machinery, New York, NY, USA, 309--320. https://doi.org/10.1145/3064663.3064768
[88]
Jeni Paay, Dimitrios Raptis, Jesper Kjeldskov, Mikael B. Skov, Eric V. Ruder, and Bjarke M. Lauridsen. 2017. Investigating Cross-Device Interaction between a Handheld Device and a Large Display. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 6608--6619. https://doi.org/10.1145/3025453.3025724
[89]
Seonwook Park, Christoph Gebhardt, Roman R"adle, Anna Maria Feit, Hana Vrzakova, Niraj Ramesh Dayama, Hui-Shyong Yeo, Clemens N. Klokmose, Aaron Quigley, Antti Oulasvirta, and Otmar Hilliges. 2018. AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3173574.3173758
[90]
Julian Petford, Miguel A Nacenta, Carl Gutwin, Joseph Eremondi, and Cody Ede. 2016. The ASPECTA toolkit: affordable full coverage displays. In Proceedings of the 5th ACM International Symposium on Pervasive Displays. 87--105.
[91]
Thomas Plank, Hans-Christian Jetter, Roman R"adle, Clemens N Klokmose, Thomas Luger, and Harald Reiterer. 2017. Is Two Enough? ! Studying Benefits, Barriers, and Biases of Multi-Tablet Use for Collaborative Visualization. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 4548--4560.
[92]
Shankar R Ponnekanti, Brad Johanson, Emre Kiciman, and Armando Fox. 2003. Portability, extensibility and robustness in iROS. In Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003.(PerCom 2003). IEEE, 11--19.
[93]
Roman R"adle, Hans-Christian Jetter, Nicolai Marquardt, Harald Reiterer, and Yvonne Rogers. 2014. Huddlelamp: Spatially-aware mobile displays for ad-hoc around-the-table collaboration. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces. 45--54.
[94]
Gonzalo Ramos, Kenneth Hinckley, Andy Wilson, and Raman Sarin. 2009. Synchronous gestures in multi-display environments. Human-Computer Interaction, Vol. 24, 1--2 (2009), 117--169.
[95]
Umar Rashid, Miguel A Nacenta, and Aaron Quigley. 2012a. The cost of display switching: a comparison of mobile, large display and hybrid UI configurations. In Proceedings of the International Working Conference on Advanced Visual Interfaces. 99--106.
[96]
Umar Rashid, Miguel A Nacenta, and Aaron Quigley. 2012b. Factors influencing visual attention switch in multi-display user interfaces: A survey. In Proceedings of the 2012 international symposium on pervasive displays. 1--6.
[97]
Jun Rekimoto. 1997. Pick-and-drop: a direct manipulation technique for multiple computer environments. In Proceedings of the 10th annual ACM symposium on User interface software and technology. 31--39.
[98]
Jun Rekimoto and Masanori Saitoh. 1999. Augmented surfaces: a spatially continuous work space for hybrid computing environments. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 378--385.
[99]
Reinout Roels, Arno De Witte, and Beat Signer. 2018. INFEX: A Unifying Framework for Cross-Device Information Exploration and Exchange. Proc. ACM Hum.-Comput. Interact., Vol. 2, EICS, Article 2 (jun 2018), bibinfonumpages26 pages. https://doi.org/10.1145/3179427
[100]
Yvonne Rogers, Youn-kyung Lim, William R. Hazlewood, and Paul Marshall. 2009. Equal Opportunities : Do Shareable Interfaces Promote More Group Participation Than Single User Displays ? Human-Computer Interaction, Vol. 24, 1 (2009), 79. https://doi.org/10.1080/07370020902739379
[101]
Manuel Roman and Roy H Campbell. 2000. Gaia: Enabling active spaces. In Proceedings of the 9th workshop on ACM SIGOPS European workshop: beyond the PC: new challenges for the operating system. 229--234.
[102]
Mark Roseman and Saul Greenberg. 1996. Building real-time groupware with GroupKit, a groupware toolkit. ACM Transactions on Computer-Human Interaction (TOCHI), Vol. 3, 1 (1996), 66--106.
[103]
Satoshi Sakurai, Yuichi Itoh, Yoshifumi Kitamura, Miguel A Nacenta, Tokuo Yamaguchi, Sriram Subramanian, and Fumio Kishino. 2008. A middleware for seamless use of multiple displays. In Interactive Systems. Design, Specification, and Verification: 15th International Workshop, DSV-IS 2008 Kingston, Canada, July 16--18, 2008 Revised Papers 15. Springer, 252--266.
[104]
Mario Schreiner, Roman Rädle, Hans-Christian Jetter, and Harald Reiterer. 2015. Connichiwa: A Framework for Cross-Device Web Applications. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, New York, NY, USA, 2163--2168. https://doi.org/10.1145/2702613.2732909
[105]
Stacey D Scott, M Sheelagh T Carpendale, and Kori Inkpen. 2004. Territoriality in collaborative tabletop workspaces. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. 294--303.
[106]
Stacey D Scott, Jordan Wan, Alma Rico, Carina Furusho, and ML Cummings. 2007. Aiding team supervision in command and control operations with large-screen displays. In HSIS 2007: ASNE Human Systems Integration Symposium. 19--21.
[107]
Teddy Seyed, Alaa Azazi, Edwin Chan, Yuxi Wang, and Frank Maurer. 2015. SoD-Toolkit: A Toolkit for Interactively Prototyping and Developing Multi-Sensor, Multi-Device Environments. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). Association for Computing Machinery, New York, NY, USA, 171--180. https://doi.org/10.1145/2817721.2817750
[108]
Teddy Seyed, Chris Burns, Mario Costa Sousa, Frank Maurer, and Anthony Tang. 2012. Eliciting usable gestures for multi-display environments. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces. 41--50.
[109]
Teddy Seyed, Mario Costa Sousa, Frank Maurer, and Anthony Tang. 2013. SkyHunter: a multi-surface environment for supporting oil and gas exploration. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces. 15--22.
[110]
Maurício Sousa, Daniel Mendes, Rafael Kuffner Dos Anjos, Daniel Medeiros, Alfredo Ferreira, Alberto Raposo, João Madeiras Pereira, and Joaquim Jorge. 2017. Creepy Tracker Toolkit for Context-aware Interfaces. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17). Association for Computing Machinery, New York, NY, USA, 191--200. https://doi.org/10.1145/3132272.3134113
[111]
Maximilian Speicher, Brian D. Hall, Ao Yu, Bowen Zhang, Haihua Zhang, Janet Nebeling, and Michael Nebeling. 2018. XD-AR: Challenges and Opportunities in Cross-Device Augmented Reality Application Development. Proceedings of the ACM on Human-Computer Interaction, Vol. 2, EICS (June 2018), 7:1--7:24. https://doi.org/10.1145/3229089
[112]
Mark Stefik, Daniel G Bobrow, Gregg Foster, Stan Lanning, and Deborah Tatar. 1987 a. WYSIWIS revised: Early experiences with multiuser interfaces. ACM Transactions on Information Systems (TOIS), Vol. 5, 2 (1987), 147--167.
[113]
Mark Stefik, Gregg Foster, Daniel G Bobrow, Kenneth Kahn, Stan Lanning, and Lucy Suchman. 1987 b. Beyond the chalkboard: Computer support for collaboration and problem solving in meetings. Commun. ACM, Vol. 30, 1 (1987), 32--47.
[114]
Norbert A Streitz, Jörg Geißler, Torsten Holmer, Shin'ichi Konomi, Christian Müller-Tomfelde, Wolfgang Reischl, Petra Rexroth, Peter Seitz, and Ralf Steinmetz. 1999. i-LAND: an interactive landscape for creativity and innovation. In Proceedings of the SIGCHI conference on Human factors in computing systems. 120--127.
[115]
Wolfgang Stuerzlinger, Olivier Chapuis, Dusty Phillips, and Nicolas Roussel. 2006. User interface facc ades: towards fully adaptable user interfaces. In Proceedings of the 19th annual ACM symposium on User interface software and technology. 309--318.
[116]
Peter Tandler. 2004. The BEACH application model and software framework for synchronous collaboration in ubiquitous computing environments. Journal of Systems and Software, Vol. 69, 3 (2004), 267--296.
[117]
Lucia Terrenghi, Aaron Quigley, and Alan Dix. 2009. A taxonomy for and analysis of multi-person-display ecosystems. Personal and Ubiquitous Computing, Vol. 13 (2009), 583--598.
[118]
James R Wallace, Stacey D Scott, and Carolyn G MacGregor. 2013. Collaborative sensemaking on a digital tabletop and personal tablets: prioritization, comparisons, and tableaux. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3345--3354.
[119]
James R Wallace, Stacey D Scott, Taryn Stutz, Tricia Enns, and Kori Inkpen. 2009. Investigating teamwork and taskwork in single-and multi-display groupware systems. Personal and Ubiquitous Computing, Vol. 13 (2009), 569--581.
[120]
Mark Weiser. 1999. The computer for the 21st century. ACM SIGMOBILE mobile computing and communications review, Vol. 3, 3 (1999), 3--11.
[121]
Greg Welch, Henry Fuchs, Ramesh Raskar, Herman Towles, and Michael S Brown. 2000. Projected Imagery in Your. IEEE Computer Graphics and Applications, Vol. 20, 04 (2000), 62--67.
[122]
Daniel Wigdor, Hao Jiang, Clifton Forlines, Michelle Borkin, and Chia Shen. 2009. WeSpace: the design development and deployment of a walk-up and share multi-surface visual collaboration system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1237--1246.
[123]
Daniel Wigdor, Chia Shen, Clifton Forlines, and Ravin Balakrishnan. 2006. Table-centric interactive spaces for real-time collaboration. In Proceedings of the working conference on Advanced visual interfaces. 103--107.
[124]
Andrew D Wilson and Raman Sarin. 2007. BlueTable: connecting wireless mobile devices on interactive surfaces using vision-based handshaking. In Proceedings of Graphics interface 2007. 119--125.
[125]
Jacob O Wobbrock, Andrew D Wilson, and Yang Li. 2007. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In Proceedings of the 20th annual ACM symposium on User interface software and technology. 159--168.
[126]
Robert Xiao, Miguel A. Nacenta, Regan L. Mandryk, Andy Cockburn, and Carl Gutwin. 2011. Ubiquitous cursor: a comparison of direct and indirect pointing feedback in multi-display environments. In Proceedings of Graphics Interface 2011. 135--142.
[127]
Divyani Yadav, Deeksha Gupta, Dhananjay Singh, Devendra Kumar, and Upasana Sharma. 2018. Vulnerabilities and security of web applications. In 2018 4th International Conference on Computing Communication and Automation (ICCCA). IEEE, 1--5.
[128]
Jishuo Yang and Daniel Wigdor. 2014. Panelrama: Enabling Easy Specification of Cross-device Web Applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2783--2792. https://doi.org/10.1145/2556288.2557199
[129]
Fengyuan Zhu and Tovi Grossman. 2020. Bishare: Exploring bidirectional interactions between smartphones and head-mounted augmented reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--14. io
Index Terms
- WAMS: A Flexible API for Visual Workspaces Across Multiple Surfaces
Recommendations
Eclipse API usage: the good and the bad
Today, when constructing software systems, many developers build their systems on top of frameworks. Eclipse is such a framework that has been in existence for over a decade. Like many other evolving software systems, the Eclipse platform has both ...
distributed user interfaces: collaboration and usability
CHI EA '12: CHI '12 Extended Abstracts on Human Factors in Computing SystemsThis document describes the most relevant issues regarding collaboration and usability when using distributed user interfaces (DUIs). The goal of this workshop is to promote the discussion about the emerging topic of DUIs, answering a set of key ...
Comments
Information & Contributors
Information
Published In
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 19 June 2023
Published in PACMHCI Volume 7, Issue EICS
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 172Total Downloads
- Downloads (Last 12 months)127
- Downloads (Last 6 weeks)4
Reflects downloads up to 11 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in