Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3411764.3445722acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Open access

Combining Touchscreens with Passive Rich-ID Building Blocks to Support Context Construction in Touchscreen Interactions

Published: 07 May 2021 Publication History
  • Get Citation Alerts
  • Abstract

    This research investigates the design space of combining touchscreens with passive rich-ID building block systems to support the physical construction of contexts in touchscreen interactions. With two proof-of-concept systems, RFIPillars and RFITiles, we explore various schemes for using tangible inputs for context enrichment in touchscreen interactions. Instead of incorporating an electronic touchscreen module that requires per-module maintenance, this work intentionally makes each tangible object passive. We explore rear-projection solutions to integrate touchscreen interactions into these passive building blocks with capacitive touch sensing techniques and deliberate physical forgiving to retain the merits of being both batteryless and wireless. The presented research artifacts embody the interaction designs and elucidate scalability challenges in integrating touchscreen interactions into this emerging tangible user interface.

    Supplementary Material

    MP4 File (3411764.3445722_videofigure.mp4)
    Supplemental video
    MP4 File (3411764.3445722_videopreview.mp4)
    Preview video

    References

    [1]
    Masahiro Ando, Yuichi Itoh, Toshiki Hosoi, Kazuki Takashima, Kosuke Nakajima, and Yoshifumi Kitamura. 2014. StackBlock: Block-shaped Interface for Flexible Stacking. In Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST’14 Adjunct). ACM, New York, NY, USA, 41–42. https://doi.org/10.1145/2658779.2659104
    [2]
    Tom Bartindale and Chris Harrison. 2009. Stacks on the Surface: Resolving Physical Order Using Fiducial Markers with Structured Transparency. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (Banff, Alberta, Canada) (ITS ’09). ACM, New York, NY, USA, 57–60. https://doi.org/10.1145/1731903.1731916
    [3]
    Patrick Baudisch, Torsten Becker, and Frederik Rudeck. 2010. Lumino: Tangible Blocks for Tabletop Computers Based on Glass Fiber Bundles. In ACM CHI 2010 Conference Proceedings (Atlanta, Georgia, USA). 1165–1174.
    [4]
    Wolfgang Büschel, Ulrike Kister, Mathias Frisch, and Raimund Dachselt. 2014. T4-transparent and translucent tangibles on tabletops. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces. ACM, 81–88.
    [5]
    Liwei Chan, Stefanie Müller, Anne Roudaut, and Patrick Baudisch. 2012. CapStones and ZebraWidgets: Sensing Stacks of Building Blocks, Dials and Sliders on Capacitive Touch Screens. In ACM CHI 2012 Conference Proceedings (Austin, Texas, USA). 2189–2192.
    [6]
    Paul Dietz and Darren Leigh. 2001. DiamondTouch: A Multi-user Touch Technology. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (Orlando, Florida) (UIST ’01). ACM, New York, NY, USA, 219–226. https://doi.org/10.1145/502348.502389
    [7]
    James J Gibson. [n.d.]. The theory of affordances. ([n. d.]).
    [8]
    Audrey Girouard, Aneesh Tarun, and Roel Vertegaal. 2012. DisplayStacks: Interaction Techniques for Stacks of Flexible Thin-film Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). ACM, New York, NY, USA, 2431–2440. https://doi.org/10.1145/2207676.2208406
    [9]
    Alix Goguey, Cameron Steer, Andrés Lucero, Laurence Nigay, Deepak Ranjan Sahoo, Céline Coutrix, Anne Roudaut, Sriram Subramanian, Yutaka Tokuda, Timothy Neate, Jennifer Pearson, Simon Robinson, and Matt Jones. 2019. PickCells: A Physically Reconfigurable Cell-Composed Touchscreen. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300503
    [10]
    Jefferson Y. Han. 2005. Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology(Seattle, WA, USA) (UIST ’05). Association for Computing Machinery, New York, NY, USA, 115–118. https://doi.org/10.1145/1095034.1095054
    [11]
    Chris Harrison, Hrvoje Benko, and Andrew D. Wilson. 2011. OmniTouch: Wearable Multitouch Interaction Everywhere. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). ACM, New York, NY, USA, 441–450. https://doi.org/10.1145/2047196.2047255
    [12]
    Florian Heller, Simon Voelker, Chat Wacharamanotham, and Jan Borchers. 2015. Transporters: Vision & Touch Transitive Widgets for Capacitive Screens. In CHI ’15: Extended Abstracts of the SIGCHI Conference on Human Factors in Computing Systems(Seoul, Republic of Korea). 1603–1608. https://doi.org/10.1145/2702613.2732707
    [13]
    Toshiki Hosoi, Kazuki Takashima, Tomoaki Adachi, Yuichi Itoh, and Yoshifumi Kitamura. 2014. A-blocks: Recognizing and Assessing Child Building Processes During Play with Toy Blocks. In SIGGRAPH Asia 2014 Emerging Technologies (Shenzhen, China) (SA ’14). ACM, New York, NY, USA, Article 1, 2 pages. https://doi.org/10.1145/2669047.2669061
    [14]
    Meng-Ju Hsieh, Rong-Hao Liang, Jr-Ling Guo, and Bing-Yu Chen. 2018. RFIDesk: An Interactive Surface for Multi-Touch and Rich-ID Stackable Tangible Interactions. In SIGGRAPH Asia 2018 Emerging Technologies (Tokyo, Japan) (SA ’18). Association for Computing Machinery, New York, NY, USA, Article 11, 2 pages. https://doi.org/10.1145/3275476.3275491
    [15]
    Meng-Ju Hsieh, Rong-Hao Liang, Da-Yuan Huang, Jheng-You Ke, and Bing-Yu Chen. 2018. RFIBricks: Interactive Building Blocks Based on RFID. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3173574.3173763
    [16]
    Scott E Hudson and Jennifer Mankoff. 2014. Concepts, values, and methods for technical human–computer interaction research. In Ways of Knowing in HCI. Springer, 69–93.
    [17]
    Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’97). ACM, New York, NY, USA, 234–241. https://doi.org/10.1145/258549.258715
    [18]
    Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). ACM, New York, NY, USA, 559–568. https://doi.org/10.1145/2047196.2047270
    [19]
    Robert J.K. Jacob, Audrey Girouard, Leanne M. Hirshfield, Michael S. Horn, Orit Shaer, Erin Treacy Solovey, and Jamie Zigelbaum. 2008. Reality-Based Interaction: A Framework for Post-WIMP Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). Association for Computing Machinery, New York, NY, USA, 201–210. https://doi.org/10.1145/1357054.1357089
    [20]
    Kunihiro Kato and Homei Miyashita. 2015. ExtensionSticker: A Proposal for a Striped Pattern Sticker to Extend Touch Interfaces and Its Assessment. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). ACM, New York, NY, USA, 1851–1854. https://doi.org/10.1145/2702123.2702500
    [21]
    Yoshifumi Kitamura, Yuichi Itoh, and Fumio Kishino. 2001. Real-Time 3D Interaction with ActiveCube. In CHI ’01 Extended Abstracts on Human Factors in Computing Systems (Seattle, Washington) (CHI EA ’01). Association for Computing Machinery, New York, NY, USA, 355–356. https://doi.org/10.1145/634067.634277
    [22]
    Hideki Koike, Wataru Nishikawa, and Kentaro Fukuchi. 2009. Transparent 2-D markers on an LCD tabletop system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 163–172.
    [23]
    Jinha Lee, Yasuaki Kakehi, and Takeshi Naemura. 2009. Bloxels: Glowing Blocks as Volumetric Pixels. In ACM SIGGRAPH 2009 Emerging Technologies (New Orleans, Louisiana) (SIGGRAPH ’09). Association for Computing Machinery, New York, NY, USA, Article 5, 1 pages. https://doi.org/10.1145/1597956.1597961
    [24]
    Kevin Lefeuvre, Soeren Totzauer, Michael Storz, Albrecht Kurze, Andreas Bischof, and Arne Berger. 2018. Bricks, Blocks, Boxes, Cubes, and Dice: On the Role of Cubic Shapes for the Design of Tangible Interactive Devices. In Proceedings of the 2018 Designing Interactive Systems Conference. ACM, 485–496.
    [25]
    Jakob Leitner and Michael Haller. 2011. Geckos: Combining Magnets and Pressure Images to Enable New Tangible-object Design and Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). ACM, New York, NY, USA, 2985–2994. https://doi.org/10.1145/1978942.1979385
    [26]
    Joanne Leong, Florian Perteneder, Hans-Christian Jetter, and Michael Haller. 2017. What a Life! Building a Framework for Constructive Assemblies. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction(Yokohama, Japan) (TEI ’17). Association for Computing Machinery, New York, NY, USA, 57–66. https://doi.org/10.1145/3024969.3024985
    [27]
    Rong-Hao Liang, Liwei Chan, Hung-Yu Tseng, Han-Chih Kuo, Da-Yuan Huang, De-Nian Yang, and Bing-Yu Chen. 2014. GaussBricks: Magnetic Building Blocks for Constructive Tangible Interactions on Portable Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 3153–3162. https://doi.org/10.1145/2556288.2557105
    [28]
    Rong-Hao Liang, Han-Chih Kuo, Liwei Chan, De-Nian Yang, and Bing-Yu Chen. 2014. GaussStones: Shielded Magnetic Tangibles for Multi-Token Interactions on Portable Displays. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST ’14). Association for Computing Machinery, New York, NY, USA, 365–372. https://doi.org/10.1145/2642918.2647384
    [29]
    Thomas W. Malone. 1983. How Do People Organize Their Desks?: Implications for the Design of Office Information Systems. ACM Trans. Inf. Syst. 1, 1 (Jan. 1983), 99–112. https://doi.org/10.1145/357423.357430
    [30]
    Nobuyuki Matsushita and Jun Rekimoto. 1997. HoloWall: Designing a Finger, Hand, Body, and Object Sensitive Wall. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology (Banff, Alberta, Canada) (UIST ’97). Association for Computing Machinery, New York, NY, USA, 209–210. https://doi.org/10.1145/263407.263549
    [31]
    David Merrill, Jeevan Kalanithi, and Pattie Maes. 2007. Siftables: Towards Sensor Network User Interfaces. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (Baton Rouge, Louisiana) (TEI ’07). Association for Computing Machinery, New York, NY, USA, 75–78. https://doi.org/10.1145/1226969.1226984
    [32]
    David Merrill, Emily Sun, and Jeevan Kalanithi. 2012. Sifteo Cubes. In CHI ’12 Extended Abstracts on Human Factors in Computing Systems (Austin, Texas, USA) (CHI EA ’12). Association for Computing Machinery, New York, NY, USA, 1015–1018. https://doi.org/10.1145/2212776.2212374
    [33]
    Rafael Morales González, Caroline Appert, Gilles Bailly, and Emmanuel Pietriga. 2016. Touchtokens: guiding touch patterns with passive tokens. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 4189–4202.
    [34]
    Florian Perteneder, Kathrin Probst, Joanne Leong, Sebastian Gassler, Christian Rendl, Patrick Parzer, Katharina Fluch, Sophie Gahleitner, Sean Follmer, Hideki Koike, and et al.2020. Foxels: Build Your Own Smart Furniture. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (Sydney NSW, Australia) (TEI ’20). Association for Computing Machinery, New York, NY, USA, 111–122. https://doi.org/10.1145/3374920.3374935
    [35]
    M. Philipose, J. R. Smith, B. Jiang, A. Mamishev, Sumit Roy, and K. Sundara-Rajan. 2005. Battery-free wireless identification and sensing. IEEE Pervasive Computing 4, 1 (Jan 2005), 37–45. https://doi.org/10.1109/MPRV.2005.7
    [36]
    Narjes Pourjafarian, Anusha Withana, Joseph A. Paradiso, and Jürgen Steimle. 2019. Multi-Touch Kit: A Do-It-Yourself Technique for Capacitive Multi-Touch Sensing Using a Commodity Microcontroller. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 1071–1083. https://doi.org/10.1145/3332165.3347895
    [37]
    Jun Rekimoto. 2002. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 113–120.
    [38]
    Jun Rekimoto, Brygg Ullmer, and Haruo Oba. 2001. DataTiles: A Modular Platform for Mixed Physical and Graphical Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI ’01). ACM, New York, NY, USA, 269–276. https://doi.org/10.1145/365024.365115
    [39]
    Toshiki Sato, Haruko Mamiya, Hideki Koike, and Kentaro Fukuchi. 2009. PhotoelasticTouch: Transparent Rubbery Tangible Interface Using an LCD and Photoelasticity. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (Victoria, BC, Canada) (UIST ’09). ACM, New York, NY, USA, 43–50. https://doi.org/10.1145/1622176.1622185
    [40]
    Martin Schmitz, Mohammadreza Khalilbeigi, Matthias Balwierz, Roman Lissermann, Max Mühlhäuser, and Jürgen Steimle. 2015. Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (Charlotte, NC, USA) (UIST ’15). ACM, New York, NY, USA, 253–258. https://doi.org/10.1145/2807442.2807503
    [41]
    Marc Teyssier, Gilles Bailly, Catherine Pelachaud, Eric Lecolinet, Andrew Conn, and Anne Roudaut. 2019. Skin-On Interfaces: A Bio-Driven Approach for Artificial Skin Design to Cover Interactive Devices. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 307–322. https://doi.org/10.1145/3332165.3347943
    [42]
    Brygg Ullmer and Hiroshi Ishii. 1997. The metaDESK: Models and Prototypes for Tangible User Interfaces. In ACM UIST 1997 Conference Proceedings (Banff, Alberta, Canada). 223–232.
    [43]
    Brygg Ullmer, Hiroshi Ishii, and Robert JK Jacob. 2005. Token+ constraint systems for tangible interaction with digital information. ACM Transactions on Computer-Human Interaction (TOCHI) 12, 1(2005), 81–118.
    [44]
    Mark Weiser. 1991. The Computer for the 21 st Century. Scientific american 265, 3 (1991), 94–105.
    [45]
    Malte Weiss, Florian Schwarz, Simon Jakubowski, and Jan Borchers. 2010. Madgets: Actuating Widgets on Interactive Tabletops. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). ACM, New York, NY, USA, 293–302. https://doi.org/10.1145/1866029.1866075
    [46]
    Malte Weiss, Julie Wagner, Yvonne Jansen, Roger Jennings, Ramsin Khoshabeh, James D. Hollan, and Jan Borchers. 2009. SLAP Widgets: Bridging the Gap Between Virtual and Physical Controls on Tabletops. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). ACM, New York, NY, USA, 481–490. https://doi.org/10.1145/1518701.1518779
    [47]
    Cary Williams, Xing Dong Yang, Grant Partridge, Joshua Millar-Usiskin, Arkady Major, and Pourang Irani. 2011. TZee: Exploiting the Lighting Properties of Multi-touch Tabletops for Tangible 3D Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). ACM, New York, NY, USA, 1363–1372. https://doi.org/10.1145/1978942.1979143
    [48]
    Karl Willis, Eric Brockmeyer, Scott Hudson, and Ivan Poupyrev. 2012. Printed Optics: 3D Printing of Embedded Optical Elements for Interactive Devices. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology(Cambridge, Massachusetts, USA) (UIST ’12). ACM, New York, NY, USA, 589–598. https://doi.org/10.1145/2380116.2380190
    [49]
    Andrew D. Wilson. 2005. PlayAnywhere: A Compact Interactive Tabletop Projection-vision System. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology(Seattle, WA, USA) (UIST ’05). ACM, New York, NY, USA, 83–92. https://doi.org/10.1145/1095034.1095047
    [50]
    Robert Xiao, Scott Hudson, and Chris Harrison. 2017. Supporting responsive cohabitation between virtual interfaces and physical objects on everyday surfaces. Proceedings of the ACM on Human-Computer Interaction 1, EICS(2017), 12.
    [51]
    Kentaro Yasu. 2019. Magnetact: Magnetic-sheet-based Haptic Interfaces for Touch Devices. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). ACM, New York, NY, USA, Article 240, 8 pages. https://doi.org/10.1145/3290605.3300470
    [52]
    John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through Design as a Method for Interaction Design Research in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 493–502. https://doi.org/10.1145/1240624.1240704

    Cited By

    View all
    • (2022)NFCStack: Identifiable Physical Building Blocks that Support Concurrent Construction and Frictionless InteractionProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545658(1-12)Online publication date: 29-Oct-2022
    • (2022)Recentering Reframing as an RtD ContributionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517789(1-14)Online publication date: 29-Apr-2022

    Index Terms

    1. Combining Touchscreens with Passive Rich-ID Building Blocks to Support Context Construction in Touchscreen Interactions
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
      May 2021
      10862 pages
      ISBN:9781450380966
      DOI:10.1145/3411764
      This work is licensed under a Creative Commons Attribution International 4.0 License.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 May 2021

      Check for updates

      Author Tags

      1. RFID
      2. building blocks
      3. capacitive sensing
      4. modular interface
      5. rear-projection
      6. rich-ID
      7. stackable
      8. tangible user interface
      9. touchscreen

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      CHI '21
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)154
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 10 Aug 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)NFCStack: Identifiable Physical Building Blocks that Support Concurrent Construction and Frictionless InteractionProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545658(1-12)Online publication date: 29-Oct-2022
      • (2022)Recentering Reframing as an RtD ContributionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517789(1-14)Online publication date: 29-Apr-2022

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Get Access

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media