Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

PrivacyShield: A Mobile System for Supporting Subtle Just-in-time Privacy Provisioning through Off-Screen-based Touch Gestures

Published: 05 July 2018 Publication History

Abstract

Current in-situ privacy solution approaches are inadequate in protecting sensitive information. They either require extra configuration effort or lack the ability to configure user desired privacy settings. Based on in-depth discussions during a design workshop, we propose PrivacyShield, a mobile system for providing subtle just-in-time privacy provisioning. PrivacyShield leverages the screen I/O device (screen digitizer) of smartphones to recognize gesture commands, even when the phone's screen is turned off. Based on gesture command inputs, various privacy-protection policies can be configured on-the-fly. We develop a novel stroke-based approach to address the challenges in segmenting and recognizing gesture command inputs, which helps the system in achieving good usability and performance. PrivacyShield also provides developers with APIs to enable just-in-time privacy provisioning in their applications. We have implemented an energy efficient PrivacyShield prototype on the Android platform, including smartphones with and without a low-power co-processor. Evaluation results show that our gesture segmentation algorithm is fast enough for real-time performance (introducing less than 200ms processing latency) and accurate (achieving an accuracy of 95% for single-character gestures and 89% for even three-character gestures). We also build a non-touch-screen-based just-in-time privacy provisioning prototype called the wrist gesture method. We compare the performance of the two prototypes by doing a 6-week field study with 12 participants and show why a simplistic solution falls short in providing privacy configurations. We also report the participants' perceptions and reactions after the field study.

Supplementary Material

pushp (pushp.zip)
Supplemental movie, appendix, image and software files for, PrivacyShield: A Mobile System for Supporting Subtle Just-in-time Privacy Provisioning through Off-Screen-based Touch Gestures

References

[1]
1999. IRONOFF DATABASE UMR CNRS 6597. http://www.irccyn.ec-nantes.fr/~viardgau/IRONOFF/IRONOFF.html. {Online; accessed 18-April-2018}.
[2]
2009. Protractor's Java implementation in the Android core framework. https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/gesture/GestureUtils.java. {Online; accessed 18-April-2018}.
[3]
2010. Kakao Talk. http://www.kakao.com/services/8. {Online; accessed 18-April-2018}.
[4]
2011. WeChat. http://www.wechat.com/en/. {Online; accessed 18-April-2018}.
[5]
2013. LG Nexus 5. http://www.lg.com/us/cell-phones/lg-D820-Sprint-Black-nexus-5. {Online; accessed 18-April-2018}.
[6]
2014. Android Lollipop. https://www.android.com/versions/lollipop-5-0/. {Online; accessed 18-April-2018}.
[7]
2014. Android Lollipop Features. https://www.android.com/versions/lollipop-5-0/#features. {Online; accessed 18-April-2018}.
[8]
2014. HTC one (M8). http://www.htc.com/us/smartphones/htc-one-m8/. {Online; accessed 18-April-2018}.
[9]
2015. Android Marshmallow. https://www.android.com/versions/marshmallow-6-0/. {Online; accessed 18-April-2018}.
[10]
2015. Smart Gallery - Photography. https://play.google.com/store/apps/details?id=com.ta.voicetag.main. {Online; accessed 18-April-2018}.
[11]
2016. Nougat. https://www.android.com/versions/nougat-7-0/. {Online; accessed 18-April-2018}.
[12]
2016. Project Oxford. https://www.projectoxford.ai/doc/speech/Get-Started/android/. {Online; accessed 18-April-2018}.
[13]
2017. Apple - Change alert styles and settings for notifications. https://support.apple.com/en-au/HT201925. {Online; accessed 18-April-2018}.
[14]
2017. iPhone X. https://www.apple.com/lae/iphone-x/specs/. {Online; accessed 18-April-2018}.
[15]
2017. OpenCamera. http://opencamera.sourceforge.net/. {Online; accessed 18-April-2018}.
[16]
2017. Power Monitor. Monsoon Solution Inc. https://www.msoon.com/LabEquipment/PowerMonitor/. {Online; accessed 18-April-2018}.
[17]
2018. Gallery Vault. https://play.google.com/store/apps/details?id=com.thinkyeah.galleryvault&hl=en. {Online; accessed 18-April-2018}.
[18]
2018. Hide Pictures Videos - Vaulty. https://play.google.com/store/apps/details?id=com.theronrogers.vaultyfree&hl=en. {Online; accessed 18-April-2018}.
[19]
2018. Indexing and retrieving contact IDs. https://developer.android.com/training/contacts-provider/retrieve-names.html. {Online; accessed 18-April-2018}.
[20]
2018. Smart Lock - Security simplified. https://get.google.com/smartlock/. {Online; accessed 18-April-2018}.
[21]
Fraser Anderson, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice. 2015. Supporting Subtlety with Deceptive Devices and Illusory Interactions. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1489--1498.
[22]
Jeremy Andrus, Christoffer Dall, Alexander Van't Hof, Oren Laadan, and Jason Nieh. 2011. Cells: A Virtual Mobile Smartphone Architecture. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles (SOSP '11). ACM, New York, NY, USA, 173--187.
[23]
Daniel Ashbrook, Patrick Baudisch, and Sean White. 2011. Nenya: Subtle and Eyes-free Mobile Input with a Magnetically-tracked Finger Ring. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2043--2046.
[24]
Alexander De Luca, Emanuel von Zezschwitz, Ngo Dieu Huong Nguyen, Max-Emanuel Maurer, Elisa Rubegni, Marcello Paolo Scipioni, and Marc Langheinrich. 2013. Back-of-device Authentication on Smartphones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2389--2398.
[25]
Thomas Deselaers, Daniel Keysers, Henry Rowley, Li-Lun Wang, Victor Cărbune, Ashok Popat, and Dhyanesh Narayanan. 2017. Google Handwriting Input in 82 languages on your Android mobile device. https://research.googleblog.com/2015/04/google-handwriting-input-in-82.html. {Online; accessed 18-April-2018}.
[26]
Dmitriy Genzel, Ashok C. Popat, Nemanja Spasojevic, Michael Jahr, Andrew Senior, Eugene Ie, and Frank Yung-Fong Tang. 2011. Translation-Inspired OCR. In ICDAR-2011.
[27]
Nicholas Gillian, R. Benjamin Knapp, and Sile O'Modhrain. 2011. Recognition Of Multivariate Temporal Musical Gestures Using N-Dimensional Dynamic Time Warping. In Proceedings of the International Conference on New Interfaces for Musical Expression. 337--342.
[28]
Otto Huhta, Swapnil Udar, Mika Juuti, Prakash Shrestha, Nitesh Saxena, and N. Asokan. 2017. Pitfalls in Designing Zero-Effort Deauthentication: Opportunistic Human Observation Attacks. In 23nd Annual Network and Distributed System Security Symposium, NDSS 2017, San Diego, California, USA, February 21-24, 2017.
[29]
Amy K. Karlson, A.J. Bernheim Brush, and Stuart Schechter. 2009. Can I Borrow Your Phone?: Understanding Concerns when Sharing Mobile Phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 1647--1650.
[30]
Wolf Kienzle and Ken Hinckley. 2013. Writing Handwritten Messages on a Small Touchscreen. In Proceedings of the 15th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '13). ACM, New York, NY, USA, 179--182.
[31]
Kenrick Kin, Björn Hartmann, Tony DeRose, and Maneesh Agrawala. 2012. Proton++: A Customizable Declarative Multitouch Framework. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, New York, NY, USA, 477--486.
[32]
S. Kurkovsky and E. Syta. 2010. Digital natives and mobile phones: A survey of practices and attitudes about privacy and security. In 2010 IEEE International Symposium on Technology and Society. 441--449.
[33]
Yang Li. 2010. Gesture Search: A Tool for Fast Mobile Data Access. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, New York, NY, USA, 87--96.
[34]
Yunxin Liu, Ahmad Rahmati, Yuanhe Huang, Hyukjae Jang, Lin Zhong, Yongguang Zhang, and Shensheng Zhang. 2009. xShare: Supporting Impromptu Sharing of Mobile Phones. In Proceedings of the 7th International Conference on Mobile Systems, Applications, and Services (MobiSys '09). ACM, New York, NY, USA, 15--28.
[35]
Hao Lü, James A. Fogarty, and Yang Li. 2014. Gesture Script: Recognizing Gestures and Their Structure Using Rendering Scripts and Interactively Trained Parts. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1685--1694.
[36]
Hao Lu and Yang Li. 2015. Gesture On: Enabling Always-On Touch Gestures for Fast Mobile Access from the Device Standby Mode. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3355--3364.
[37]
Diogo Marques, Ildar Muslukhov, Tiago Guerreiro, Luís Carriço, and Konstantin Beznosov. 2016. Snooping on Mobile Phones: Prevalence and Trends. In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016). USENIX Association, Denver, CO, 159--174. https://www.usenix.org/conference/soups2016/technical-sessions/presentation/marques
[38]
Chulhong Min, Saumay Pushp, Seungchul Lee, Inseok Hwang, Youngki Lee, Seungwoo Kang, and Junehwa Song. 2014. Uncovering Embarrassing Moments in In-situ Exposure of Incoming Mobile Messages. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp '14 Adjunct). ACM, New York, NY, USA, 1045--1054.
[39]
Patrick Mochel. 2005. The sysfs Filesystem. In Proceedings of the Linux Symposium, Vol. 1. 313--326.
[40]
Miguel A. Nacenta, Yemliha Kamber, Yizhou Qiang, and Per Ola Kristensson. 2013. Memorability of Pre-designed and User-defined Gesture Sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 1099--1108.
[41]
Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. FingerIO: Using Active Sonar for Fine-Grained Finger Tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1515--1525.
[42]
Xudong Ni, Zhimin Yang, Xiaole Bai, A. C. Champion, and D. Xuan. 2009. DiffUser: Differentiated user access control on smartphones. In 2009 IEEE 6th International Conference on Mobile Adhoc and Sensor Systems. 1012--1017.
[43]
Taiwoo Park, Jinwon Lee, Inseok Hwang, Chungkuk Yoo, Lama Nachman, and Junehwa Song. 2011. E-Gesture: A Collaborative Architecture for Energy-efficient Gesture Recognition with Hand-worn Sensor and Mobile Devices. In Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems (SenSys '11). ACM, New York, NY, USA, 260--273.
[44]
Lukasz Piwek and Adam Joinson. 2016. "What do they snapchat about?" Patterns of use in time-limited instant messaging service. Computers in Human Behavior 54 (2016), 358--367.
[45]
Julian Seifert, Alexander De Luca, Bettina Conradi, and Heinrich Hussmann. 2010. TreasurePhone: Context-Sensitive User Data Protection on Mobile Phones. Springer Berlin Heidelberg, Berlin, Heidelberg, 130--137.
[46]
Longfei Shangguan, Zimu Zhou, and Kyle Jamieson. 2017. Enabling Gesture-based Interactions with Objects. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '17). ACM, New York, NY, USA, 239--251.
[47]
Michael Sherman, Gradeigh Clark, Yulong Yang, Shridatt Sugrim, Arttu Modig, Janne Lindqvist, Antti Oulasvirta, and Teemu Roos. 2014. User-generated Free-form Gestures for Authentication: Security and Memorability. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '14). ACM, New York, NY, USA, 176--189.
[48]
H. Shimodaira, T. Sudo, M. Nakai, and S. Sagayama. 2003. On-line overlaid-handwriting recognition based on substroke HMMs. In Document Analysis and Recognition, 2003. Proceedings. Seventh International Conference on. 1043--1047.
[49]
Anselm Strauss and Juliet Corbin. 1994. Grounded theory methodology. Handbook of qualitative research (1994), 273--285.
[50]
Emanuel von Zezschwitz, Sigrid Ebbinghaus, Heinrich Hussmann, and Alexander De Luca. 2017. You Can'T Watch This!: Privacy-Respectful Photo Browsing on Smartphones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 4320--4324.
[51]
Yang Li. 2010. Protractor: A Fast and Accurate Gesture Recognizer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 2169--2172.
[52]
Wei Wang, Alex X. Liu, and Ke Sun. 2016. Device-free Gesture Tracking Using Acoustic Signals. In Proceedings of the 22Nd Annual International Conference on Mobile Computing and Networking (MobiCom '16). ACM, New York, NY, USA, 82--94.
[53]
Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures Without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST '07). ACM, New York, NY, USA, 159--168.
[54]
Yina Ye and Petteri Nurmi. 2015. Gestimator: Shape and Stroke Similarity Based Gesture Recognition. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). ACM, New York, NY, USA, 219--226.
[55]
Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand Writing on Stylus Keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 97--104.
[56]
Y. Zou, Y. Liu, Y. Liu, and K. Wang. 2011. Overlapped Handwriting Input on Mobile Phones. In 2011 International Conference on Document Analysis and Recognition. 369--373.

Cited By

View all
  • (2022)Sharing without scaringProceedings of the Eighteenth USENIX Conference on Usable Privacy and Security10.5555/3563609.3563645(671-685)Online publication date: 8-Aug-2022
  • (2021)Notification privacy protection via unobtrusive gripping hand verification using media soundsProceedings of the 27th Annual International Conference on Mobile Computing and Networking10.1145/3447993.3483277(491-504)Online publication date: 25-Oct-2021
  • (2020)Context Aware Photo Protection for In-Situ Sharing22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3406324.3409261(1-3)Online publication date: 5-Oct-2020
  • Show More Cited By

Index Terms

  1. PrivacyShield: A Mobile System for Supporting Subtle Just-in-time Privacy Provisioning through Off-Screen-based Touch Gestures

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 2, Issue 2
      June 2018
      741 pages
      EISSN:2474-9567
      DOI:10.1145/3236498
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 05 July 2018
      Accepted: 01 April 2018
      Revised: 01 April 2018
      Received: 01 February 2018
      Published in IMWUT Volume 2, Issue 2

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Gesture recognition
      2. Gesture segmentation
      3. In-situ privacy
      4. In-situ usability
      5. Just-in-time privacy provisioning
      6. Mobile Systems

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)26
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 01 Nov 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)Sharing without scaringProceedings of the Eighteenth USENIX Conference on Usable Privacy and Security10.5555/3563609.3563645(671-685)Online publication date: 8-Aug-2022
      • (2021)Notification privacy protection via unobtrusive gripping hand verification using media soundsProceedings of the 27th Annual International Conference on Mobile Computing and Networking10.1145/3447993.3483277(491-504)Online publication date: 25-Oct-2021
      • (2020)Context Aware Photo Protection for In-Situ Sharing22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3406324.3409261(1-3)Online publication date: 5-Oct-2020
      • (2018)Exploring Relationship-aware Dynamic Message Screening for Mobile MessengersProceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers10.1145/3267305.3267673(134-137)Online publication date: 8-Oct-2018

      View Options

      Get Access

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media