Preserving Privacy – More Than Reading a Message Susanne Furman and Mary Theofanos National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, Maryland, 20899, USA {Susanne.Furman,Mary.Theofanos}@nist.gov Abstract. Social media has become a mainstream activity where people share all kinds of personal and intimate details about their lives. These social networking sites (SNS) allow users to conveniently authenticate to the third-party website by using their SNS credentials, thus eliminating the need of creating and remembering another username and password but at the same time agreeing to share their personal information with the SNS site. Often this is accomplished by presenting the user with a dialog box informing them that they will be sharing information. We were interested in determining if SNS users authenticating to a third-party website with their SNS credentials, were reading the informational message and if changing the message format would impact the choice to continue or cancel. Format type did not alter the participant’s choice to continue. Eye-tracking data suggests that the participants who chose to continue read some of the words in the message. Keywords: Access to the Web, privacy, eye tracking, authentication. 1 Background By creating an identity on a website, users gain the ability to engage in a wide range of activities like e-mail, online shopping and banking, or in social networking activities. Today’s Web is very site-centric where the user is expected to maintain a separate copy of their username and corresponding password for each site they join [14]. To create and maintain these separate identities, users must create and remember their usernames and passwords; populate a profile often with the same data across sites; and remember each site’s different password rules [9]. Federated identity management offers the user a solution to reduce the burden of multiple identities. It provides a technology that websites can offer to users as a single sign-on (SSO) experience. SSO offers the user an easier experience through a more consistent and less frequent log-on process. The user can authenticate once and access their protected resources across multiple websites [9]. Facebook1 offers its users a type of SSO that allows users to connect their SNS identity, friends, and personal information to any site that offers this service. 1 Disclaimer: Any mention of commercial products is for information only; such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that these entities, materials, or equipment are necessarily the best for the purpose. C. Stephanidis and M. Antona (Eds.): UAHCI/HCII 2014, Part IV, LNCS 8516, pp. 14–25, 2014. © Springer International Publishing Switzerland 2014 Preserving Privacy – More Than Reading a Message 15 Through this technology, a user’s SNS ID becomes a gateway that provides access to the digital world. The third-party website offering this service benefits through distributing their services with minimal effort to SNS users and the SNS users benefit by eliminating the cumbersome process of registering, and creating and remembering a username and password [8]. Social media has become a mainstream activity where people share all kinds of personal and intimate details about their lives; photos of their children, family and friends; their email address; and even their physical location. Many of these users are unaware that both their personal information as well as that of their friends is passed to these third parties when they authenticate using their SNS credentials. SNS websites do inform users (through a pop-up dialog window) that authentication to a third-party website using their SNS credentials results in their personal information being shared with the website. The user is typically given the option to agree to or cancel the authentication request. Privacy advocates are concerned that these SNS authentication services pose new sets of concerns about how data are collected and shared among websites. One major concern is that because users don’t completely understand how these services work and have the mistaken impression that their data isn’t being collected [8]. Many popular SNS applications transmit users’ identifying information to dozens of ad and internet tracking companies. This practice affects millions of SNS application users, even those who implement the strictest privacy settings. Ten of the most popular applications reviewed transmitted not only a user’s ID and personal information but also personal information about the user’s friends to outside companies [13]. Approximately 850 million times per month, apps have asked SNS users to release their basic information [17]. Most SNS do not make it clear exactly what user information is shared and to whom it is given, so many SNS users make the assumption that their information is safe [12]. Also, SNS privacy settings are often difficult to manage and users often do not change them from their defaults, which are generally set to maximize the visibility and sharing of users’ profile information [7]. So, it is not surprising that SNS users are unaware of and don’t understand the privacy dangers from data sharing with third-party apps or websites that they connect to using their SNS accounts. Many users believe that online communities are safe and don’t understand privacy policies [16]. They do not realize the extent to which they share not only their own information, but that of their friends as well [2]. As a result, they feel their risk of unwilling or inadvertent disclosure of personal information is very low [15]. Motivation is directly tied to attention so the more motivated a person is to perform a particular task, the more effort and attention they will devote [3]. Authentication tasks (i.e., entering a username and password) or privacy and security tasks (e.g., reading a privacy policy) are not considered primary tasks but are viewed as secondary and a requirement for completion of primary tasks. 16 S. Furman and M. Theofanos Another factor impacting attention is that often security messages resemble other non-essential pop-up dialogs. As a result users often fail to realize that they contain important information about how their privacy and security will be affected by taking a certain action. Often users disregard or do not read these dialogs and only click “Agree” or “OK” to close the dialog message [18]. Message format can also impact reading. Information presented in a list format can benefit users who are searching for information that is embedded in other text. List formats facilitate easier information acquisition than paragraph formats, due to their lower print density [19]. The results of a 1995 study demonstrated that older patients were better able to find, understand, and later recall medication instructions that were presented in list format rather than paragraph format [10]. 2 Research Goal Millions of SNS users are utilizing their SNS identities to authenticate to third-party websites. Many are unaware that both their personal information and that of their friend’s is passed to these third parties. SNS websites typically inform the user via a dialog message in a pop-up window that by agreeing they will be sharing personal data with the third-party website. However, the user does have the option to agree or cancel the authentication request. We wanted to investigate if users read the authentication dialog message explaining that their personal information would be shared with a third-party website. Also we wondered if manipulating the format of the dialog message (i.e., sentence-style, list-style, and list with personal data displayed) would impact whether participants would decide to continue with the authentication process. Previous research testing message formats explains the lack of observable effects by claiming that participants did not read or were habituated to the dialog messages. The study supported this explanation with screen capture videos that show the message was displayed for a period of time but clarifies that without an eye tracker they could not be certain users read the message [5]. Specifically we were interested in: • Will SNS users authenticate to a third-party website using their SNS credentials after being informed that their personal information will be shared with the website? • Will manipulating the format of the dialog message (i.e., sentence-style, liststyle, and list with personal data displayed) influence a participant’s choice to continue with the authentication process? Figure 1 shows the three formatted messages. • What information does the eye-gaze data show about the content participants viewed in the dialog message if any? Does the format of the message make any difference in a participant’s authentication choice to continue? Preserving Privacy – More Than Reading a Message 17 Fig. 1. Format message conditions 3 Methodology 3.1 Participants A study was conducted at NIST for a period of four weeks to examine these three message conditions. Participants were required to have an SNS account and received $50 for participation in a 30 min session. Of the 120 participants initially recruited, 117 completed the study. One participant withdrew, and hardware malfunctions prevented two others from completing the study. Participants ranged in age from 17 to over 60 and age groups were fairly equally represented. There were fewer males (35 %) than females (65 %). Most of the participants (92 %) had at least some college experience. 3.2 Instruments and Apparatus Software code was written to intercept and replace the original SNS authentication message dialog. The code replaced the original dialog message with a similar message in one of three formats identified above. Participants were randomly assigned to one of the three format conditions at the start of their session. 18 S. Furman and M. Theofanos The study used a PC integrated with the Tobii T60XL1 eye tracker paired with a high-resolution 24” TFT wide screen monitor having a display pixel resolution of 1900 x 1200. The eye tracker looked like a normal computer display, and its high resolution cameras were invisible to the participant. The online version of the Wall Street Journal (WSJ.com website)1 was shown in Internet Explorer 9. All precision measurements were done at 60 Hz sampling rate, and all participants completed a nine point eye calibration prior to the start of the session. Gaze data was logged by Tobii Studio software. 3.3 Sessions After signing the consent form, participants logged into their SNS account. The researcher started the intercept software, assigned the participant to one of the three format conditions, and started the eye-tracking device. The intercept software launched the online version of the third-party website. The researcher asked participants to find a news story of interest, and once selected, the researcher provided the following scenario: You are sitting at home and are very interested in reading this news story but you don’t have time right now. Given the number of times that news sites refresh their content, you are concerned that you may not find the story when you come back. Please save the news story so that you can read it later. The researcher instructed the participant to do whatever they would do at home. Saving the news story on the site required the participant to set up an account or use their SNS credentials to authenticate to the third-party website. If the participant chose not to use their SNS credentials to authenticate, the researcher stopped the eye tracker and instructed them to complete an online survey about their SNS use and privacy concerns. If the participant chose to use their SNS credentials to authenticate, the choice was recorded, the eye tracker captured the participant’s interaction with the dialog message, and particpants completed the online survey. After session completion, the researcher logged the participant out of their SNS account, removed the intercept application from their SNS apps, asked if there were any questions, and thanked the participant for their time. Participants were compensated for their time prior to leaving the laboratory. 4 Results 4.1 Authentication Using SNS Credentials The participants were randomly assigned to one of the format conditions at the start of their session resulting in 40 participants in the sentence-style format, 38 in the list, Preserving Privacy – More Than Reading a Message 19 and 39 in the list with data displayed formats. Fifty-nine (51 %) of the 117 participants chose to use SNS credentials to authenticate to the third-party website. Of the 40 participants in the sentence-style format, 23 (58 %) authenticated using their SNS credentials; of the 38 participants in the list-style, 17 (45 %) authenticated; and of the 39 in list with data format, 19 (49 %) authenticated. A chi squared test was performed to determine whether the format conditions were equal with respect to authentication choice. Although there were more participants using their SNS credentials to authenticate in the sentence-style format, use/did not use data comparing the three conditions were not significant, χ2 (2, N= 59) = 1.3382, p > 0.5. 4.2 Eye-Tracking Data Tobii Studio software captured the gaze data as participants viewed the third-party website. Of particular interest was where participants allocated their visual attention when the SNS SSO dialog box appeared. For that reason, only those participants who chose to use their SNS credentials and those who started to use their SNS credentials but did not continue were included in the eye-tracking analyses. AOI 1 – Dialog Message Analyses. An area of interest (AOI) is a user-defined area on the stimulus that is used for capturing and analyzing the eye-tracking data (see Figure 2). The message content, title, and action buttons on the SNS authentication dialog message were marked as the initial AOI. Of particular interest was the amount of attention the AOI received. Therefore, the mean and standard deviations were calculated for the following metrics: (1) fixation count: the number of times the participant fixates or pauses over areas of interest - this indicates how many times the participant looks at the area; (2) total fixation duration: measures the sum of the length of time for all fixations or pauses within an AOI measured in seconds; (3) total visit duration: how much time in seconds the participant spent within an area of interest - this indicates the level of the participant’s involvement with the area. The overall mean total visit duration was 5.56 s. Participants in the list format condition spent less time viewing the AOI than those in the sentence-style or list with data displayed format conditions. Individuals who chose not to connect had the longest total visit duration. Table 1 shows the total visit duration, fixation counts, and fixation duration means and standard deviations across format conditions. The mean fixation count for all participants was 21.04 fixations. The mean fixation count showed similar results with the sentence-style format having fewer fixations than the list or the list with data displayed formats. However, the participants who did not choose to connect had the most fixations. The mean fixation duration for all participants was 4.39 s. The results across format condition show that the participants in the sentence format condition spent less time fixating within the AOI than the list format participants. 20 S. Furman and M. Theofanos Table 1. AOI total fixation, fixation count, and total fixation duration (seconds) mean and standard deviations across format condition Format Condition Sentence-style List List with Data Did Not Connect Total Visit Duration(s) Mean St Dev 4.85 4.35 4.70 3.74 8.07 6.42 13.01 5.94 Fixation Count Mean 16.75 17.8 26.67 31 St Dev 14.11 11.16 21.8 16.02 Fixation Duration(s) Mean St Dev 3.28 2.78 3.59 3.97 6.16 5.1 6.03 2.6 Multiple AOI Analyses. We separated the dialog box into separate AOIs, including the basic information area (e.g., name, email, gender), the profile picture, and the participant’s photos of friends. The sentence-style format did not include any shared photos and is not included in these analyses. The main content AOI included the personal information that the participants shared from their profile. Participants spent less time fixating within the list format AOI than within the AOIs in the sentence-style, and the “did not connect“ format conditions. The list format had fewer fixations than the sentence-style format, list with data format and didn’t connect format groups. They also had a shorter visit duration than the sentence-style format, list with data format, and didn’t connect format conditions. Table 2 shows the fixation duration, fixation count, and total visit duration means and standard deviations across format conditions. Table 2. Multiple AOI total fixation duration, fixation count, and total fixation duration means and standard deviations Format Condition Total Visit Duration (s) Mean St Dev Fixation Count Mean St Dev Fixation Duration (s) Mean St Dev Personal Information Message Content Sentence 1.95 2.53 8.93 10.17 1.63 List 1.66 1.64 7.25 6.21 1.45 List with Data 4.13 4.82 17 19.93 3.68 Did Not Connect 6.85 3.38 18.6 11.59 3.6 Photos of Friends Sentence Did not have photos on their message dialogs List .75 .56 3.83 2.17 .62 List with Data 1.47 1.25 4.86 3.44 1.32 Profile Photo Sentence .61 .13 3.33 .58 .58 List .21 .13 1.25 .05 .21 List with Data .38 .21 2.33 1.53 .38 2.19 1.37 4.44 1.79 .55 1.25 .16 .13 .21 Preserving Privacy – More Than Reading a Message 21 Large within group standard deviations precluded significant results for any pairwise comparisons, but there appeared to be a trend in the data across formats. A trend analysis attempts to spot a pattern or trend in the data. A Likelihood ratio test was conducted and showed a significant monotonic trend for fixation count and total visit duration for both the dialog box AOI and personal content AOI, p < 0.05. This was also the case for the fixation count for the dialog AOI. A heat map was created to visualize the eye-tracking data (see Figure 2). The heat maps were consistent with the eye-tracking data and showed that the majority of participants did read some of the content of the dialog message that indicated their personal information would be shared with the third-party website. Fig. 2. AOI and heat map example Survey Responses. Our online SNS use and privacy survey data shows that 57 % of the 117 participants change their SNS privacy settings once a year and approximately 80 % of those individuals set the level at strict or extremely strict where only their friends can see their information. The majority of participants were very concerned about having their cell phone (91 %), postal address (91 %), email address (78 %), and picture (64 %) shared with a third-party website without their permission. That concern does not appear to transfer to SNS accounts where the respondents are willing to share their name (83 %), gender (80 %), birthday (61 %), picture (72 %), interests (71 %), friends (64 %), current city (65 %), and education (66 %). 22 5 S. Furman and M. Theofanos Discussion We asked participants to save a news story they selected from a third-party website requiring authentication either by setting up an account or using their SNS credentials. Approximately half of the participants chose to use their SNS credentials to authenticate to the third-party website. We hypothesized that participants who were presented with the list with data format would spend more time reading the dialog box than the other formats. Unfortunately the standard deviations precluded finding any significant differences for pairwise comparisons. However, we did see a trend showing that there is a significant increase in reading times across the formats. We also hypothesized that these participants upon seeing their personal information displayed would cancel authentication. While these participants spent more time reading the dialog message, they did continue with authentication. One might assume with eye-tracking data that there is a relationship between what the participant is looking at and what they are thinking about. Unfortunately this isn’t necessarily true. What eye tracking provides is an objective way of determining where the participant’s visual attention is located. Our data showed that most of the participants using their SNS credentials actually read some part of the message dialog pop-up informing them that their personal information would be shared. College students read about 300 words per minute with an average of 200 microseconds per word [19]. The dialog box for the sentence-style format contained about 30 words, while the other formats were dependent on the information the participant displayed in their profile. Word processing in text is reflected in fixation time where short, regularly spelled, frequent in occurrence, and semantically/ syntactically predictable words are fixated for a shorter period of time. Words that are predictable are often skipped, and better readers average about 84 fixations per 100 words, often skip more words, and have shorter fixations [11]. The total mean visit duration across format conditions was approximately eight seconds, giving the participants enough time to read approximately 10 words and the content was also predictable. The mean fixation count across formats was 23 accounting for approximately 19 words which leads us to believe that participants read the content in the dialog box. 6 Conclusions Our eye-tracking data shows that participants read content in the SNS authentication dialog message box but also continued with authentication even though they were informed their personal information was about to be shared. Although this does not indicate they processed any of the words or content, it does seem to suggest that they did not habituate to the dialog box or just select the agree button to close the dialog so that they could continue with their main task. Preserving Privacy – More Than Reading a Message 23 There was a signficant trend in reading times across formats. But the format style did not impact participants’ decision to continue with authentication. Participants chose to share their personal information by selecting to ‘allow’ the SNS to authenticate to the third-party website. Participants comments were inconsistent with their actions. Participants admit to sharing many types of personal information on their SNS pages but are more concerned with sharing personal information with others (e.g., ad companies or marketers). They were adamantly opposed to sharing their personal information with a third-party website without their consent. Even though participants gave consent, they didn’t seem to realize that their SNS privacy settings did not apply to the sharing of their personal information when they authenticated using their SNS credentials. Some researchers think that the SNS authentication message is too generic and does not adequately convey this data sharing. They believe that users do not realize they share so much information with these third-party websites and applications [2]. While others believe that the average user is reacting to these interruption dialog messages with responses that range from mild irritation to annoyance, and users quickly learn to visually and cognitively dismiss them [1]. Still others think that over the years, users are trained to click dialogs away to complete the primary task. Because these interception dialogs interrupt the user’s primary task, the users do not bother to read them and as a result, do not heed warnings [4]. Our participants may not have fully understood the risks associated with innocentappearing disclosure of information like their hometown or current city. As we are quite aware, users’ attention is scarce, and identity management is seldom a user’s primary goal. The participants in our study may have quickly scanned the message and did not fully understood what they were consenting to. Or quite simply, our participants may have continued authentication even though they were informed that their personal information was being shared because they believed the third-party website to be a reputable site that would protect their information. Having a single sign-on method such as SNS identity eases the burden for users trying to authenticate to a new online service. Often users are willing to trade-off some possible or unknown risks for the convenience of not having to set up another account, and create and remember another username and password [6]. The Federal Trade Commission and others have policy goals of adequately informing users of when their personal information might be shared. Future research should explore other types of alerts and explore whether users understand the potential implications and consequences of their choices. References 1. Bahr, G.S., Ford, R.A.: How and why pop-ups don’t work: Pop-up prompted eye movements, user affect and decision making. Computers in Behavior 27(2), 776–783 (2010) 2. Besmer, A., Lipford, H.R.: Users’ (mis)conceptions of social applications. In: Mould, D., Noël, S. (eds.) Graphics Interface, pp. 63–70. ACM, New York (2010), http://hci.uncc.edu/pubs/Misconceptions.pdf (retrieved ) 24 S. Furman and M. Theofanos 3. Bitgood, S.: The role of attention in designing effective interpretive labels. Journal of Interpretation Research 5(2), 31–45 (2000), http://www.jsu.edu/psychology/docs/5.1-role_of_attention.pdf (retrieved ) 4. Bohme, R., Kopsell, S.: Trained to accept?: a field experiment on consent dialogs. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 2403–2406. ACM (2010) 5. Egelman, S.: My profile is my password, verify me! The privacy/convenience tradeoff of facebook connect. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2369–2378. ACM (2013) 6. Good, N.S., Grossklags, J., Mulligan, D.K., Konstan, J.A.: Noticing notice: a large-scale experiment on the timing of software license agreements. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 607–616. ACM (April 2007) 7. Gross, R., Acquisiti, A.: Information revelation of the privacy in online social networks. In: Proceedings of the 2005 ACM Workshop on Privacy in Electronic Society, pp. 71–80. ACM, New York (2005), doi:10.1145/1102199.1102214 8. Ko, M.N., Cheek, G.P., Shehab, M.: Social-networks connect services. Computer 43(8), 37–43 (2010), doi:10.1109/MC.2010.239; MacMillan, D.: FB connect: Your 8,000 hidden friends. Bloomburg BusinessWeek: Technology (April 2, 2009) http://www.businessweek.com/technology/content/apr2009/ tc2009041_649562.htm (retrieved ) 9. Maler, E., Reed, D.: The venn of identity, options and issues in federated identity management. IEEE Security & Privacy 6(2), 16–23 (2008), doi:10.1109/MSP.2008.50 10. Morrow, D., Leirer, V., Altieri, P.: List formats improve medication instructions for older adults. Educational Gerontology: An International Quarterly 21(2), 151–166 (1995), doi:10.1080/0360127950210204 11. Rayner, K., Juhasz, B.J., Pollatsek, A.: Eye Movements During Reading. In: Snowling, M.J., Hulme, C. (eds.) The Science of Reading: A Handbook, pp. 79–97. Blackwell Publishing Ltd., Oxford (2008), doi:10.1002/978047-757642.ch5 12. Roberts, K.K.: Privacy & perceptions: How facebook advertising affects its users. The Elon Journal of Undergraduate Research in Communications 1(1), 24–34 (2010), http://www.elon.edu/docs/e-web/academics/communications/ research/03RobertsEJSpring10.pdf 13. Steel, E., Fowler, G.A.: Facebook in privacy breach: Top-ranked applications transmit personal IDs, a journal investigation finds. The Wall Street Journal (October 17, 2010), http://online.wsj.com/article/SB1000142405270230477280457555 8484075236968.html (retrieved) 14. Sun, S.T., Boshmaf, Y., Hawkey, K., Beznosov, K.: A billion keys, but few locks: The crisis of web single sign-on. In: Proceedings of the 2010 Workshop on New Security Paradigms, pp. 61–72. ACM, New York (2010), doi:10.1145/1900546.1900556 15. Tow, W., Newk-Fon, H., Dell, P., Venable, J.: Understanding information disclosure behavior in Australian Facebook users. Journal of Information Technology 25(2), 126–136 (2010), doi:10.1057/jit.2010.18 16. Tuunainen, V.K., Pitkanen, O., Hovi, M.: Users’ awareness of privacy on online social networking sites – case Facebook. In: BLED 2009 Proceedings (Paper 42) (2009), http://aisel.aisnet.org/bled2009/42 (retrieved) Preserving Privacy – More Than Reading a Message 25 17. Wang, N., Xu, H., Grossklags, J.: Third party apps on facebook: Privacy and the illusion of control. In: Proceedings of the 5th ACM Symposium on Computer Human Interaction for Management of Information Technology (Article No. 4), ACM, New York (2011), doi:10.1145/2076444.2076448 18. West, R.: The psychology of security. Communications of the ACM 51(4), 34–40 (2008), doi:10.1145/1330311.1330320 19. Wolgater, M.S., Shaver, E.F.: Evaluation of list vs. paragraph text format on search time for warnings symptoms in a product manual. Advances in Occupational Ergonomics and Safety 4, 434–438 (2001)