Security and Usability of a Personalized User Authentication Paradigm: Insights from a Longitudinal Study with Three Healthcare Organizations

We examined papers from the ACM Digital Library and IEEE Xplore and used the following keywords in our queries: authentication; password; biometric; locimetric; drawmetric; healthcare; health. We reviewed 40 papers based on our inclusion criteria, which were published from January 1, 2015, to January 6, 2021.

Several literature reviews on user authentication research and practices in the healthcare domain exist. For example, Jayabalan and O'Daniel [56] and Fernández-Alemán et al. [41] reported a study on authentication factors in electronic health records. Mason et al. [74], Fatima et al. [40], and Okoh and Awad [79] reviewed the current state in biometric authentication in healthcare environments. Schwartze et al. [90] conducted a systematic literature review on authentication systems for securing clinical documentation workflows. Kumar et al. [65] conducted a review on user authentication in gadget-free healthcare environments.

The healthcare domain embraces unique constraints and characteristics [64] that are related to different access control scenarios that need to be supported not only from a healthcare staff perspective but also from a patient-centric approach. Starting from the medical staff (e.g., doctors, nurses, caregivers), there are numerous scenarios in which stakeholders interact with medical systems that are deployed on heterogeneous devices and within different contexts of use [38, 108]. For example, when doctors visit patients within the hospital, they typically have access to the patients’ records through a smartphone or tablet device [7], whereas accessing more controlled places like surgery rooms, intensive care rooms, and such necessitates a multi-factor and/or biometric-based authentication approach. From a patients’ perspective, access to services and data is limited to a Web-based solution in which patients access their personal health records using a textual password [38], in combination with a one-time password as a second layer for authentication. These authentication methods are analyzed in the next section under the following perspectives: security, privacy, usability, memorability, user experience, user acceptance, and trust.

The majority of healthcare organizations currently employ traditional textual password solutions [61, 63, 110]. However, textual password schemes have known security issues and are constantly becoming less usable and less memorable due to strict password policies [38, 98]. To address memorability issues of using multiple textual passwords in different services within the hospital, healthcare organizations deploy Single Sign-On (SSO) solutions that allow the end users to enter their password credentials once in the beginning, and then access several independent services within their organization, without requiring them to re-enter their credentials. However, studies have shown that although SSO is effective in administrative contexts of medical staff, it creates difficulties in collaborative contexts due to the strict password policies of the healthcare organization. For example, medical staff need to frequently logout and login when they change location in the hospital, or after a small period of inactivity [38, 49, 76]. According to Heckle and Lutters [49], given that the clinical staff are frequently changing locations in the hospital when taking care of patients, they are required to continuously login and logout from the system (e.g., due to system timeouts, different access control depending on the system used in the corresponding location, walking away from the screen). In addition, studies have shown that medical staff may typically utilize SSO features within their network; however, they also utilize different textual password credentials for accessing systems that are off the network of their healthcare organization.

Furthermore, the literature reveals other proposals for improving password security and usability, such as through mutual authentication or two-way authentication [52, 66]; group-based authentication for assisting the access and sharing of electronic health records among trusted members [69]; approaches suggesting practical recommendations for the creation of strong and usable passwords that combine minimum-strength, minimum-length, and blocklist requirements [98]; providing guidance and feedback during password creation [91]; and proposing alternative mechanisms, such as graphical user authentication schemes, which require from users to draw secret gestures on an image or select a sequence of images as their secret key [1, 11, 13, 28]. In addition, healthcare environments entail unique constraints and characteristics with regard to the end user activities, workflows, and context of use. For example, given that patients and medical staff access data and services from different locations (e.g., within the hospital or through a trusted location), hence several works have proposed location-based authentication approaches in healthcare environments [56, 72].

In the past years, a variety of directives and regulations have been proposed that require the deployment of two-factor or multi-factor authentication solutions in healthcare environments aiming to add multiple layers of security in the user authentication process (e.g., U.S. Health Insurance Portability Accountability Act (HIPAA), European Union Agency for Network and Information Security (ENISA)). Multi-factor authentication solutions typically combine a knowledge-based authentication method (e.g., a textual password) along with a token-based authentication method (e.g., smart card, one-time password sent to the users’ smartphone) [66]. Token-based authentication utilizing smart cards is one of the most common methods for multi-factor authentication in healthcare systems with numerous proposals in the literature [44, 48, 56, 60, 66, 71]. Works have also proposed three-factor authentication combining textual passwords, smart cards, and biometric technology for increased security and usability [33, 35, 57], and the work by Amin et al. [5] proposed a remote patient mutual authentication scheme using smart cards and elliptic curve cryptography. Furthermore, with the advent of smartphone technology in recent years, token-based authentication is achieved with the usage of the user's smartphone device that acts as a trusted token for multi-factor authentication [1, 45, 56, 92].

Finally, biometric-based authentication is constantly gaining market share, aiming to provide increased usability for accessing medical records without compromising the patients’ privacy and security [79]. Biometric technologies are typically based on information about the users’ physical characteristics (e.g., fingerprint, iris, face, voice) and/or behavioral characteristics (e.g., typing patterns, interaction and engagement patterns) [54]. Numerous biometric-based authentication schemes have been proposed that retrieve the users’ biometric characteristics either (1) through their interaction device (smartphone, laptop, etc.) [46, 111] or (2) through surroundings within smart environments, such as smart healthcare, automated monitoring, and smart manufacturing [47, 65]. Various biometric-based approaches have been proposed in the literature for granting access to medical records by utilizing voice acoustics’ analysis and audio-visual identity verification [94], physiological signals analysis (e.g., photoplethysmogram signals) [23, 112], face and voice analysis [75], hand geometry analysis [78], hand gesture spatial interaction analysis based on fingertips and joints [53], and periocular-based analysis [74].

Stakeholders from three public European healthcare organizations, which support thousands of patients and users annually, have participated in the survey: Zuyderland Medical Center,¹ Sittard, the Netherlands; Hospital Clinic Barcelona,² Barcelona, Spain; and Western General Hospital within NHS Lothian,³ Edinburgh, Scotland. A total of nine individuals participated in the user survey with varying roles in the aforementioned organization (i.e., chief information security officers, enterprise architects, IT department managers, security experts, doctors, and project managers). Each stakeholder participated in a semi-structured interview that lasted for approximately 45 minutes each. Participation in the interviews was voluntary and could be canceled at any time.

A series of semi-structured interviews was conducted with key stakeholders from the participating healthcare organizations. The interviews were split into two parts. In Part A, participants were initially guided to an online consent form, and each one read and agreed to participate. Participants were then introduced to the survey, its purpose, and its objectives. In Part B, we conducted an initial profiling (approximately 5 minutes) of the participants asking questions that relate to the participant's background and position in the organization, with the aim to understand the background of the interviewee and the context of her or his answers. Then we discussed two main topics: Topic 1—User Authentication Policy (approximately 20 minutes), which was focused on eliciting details about the user authentication policy and procedures of the organization (e.g., how the policy was derived, since when the policy is valid), and Topic 2—Technical Details and Workflows (approximately 20 minutes), which was focused on eliciting details with regard to technical and security matters of the currently applied user authentication scheme and policy (e.g., what is the current password complexity of the applied authentication policy, which is the maximum number of days a password may be used).

Responses of the interviewees and the organizations were anonymized. All organizations reported that their user authentication policy is based on current industry standards and best practices, and that they primarily apply textual passwords as their core means for authentication. The main password policy is based on a widely applied policy—that is, a textual password with a minimum length of eight characters containing no part of the user's real name or username, and including a minimum of one uppercase, one lowercase, one symbol, and one numeric character. The policy had variations across organizations in terms of character type and their combination.

Furthermore, one out of three organizations employed multi-factor authentication in certain scenarios: (1) when accessing the patients’ database, medical staff uses an RFID badge, combined with a four-digit PIN code, and (2) when accessing the healthcare system from outside the organization's network, medical staff and patients are required to login with their textual password, combined with a second factor for authentication based on a one-time password and/or a push notification that is sent on a third-party mobile application. In this respect, one participant stated: “Active directory is your first line of defense so that's why when you are internally it's ok and you login with your badge, so your badge is your second factor. If you are outside of the hospital that's also possible, you can get a remote reader at home, so you get an SMS or via the Microsoft app, you can authenticate and get your session” ∼ Security Expert. The same organization applies variations in the policy depending on the role of the user as well as the context of use. For example, exceptions for policies can be requested by end users. In addition, doctors may use their RFID badge to enter the emergency room. In this respect, one participant stated: “People can request exceptions on a policy and then we look at the case and decide whether we can change the policy” ∼ Security Expert.

We further asked participants whether their organization considers investing and deploying biometric technology, and the majority reported that they have considered this technology; however, this has not been implemented yet due to increased costs and known security and privacy issues within biometric technologies. For instance, facial recognition was given a trial by one organization, but there were problems in some cases. One participant stated: “The problem is that if you go a little away from the screen, or two persons are standing, one person is close and one is standing behind the screen, the system did not know which one is the user” ∼ Security Expert.

Moreover, interviews with end users (e.g., administrators, doctors) reveal that a high number of users expressed complaints on the authentication policy: “There are complaints about the complexity of the passwords, the amount of passwords they have to use, changing the passwords, so it's not a very nice picture” ∼ Administrator. Another participant reported that the users easily forget their passwords, either due to holidays or due to the frequent password changing, so there is often the need to reset them via the helpdesk: “They have problems to remember and sometimes they have to put the password in a post-it and the password is not hidden from the public when they are working in their desk” ∼ Security Expert. Several users of the organization also stated that they must remember and use more than one password, a factor that renders the authentication process harder to complete. In addition, the Web browser of these systems does not allow saving the password for the organization: “It feels like quite a large number. I would say at least 10 [passwords]” ∼ Doctor. Finally, users in general feel like they are putting a lot of effort to remember passwords and need to login several times per day. Some participants stated that they are more than willing to change their current authentication scheme, as long as it applies across multiple systems and it is not too complicated to be used: “I certainly will be willing to change as long as it is applied across multiple systems. But if it's a new authentication type that's different for each system then that would cause problems” ∼ Doctor.

Finally, we asked participants to provide details about the “perfect authentication scheme” and a wish list for “better passwords”. The majority responded that they would like to have a secure system that respects their privacy and usability. One participant responded: “I would really like to leave our employees free and choosing what mechanism they want, the only concern is the level of security and its usability” ∼ Manager. Interest was expressed on the deployment of two-factor authentication methods utilizing the users’ smartphones. Another participant was very interested in the integration of alternative and usable authentication schemes; however, concerns relate to the increased complexity and cost of applying new policies and systems in the organization's production line: “There are many procedures in order to make small changes. It is very difficult to implement. We are now testing another user authentication but this takes a lot of time and it will take as much time to implement it” ∼ Security Expert.

To investigate RQ₁, we ran two security analyses to investigate whether there are differences in security strength of the user-created graphical passwords between the control and experimental user groups. The first analysis compared password guessability that was based on a naïve brute-force attack, whereas the second analysis compared password guessability that was based on the PoI-assisted brute-force attack. Figure 2 (left) illustrates the means of password guessability among user groups, as assessed by the naïve and the PoI-assisted brute-force attack model. For the naïve brute-force attack, we ran an independent samples t-test to determine whether the two user groups (control vs. experimental) generated different password strengths in terms of password guessability. The assumption of homogeneity of variances was not violated, as assessed by Levene's test for equality of variances (p = .075). There were no significant outliers in the data, as assessed by inspection of boxplots, and data were normally distributed, as assessed by Shapiro-Wilk's test (p > .05). Results revealed significant differences with a mean difference of 22 million guesses (95% CI, –5.7 million to 1.28 million), t(66) = –1.261, p = .021. In particular, user-chosen graphical passwords of the experimental group required 53 million guesses to crack, whereas for the control group, 31 million guesses were required.

For the PoI-assisted brute-force attack, we ran a Welch t-test to determine whether the two user groups (control vs. experimental) generated different password strengths in terms of password guessability, due to the assumption of homogeneity of variances being violated, as assessed by Levene's test for equality of variances (p = .048). There were no significant outliers in the data, as assessed by inspection of boxplots, and data were normally distributed, as assessed by Shapiro-Wilk's test (p > .05). Results revealed significant differences with a mean difference of 30 million guesses (95% CI, –6 million to –346,000), t(36.165) = –2.140, p = .039. In particular, user-chosen graphical passwords of the experimental group required 47 million guesses to crack, whereas those of the control group required 17 million guesses to crack. Figure 2 (right) illustrates the percentage of passwords cracked indicating that users from the control group exhibited a higher percentage of passwords cracked than users from the experimental group. The percentage of graphical passwords cracked reached 100% for the control group within 2²⁶ guesses, and for the experimental group within 2²⁹ guesses.

To further verify the security strength of the created passwords, we took an extra step to analyze the users’ individual gestures with respect to PoI regions (i.e., regions that attract the users’ attention and are prone to automated guessing attacks) [28]. To identify the PoIs of each image, we followed a semi-automated image analysis approach by applying saliency maps and saliency filters [81] to detect salient regions on the images and computer vision techniques to detect PoIs [28]. Figure 3 (left) illustrates an example of a hospital image used and its corresponding salient regions as detected through the image analysis (Figure 3, right).

A Mann-Whitney U test was run to determine if there were differences in number of PoI selections between the control and experimental group. Distributions of values for the two groups were similar, as assessed by visual inspection. The median number of PoI selections for the control group (1.72) was statistically significantly higher compared to the experimental group (1.53), U = 224.000, z = – 4.561, p < .001, using an exact sampling distribution for U [36]. We further ran an independent-samples t-test, with the user group (control vs. experimental) as the independent variable, and the proportion of gestures falling into PoI regions as the dependent variable. The analysis (Figure 4) revealed that users of the experimental group made a lower proportion of selections falling into PoI regions (0.45 ± 0.04) than users of the control group (0.71 ± 0.04), a statistically significant difference of 0.26 ± 0.05 (95% CI, .15 to .37), t(65) = 4.93, p < .001. In addition, the effect size (Hedge's g = 1.112 [50]) indicates a large effect since it is greater than 0.8 [24].

To further investigate whether individuals, who share common experiences within the sceneries depicted in the location-aware images, tend to create similar passwords when they use the same image during password creation, we first split the participants from the experimental group into subgroups based on the image they used. In the sample of the experimental group (n = 36), one out of nine images was not used by any participant. From the remaining eight images that were selected by participants, two images were selected by only one participant, and six images were selected by more than one participant, thus forming six subgroups of participants.

Given that the implementation of the DuoPass graphical password mechanism takes into consideration the order and the type of gestures (e.g., circles are more complex than simple taps but less complex than lines⁴), to understand the similarities of users’ password selections, we have disregarded the order and the type of the gestures and rather focused on the positions of the password selections. To do so, we simplified the gesture type as follows: for circles, we disregarded the radius and the directionality and kept only the center of the circle as an x, y segment, whereas for lines, we considered only the x, y segment of the starting point of the line. Table 2 summarizes the similarities in image regions across users who created their password on the same image. Accordingly, out of 102 gestures made by 34 users (n = 36, but we exclude the two images that were selected by only one participant), 6 users chose one same region, 2 users chose two same regions, and no user selected all three same regions. We would also like to note that when we consider the exact order of the gestures, there are no observed similarities in image regions across all subgroups and all participants.

To investigate RQ₂, we ran a two-way mixed analysis of variance (ANOVA) with the user group (control vs. experimental) and users’ password selections (three consecutive selections) as the independent variables, and the time to make each password selection as the dependent variable. There were no significant outliers, as assessed by inspection of boxplots. The data were normally distributed, as assessed by Shapiro-Wilk's test of normality (p > .05). There was homogeneity of variances (p > .05), as assessed by Levene's test of homogeneity of variances. The analysis revealed significant differences between the three users’ password selections on the time to compose the graphical password, F(1, 66) = 86.942, p < .01, partial η² = .568. The analysis revealed that there was no interaction between the user group and users’ password selections on the time to compose the graphical password, F(1, 66) = 1.459, p = .231, partial η² = .022. Figure 5 (left) depicts the time to make each of the three graphical password selections.

We further examined simple main effects for each password selection. Data are mean ± standard error unless otherwise stated. The analysis revealed that the time to create the last (third) selection between the two groups was statistically significant (control: 1,109.81 ± 1,222.91 msec vs. experimental: 650.83 ± 509.97 msec) with a mean difference of 458.97 msec, F(1, 66) = 4.161, p = .043, partial η² = .06. With regard to the first and second selections, there were no significant differences between the control and experimental groups (p > .05).

To investigate RQ₃, we measured login task completion time for each of the login sessions over the 7-day period, and memory time, which is the maximum amount of time (in hours) someone could effectively remember their password from the day of creation. Accordingly, we initially analyzed login task completion time using a mixed-effects analysis (with the lme4 package in R) [10] since this enabled us to handle all variables of the study while accounting for repeated measures of individuals (four login sessions over a period of 7 days) and for handling missing data of users—for example, a user who has not participated in some sessions across the 7 days of the study can be used in the analysis without requiring removing the user from the sample [82]. In this respect, we performed a mixed-effects analysis of the relationship between the time to successfully authenticate (by also including any failed attempts that eventually ended in a successful authentication) and the user group. As fixed effects, we entered the user group (control and experimental) into the model. As random effects, we used subjects to account for non-independence of measures. Visual inspection of residual plots revealed that linearity and homoscedasticity were not violated. p-Values were obtained by likelihood ratio tests of the full model with the effect in question against the model without the effect in question [107]. The analysis revealed that the user group had no impact on the time needed to authenticate (x²(1) = .171, p = .679). The mean login time for the users of the control group was 7.47 ± 4.78 seconds, whereas for the users of the experimental group, it was 7.63 ± 6.3 seconds. Figure 5 (right) depicts the mean login time across the user group for each of the four sessions. We further analyzed the users’ login attempts, indicating that overall, most login attempts were completed using the graphical password. Regarding the textual password login attempts, 5 out of 36 individuals from the experimental group also logged in once using a textual passphrase with a mean login time of 5.21 ± 2.43 seconds, whereas 10 out of 32 individuals from the control group also logged in once using a textual passphrase with a mean login time of 8.66 ± 3.05 seconds.

The maximum memory time that someone could achieve was approximately 168 hours (7 days x 24 hours). To investigate memorability, we conducted an independent-samples t-test, with the user group (control vs. experimental) as the independent variable and the memory time as the dependent variable. The analysis revealed that memory time between the two user groups was not statistically significant different, t(66) = –.961, p = .340. Memory time of the control group was 106.13 ± 76.8 hours, whereas memory time of the experimental group was 121.58 ± 55.2 hours.

To investigate RQ₄, we conducted a post-study survey to elicit the users’ perceptions (experimental group) on the security, memorability, trust, usability, and likeability toward the DuoPass system based on their interactions. For this purpose, we designed a questionnaire by following state-of-the-art works and guidelines on eliciting perceived security, trust, memorability, usability, and user experience [8, 17, 21]. Some of the example statements of the survey were “Overall, how secure do you find the DuoPass password system?”, “How mentally demanding was the login task?”, and “I trust in the ability of the DuoPass password system to protect my privacy”. Users rated the statements through a 5-point Likert scale, with the labels changing depending on the question (e.g., 1: Strongly disagree to 5: Strongly agree; 1: Very insecure to 5: Very secure). Perceived usability was measured through the SUS [17], which is an accredited and widely applied system usability instrument and widely used in password studies [21]. The survey also investigated the likeability toward the DuoPass personalized and flexible approach of DuoPass with users rating the statements through a 5-point Likert scale (1: Not at all to 5: Absolutely). Figure 6 illustrates the responses of participants toward perceived security, memorability, trust, and likeability.

Results revealed that the majority of participants perceive the DuoPass system as secure (75%), with low mental demand (77%) in recalling the password and users could effectively recall their password (84%), whereas 80% of the participants trust the technology and its ability to keep their data private and secure. Furthermore, when participants were asked whether they like the flexible and personalized approach for user authentication, the majority of participants (91%) extremely (21/36) or very much (12/36) liked the idea, with three users either moderately (1/36) and slightly (2/36) liking the idea. Table 3 also summarizes the likeability scores per healthcare organization, indicating that patients across organizations had a consensus on liking the suggested approach and increase ecological validity.

Representative positive responses from participants included the following: “I like it very much, [it's] a great new approach”, “easy yet hard to crack by (hackers)”, “no one knows what I see in it”, “easier to remember”, “it's a much trickier way for criminals to find out what someone has entered”, “accessible to everyone”, “easy to use, especially through the use of images that capture the imagination of the user and therefore easier to remember and harder to find for people who want to harm. Each image is basically different and very personal”, “very personal password”, “I would like a lot that the images displayed were personal images that I could upload”, “genius”, “a sentence is better for me to remember”, “point out the many possibilities with regard to places in the photo”, “it was very easy to remember”, “Very individual. Less prompts requirements usual password and feels very secure because of that”, “pictures are good”, “Easy to use and bespoke”, “Works for people who learn using pictures. Hopefully easier to remember”, “great not to have to remember yet another password”, “it is very personalized”, “not having to remember text”, “It is nice and quick to be able to log with 3 clicks you remember from an almost infinite possibility of clicks and orders”, “For people who have visual memory, using images instead of text it's probably easy to remember”, “Better for dyslexic people”, “I like the concept of using picture passwords, it can make it easier to remember”.

Negative responses from participants included the following: “there are some places in the images that obviously pop out more and possibly people is more prone to use them in their password, making this system more insecure”, “people will use the easiest gestures and this could be a safety concern”, “if you don't use it often would it be still memorable?”, “with a photo you have to be able to use the same photo everywhere, because otherwise I can't remember the login code”, “My main concern is that choices of pass gestures and locations are not truly random and can be figured out using a user's publicly available data, such as for instance a social network profile”, “Perhaps tapping on the heads is too obvious, people might create weak passwords”, “security, if I pick 3 simple gestures it may be easier for a criminal to guess”.

Finally, we measured system usability based on SUS with participants scoring an overall SUS score of 74.77%. Table 4 summarizes the SUS scores of patients across the participating healthcare organizations. Based on the literature, the average SUS score is 68% [89]. In case the score is under 68%, the system entails various usability issues that need improvement, whereas a score above 68%,indicates that the system entails good usability practices. Accordingly, the scores across the three healthcare organizations ranged between 72.14% for Healthcare Organization 1, 74.58% for Healthcare Organization 2, and 80% for Healthcare Organization 3, with an overall score of 74.77%. Such results are encouraging for further investigating and improving the system since the score suggests that the DuoPass system scores very well in usability, end users like the system, and they can easily complete the authentication-related tasks. Nevertheless, given that the score is below 80%, there are still aspects that require improvements. For example, during the studies, some patients had difficulties entering their graphical password through the developed gesture input mechanism, hence next steps entail improving the gesture input functionality. In addition, we conducted a one-way ANOVA to determine if the SUS score was different for people belonging to different healthcare organizations. There were no outliers, as assessed by boxplot; data was normally distributed for each group, as assessed by Shapiro-Wilk's test (p > .05), and there was homogeneity of variances, as assessed by Levene's test of homogeneity of variances (p = .843). Data is presented as mean ± standard error. The SUS score increased from 72.14 ± 4.23% to 74.58 ± 5.59% to 80 ± 5.10%, in that order, but the differences between the healthcare organizations was not statistically significant, F(2, 36) = 0.629, p = .538.

The experimental evaluation study revealed interesting insights related to security, memorability, and user experience with regard to the suggested DuoPass approach. Table 5 provides a summary of the main findings. Users make arbitrary choices in knowledge-based user authentication, which decreases the security. In locimetric passwords, this is a well-known and highly researched issue. The DuoPass approach aims to overcome this issue by recommending personalized images that are related to the users’ prior experiences in the healthcare environment. As such, we expected to improve security and at least retain memorability and user acceptance. Both security and memorability analyses provide evidence that the DuoPass approach assisted end users in making password choices based on their experiences, overcoming arbitrary choices, which are one of the main reasons for decreased security and memorability in locimetric approaches.

From a security perspective, we report DuoPass’ superiority against the state-of-the-art approach given that the experimental user group scored significantly higher guessability compared to the control group. This can be accredited to the fact that users from the experimental group created graphical passwords on images that were related to their prior experiences within the hospital and hence created selections on regions based on their experiences, rather than generic regions, which may be susceptible to a brute-force attack. Such a finding is in line with existing research, which revealed that depicting images related to the users’ prior sociocultural experiences increases the security of user-selected graphical secrets [28]. Furthermore, task completion efficiency analyses revealed that during the last password selections (third gesture), there were significant differences in user selections, with users from the experimental group making a significantly faster selection compared to the users from the control group. This can be explained by the fact that users from the control group needed more time to reason about a secret story on an image that was rather not familiar to them, whereas in the experimental group, users created a story based on their familiarity with the image, and consequently, as they composed their password, they were faster in making their last selections.

From a memorability and user experience perspective, descriptive statistics reveal that users from the experimental group scored higher memory time compared to the control group, indicating good memorability aspects of the approach; however, this difference was not statistically significant. This finding was triangulated with end users’ qualitative feedback in which participants perceived the DuoPass secrets as highly memorable, users were able to memorize their secret for the whole period of the study, the majority reported low mental demand (77%) in recalling their password, and they could effectively recall their password (84%).

Finally, DuoPass scores well in usability based on participant responses to the SUS (74.77%), but indicating that there is still room for improvement given that best SUS scores should be 80% and above. From feedback received during the studies, some patients had difficulties in entering their graphical password through the developed gesture input mechanism, hence our efforts are focused on improving the interaction design of gestures to address cross-compatibility issues and heterogeneity of devices. When users were asked about likeability aspects of the approach, the significant majority of users (90%) extremely and very much like the flexible and personalized approach, and the majority would like to use DuoPass as an alternative password system (75%).

To investigate the RQ, we conducted three analyses: (1) we calculated the Euclidean distance of the attackers’ guessing selections from the legitimate end users’ password secret selections; (2) based on the first analysis (Euclidean distance), we adjusted the brute-force attack performed in Section 5.5.1 to investigate whether users who share common experiences were able to run a more effective attack by starting to guess regions they suspected that the users selected their password; and (3) we performed a qualitative analysis based on the participants’ feedback at the end of the human guessing attack study to better understand the approach followed by attackers on graphical passwords created on location-aware images. In the analyses that follow, data are mean ± standard error. There were no significant outliers in the data.

A. Disregarding the type of the gesture and the exact order. Figure 8 depicts the Euclidean distance of each gesture of each participant by disregarding the type and the exact order of the attackers’ gestures and the end user's gestures. For the analysis, we adopted a threshold of three segments by considering the allowed tolerance of the graphical password mechanism.⁴ Accordingly, among 276 gestures (3 gestures x 92 participants), 49 gestures (17.7%) were in close proximity with the attacker's guessed selections. Furthermore, we conducted a one-way multivariate analysis of variance (MANOVA) to determine the effect of relationship between attackers and legitimate end users on how far the attackers’ password selections were from the legitimate end users’ password selections. Three measures were assessed: Euclidean distance of the legitimate users and the attackers on the first gesture, second gesture, and third gesture of the graphical password. Participants belonged to one of the following categories: family member, friend, medical staff, patient, and nurse. Preliminary assumption checking revealed that data was normally distributed, as assessed by Shapiro-Wilk's test (p > .05); there were no univariate or multivariate outliers, as assessed by boxplot and Mahalanobis distance (p > .001), respectively; there were linear relationships, as assessed by scatterplot; no multicollinearity (r = .117, p = .004 between gesture one and gesture two; r = .021, p = .007 between gesture one and gesture three; and r = .133, p = .010 between gesture two and gesture three); and there was homogeneity of variance-covariance matrices, as assessed by Box's M test (p = .007). The analysis revealed that the differences between groups on the combined dependent variables were statistically significant, F(12, 225.180) = 4.356, p < .0005; Wilks’ Λ = .576; partial η² = .168. Follow-up univariate ANOVAs revealed that all three gestures were statistically significantly different between the participants from different relationship groups (first gesture: F(4, 87) = 5.351, p = .001; partial η² = .197; second gesture: F(4, 87) = 3.305, p = .014; partial η² = .132; third gesture: F(4, 87) = 4.550, p = .002; partial η² = .173; Tukey-Kramer post hoc tests showed that for the first gesture, participants from the family group had statistically significantly lower mean scores than participants from either the patient group (p = .002) or the nurse group (p = .050), whereas participants from the friend group had statistically significantly lower mean scores than participants from the patient group (p = .006). Regarding the second gesture, participants from the family group had statistically significantly lower mean scores than participants from the nurse group (p = .020). Regarding the third gesture, participants from the family group had statistically significantly lower mean scores than participants from the medical staff group (p = .036), the patient group (p = .025), and the nurse group (p = .001).

B. Disregarding the type of the gesture but considering the exact order. Figure 9 depicts the Euclidean distance by disregarding the type of selections but considering the exact order of the attackers’ gestures and the end users’ gestures. Applying the same threshold of three segments, the analysis revealed that among 276 gestures (3 gestures x 92 participants) made by the participants, 19 gestures (6.8%) were in close proximity with the attacker's guessed selections. Furthermore, we conducted a one-way MANOVA to determine the effect of relationship between attackers and legitimate end users on how far the attackers’ password selections were from the legitimate end users’ password selections. Three measures were assessed: Euclidean distance of the legitimate users and the attackers on the first gesture, second gesture, and third gesture of the graphical password. Participants belonged to one of the following categories: family member, friend, medical staff, patient, nurse. Preliminary assumption checking revealed that data was normally distributed, as assessed by Shapiro-Wilk's test (p > .05); there were no univariate or multivariate outliers, as assessed by boxplot and Mahalanobis distance (p > .001), respectively; there were linear relationships, as assessed by scatterplot; no multicollinearity (r = –.089, p = .004 between gesture one and gesture two; r = .253, p = .008 between gesture one and gesture three; and r = .055, p = .009 between gesture two and gesture three); and there was homogeneity of variance-covariance matrices, as assessed by Box's M test (p < . 0005). The analysis revealed that the differences between groups on the combined dependent variables were statistically significant, F(12, 225.180) = 11.500, p < .0005; Wilks’ Λ = .282; partial η² = .344. Follow-up univariate ANOVAs revealed that all three gestures were statistically significantly different between the participants from different relationship groups (first gesture: F(4, 87) = 12.567, p < .0005; partial η² = .366; second gesture: F(4, 87) = 3.930, p = .006; partial η² = .153; third gesture: F(4, 87) = 13.832, p < .0005; partial η² = .389; Tukey-Kramer post hoc tests showed that for the first gesture, participants from the family group had statistically significantly lower mean scores than participants from the medical staff group (p < .0005), the patient group (p = .001), and the nurse group (p < .0005), whereas participants from the friend group had statistically significantly lower mean scores than participants from the medical staff group (p = .001), the patient group (p = .004), and the nurse group (p < .0005). Regarding the second gesture, participants from the family group had statistically significantly lower mean scores than participants from the nurse group (p = .002). Regarding the third gesture, participants from the family group had statistically significantly lower mean scores than participants from the medical staff group (p = .011), the patient group (p < .0005), and the nurse group (p < .005), whereas participants from the friend group had statistically significantly lower mean scores than participants from the patient group (p < .0005) and the nurse group (p < .0005).

C. Considering the type of the gesture and exact order. We compared the three attempts of each attacker with the end user's stored password from the same pair of participants. From a total of 276 attacking guesses (3 attempts of each attacker x 92 participants), there was no successful attempt, yielding an online success guessing rate of 0%.

To investigate whether the suggested location-aware image approach holds against attacks when considering the experience-spots indicated by each participant who acted as an attacker, we conducted an offline attack comparing a PoI-assisted brute-force attack (the same attack that considers PoIs as described in Section 5.5.1) and a personalized PoI-assisted brute-force attack that was further enhanced to consider the experience-spots regions as indicated by the human attacker.

A. Disregarding order and type of gestures across all participants. Given that the implementation of PGA-like mechanisms takes into consideration the order and the type of gestures, which could impact the total guesses required to crack a graphical password (e.g., circles are more complex than simple taps but less complex than lines⁴), it is interesting to first understand how each attack type (PoI-assisted brute-force attack vs. personalized PoI-assisted brute-force attack) performs when we disregard the order and the type of the gestures and rather focus on the positions of the password selections. To do so, we simplify the gesture type as follows: for circles, we disregard the radius and the directionality and keep only the center of the circle as an x, y segment, whereas for lines, we consider only the x, y segment of the start of the line.

A one-way MANOVA was run to determine the effect of relationship between attackers and legitimate end users on the number of guesses required to crack the passwords when using a PoI-assisted brute-force attack vs. a personalized PoI-assisted brute-force attack by considering also the experience-spots provided by the attackers. Two measures were assessed: number of guesses required to crack the passwords when using a PoI-assisted brute-force attack and number of guesses when using a personalized PoI-assisted brute-force attack. Participants belonged to one of the following categories: family member, friend, medical staff, patient, nurse. Preliminary assumption checking revealed that data was normally distributed, as assessed by Shapiro-Wilk's test (p > .05); there were no univariate or multivariate outliers, as assessed by boxplot and Mahalanobis distance (p > .001), respectively; there were linear relationships, as assessed by scatterplot; no multicollinearity (r = .394, p < .0005); and there was homogeneity of variance-covariance matrices, as assessed by Box's M test (p = .002). The analysis revealed that the differences between groups on the combined dependent variables were statistically significant, F(8, 172) = 4.546, p < .0005; Wilks’ Λ = .681; partial η² = .175. Follow-up univariate ANOVAs revealed that in both types of attacks, the number of guesses required to crack the passwords was statistically significantly different between the participants from different relationship groups (PoI-assisted brute-force attack: F(4, 87) = 4.650, p = .002; partial η² = .176; personalized PoI-assisted brute-force attack: F(4, 87) = 7.473, p < .0005; partial η² = .256; Tukey-Kramer post hoc tests showed that for the PoI-assisted brute-force attack, participants from the family group had statistically significantly lower mean scores than participants from the nurse group (p = .001), whereas participants from the friend group had statistically significantly lower mean scores than participants from the nurse group (p = .024). Regarding the personalized PoI-assisted brute-force attack, participants from the family group had statistically significantly lower mean scores than participants from the nurse group (p < .0005), participants from the friend group had statistically significantly lower mean scores than participants from the nurse group (p = .002), participants from the medical staff group had statistically significantly lower mean scores than participants from the nurse group (p = .003), and participants from the patient group had statistically significantly lower mean scores than participants from the nurse group (p = .008). In the PoI-assisted brute-force attack, the mean number of guesses required to crack the passwords per group was as follows: (1) family member: 43,576.94 ± 7,488.35; (2) friend: 71,732.66 ± 17,131.09; (3) medical staff: 99,921.83 ± 18,620.08; (4) patient: 112,232.45 ± 22,590.04; and (5) nurse: 159,116.55 ± 27,663.24. In the personalized PoI-assisted brute-force attack, the mean number of guesses required to crack the passwords per group was as follows: (1) family member: 28,694.61 ± 4,596.83; (2) friend: 52,546.77 ± 19,951.18; (3) medical staff: 55,511.50 ± 11,560.53; (4) patient: 62,751.50 ± 11,903.80; and (5) nurse: 124,987.88 ± 12,485.52. Figure 10 depicts the means of password strength among attack types by disregarding the order and the type of gestures across all participants.

B. Disregarding order and type of gestures across participants with at least one gesture containing an experience-spot. A one-way MANOVA was run to determine the effect of relationship between attackers and legitimate end users on the number of guesses required to crack the passwords when using a PoI-assisted brute-force attack vs. a personalized PoI-assisted brute-force attack by considering participants with at least one gesture containing an experience-spot. Two measures were assessed: number of guesses required to crack the passwords when using a PoI-assisted brute-force attack and number of guesses when using a personalized PoI-assisted brute-force attack. Participants belonged to one of the following categories: family member, friend, medical staff, patient, nurse. Preliminary assumption checking revealed that data was normally distributed, as assessed by Shapiro-Wilk's test (p > .05); there were no univariate or multivariate outliers, as assessed by boxplot and Mahalanobis distance (p > .001), respectively; there were linear relationships, as assessed by scatterplot; no multicollinearity (r = .764, p < .0005); and there was homogeneity of variance-covariance matrices, as assessed by Box's M test (p = .001). The analysis revealed that the differences between groups on the combined dependent variables were statistically significant, F(8, 96) = 43.855, p < .0005; Wilks’ Λ = .046; partial η² = .785. Follow-up univariate ANOVAs revealed that in both types of attacks, the number of guesses required to crack the passwords was statistically significantly different between the participants from different relationship groups (PoI-assisted brute-force attack: F(4, 49) = 66.535, p < .0005; partial η² = .845; personalized PoI-assisted brute-force attack: F(4, 49) = 81.915, p < .0005; partial η² = .870; Tukey-Kramer post hoc tests showed that for the PoI-assisted brute-force attack, participants from the family group had statistically significantly lower mean scores than participants from the friend group (p = .032), the medical staff group (p < .0005), the patient group (p < .0005), and the nurse group (p < .0005). Participants from the friend group had statistically significantly lower mean scores than participants from the medical staff group (p = .001), the patient group (p = .020), and the nurse group (p < .0005). Participants from the medical staff group had statistically significantly lower mean scores than participants from the nurse group (p < .0005), whereas participants from the patient group had statistically significantly lower mean scores than participants from the nurse group (p < .0005). Regarding the personalized PoI-assisted brute-force attack, participants from the family group had statistically significantly lower mean scores than participants from the medical staff group (p = .001), the patient group (p < .0005), and the nurse group (p < .0005). Participants from the friend group had statistically significantly lower mean scores than participants from the medical staff group (p = .002), the patient group (p < .0005), and the nurse group (p < .0005). Participants from the medical staff group had statistically significantly lower mean scores than participants from the patient group (p < .0005) and the nurse group (p < .0005), whereas participants from the patient group had statistically significantly lower mean scores than participants from the nurse group (p = .050). In the PoI-assisted brute-force attack, the mean number of guesses required to crack the passwords per group was as follows: (1) family member: 25,308.50 ± 1,454.21; (2) friend: 32,241.70 ± 1,330.90; (3) medical staff: 41,972.60 ± 2,062.57; (4) patient: 39,267.41 ± 1,436.24; and (5) nurse: 58,851.83 ± 1,498.21. In the personalized PoI-assisted brute-force attack, the mean number of guesses required to crack the passwords per group was as follows: (1) family member: 7,536.10 ± 351.76; (2) friend: 7,815.60 ± 308.75; (3) medical staff: 12,074.80 ± 394.72; (4) patient: 19,233.83 ± 969.97; and (5) nurse: 22,047.91 ± 1,000.61. Figure 11 depicts the means of password strength among attack types by disregarding the order and the type of gestures across participants with at least one gesture containing an experience-spot.

C. Considering order and type of gestures across all participants. A one-way MANOVA was run to determine the effect of relationship between attackers and legitimate end users on the number of guesses required to crack the passwords when using a PoI-assisted brute-force attack vs. a personalized PoI-assisted brute-force attack by taking into account the order and type of gestures across all participants. Two measures were assessed: number of guesses required to crack the passwords when using a PoI-assisted brute-force attack and number of guesses when using a personalized PoI-assisted brute-force attack. Participants belonged to one of the following categories: family member, friend, medical staff, patient, nurse. Data are expressed as mean ± standard error. Preliminary assumption checking revealed that data was normally distributed, as assessed by Shapiro-Wilk's test (p > .05); there were no univariate or multivariate outliers, as assessed by boxplot and Mahalanobis distance (p > .001), respectively; there were linear relationships, as assessed by scatterplot; no multicollinearity (r = .183, p = .008); and there was homogeneity of variance-covariance matrices, as assessed by Box's M test (p = .012). The analysis revealed that the differences between groups on the combined dependent variables were not statistically significant, F(8, 172) = .020, p = 0.99; Wilks’ Λ = .998; partial η² = .001. In the PoI-assisted brute-force attack, the mean number of guesses required to crack the passwords per group was as follows: (1) family member: 1,660,935.38 ± 348,113.52; (2) friend: 1,687,709.11 ± 447,019.45; (3) medical staff: 1,629,472.16 ± 124,750.15; (4) patient: 1,624,675.54 ± 849,169.82; and (5) nurse: 1,622,255.16 ± 66,263.27. In the personalized PoI-assisted brute-force attack, the mean number of guesses required to crack the passwords per group was as follows: (1) family member: 1,745,307.61 ± 253,165.96; (2) friend: 1,690,173.83 ± 126,111.69; (3) medical staff: 1,789,121.611 ± 217,395.14; (4) patient: 1,797,917.49 ± 475,354.52; and (5) nurse: 1,818,772.27 ± 164,674.17. Figure 12 depicts the means of password strength among attack types by considering the order and the type of gestures across all participants.

To further shed light and understand the approach followed by attackers on graphical passwords created on location-aware images, we used the data gathered from the feedback mechanism at the end of the study, as well as observations made by the researchers during the attack phase. In many cases, attackers used knowledge about the end user under attack, related to their habits, preferences, and facts about their personality: “My colleague is a great storyteller and I believe he would try to create a story for the password, like arriving at the entrance of the hospital, then entering by the stairs, then reading information on the panel. So, I decided to make my attack having this story-telling process in mind”. – P19; “I thought she will have used the three possible gestures (instead of just one or two of the options), and marked colorful items, because of her personality”. – P28; “Most of the times he has his coffee in the front yard outside the emergency room during his shift break. I would be surprised if he hadn't selected this particular area”. – P56; “My colleague likes flowers and plants. I think that some of her selections must be on the flowers”. – P39; “She likes painting at her free time and I think she drew straight lines on objects that have bright colors”. – P11.

In other cases, it is evident that the scenery depicted on the location-aware images impacted the selections of the attackers. In particular, attackers used a more personalized approach by considering specific information related to their common shared experiences with the end user under attack within the places depicted on the location-aware images: “Usually we have lunch together at the hospital's cafeteria and we tend to be seated at the tables near the entrance. Hence, I made my selections around these tables”. – P7; “Considering direction of movement; places he usually sits or similar”. – P14; “The pictures are from daily routines, so I'm trying to guess where he is going on a daily basis or important aspects for him in the photo”. – P29; “I work at the emergency department and the colleague that I am requested to guess his password is the ambulance driver. I think it is very possible that he made some of his password selections near or on the ambulance outside of the emergency department”. – P8; “The photo shows two parking lots, but I know that she usually parks her car to the one next to the left entrance of the hospital because it is more convenient for her. I guess that some of her selections must be within this specific parking lot area”. – P16; “Being a hospital receptionist and a friendly person that interacts daily with many patients, I think that she must have selected the people standing in front of the reception desk”. – P33.

In very few cases, attackers did not employ any sophisticated attack but rather focused on the obvious PoIs of the images: “Tried to guess likely features in the images, but type of gesture I used was just random”. – P23; “I clicked on the most dominant views. I chose them because they caught my eye”. – P34; “I believe he must have selected the chairs because these are the most visible points in the image”. – P42.

The preceding observations were concentrated in a coding schema relevant to the approach followed by the attackers as follows:

Table 7 summarizes the responses about the approach employed by the attackers based on the aforementioned coding schema.