research-article

Teaching Johnny not to fall for phish

Authors:

Ponnurangam Kumaraguru,

Alessandro Acquisti,

Lorrie Faith Cranor,

Jason HongAuthors Info & Claims

ACM Transactions on Internet Technology (TOIT), Volume 10, Issue 2

Article No.: 7, Pages 1 - 31

https://doi.org/10.1145/1754393.1754396

Published: 10 June 2010 Publication History

Abstract

Phishing attacks, in which criminals lure Internet users to Web sites that spoof legitimate Web sites, are occurring with increasing frequency and are causing considerable harm to victims. While a great deal of effort has been devoted to solving the phishing problem by prevention and detection of phishing emails and phishing Web sites, little research has been done in the area of training users to recognize those attacks. Our research focuses on educating users about phishing and helping them make better trust decisions. We identified a number of challenges for end-user security education in general and anti-phishing education in particular: users are not motivated to learn about security; for most users, security is a secondary task; it is difficult to teach people to identify security threats without also increasing their tendency to misjudge nonthreats as threats. Keeping these challenges in mind, we developed an email-based anti-phishing education system called “PhishGuru” and an online game called “Anti-Phishing Phil” that teaches users how to use cues in URLs to avoid falling for phishing attacks. We applied learning science instructional principles in the design of PhishGuru and Anti-Phishing Phil. In this article we present the results of PhishGuru and Anti-Phishing Phil user studies that demonstrate the effectiveness of these tools. Our results suggest that, while automated detection systems should be used as the first line of defense against phishing attacks, user education offers a complementary approach to help people better recognize fraudulent emails and websites.

References

[1]

Abu-Nimeh, S., Nappa, D., Wang, X., and Nair, S. 2007. A comparison of machine learning techniques for phishing detection. e-Crime Researchers Summit, Anti-Phishing Working Group.

Digital Library

[2]

Account Guard. 2006. Account Guard. http://pages.ebay.com/ebay_toolbar/.

[3]

Adams, A. and Sasse, M. A. 1999. Users are not the enemy. Comm. ACM 42, 12, 40--46. DOI=http://doi.acm.org/10.1145/322796.322806.

Digital Library

[4]

Aleven, V. and Koedinger, K. R. 2002. An effective metacognitive strategy: learning by doing and explaining with a computer-based cognitive tutor. Cogn. Sci. 26, 2, 147--179.

[5]

Anandpara, V., Dingman, A., Jakobsson, M., Liu, D., and Roinestad, H. 2007. Phishing IQ tests measure fear, not ability. Usable Security Workshop (USEC'07). http://usablesecurity.org/papers/anandpara.pdf.

Digital Library

[6]

Anderson, J. R. 1993. Rules of the Mind. Lawrence Erlbaum Associates, Inc.

[7]

Anderson, J. R., Corbett, A. T., Koedinger, K. R., and Pelletier, R. 1995. Cognitive tutors: Lessons learned. J. Learn. Sci. 4, 2, 167--207.

[8]

Anderson, J. R. and Simon, H. A. 1996. Situated learning and education. Educ. Resear. 25, 5--11.

[9]

Anti-Phishing Working Group. 2007. Anti-Phishing Working Group. http://www.antiphishing.org/.

[10]

Bahrick, H. P. 1979. Maintenance of knowledge: Questions about memory we forgot to ask. J. Exper. Psych. 108, 3, 296--308.

[11]

Baker, R., Habgood, J., and Ainsworth, S. E. 2007. Modeling the acquistion of fluent skill in educational action games. Proceedings of the Conference on User Modeling, 17--26.

Digital Library

[12]

Barnett, S. M. and Ceci, S. J. 2002. When and where do we apply what we learn? a taxonomy for far transfer. Psych. Bull. 128, 612--637.

[13]

Bransford, J. D. and Schwartz, D. L. 2001. Rethinking transfer: A simple proposal with multiple implications. In Review of Research in Education, A. Iran-Nejad and P. D. Pearson., Eds. Vol. 24, American Educational Research Association (AERA), Washington, DC, 61--100.

[14]

Burmester, G. M., Stottler, D., and Hart, J. L. 2005. Embedded training intelligent tutoring systems (ITS) for the future combat systems (FCS) command and control (C2) vehicle. Tech. rep., Defense Technical Information Center. http://www.stottlerhenke.com/papers/IITSEC-02-ITSFCS.pdf.

[15]

Chandrasekaran, M., Narayanan, K., and Upadhyaya, S. 2006. Phishing email detection based on structural properties. Proceedings of the NYS Cyber Security Conference.

[16]

Clark, R. C. 1989. Developing Technical Training: A Structured Approach for the Development of Classroom and Computer-Based Instructional Materials. Addison Wesley Publishing Company.

[17]

Clark, R. C. and Mayer, R. E. 2002. E-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. John Wiley & Sons, Inc.

Digital Library

[18]

Committee on Developments in the Science of Learning and National Research Council. 2000. How People Learn: Bridging Research and Practice. National Academies Press.

[19]

Corbett, A. T. and Anderson, J. R. 2001. Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'01). ACM Press, New York, NY, 245--252.

Digital Library

[20]

Cordova, D. I. and Lepper, M. R. 1996. Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. J. Educ. Psych. 88, 4, 715--730.

[21]

Cranor, L. F. 2008. A framework for reasoning about the human in the loop. In Proceedings of the Conference on Usability, Psychology and Security.

Digital Library

[22]

Cranor, L. F. and Garfinkel, S. Aug, 2005. Security and Usability: Designing Secure Systems that People Can Use. O'Reilly, Sebastopol, CA.

Digital Library

[23]

Dhamija, R. and Tygar, J. 2005. The battle against phishing: Dynamic security skins. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS'05). ACM Press, New York, NY, 77--88.

Digital Library

[24]

eBay. 2006. Spoof email tutorial. http://pages.ebay.com/education/spooftutorial.

[25]

Eberts, R. E. 1997. Handbook of Human-Computer Interaction. Elsevier Science, 825--847.

[26]

Egelman, S., Cranor, L. F., and Hong, J. 2007. You've been warned: An empirical study of the effectiveness of Web browser phishing warnings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'08).

Digital Library

[27]

Emigh, A. 2005. Online identity theft: Phishing technology, chokepoints and countermeasures. Tech. rep., Radix Labs. October. http://www.antiphishing.org/Phishing-dhs-report.pdf.

[28]

Evers, J. 2006. User education is pointless. http://news.com.com/2100-7350_3-6125213.html.

[29]

Federal Trade Commission. 2006a. An e-card for you game. http://www.ftc.gov/bcp/conline/ecards/phishing/index.html.

[30]

Federal Trade Commission. 2006b. How not to get hooked by a phishing scam. Consumer alert news. http://www.ftc.gov/bcp/edu/pubs/consumer/alerts/alt127.htm.

[31]

Ferguson, A. J. 2005. Fostering e-mail security awareness: The west point carronade. EDUCASE Quart. 1. http://www.educause.edu/ir/library/pdf/eqm0517.pdf.

[32]

Fette, I., Sadeh, N., and Tomasic, A. 2006. Learning to detect phishing emails. In Proceedings of the 16th International Conference on World Wide Web.

Digital Library

[33]

Florencio, D. and Herley, C. 2005. Stopping a phishing attack, even when the victims ignore warnings. Tech. rep., Microsoft.

[34]

Fong, G. T. and Nisbett, R. E. 1991. Immediate and delayed transfer of training effects in statistical reasoning. J. Exper. Psych. 120, 34--45.

[35]

Gagne, R. M., Foster, H., and Crowley, M. E. 1948. The measurement of transfer of training. Psych. Bull. 45, 2, 97--130.

[36]

Gordon, L. A., Loeb, M. P., Lucyshyn, W., and Richardson, R. 2006. CSI/FBI computer crime and security survey. Report, Computer Security Institute.

[37]

Gorling, S. 2006. The myth of user education. In Proceedings of the 16th Virus Bulletin International Conference.

[38]

Hight, S. D. 2005. The importance of a security, education, training and awareness program. http://www.infosecwriters.com/text_resources/pdf/SETA_SHight.pdf.

[39]

Jackson, C., Simon, D., Tan, D., and Barth, A. 2007. An evaluation of extended validation and picture-in-picture phishing attacks. In Proceedings of the Usable Security Workshop (USEC'07). http://usablesecurity.org/papers/jackson.pdf.

Digital Library

[40]

Jagatic, T., Johnson, N., Jakobsson, M., and Menczer, F. 2007. Social phishing. Comm. ACM 50, 10, 94--100.

Digital Library

[41]

Jakobsson, M. 2007. The human factor in phishing. In Privacy & Security of Consumer Information. http://www.informatics.indiana.edu/markus/papers/aci.pdf.

[42]

Jakobsson, M. and Myers, S., Eds. 2006. Phishing and Countermeasures: Understanding the Increasing Problem of Electronic Identity Theft. Wiley-Interscience.

Digital Library

[43]

James, L. 2005. Phishing Exposed. Syngress Publishing, Canada.

Digital Library

[44]

Johnson, B. R. and Koedinger, K. R. 2002. Comparing instructional strategies for integrating conceptual and procedural knowledge. In Proceedings of the Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education. Vol. 1--4. 969--978.

[45]

Kirkley, J. R. and et al. 2003. Problem-based embedded training: An instructional methodology for embedded training using mixed and virtual reality technologies. In Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). http://www.iforces.org/downloads/problem-based.pdf.

[46]

Koedinger, K. R. 2002. Toward evidence for instruction design principles: Examples from cognitive tutor math 6. Proocedings of the Annual Meeting of the Norh American Chapter of the International Group for the Psychology of Mathematics Education 1--4.

[47]

Kumaraguru, P., Rhee, Y., Acquisti, A., Cranor, L. F., Hong, J., and Nunge, E. 2007a. Protecting people from phishing: the design and evaluation of an embedded training email system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'07). ACM Press, New York, NY, 905--914.

Digital Library

[48]

Kumaraguru, P., Rhee, Y., Sheng, S., Hasan, S., Acquisti, A., Cranor, L. F., and Hong, J. 2007b. Getting users to pay attention to anti-phishing education: Evaluation of retention and transfer. e-Crime Researchers Summit, Anti-Phishing Working Group.

Digital Library

[49]

Lininger, R. and Vines, R. D. 2005. Phishing: Cutting the Identity Theft Line. IN. John Wiley and Sons.

Digital Library

[50]

Macmillan, N. A. and Creelman, C. D. 2004. Detection Theory: A User's Guide. Lawrence Erlbaum.

[51]

Mail Frontier. 2006. Mailfrontier phishing IQ test. http://survey.mailfrontier.com/survey/quiztest.html.

[52]

Mandl, H. and Levin, J. R. 1989. Knowledge Acquisition from Text and Pictures. North-Holland.

[53]

Mathan, S. A. and Koedinger, K. R. 2003. Artificial Intelligence in Education: Shaping the Future of Learning Through Intelligent Technolgis. IOS Press, 13--20.

[54]

Mathan, S. A. and Koedinger, K. R. 2005. Fostering the intelligent novice: Learning from errors with metacognitive tutoring. Educ. Psych. 40, 4, 257--265.

[55]

Mayer, R. E. 2001. Multimedia Learning. Cambridge University Press, Cambidge, UK.

Digital Library

[56]

Mayer, R. E. and Anderson, R. B. 1992. The instructive animation: Helping students build connections between words and pictures in multimedia learning. J. Educ. Psych. 84, 4, 444--452.

[57]

McBride, C. M., Emmons, K. M., and Lipkus, I. M. 2003. Understanding the potential of teachable moments: the case of smoking cessation. Health Educ. Resear. 18, 2, 156--170.

[58]

Merrienboer, J. V., de Croock, M., and Jelsma, O. 1997. The transfer paradox: Effects of contextual interference on retention and transfer performance of a complex cognitive skill. Percept. Motor Skills 84, 784--786.

[59]

Microsoft Corporation. 2006. Consumer awareness page on phishing. http://www.microsoft.com/athome/security/email/phishing.mspx.

[60]

Miller, R. C. and Wu, M. 2005. Fighting phishing at the user interface. In L. Cranor and S. Garfinkel Eds. Security and Usability: Designing Secure Systems that People Can Use. O'Reilly.

[61]

Moreno, R. and Mayer, R. E. 1999. Cognitive principles of multimedia learning: The role of modality and contiguity. J. Educ. Psych. 91, 358--368.

[62]

MySecureCyberspace. 2007. Uniform resource locator (URL). http://www.mysecurecyberspace.com/encyclopedia/index/uniform-resource-locator-url-.html.

[63]

Netcraft. 2006. Netcraf. http://toolbar.netcraft.com/.

[64]

New York State Office of Cyber Security & Critical Infrastructure Coordination. 2005. Gone phishing&ldots;&ldots; a briefing on the anti-phishing exercise initiative for New York State government. Aggregate Exercise Results for public release.

[65]

Nielsen, J. 2004. User education is not the answer to security problems. http://www.useit.com/alertbox/20041025.html.

[66]

Robila, S. A. and Ragucci, J. W. 2006. Don't be a phish: steps in user education. In Proceedings of the 11th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITICSE'06). ACM Press, New York, NY, 237--241. DOI=http://doi.acm.org/10.1145/1140124.1140187.

Digital Library

[67]

Rubin, D. C. and Wenzel, A. E. 1996. One hundred years of forgetting: A quantitative description of retention. Psych. Rev. 103, 4, 734--760.

[68]

Salkind, N. J. 2006. Encyclopedia of Measurement and Statistics. Sage Publications.

[69]

Schmidt, R. A. and Bjork, R. A. 1992. New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psych. Sci. 3, 4, 207--217.

[70]

Schneier, B. 2000. Semantic attacks: The third wave of network attacks. Crypto-Gram Newsletter. http://www.schneier.com/crypto-gram-0010.html#1.

[71]

Schwartz, D. L. and Bransford, J. D. 1998. A time for telling. Cogn. Instruc., 475--522.

[72]

Sender Policy Framework. 2006. Sender Policy Framework. http://www.openspf.org/.

[73]

Sheng, S., Magnien, B., Kumaraguru, P., Acquisti, A., Cranor, L. F., Hong, J., and Nunge, E. 2007. Anti-phishing phil: The design and evaluation of a game that teaches people not to fall for phish. Proceedings of the Symposium on Usable Privacy and Security.

Digital Library

[74]

Sheng, S., Wardman, B., Warner, G., Cranor, L., Hong, J., and Zhang, C. 2009. An empirical analysis of phishing blacklists. Proceedings of the 6th Conference on Email and Anti-Spam.

[75]

Singley, M. and Anderson, J. R. 1989. The Transfer of Cognitive Skill. Harvard University Press.

Digital Library

[76]

SpoofGuard. 2006. Spoofguard. http://crypto.stanford.edu/SpoofGuard/.

[77]

SpoofStick. 2006. Spoofstick. http://www.spoofstick.com/.

[78]

Whitten, A. 2004. Making security usable. Ph.D. thesis, Carnegie Mellon University.

[79]

Wu, M., Miller, R. C., and Garfinkel, S. L. 2006. Do security toolbars actually prevent phishing attacks? In Proceedings of the Conference on Human Factors in Computing Systems (CHI), 601--610.

Digital Library

[80]

Yahoo. 2007. DomainKeys: Proving and Protecting Email Sender Identity. http://antispam.yahoo.com/domainkeys.

[81]

Ye, Z. E. and Smith, S. 2002. Trusted paths for browsers. In Proceedings of the 11th USENIX Security Symposium. USENIX Association, Berkeley, CA, 263--279.

Digital Library

[82]

Zhang, Y., Egelman, S., Cranor, L., and Hong, J. 2007. Phinding phish: Evaluating anti-phishing tools. In Proceedings of the 14th Annual Network and Distributed System Security Symposium. http://lorrie.cranor.org/pubs/ndss-phish-tools-final.pdf.

Cited By

Agha ZAli NPark JWisniewski P(2025)A systematic review on design-based nudges for adolescent online safetyInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2024.10070243(100702)Online publication date: Mar-2025
https://doi.org/10.1016/j.ijcci.2024.100702
Yasin AFatima RWen LJiangBin ZNiazi M(2025)What goes wrong during phishing education? A probe into a game-based assessment with unfavorable resultsEntertainment Computing10.1016/j.entcom.2024.10081552(100815)Online publication date: Jan-2025
https://doi.org/10.1016/j.entcom.2024.100815
Sturman DAuton JMorrison B(2025)Security awareness, decision style, knowledge, and phishing email detection: Moderated mediation analysesComputers & Security10.1016/j.cose.2024.104129148(104129)Online publication date: Jan-2025
https://doi.org/10.1016/j.cose.2024.104129
Show More Cited By

Index Terms

Teaching Johnny not to fall for phish
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Security and privacy
  1. Systems security
    1. Operating systems security

Recommendations

Anti-Phishing Phil: the design and evaluation of a game that teaches people not to fall for phish
SOUPS '07: Proceedings of the 3rd symposium on Usable privacy and security

In this paper we describe the design and evaluation of Anti-Phishing Phil, an online game that teaches users good habits to help them avoid phishing attacks. We used learning science principles to design and iteratively refine the game. We evaluated the ...
Getting users to pay attention to anti-phishing education: evaluation of retention and transfer
eCrime '07: Proceedings of the anti-phishing working groups 2nd annual eCrime researchers summit

Educational materials designed to teach users not to fall for phishing attacks are widely available but are often ignored by users. In this paper, we extend an embedded training methodology using learning science principles in which phishing education ...
School of phish: a real-world evaluation of anti-phishing training
SOUPS '09: Proceedings of the 5th Symposium on Usable Privacy and Security

PhishGuru is an embedded training system that teaches users to avoid falling for phishing attacks by delivering a training message when the user clicks on the URL in a simulated phishing email. In previous lab and real-world experiments, we validated ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology

ACM Transactions on Internet Technology Volume 10, Issue 2

May 2010

123 pages

ISSN:1533-5399

EISSN:1557-6051

DOI:10.1145/1754393

Issue’s Table of Contents

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 June 2010

Accepted: 01 November 2009

Revised: 01 September 2009

Received: 01 March 2009

Published in TOIT Volume 10, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

239
Total Citations
View Citations
3,610
Total Downloads

Downloads (Last 12 months)354
Downloads (Last 6 weeks)41

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Agha ZAli NPark JWisniewski P(2025)A systematic review on design-based nudges for adolescent online safetyInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2024.10070243(100702)Online publication date: Mar-2025
https://doi.org/10.1016/j.ijcci.2024.100702
Yasin AFatima RWen LJiangBin ZNiazi M(2025)What goes wrong during phishing education? A probe into a game-based assessment with unfavorable resultsEntertainment Computing10.1016/j.entcom.2024.10081552(100815)Online publication date: Jan-2025
https://doi.org/10.1016/j.entcom.2024.100815
Sturman DAuton JMorrison B(2025)Security awareness, decision style, knowledge, and phishing email detection: Moderated mediation analysesComputers & Security10.1016/j.cose.2024.104129148(104129)Online publication date: Jan-2025
https://doi.org/10.1016/j.cose.2024.104129
Dubovecka K(2024)Vulnerability of Students of Masaryk University to Two Different Types of PhishingApplied Cybersecurity & Internet Governance10.60097/ACIG/190268Online publication date: 24-Jun-2024
https://doi.org/10.60097/ACIG/190268
Schöni LCarles VStrohmeier MMayer PZimmermann V(2024)You Know What? - Evaluation of a Personalised Phishing Training Based on Users' Phishing Knowledge and Detection SkillsProceedings of the 2024 European Symposium on Usable Security10.1145/3688459.3688460(1-14)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3688459.3688460
Lain DJost TMatetic SKostiainen KCapkun SLuo BLiao XXu JKirda ELie D(2024)Content, Nudges and Incentives: A Study on the Effectiveness and Perception of Embedded Phishing TrainingProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690348(4182-4196)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3690348
Burda PAllodi LZannone N(2024)Cognition in Social Engineering Empirical Research: A Systematic Literature ReviewACM Transactions on Computer-Human Interaction10.1145/363514931:2(1-55)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1145/3635149
Agha ZPark JWan RAli NWang YDifranzo DBadillo-Urquiola KWisniewski P(2024)Tricky vs. Transparent: Towards an Ecologically Valid and Safe Approach for Evaluating Online Safety Nudges for TeensProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642313(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642313
Wang CJia ZBenkraouda HZevnik CHeuermann NFoulger RHandler JWang G(2024)VeriSMS: A Message Verification System for Inclusive Patient Outreach against Phishing AttacksProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642027(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642027
Chen XSacré MLenzini GGreiff SDistler VSergeeva A(2024)The Effects of Group Discussion and Role-playing Training on Self-efficacy, Support-seeking, and Reporting Phishing Emails: Evidence from a Mixed-design ExperimentProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641943(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641943
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents