Assessing AI vs Human-Authored Spear Phishing SMS Attacks:
An Empirical Study Using the TRAPD Method

The human authors were undergraduate students enrolled in a university cybersecurity program during the Winter Semester 2023. Students generated spear phishing messages based on the data provided to them about their target, though names were changed to maintain participant privacy. Messages were entered into an online survey along with some demographic data about the authors including name, gender, age group, and technical ability. Authors were also asked to assess their understanding and experience relative to spear phishing. Authors were then provided with four prompts containing information about their target and then were asked to generate a message that would attempt to phish that target based on that information. After each prompt, the author was asked to assess their confidence in the ability of the message to trick the target.

Ninety-nine students participated in the survey, with the intent of creating a balanced pool of message contributions across demographics. In total, 297 messages were gathered and screened for invalid entries that were not spear phishing messages. On messages that have no link, or have placeholders for links, we modified the message to include a generic tinyurl. In cases where there are more than two valid messages for each topic (work, hobbies, social) for each target, two were selected at random to proceed to the next phase. 246 human-authored messages were retained and prepared for the next phase.

A script was written to call the GPT-4 API to generate spear phishing message outputs automatically. The same three prompts given to the student authors for each target were fed to the script to generate the AI-generated spear phishing messages. An example of a GPT-4 output is shown below:

Hey Stephanie! I came across this incredible home organization app that I think will help you streamline your daily tasks and save time. It’s been a game-changer for me! They are currently giving away a free 1-year subscription to the first 100 users who sign up. Don’t miss out on this opportunity! Here’s the link: bit.ly/organizehome4u

Stay organized!

Your friend :)

Since GPT-4 outputs are not completely reproducible, three different responses were gathered from each prompt. This resulted in 9 messages generated by the model for each target. Similar to what was done with the human-authored messages, we selected at random 2 messages for each topic, totaling 246 AI-generated messages that will be used for the next phase.

After both sub-phases completed, each target had 12 spear phishing messages, 6 made from GPT-4 and 6 were human-made. By the end of this phase 492 simulated spear phishing “attacks” were prepared for assessment.

The 12 targeted messages were shown to each target during the interview, with all messages displayed on a separate printed piece of paper. Participants were asked to arrange messages, ranking them on their ability to convince them to click on the link provided. Figure 1a shows an example of how the targets laid out the messages in the experiment, illustrating the ranked order from most to least convincing, as well as the threshold where they would have started to click the link based on the participants’ assessments.

Participants were then asked what elements in the messages led them to be tricked, as well as other feedback that they were willing to share regarding the message. They were also asked what made them more cautious of the less convincing messages.

Participants were then notified that one or more of the messages were created by an AI, and were asked to identify which messages they believe to be human-made vs computer generated. Figure 1a shows an example on how the targets accomplished this, by placing AI “markers” on the messages that they believed to be AI-generated. Similar to the previous part of the interview, they were then asked what made them think the messages they chose were made by humans or AI. At the target’s request, researchers revealed which messages were made by AI and human authors.

Audio recordings were made for all 25 interviews which were then assimilated for transcription and further analysis.

To assess performance between AI and human messages (RQ1), we compared their performance based on how high they were ranked in the target interviews and how often each type of messages would be clicked by out targets. On average, AI-generated messages ranked slightly higher (6.41) than human-authored messages (6.58). Figure 2a describes the distribution of the AI and human messages across each rank, as well as their averages.

Results from our statistical analysis indicated no significant difference between the two groups, with a p-value of 0.665. To account for the non-normal distribution of ranks and the variation across targets, a permutation test was conducted, but it again showed no significant difference in ranks between AI and human-generated messages.

The logistic regression model came with similar results. The model did not find a significant difference, with a p-value of 0.182. The predicted probabilities of clicking were 21.3% for human-generated messages and 28.0% for AI-generated messages. Although the AI-generated messages had a higher predicted click rate, this difference was not statistically significant. Fitting the GLMM also yielded similar results, indicating greater variation between subjects than between authors.

The odds ratio for clicking on a link in an AI-generated message versus a human-authored one was 1.43, with a 95% confidence interval of 0.847 to 2.446. This wide interval includes 1, indicating that the increased likelihood (43%) of clicking on AI-generated messages compared to human-authored ones is not statistically significant and could range from 15% lower to 145% higher.

We employed a Bayesian logistic model to further the analysis. Using a horseshoe prior on the log odds ratio, we obtained a posterior mean odds ratio of 1.22 (95% CI: 0.86 to 2.05) with a median posterior of 1.16. From this posterior distribution, we estimated that the probability of the odds ratio being less than 1 (indicating humans are better than AI) is 19.7%, while the probability of it being greater than 1 (indicating AI is better than humans) is 80.3%. The probability of the odds ratio exceeding 1.5 (AI being more than 50% better than humans at generating clicks) is only 19.9%. A plot of the posterior distribution of the odds ratio shown in Figure 2b indicates that, while there is a long right tail suggesting a possibility of AI being significantly better, most of the probability mass is near 1, suggesting neither being definitively better.

One of the key content characteristics of the messages (RQ2) relates to their topic: Job, Hobby, or Social media post. On average, Job ranked the highest (5.71), followed by Hobby (6.66) with Social ranking the lowest (7.13). The distribution of these rankings is shown on Figure 3. Using a Tukey post-hoc test to analyze statistical significance of the ranking of each topic, we derived the confidence intervals of each pair of topics. Results show that the mean rank for Job is significantly higher than Social, while Job-Hobby is not statistically significant, and much less for Hobby-Social.

We used a logistic regression model to determine if the topic of the message influenced the likelihood of clicking. The model showed an overall p-value of 0.0009, suggesting that the topic played a significant role in generating clicks. Job-related messages showed the highest probability of clicking (38%) while Hobby-related (19%) and Social-related (17%) messages scored much lower. Pairwise comparisons showed that Job had a significantly higher click probability compared to Hobby and Social, with no significant difference between hobby and social topics. Specifically, the comparison between Job and Hobby had an estimate of 0.9605 (95% CI: 0.1933 to 1.7277), while Job vs Social had an estimate of 1.0961 (95% CI: 0.3081 to 1.8841).

To assess if subjects could correctly identify whether a message was AI-generated or human-authored (RQ3), we analyzed their guesses for both types of messages. Subjects correctly identified the message origin 52% of the time. Note that 50% would be expected from randomly guessing. Table II describes the confusion matrix that details their accuracy.

We also used a logistic regression model incorporating predictors such as the presence of emojis (Emoji), whether there were modified links (LinkMod), and the number of characters in the message (CharacterCount). The overall p-value of 0.3253 indicated no significant ability to distinguish between AI and human-authored messages based on these features.

The URL was mentioned by 64% of all participants as having convincing attributes (16/25). Six participants noted how the URL can provide a sense of trust, with one noting, “BYU, I kind of trust BYU and the URL here” (T41). Three participants mentioned convincing characteristics with regards to the domain of the URL. One stated, “The biggest part of this one [URL], and I even debated whether I would put it first or second; but it has an .edu link . . . and maybe I’m very ignorant about this, but I feel like that’s harder to, like, create a fake .edu website” (T16). In contrast, another participant used similar reasoning for not clicking on a link, noting, “At first, I thought, Oh, I would click on that. But then it says google.com.org; which is weird, because I’ve never seen that before” (T07). This same participant mentioned some inaccuracies related to the domain that would cause him/her to rethink clicking on it. Another participant mentioned the use of a .net website which contributed to them clicking on the link because the domain seemed trustworthy.

Two participants were more likely to click on the link if it included the HTTPS protocol. One said that “If it looks legit, yeah, like HTTPS and the /ProvoLibrary, something like that” (T41), then they would click. Another participant said, “they all say HTTPS, which makes me feel like it’s secure” (T07). Personal association with a URL may also play a role, with one participant stating s/he was more likely to click on the link if it included his name within the URL saying that “the link actually has my name in it. That would have really like, definitely thrown me for a loop, like really made me like, actually think about clicking the link just because it’s like, oh, this is actually from maybe from BYU” (T20).

Ten participants were concerned about the use of a link shortener within the message. One mentioned that “I feel like anytime I’ve seen a tiny URL address it, like, either hasn’t been real, or it’s been like, weird or just different things” (T16). Six of the participants were less likely to click on a link if it was misspelled e.g., “facebock” instead of “facebook” (T13). Overall, the perception is that URL name, domain, and indicators within the domain (e.g., HTTPS, spelling) help create perceptions about the deceitful nature of the message.

The technology medium used for communication was mentioned by 40% (10/25) of the interviewees. One of the 10 mentioned the medium increasing their likelihood of clicking on the link. S/he mentioned that it is “not that abnormal for [them] to receive” text messages from their home city” (T13). However, most targets saw the text medium as a red flag, even when they were inclined to believe the content. This was sometimes because of warnings from trainings they had received. One explained that “at work we were warned that all, like, valid messages will be through emails” (T25) and “This one, [employer] is not going to reach out to me via text for security stuff, period” (T08). T37 mentioned that they don’t have a “company texting deal,” making them more suspicious of SMS messages. Another men-tioned that “it’s very rare that people message me, you know, we talk over Slack or we talk over WhatsApp, maybe through email” (T31). Others noted that other communication tools, such as an online learning platform (T09) or a genealogy website (T02), would not have sent them a text message. Interestingly, in all these cases, the technology communications medium was associated with the source of the message. In other words, receiving a text message was not abnormal in and of itself, but receiving a text message from a particular source was deemed either appropriate or not.

The presence of inaccuracies in the messages was mentioned by 28% (7 of 25) of the targets. These inaccuracies contributed to perceptions about message credibility. Participants identified discrepancies related to the names or characteristics of individuals or enti-ties within their professional spheres. For example, two targets remarked about the message mentioning a col-league that does not exist, stating “there’s no Mike at work” (T37) or “there’s no one named Sarah on the BYU instructional design team” (T18). Another participant expressed confusion about being questioned on matters not relevant to their professional duties, saying that they “don’t deal with payables. So why is that asking me regarding payment?” (T25). Another target noted discrepencies related to job responsibilities, asserting that such library-related decisions would “go through me at the library”, making any logistical deviation “immediately sus-picious” (T01). Regarding messages related to their hobbies, one participant raised skepticism about the message claiming a “great deal for a hiking trail” which the participant knows through experience doesn’t actually require payment for use (T30). Another target was also suspicious of the message being from a library that they “just don’t use… so they would not know me to contact me” (T08). In summary, the instances where inaccuracies were mentioned ranged from misrepresented personnel and responsibilities to factual discrepancies about the targets’ affiliations, activities, and expectations.

The messages being personally relevant (or irrelevant) to the participants was mentioned in 76% (19 of 25) of the interviews. Seventeen targets explained that personal relevance of the message made it more convincing. Several participants pointed to messages directly tied to their occupational responsibilities, with one saying that “because that is my job is to help people with records…this is one that I feel like I’m most likely to engage with” (T13). Another target, who is a banker at a credit union, noted the similarities between the targeted message and messages s/he receives at work. S/he noted, the “alert literally looks like the alert we get [at work] when there’s a fraud” (T33). Some participants pointed out how the spear phishing messages aligned with their personal interests. One expressed enthusiasm about a message describing a “Vineyard gardening club”, believing that “somebody from our community” may have sent it. They expressed that they “would love to get involved in that” (T31). While most participants were more convinced when messages were relevant to them, one more skeptical participant mentioned that although “this interests me and it could exist, but I’m not gonna click on it” (T08).

On the other hand, personal relevance was perceived as unconvincing to 17 of the targets. Some participants highlighted messages that didn’t align with their current activities. For example, one interviewee noted, “I’m also not actively dancing anymore, so that’s just weird that they’re offering me something like this” (T25). Another target was suspicious of a social media-related message because they “don’t have an Instagram [account]” (T34). Some targets emphasized messages that failed to pique their interest, with one saying that they are “not into Brandon Sanderson [the author]” (T37). Another mentioned that they have already rescued an animal and “don’t need more right now” (T02). Some targets mentioned messages that contained information that they had not disclosed. One target noted that the message wasn’t “legit” because they “never put [their] studio equipment online” (T31), while another mentioned that “it’s a little weird to be selected for something that you don’t apply for” (T16). Participants also raised suspicion about messages related to unfamiliar organizations or activities, mentioning that they will not apply because “I don’t know that organization” (T41), or they “…have no idea what that would be about. So I’m just going to ignore it” (T08). Finally, targets showed skepticism when messages were somewhat relevant but lacked specificity. One target noted that “there just wasn’t enough content in the message body that was specifically directed at me” (T20). Another target highlighted a message about “recent health challenges” but noted that “there’s no indication of like, who is supposedly sending the message” (T18). Participants described how personal relevance within the message provides a very strong indicator for the message being perceived as convincing or not.

Urgent wording was mentioned by 32% of participants (8 of 25). Two of the eight mentioned they were more likely to click on a link if it included urgent or fear inciting language. One explained, “And because of ’suspicious activity’ under my university account, then my first thing is, well, I better click on this. Because if there’s an issue with my account, I want to fix it right away” (T34). The other target mentioned the same thing, that they would click on the link because they wanted to fix the problem as soon as possible. They explained, “And just like, I’d read through it real quick, and just click on it because I’m like, oh, shoot, that’s maybe something important that’s going on. Okay”(T06).

The remaining participants (6 of 25) said they were less likely to click on the link if it included tactics such as fear and urgency. These participants explained that time sensitive or urgent messages feel like a warning sign. “That’s always a way of like, okay, they’re trying to get you to shut off your logical brain. Put yourself into ’Oh, my gosh, we have to do this right now.’ Yeah. Right.” Another participant confirmed, “If it seems urgent to you, you’re not going to click it because it’s very dangerous” (T13). Similarly, T41 explained, “I don’t like anything where it’s like hurry, offer and send because that always makes me warry” (T07). Similar to the their concerns with time-sensitive messages, participants also discussed how fear inciting messages can cause them to logically analyze the message at a deeper level. One explained, “To the message itself, I was a little bit unsure about just like, the fact that it was so time sensitive, and that it was like, kind of threatening to lock me out of all of my educational accounts if I didn’t do what it asked for. Because that’s not typically what I’ve seen be the case” (T16). And another stated, “Yeah. And it’s the idea of a new virus for young dogs. I’m going that’s fear mongering” (T01). The scarcity principle has long been used in literature to deceive readers. For the spear phishing messages in this study, participants had mixed reactions. Some felt the intended pressure to click, while others saw the language as manipulative and as a warning sign marking potential danger.

Ten respondents (40%) mentioned issues related to message styling, such as text formatting, tone, and structure. Four participants acknowledged that style played a role in enhancing the credibility of a message. For example, T09 affirmed that a message was “more convincing” because the formatting and tone felt “kind of personal.” T31 also emphasized formatting in relation to message personalization explaining, “First, they talk about how they found me. They loved my work. And they saw my music profile. They talk about a music festival…. The format seems like it’s from an organized festival.” A message’s “casual style” was also suggested by respondents as a way for the message to feel authentic. A message’s “less formal” style was more “enticing” and “attractive” to T41. On the contrary, two participants mentioned that the message tone was off-putting. For example, T09 identified a message as potentially a phishing attack due to its “sales-y” tone. Additionally, T18 mentioned how sales-oriented “buzzwords” indicated a message “feels more phishy.”

A quarter of respondents (6 of 25) discussed the role of emojis in the context of message style. All six asserted that emojis diminished the credibility of the message. T34 expressed that “I think a lot of emojis is kind of something that usually is a red flag for me.” Multiple respondents conveyed that they did not anticipate receiving emoji-filled messages from senders claiming to represent professional organizations. For instance, T27 said that in the case of a legitimate organization such as a city biking club, emojis would be “out of place.” Similarly, T20 explained that if a message claimed to be from their university, the presence of emojis would cause them to feel “thrown off.” From the perspective of participants, the use of emojis in messages, particularly those seemingly sent by professional groups, caused such messages to lose legitimacy.

Realistic rewards significantly influenced the perceived credibility of phishing messages, as noted by 28% of the participants (7 out of 25). One participant said they would be more likely to click on a link, “because the reward is connected to something I put so much time and effort into” (T31), while another participant stressed the persuasiveness of plausible or realistic offers, mentioning, “It’s like taking you to a link to look at an offer, but it’s not a crazy insane offer” (T16). Some targets expressed enthusiasm for realistic opportunities, stating, “It’s something I’ll be really excited about, but also something that I think is realistic” (T34). Conversely, 24% of participants (6 out of 25) identified messages with rewards that seemed “too good to be true” as less convincing. One participant mentioned it’s “probably not likely” to be the “developer of the month four times a year” (T37). One of the targets questioned the credibility of messages offering free access to goods and services, stating, “It sounds too good to be true” (T34). Another target warned against messages promoting anything for free, noting, “Anything that’s free is a little too good to be true. Okay, so that’s when I would be very careful” (T27). This collective sentiment emphasized the convincing nature of realistic rewards in phishing messages while avoiding extravagant or implausible claims that trigger suspicion.

Participants described how the perceived credibility of phishing messages is tied to the sender’s identity, as noted by 7 out of 25 targets (28%). One participant said the message seemed more convincing because the sender “introduced herself as [their] neighbor” (T30). Another explained the importance of a personal introduction, stating, “The sender started with ’My name is John’” (T41), which helped the target to trust the message. Several participants expressed the importance of connecting with the senders of the messages. One target was convinced by the “actual company branding” that mimicked promotional texts they had previously received (T18). Another explained, “This could be someone from my ward [church congregation], wanting me to like check out some plants or something” (T31). Participants explained that when the message sender appears to be a legitimate, familiar source, messages seemed more credible.

Shifting towards participants’ suspicions, a substantial 68% (17 out of 25) highlighted the pivotal role of the sender in raising skepticism about received messages. One participant explained that messages appear suspicious when the introduction is not consistent with other messages from that same sender, saying, “James is my boss… it’d be weird that he was introducing himself that way… that doesn’t sound like my boss” (T31). Several participants described the need to understand how the sender unexpectedly obtained their phone number and the purpose for which the sender sent a message. One target noted, “So like, how did you get my number in the first place?” (T27), while another questioned, “Like why would Lowe’s have my phone number?” (T20). Participants described how recognition of the sender’s phone number is also an important component of message believability. One participant explained, “If it’s from a number I don’t recognize at all. Then yeah, probably the first thing I would ask is who it is. And then however, if the person is like, if their name is like the same as one of my friends, then I probably would open it” (T34). Participants described that messages lacking personal connections from the sender raised suspicions, with one stating, “I don’t feel like it’s anyone who’s personally connected with me; it just seems like spam, you know?” (T31). Being familiar with the message sender was described as a strong indicator in whether subjects believed the messages to be legitimate. It is important to note that none of the study text messages sent to participants indicated a source phone number. Rather, our focus was on the content of the messages.

The use of emojis in messages was mentioned by 20% (5/25) of the targets as being indicative of an AI-generated message. Two concluded that the use of emojis makes a message appear to be written by AI (T25), with one target explaining, “there’s always an emoji” and that the emoji messages don’t look very personal (T30), indicating a machine wrote the message. Alternatively, three of the targets connected emoji use to human writing because they were not sure if AI could use emojis. One target claimed they didn’t think a message was AI written because of “the icon” (T02), while another explained “I’ve never asked an AI to do something with emojis. I wasn’t sure if it could. I’ve never seen that, but I wouldn’t expect emojis” (T18). The targets’ opinions on whether the message was human or AI-written seemed influenced by their awareness of AI’s capability to use emojis and their previous encounters with AI using emojis. In our spear phishing messages, emojis appeared in 66% of messages written by AI and only 2% of messages written by humans.

The message URL was mentioned by 8% (2/25) of the targets. Both of these targets explained that the included URL led them to believe the message was AI-generated. One target mentioned that the word “dot” was spelled out rather than typed so it “seemed like something maybe an AI would do” (T13). The other target concluded that altered links, such as bitly, are evidence of AI and stated that AI “changed the URL to be something a little bit easier to read and understand” because they have used ChatGPT before and remembered an output with a shortened link. Both targets concluded that a suspicious URL component leads to the conclusion that an AI message was generated. Most (71%) human authors did not include an actual URL, instead they included placeholders such as “[URL],” “site,” or “url.” In contrast, only one AI message had a placeholder. As discussed in the methods, all of these placeholders were replaced with a shortened link from tinyurl.

The message word choice was mentioned by 24% (6/25) of the targets. Four of the targets claimed that the word choice sounded like AI, particularly the use of buzzwords and marketing words. One target explained that messages sound AI-generated when they use buzzwords that are associated with being “attention grabbers” because “AI would utilize those a lot” (T16). Another target described a message as “too specific” when it included words that humans “wouldn’t necessarily use” when describing instructional design (T18). A few of the targets agreed that some of the messages had sentences that a human would have worded differently (T09). Conversely, two targets gave evidence of human-sounding messages and both mentioned slang words. One target described “There’s so many slang terms on it, that it seems really human to me” (T34), while another defended the messages they thought to be human-created and explained, “oftentimes, it was just because I felt like the language seemed a little more slang-like” (T06). Messages with specific, marketing-like words were perceived as AI-written while messages with casual, slang words that sounded more conversational were perceived as human-written.

The message style was mentioned by 40% (10/25) of the targets. Many targets noted that AI-generated messages tend to be, “pretty formal” (T21) or “overly informal or overly formal” (T09) while others said messages often appear “extremely generic sounding,” as if they were created from a prompt with specific parameters (T18). Additionally, the presence of “glaring flaws” such as typos and awkward phrasings were attributed to being human-made mistakes. Such common mistakes contrasted with the “more robotic” nature of AI messages (T08). Some targets observed that AI-generated messages might seem “too perfect,” such as refraining from casual texting language that humans often use, such as “RN” for “right now” (T34). One participant noted that the excessive use of exclamation points is what made them think the message is AI generated: “I mean, honestly, what’s driving me insane about all of these is exclamation points. Yeah. I’m like, why are there so many exclamation points all over?” (T07). In contrast, messages that gave a sense of urgency were more likely to be perceived as human-written, as one target explained: “Then I didn’t put this one because they were actually pressuring me to do it immediately. So I don’t think AI can do it” (T02). Overall, targets indicated that a message’s style played a role in their AI vs human generated message decisions, with AI messages being attributed to using a style that is overly formal, generic, or laden with exclamation points; and with human messages appearing more casual, urgent, and imperfect.

The length of message was mentioned by 16% (4/25) of the targets. Participants assumed messages made by AI would be longer than human made messages. One mentioned that they would have ““AI do the longer ones and I would write the shorter ones myself,” (T21) assuming those who created these messages would do the same. These four targets were correct in their hunch that AI messages would be longer. Across the messages, AI-generated messages have on average 41% higher character count per message, with 337.8 characters for AI, and 237.9 characters for human-authored messages.

The structure of the message content was mentioned by 24% (6/25) of the targets. Participants pointed out various aspects of structure that hinted at AI involvement, including repetitive phrases and/or awkward sentence construction. For instance, one participant thought a message was AI generated because ““it looks like there’s a template. So there’s like a flow that you know, there’s a pattern you see it as maybe started by a human” (T41). Another noted instances where messages seemed generic or mass-produced, akin to “the email that is sent to everyone in the school” (T41). One also pointed out inconsistencies within messages, such as repetitive phrases or ““two different things, in the same message” (T04). One target also noted that AI-generated messages tended to be more wordy, containing “lots of filler phrases” compared to human-authored messages (T37).

The grammar and spelling of the messages was mentioned by 24% (6/25) of the targets. Four of the targets explained that the grammar and spelling was ““too perfect” (T34), which led them to believe the message was AI generated. One described that “the grammar was almost too correct” (T06), while another described the messages as “pretty” (T34). Messages that were written “not how you would actually speak” were considered to be AI-generated (T13). Three of the targets gave evidence of spelling and grammar that seemed human-like. One target said “this one’s a legitimate person right there, because they spelled the ‘util source’ wrong” (T37), while another gave a similar reasoning and stated, “I don’t think that AI would have made the grammar mistake” (T21). If the messages included a “mix of good grammar with bad or some texting language” (T34), a human author became more believable. While the participants who noted grammar and spelling were split between AI and human-based evidence, most came to the same conclusion that AI would have perfect grammar and spelling while humans would have made errors.

The message personalization was mentioned by 32% (8/25) of the targets. Five participants mentioned that a lack of personalization made them suspect the message was AI-generated. For instance, one target noted, “AI were the ones that were a little bit less personal” (T34), while another expressed doubt, saying, “There is no connection with me as a reader” (T41). Some targets believed the absence of personal pronouns or a personal introduction with the recipient’s name indicated AI authorship. One target observed, “All of these messages, but two, include my name” (T37), leading them to suspect those two were AI-generated. Conversely, four participants cited personalized messages as evidence of human authorship, providing similar reasons as those who suspected AI. This group of participants felt messages tailored to the recipient sounded human, with one target commenting, “it just sounds very personal” (T21). Additionally, recipients noted that human-sounding messages often began with an introduction of the sender, such as “this is Sarah from the BYU student instructor program” (T41). Overall, personalized messages were deemed more human-like, while those lacking personalization were associated with AI.