AI-Written Phishing Emails: New Red Flags for Teams

For years, security training told staff to look for bad grammar and odd spelling. That advice is now weak. In a December 2024 public service announcement, the FBI warned that criminals use generative AI to commit fraud on a larger scale and to make their schemes more believable, including by using AI translation to limit the grammar and spelling errors that once exposed them.

This guide covers what the evidence shows and the habits that still catch AI-polished phishing.

Try it: use ProofAI's AI checker tool, free for 10 checks a day with no login.

What the research shows

The clearest test so far is a study by Fred Heiding, Bruce Schneier and colleagues, published on arXiv in November 2024. With 101 participants, they compared four kinds of email:

Email type	Click-through rate
Generic phishing (control)
Written by human experts
Fully automated by AI
AI with a human in the loop

The fully automated AI emails did as well as the human experts. The AI's background research on targets was accurate and useful in 88% of cases, and each automated email cost roughly four cents in API fees. The sample was small and drawn from a university setting, so treat the exact figures with care. The direction is clear, though: personalised phishing no longer needs a skilled human writer.

The scale of the problem is large. The FBI's 2025 IC3 report recorded 24,768 business email compromise complaints with reported losses of about USD 3.05 billion in 2025. It also counted 22,364 complaints that referred to AI, with reported losses of about USD 893 million. These are US complaints only, but the same tactics are used against Indian companies.

How attackers use AI

The FBI PSA lists several methods. The ones most relevant to workplace phishing are:

Want to check your own content? Try ProofAI free 10 checks a day, no login needed.

Writing more messages, faster, with believable wording.
Translating into fluent English or other languages, removing the usual giveaways.
Building fake supporting content, such as investment websites, social media profiles and AI chatbots on fraud sites.
Cloning voices to impersonate a relative or an account holder.
Faking video calls with company executives or authority figures.

The last two matter because they let an email scam move to a phone or video call that seems to confirm it.

Also read: Signs Content Was Written by ChatGPT (and Their Limits)

Red flags that still work

Polished language is no longer a sign of a genuine email. Watch the request and the context instead. Pause if a message asks you to:

change bank details for a supplier, employee or customer
make an urgent payment that skips the normal approval process
buy gift cards or send cryptocurrency
share a one-time password (OTP), password or login link
keep the request secret from colleagues or your manager
open an unexpected attachment or follow a link to sign in
act before a deadline that is very close

Check the technical details too: does the reply-to address match the sender? Does the link go where the text says it does? Is the domain spelt slightly differently from the real one?

Four habits that stop AI phishing

1. Verify through a separate channel. The FBI advises verifying a caller by hanging up, finding the organisation's contact details yourself, and calling that number directly (FBI PSA). Use the same approach for emailed requests: never use the phone number or link in the suspicious message.

2. Require two people for payments and bank detail changes. A second approver who follows a written checklist makes a single convincing email much less dangerous.

3. Agree a verification phrase for high-risk requests. The FBI suggests families create a secret word to confirm identity. Finance teams can use the same idea for urgent requests from senior staff, especially by phone or video.

4. Report, don't just delete. One report lets your security team warn colleagues who received the same message and block the sender.

Update your training

Remove "look for spelling mistakes" as a primary tip.
Use well-written examples in phishing simulations, not just clumsy ones.
Teach one habit: pause and verify any request involving money, credentials or secrecy.
Make reporting quick and blame-free, so people report mistakes early.

If someone falls for it

Contact your bank immediately to try to stop or reverse the payment.
Change any exposed passwords and turn on multi-factor authentication.
In India, call the national cyber crime helpline 1930 or report at cybercrime.gov.in. Kerala Police notes the helpline runs 24 hours a day and that prompt reporting is crucial. Outside India, report to your national body; in the US, that is IC3.
Tell your IT or security team so they can check for wider compromise.

Where AI text detection fits

An AI detector cannot tell you whether an email is a scam. Many legitimate senders use AI writing tools too. Detection is useful as one extra signal: if an "urgent message from the CEO" scores as likely AI-generated and is also asking for an unusual payment, that is a good reason to verify before acting.

ProofAI's ChatGPT text detector checks pasted text of 50 words or more and shows which passages drove the score, with 10 free checks a day and no account. Teams handling a high volume of reported emails can connect ProofAI through its REST API. Either way, treat the result as a probability, not proof, and keep call-back verification as your main defence.