According to proactive cyber security specialists, Heimdal, in Copenhagen, the rapid adoption of Generative AI (GenAI) technologies has led to a significant increase in sophisticated phishing campaigns. A recent study by Abnormal Security reveals that 80% of these campaigns now leverage GenAI tools, marking a critical turning point in the fight against digital fraud.
The integration of AI in phishing attacks has led to a dramatic 1265% increase in such incidents since 2022, as reported by Infosecurity Magazine. The availability of free or trial-based AI tools, such as Chat GPT, has made it easier for cyber criminals to generate convincing phishing content, with the potential to create up to 30 templates per hour.
AI’s proficiency in generating high-quality content has significantly reduced the effectiveness of traditional phishing detection methods. AI-based proofreading tools can eliminate common phishing indicators, making attacks more challenging to identify. The rapid response rates of AI models, like Chat GPT’s 15-20 seconds and the 3.5 Turbo Model API’s under 3 seconds, further enhance the efficiency of these attacks.
The concept of ‘Malicious-AI-as-a-Service’ is gaining traction, facilitating the automation and scaling of phishing operations. This development lowers the entry barrier for cyber crime, enabling even those with minimal technical skills to execute sophisticated attacks.
Valentin Rusu, Head of Malware Research and Analysis at Heimdal, highlights the potential dangers of Reinforced Learning in black-hat hacking. “Imagine a hacker training an AI to break security systems through trial and error.” “This could lead to unprecedented cyber security challenges,” Rusu remarked.
Adelina Deaconu, Heimdal’s MXDR (SOC) Team Lead, adds that genAI has the potential to exploit personal vulnerabilities and advises people to step back if something seems suspicious. “I’m especially worried about how generative AI can now analyse and exploit personal vulnerabilities and emotions, making the emails seem more convincing. I advise people to step back, verify information, and report any concerns. If something seems suspicious, it likely is.”, says Adelina.
Brian David Crane, founder of Callersmart, an app for investigating mystery phone numbers, believes that generative AI can scale up spear phishing and vishing attacks. “With generative AI, cyber attacks can happen at scale, be relentless with malware code modification and generative chatbots using spear phishing & vishing attacks with an automated selection of targets based on publicly available data or information.”, says David.
Lukas Junokas, Chief Technology Officer at Breezit, an event planning platform, shares a challenging encounter with a phishing email that closely imitated the writing style of a high-ranking executive, asking for confidential information. This email managed to evade standard detection filters because of its authenticity. “Generative AI has undeniably transformed phishing, making attacks more personalised and harder to detect. The new challenge lies in the arms race between evolving AI capabilities in both creating and detecting sophisticated threats,” Lukas noted.
Heimdal quotes statistical insights on the growing AI threat gleaned whilst researching the topic:
● 83% of companies prioritise AI over other technologies (Notta AI).
● 51% of businesses rely on AI for threat detection and remediation (EFT Sure).
● One in five people will open AI-generated phishing emails (Sosafe Awareness).
● 69% of organisations stated that they could not avert cyber-attacks without AI (Capgemini).
As AI continues to evolve, organisations and individuals must stay informed and exert caution when opening emails.
“People should pay attention to strange email addresses, tone of the email, requests for sensitive information, signature and formatting, and should avoid clicking on URLs (hover over them first and see if the displayed URL matches the visible text),” says Adelina.
Understanding the capabilities and potential misuse of AI in phishing is the first step toward developing more effective countermeasures.
