AI phishing: How to defend AI-generated attacks

The rise of ever-dangerous AI phishing attacks

Phishing attacks have always been dangerous, but AI is taking the level of sophistication and their scale and precision. Large language models can now generate highly tailored phishing emails with just minimal data points, making it easier and faster to create convincing deceptions. The advanced use of psychological tools continues to exploit our inherent human tendency like trust in authority, fear of missing out, and or the desire to respond.

Research published this year has shown that when AI-generated emails are combined with these psychological techniques, the success rates are very high. It shows us that AI-generated phishing emails can achieve click-through rates as high as 44%, and when combined with psychological models, success rates soar to upwards of 80%. Phishing campaigns that once took significant time and skill are now automated, lowering the bar for non-technical attackers to launch targeted phishing at scale, increasing their impact while reducing the cost, time and effort.

Phishing works because it preys on common human biases, which we all have, but it’s more nuanced than that. What one person falls for, may raise an instinctive concern for another. Some are reeled in by a sense of urgency, secrecy, or simply just trusting the sender or source. Our complex diversity of potential human responses is why phishing attacks continue to be so effective despite growing awareness, especially financial scams, that exploits the same human condition.

Using AI as a Defence Against Phishing

The enhanced AI driven tools that attackers use to craft those sophisticated phishing emails can also be used to detect them, faster and more effectively intercepting the attack. Whilst the AI-enabled security tech is improving, its performance is not always perfect, and phishing detection still needs a ‘human in the loop’ security and IT professionals continue to provide essential human oversight. Combining AI detection with human verification and advanced behavioural analytics, provides a robust approach. Of course, on-going education to help recognise subtle and advanced phishing attempts, sharing and celebrating experiences.

Whilst human error continues to make up the vast percentage of data breach incidents, as reported by the ICO, I recommend avoiding calling out your workforce as the weakest link, share poor and celebrate good behaviours elevating everyone as your strongest asset. Go beyond one-and-done annual interventions, they are ineffective against this threat, only continuous supportive education on this topic genuinely supports positive security culture outcomes.

Changing behaviour requires making it easy for people to do something. It's not enough to simply tell your staff or write it in a policy. Simplifying steps to help them respond quickly and effectively to potential risks. Nudge theory, which has been around for some time, and used in society to help change or influence behaviour. Not to be confused with notification, nudging focuses on subtle behaviours to help people make better choices naturally, without force, remember when sweets and snacks were in the check-out aisle.

In our world of cyber security, instead of relying on fear tactics that can cause resistance, nudging offers a smoother approach. To try and build better security habits by making the right decision feel like the easiest one, steering people towards better outcomes without overwhelming or pressuring them.

Nudging can be effective but may face limitations in combating AI-driven phishing. As AI generates more and more sophisticated and personalised attacks, subtle nudges might not always be sufficient to counter such tailored threats. Attackers can exploit behavioural tendencies with precision, making it harder for general nudges to guide users effectively. In this fast-evolving threat landscape, stronger interventions, such as multi-layered defences, real-time AI detection, and advanced AI-enabled behavioural analysis, are necessary to maintain pace with the sophistication of AI-enhanced phishing.

Building Resilience Against AI Phishing

To help with the sophistication of AI-enhanced phishing, your multi-layered security strategy must use both tech and human insight, a powerful collaborative force. Having security teams that are skilled in understanding and collaborating with AI, will give you a competitive advantage.

Four strategies to combat AI phishing:

1. AI-Driven Behavioural Analytics

Security systems must use AI to analyse user behaviour in real-time, flagging anomalies such as unusual communication patterns or login activity. These systems need to adapt dynamically to new phishing tactics, offering proactive defence against evolving threats.

2. Advanced Email Filtering

Organisations should adopt AI-powered email filters that go beyond traditional keyword detection, focusing on behavioural signals and language patterns that may indicate phishing intent. These tools should be integrated with machine learning models that evolve with new attack vectors.

3. Personalised Staff Training

Just as attackers use AI to personalise phishing emails, organisations should consider using this approach to tailor security training. AI-driven phishing simulations should adapt to the behaviour of individual employees, with integrated nudging techniques.

4. Adopt Zero Trust Principles

As AI-enabled phishing attacks become more personalised, a Zero Trust approach assumes you are in a hostile environment, and will be compromised, while ensures that no action is trusted without verification. This includes verifying identities, permissions, and intent at every stage of interaction, helping to reduce the impact of a breach.

Evolve your approach with sophisticated defences

The convergence of AI, security and psychology in phishing has fundamentally altered the threat landscape. Attackers are no longer limited by time or skill; they can now launch highly targeted, effective campaigns at low costs and scale. Organisations must respond with equally sophisticated defences, combining AI-driven detection powered by human insight with tailored staff training and data driven telemetry for your enhanced behavioural analytics.

As AI-enhanced phishing becomes more prevalent, organisations must evolve their strategies to stay ahead. Defending against these advanced attacks requires more than traditional security measures, it needs a deep understanding of both the technological and psychological elements at play.

As we covered in our top cyber security tips article, every employee, at every level, should be empowered to make smarter cybersecurity decisions. With organisational-level cyber security awareness training, you can build robust defences in your people, processes and technology, and protect against new threats, such as AI phishing.

Only by integrating these insights can organisations effectively mitigate the rising threat of AI-driven phishing in the years ahead.

Looking to build AI security skills in your business? Learn more about our cyber security training, or get in touch to discuss your requirements.