Why defensive AI alone is not enough: the crucial role of a strong security culture

Before the rise of AI-driven cyber threats, phishing attempts were often easy to spot. Poor grammar, overly manipulative language, and unsolicited requests were telltale signs of malicious attacks.

With the implementation of offensive AI used by threat actors, these phishing attempts have become harder to identify. While Secure Email Gateways (SEGs) have also implemented defensive AI to combat these threats, these emails are still reaching users’ inboxes.

The AI email security gap

AI and Machine Learning (ML) models in SEGs are primarily trained on historical data, enabling them to recognize phishing patterns. While this retrospective approach is effective for identifying known threats, it struggles to keep pace with rapidly evolving attack techniques. Offensive AI enables threat actors to generate highly professional, industry-specific phishing emails using minimal effort or time investment. These attacks can mimic the jargon and technical terms of targeted sectors, making malicious emails appear legitimate, and allowing them to bypass SEGs.

Despite embracing AI capabilities with open arms, SEGs are still struggling to keep up with these sophisticated phishing attempts. While AI can efficiently identify repetitive patterns and filter out bulk threats, it remains reactive. This gap between offensive and defensive AI leaves organizations vulnerable to novel phishing techniques.

How attackers bypass SEGs

Cybercriminals continuously develop new methods to circumvent SEGs, often manipulating legitimate services or introducing novel techniques that AI models have yet to encounter. Some of the most effective tactics include:

QR codes: Embedding malicious links within QR codes can be challenging for AI systems to analyze automatically. This attack method requires the employee to scan a code on their phone, removing the physical protection on their enterprise systems. The most recent innovative QR code technique involves rotating and embedding one QR code within another so a SEG scanning a QR code will get a different result than a victim who is instructed to scan the code sideways.

Malicious attachments: Disguising harmful links within seemingly benign attachments, such as PDFs or Microsoft Office documents, allows attackers to exploit the trust associated with common business communication.

URL obfuscation or redirection: Threat actors use legitimate services to mask malicious links, redirecting victims to phishing sites.

SEG-encoded links: Since SEGs rewrite incoming email URLs to scan for threats, attackers can embed pre-encoded URLs from other SEGs, tricking security filters into marking them as safe.

Malicious HTML files: Attackers attach malicious HTML files that, when opened, direct users to phishing sites or prompt credential entry.

These various techniques highlight the adaptive nature of phishing threats and techniques employed by threat actors that are used to bypass email security defenses.

The necessity for a strong security culture

As phishing attacks evolve, introducing novel threats that AI tools may not yet recognize, human ingenuity becomes a vital component of a comprehensive, layered defense strategy. This makes the cultivation of a strong security culture within organizations essential. While AI excels at routine pattern recognition and data filtering, human intuition and vigilance remain indispensable for identifying and responding to complex or ambiguous threats.

Building a robust security culture starts with communicating the significance of email security and positioning employees as the first line of defense. Creating a non-punitive environment where staff feel empowered to report suspicious activity is key to enhancing overall security.

This can be achieved by implementing user-friendly reporting tools, enabling quick identification and response to live threats, and offering interactive training sessions tailored to the unique risks faced by the organization. These initiatives ensure employees are equipped with the knowledge to spot and report phishing attempts effectively.

Recognizing and rewarding proactive security behaviors not only boosts engagement but also reinforces the value of individual contributions to organizational safety. By integrating these elements of a strong security culture, organizations can leverage human ingenuity alongside AI-driven defenses to create a formidable, multi-layered approach to threat protection.

Combining the power of AI efficiency and human ingenuity

While defensive AI can offer significant advantages, it is not infallible. The most effective defense against sophisticated phishing attacks combines AI-driven capabilities with human insight. AI excels at managing repetitive tasks and flagging potential issues, but human analysis is crucial for interpreting context, assessing nuances, and making informed decisions in ambiguous situations.

As phishing strategies continue to evolve, organizations must recognize that AI alone is not enough. By investing in a strong security culture that empowers employees to serve as vigilant defenders and complementing this with the power of advanced AI tools, organizations can establish a resilient, multi-layered defense against cyber threats.

We've featured the best encryption software.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro