Updated: 6 July 2026
What are AI-enhanced attacks and how do they differ from traditional threats? AI-enhanced attacks are cyberattacks in which artificial intelligence helps attackers accelerate, scale, or improve traditional methods such as phishing, social engineering, reconnaissance, or vulnerability exploitation. The key difference is that AI removes historical barriers for attackers: time, cost, language skill, and technical skill. A phishing campaign that previously took weeks to prepare can now be built in hours. Deepfake fraud that once required specialist capability is now available through APIs. For CISOs and boards, this is a fundamental shift in threat modelling. The objective has not changed – money, data, sabotage – but the barrier to entry has fallen sharply.
Ai attacks in cybersecurity – key takeaways:
- IBM X-Force 2023: AI-generated spear phishing takes an attacker about 5 minutes, compared with about 16 hours for a manually prepared message. This compression changes the economics of attacks at scale
- Deepfake fraud is already costing companies millions. In the Arup/Hong Kong 2024 incident, an employee authorised transfers of about USD 25 million after a video conference with AI-generated colleagues
- AI is not used only by APT and organised criminal groups. Low-budget attackers can access similar tools through dark web services and publicly available models
- Defences based only on signatures and known attack patterns are becoming less effective against AI-generated, unique phishing messages
- Human vigilance must shift towards process signals, such as “does this bypass the normal workflow?”, not only language signals such as grammar errors or strange tone, because AI removes many of those signals
- Patronusec, in the vCISO role, helps organisations update threat models for AI-enhanced attacks and implement controls suited to the new threat landscape
Table of Contents
How does AI change phishing, and why might traditional training no longer be enough?
Traditional phishing training teaches employees to look for grammar mistakes, odd tone, unknown senders, and generic greetings. AI removes most of these signals.
What AI changes in phishing:
Personalisation at scale: AI can generate personalised messages for thousands of targets at once, using data from LinkedIn, social media, public job postings, and data leaks. The message can include the recipient’s name, role, employer, and references to recent projects or company events.
Elimination of language errors: AI-generated phishing in Polish or English can be grammatically correct, natural, and aligned with an organisation’s communication tone, especially if the attacker has analysed public company materials.
Contextual credibility: AI can generate messages with precise business context, such as “following the proposal submitted by your sales team on X date.” This creates credibility without requiring access to internal company data.
How to update phishing training:
Traditional “spot the mistakes in this email” training is no longer enough. The programme must evolve:
- Process signals instead of language signals: Teach employees to ask: “does this bypass the normal workflow?”, “does it create artificial urgency?”, “does it ask for secrecy?”, “does it move the conversation outside official channels?”
- Out-of-band verification: Every non-standard request, such as a transfer, bank detail change, or access request, must be confirmed through a channel other than email, such as a known phone number or corporate messenger
- AI-phishing simulations: Simulation platforms should include AI-generated messages, not only templates with obvious warning signs
- A culture where checking is normal: Employees must feel able to verify a request without being treated as difficult, especially for financial requests
Patronusec Insight: The most common mistake in phishing programmes after moving into the AI era is changing simulation templates without changing the success metric. Organisations measure “what percentage clicked” instead of “what percentage reported the suspicious message.” In an AI-phishing environment, detection and reporting matter more than recognising fixed message characteristics, because those characteristics keep changing. We deliver phishing training and simulations with reporting metrics as part of the vCISO service and as a separate phishing training and simulation service.
What is deepfake fraud and how should companies protect themselves?
Deepfake fraud is financial fraud using synthetic media, such as AI-generated voice or video, to impersonate trusted people: a CEO, CFO, lawyer, or business partner.
Scale of the threat:
The Arup/Hong Kong incident in 2024 is the most widely publicised case: an employee joined a video conference with AI-generated people impersonating known colleagues and authorised transfers totalling about USD 25 million. This was not an isolated event. FBI IC3 has reported a growing number of BEC cases involving deepfake audio and video.
How deepfake fraud works in practice:
- The attacker collects voice or video samples from public recordings such as YouTube, conference talks, or LinkedIn
- The attacker trains or uses a ready-made deepfake model, often available commercially through an API
- The attacker contacts the victim through an unexpected video call or video conference
- The attacker asks for an urgent and confidential transaction or access data
- The victim hears and sees a “trusted person” and does not recognise the fraud
Protective measures:
- Out-of-band verification for payments: Every payment above a defined threshold must be confirmed through another channel, such as SMS, an authorisation app, or a call back to a known number. Never treat a video conference as sufficient authorisation for a transaction.
- Verification phrase: Establish an internal word or phrase that confirms identity in suspicious situations and is known only inside the organisation.
- Zero-trust payment procedures: No transfer above X PLN should be initiated by one person without second approval, regardless of the authorisation channel.
- Training for finance and management: People responsible for transfers must understand deepfake fraud and verification procedures. They are the most common targets.
Do you want to update your security programme for AI-enhanced attacks?
Patronusec reviews security programmes against AI-enhanced threats and recommends updates, from phishing training to payment verification procedures and incident management.
Book an AI threats security programme review
How does AI change vulnerability exploitation and what does it mean for vulnerability management?
Beyond phishing and social engineering, AI is increasingly used in the technical parts of attacks.
AI-assisted vulnerability research:
Vulnerability researchers, both legitimate and criminal, use LLMs to analyse source code, identify potential logic flaws, and generate proof-of-concept exploits. This accelerates the cycle from vulnerability discovery to working exploit.
The implication is clear: the window between CVE publication and exploit availability, often called mean time to exploit, is shrinking. Companies with a “patch within 30 days” policy should consider 72-hour remediation for critical CVEs on edge systems.
Automated scanning and reconnaissance:
AI automates reconnaissance: gathering information about a target from public sources (OSINT), scanning attack surface, and identifying weak points. What previously took weeks of manual work can now be automated.
AI-generated malware:
Malware generated with AI assistance has been observed with capabilities intended to avoid traditional AV/EDR signatures. AI can generate code mutations that are unknown to signature databases.
What this means for vulnerability management:
- Patch prioritisation must consider exploitation speed, not only CVSS severity
- Vulnerability scans must be more frequent for internet-facing systems
- EDR with behavioural detection, not only signature detection, becomes more important
- Attack Surface Management (ASM), meaning continuous monitoring of the external attack surface, becomes essential
Patronusec Insight: Companies that patch systems “once a month during the maintenance window” are increasingly exposed. Critical CVEs in VPNs or firewalls can now be exploited within hours of publication by automated systems scanning the internet. We recommend a separate accelerated process for critical CVEs on edge systems, independent of the standard patch cycle. Regular vulnerability scans provide visibility into current exposure.
How should CISOs update their threat models for AI-enhanced attacks?
The threat model must treat AI as a force multiplier for attackers, not as a completely separate attack vector.
Threat model update:
For each threat scenario in the existing model, ask: “How does AI change the likelihood or impact of this threat?” Examples:
- Phishing: success probability increases as quality signals disappear; scale increases through mass personalisation
- BEC and deepfake: a new vector becomes more common because technical requirements are lower
- Vulnerability exploitation: exploitation windows shrink and reconnaissance becomes faster
- Insider threat: AI can help an attacker plan and hide activity
Controls that become more important:
- Zero-trust authorisation for payments and privileged actions
- Behavioural EDR that detects abnormal activity rather than only signatures
- Training focused on process signals, not only language signals
- Out-of-band verification for critical actions
- Shorter patch cycles for edge systems
Controls that become less reliable on their own:
- Signature-only malware detection
- Phishing training based only on spotting poor grammar
- Firewall-first defence without internal behavioural monitoring
FAQ – AI Attacks in Cybersecurity
Is AI used by all attackers or only by APT groups?
AI is increasingly available beyond advanced APT groups. Ready-made tools for phishing generation, voice cloning, and simple exploitation automation are available through dark web services and public APIs. This means lower barriers to entry for both sophisticated groups and less skilled criminals.
How can you recognise a deepfake video during a call?
Current deepfake video may show artefacts such as irregular blinking, distortions around the face during fast movement, unnatural hair or clothing behaviour, or delay between lip movement and audio. But the technology is improving quickly. A better defence than “recognise the deepfake” is the procedure: no payment is authorised solely through a video conference, and confirmation through a second channel is always required.
Are there tools that detect AI-generated phishing?
Yes. Tools such as Darktrace, Abnormal Security, and Microsoft Defender for Office 365 use machine learning to detect email anomalies. However, no tool guarantees 100% effectiveness. Layered protection – tooling, training, and procedures – remains necessary.
How often should phishing training be updated in the AI era?
At minimum, update it annually or after each significant change in the phishing landscape, such as a new attack type or new AI tooling used by attackers. Phishing simulations should run monthly or quarterly with varied scenarios, including AI-generated messages.
How does Patronusec help organisations adapt to AI-enhanced threats?
We review security programmes against AI-enhanced threats: threat model updates, control effectiveness assessment, recommendations for phishing training with AI scenarios, and payment verification procedures. In the vCISO model, we monitor the evolution of AI threats and recommend security programme updates continuously.
Does AI on the defender side neutralise AI in attackers’ hands?
AI is used on both sides. AI-powered EDR and SIEM can improve behavioural anomaly detection. However, attackers retain an asymmetrical advantage: they need to succeed once, while defenders need to block every attack. AI does not remove that asymmetry. It raises the level of the fight, but it does not eliminate the threat. Security culture, procedures, and training remain critical alongside tools.
AI-enhanced attacks and security programme update – free consultation
Patronusec helps organisations update their security programmes for AI-enhanced threats, from threat modelling and phishing training to payment verification procedures and vulnerability management.
In a free 30-minute consultation, we will help you:
- Assess whether your current security programme adequately addresses AI-enhanced threats
- Identify key gaps in phishing training, payment procedures, and vulnerability management
- Plan security programme updates prioritised by risk
- Choose a support model: one-off review or ongoing vCISO support
Free consultation | vCISO | Phishing training and simulations | Penetration testing | Vulnerability scans
