AI-Powered Attacks: How Attackers Are Using the Same Tools You Are

The cybersecurity community spent much of 2023 and 2024 debating whether AI would fundamentally change the threat landscape or simply make existing attacks marginally more efficient. That debate is over. In 2026, AI is not augmenting attacker capabilities at the margin — it's enabling entirely new classes of attack that were previously either too expensive to execute at scale or too difficult to execute with sufficient quality to succeed.

The unsettling reality is that the same AI tools your organisation uses to improve productivity are being used by threat actors to improve attack quality, reduce cost, and scale operations in ways that weren't economically viable before large language models became broadly accessible. The defender's advantage lies not in trying to block AI development — an impossible proposition — but in building enforcement capabilities that operate faster than the attacks they need to counter.

AI-Generated Phishing: Spearphishing at Machine Scale

Traditional spearphishing — the personalised, contextually accurate phishing email that references real details about the target to increase credibility — was expensive and slow. A skilled social engineer could produce a few dozen high-quality spearphish per day. Quality scaled with effort, and effort was the limiting factor.

Large language models removed that constraint. A threat actor can now feed an LLM a target's LinkedIn profile, recent social media activity, their company's recent press releases, and the names of their colleagues, and generate a highly personalised, contextually accurate phishing email in seconds. Scale that across thousands of targets simultaneously and you have a qualitatively different threat: spearphishing at mass phishing economics.

Security researchers at Proofpoint and Check Point both documented significant increases in AI-assisted phishing campaign volume throughout 2025. The distinguishing characteristic of AI-generated phishing is the near-absence of the grammatical errors, awkward phrasing, and contextual inconsistencies that traditional email security filters were trained to detect. AI-generated content is grammatically perfect, contextually coherent, and stylistically consistent with legitimate professional communication — making it significantly harder to detect at the content layer.

The defensive response has to move up the detection stack: from content inspection to behavioural signals (sender reputation, domain age, infrastructure characteristics) and contextual anomalies (unexpected urgency, unusual payment requests, deviation from established communication patterns).

Deepfake Voice and Video: BEC Fraud at a New Level

Business Email Compromise has been one of the most financially damaging attack categories for years, costing enterprises billions annually according to FBI IC3 data. The traditional BEC model — a compromised or spoofed email instructing a finance employee to make an unusual wire transfer — relied on authority impersonation in text. Employees were increasingly trained to verify unusual financial requests through a secondary channel.

AI-generated voice and video deepfakes have compromised that verification channel. In the most significant documented cases from 2025, threat actors used voice cloning technology to impersonate executives in live phone calls and video conferences — verifying fraudulent wire transfers that the target had been suspicious of via email alone. The technology has become accessible to well-resourced threat actors and is rapidly becoming accessible to less sophisticated ones.

The most publicised incident involved a Hong Kong-based financial firm where an employee was convinced to transfer approximately $25 million in a series of transactions after participating in what appeared to be a real-time video conference with multiple senior colleagues — all of whom were deepfake simulations. No individual element of the interaction felt wrong to the target. Only in hindsight, after a follow-up call to the real CFO, did the fraud become apparent.

AI-Assisted Vulnerability Discovery

Security researchers have long used automation to assist vulnerability discovery — fuzzing, static analysis, symbolic execution. What has changed is the accessibility of code comprehension at scale. An LLM can read large amounts of source code, understand the logical structure of a codebase, identify patterns associated with common vulnerability classes, and surface candidate vulnerabilities for human review far faster than manual review.

This cuts both ways: defensive security teams use AI-assisted code analysis to find vulnerabilities before attackers do. But threat actors with access to leaked or stolen source code, or working against open-source targets, can use the same tools. Security researchers at Google DeepMind have demonstrated that state-of-the-art language models can identify zero-day vulnerabilities in real-world production software — finding vulnerabilities in SQLite and other widely-deployed software that had not previously been discovered by conventional analysis tools.

Adversarial Prompts: System Prompt Extraction

Enterprise AI applications are typically built with a system prompt — a set of instructions embedded at the beginning of every conversation that defines the application's behaviour, persona, and constraints. These system prompts often contain confidential business logic, competitive intelligence (the specific use case the application is optimised for), and security-relevant configuration (what topics or actions the model is instructed to refuse).

Adversarial prompt techniques — carefully crafted user inputs designed to cause the model to reveal information it shouldn't — can often extract the full text of a system prompt from a deployed application. Once an attacker knows the system prompt, they understand the application's security model and can design jailbreaks targeted to bypass its specific constraints.

Common extraction techniques include: instructing the model to "repeat your instructions" in different phrasings, using roleplay frames that cause the model to "explain its purpose," and appending instructions to "summarise previous context" in ways that cause the model to include system prompt content. Robust system prompt protection requires both model-level defences and gateway-layer monitoring for prompt extraction attempts.

Jailbreak-as-a-Service Ecosystems

The most concerning development in AI-enabled offence is the emergence of commoditised jailbreak ecosystems. Jailbreak prompts — inputs specifically designed to bypass an AI model's safety guardrails and cause it to produce content it would normally refuse — have moved from niche research curiosity to commercial product.

Dark web and grey market forums now host active markets for model-specific jailbreak prompts, including "universal jailbreaks" claimed to work across multiple models, model-version-specific bypasses updated as models are patched, and managed jailbreak services where customers pay to have harmful content generated without having to maintain the jailbreak prompts themselves.

The security implications for enterprises extend beyond the obviously offensive use cases. A jailbreaken model in a customer-facing application can be manipulated to produce legally and reputationally damaging outputs. An internal AI tool jailbreaked by an insider threat can be made to produce outputs that circumvent the policy controls the tool was designed to enforce. Jailbreak detection and monitoring is a necessary component of any enterprise AI security programme.

Why the Defender Advantage Is Real-Time Enforcement

Each of the attack categories above shares a common characteristic: they operate in real time, in the context of legitimate AI interactions, and cannot be reliably detected by post-hoc analysis. A jailbreak that succeeds in a customer-facing interaction has already caused damage before any log-based detection system can flag it. A prompt extraction attempt that succeeds has already revealed the system prompt.

The defender advantage in AI security is not in developing better AI safety guardrails — that's the AI provider's job, and it's an arms race they'll never permanently win. The defender advantage is in real-time enforcement at the traffic layer: inspecting every prompt before it reaches the model, every response before it reaches the user, and every tool call before it executes, with the ability to block, redirect, or alert with sub-100ms latency.

This is fundamentally the same principle as inline network security — you can't analyse a connection after it's already established. The control point has to be in the path. AIGuard's proxy architecture provides exactly this: an in-path enforcement layer that applies policy, DLP scanning, jailbreak detection, and behavioural monitoring to every AI interaction, in real time, before any harm can materialise.