AI vs. AI: Detecting an AI-obfuscated phishing marketing campaign

Microsoft Menace Intelligence just lately detected and blocked a credential phishing marketing campaign that probably used AI-generated code to obfuscate its payload and evade conventional defenses. Showing to be aided by a big language mannequin (LLM), the exercise obfuscated its habits inside an SVG file, leveraging enterprise terminology and an artificial construction to disguise its malicious intent. In analyzing the malicious file, Microsoft Safety Copilot assessed that the code was “not one thing a human would usually write from scratch because of its complexity, verbosity, and lack of sensible utility.”

Like many transformative applied sciences, AI is being adopted by each defenders and cybercriminals. Whereas defenders use AI to detect, analyze, and reply to threats at scale, attackers are experimenting with AI to boost their very own operations, equivalent to by crafting extra convincing lures, automating obfuscation, and producing code that mimics authentic content material. Though the marketing campaign on this case was restricted in nature and primarily aimed toward US-based organizations, it exemplifies a broader development of attackers leveraging AI to extend the effectiveness and stealth of their operations. This case additionally underscores the rising want for defenders to grasp and anticipate AI-driven threats.

Regardless of the sophistication of the obfuscation, the marketing campaign was efficiently detected and blocked by Microsoft Defender for Workplace 365’s AI-powered safety programs, which analyze alerts throughout infrastructure, habits, and message context that stay largely unaffected by an attacker’s use of AI. By sharing our evaluation, we intention to assist the safety group acknowledge related ways being utilized by risk actors and reinforce that AI-enhanced threats, whereas evolving, will not be undetectable. As we talk about on this publish, an attacker’s use of AI usually introduces new artifacts that may be leveraged for detection. By making use of these insights and our beneficial finest practices, organizations can strengthen their very own defenses towards related rising, AI-aided phishing campaigns.

Phishing marketing campaign ways and payload

On August 18, Microsoft Menace Intelligence detected a phishing marketing campaign leveraging a compromised small enterprise electronic mail account to distribute malicious phishing emails meant to steal credentials. The attackers employed a self-addressed electronic mail tactic, the place the sender and recipient addresses matched, and precise targets had been hidden within the BCC area, which is completed to aim to bypass fundamental detection heuristics. The content material of the e-mail was crafted to resemble a file-sharing notification, containing the message:

Hooked up to the e-mail was a file named 23mb – PDF- 6 pages.svg, designed to appear like a authentic PDF doc although the file extension signifies it’s an SVG file. SVG recordsdata (Scalable Vector Graphics) are engaging to attackers as a result of they’re text-based and scriptable, permitting them to embed JavaScript and different dynamic content material straight inside the file. This makes it attainable to ship interactive phishing payloads that seem benign to each customers and plenty of safety instruments. Moreover, SVGs help obfuscation-friendly options equivalent to invisible parts, encoded attributes, and delayed script execution, all of which can be utilized to evade static evaluation and sandboxing.

When opened, the SVG file redirected the consumer to a webpage that prompted them to finish a CAPTCHA for safety verification, a typical social engineering tactic used to construct belief and delay suspicion. Though our visibility for this incident was restricted to the preliminary touchdown web page as a result of exercise being detected and blocked, the marketing campaign would have very probably offered a pretend register web page after the CAPTCHA to reap credentials.

An evaluation of the SVG code discovered that it used a singular technique of obfuscating its content material and habits. As a substitute of utilizing cryptographic obfuscation, which is usually used to obfuscate phishing content material, the SVG code on this marketing campaign used business-related language to disguise its malicious exercise. It did this in two methods:

First, the start of the SVG code was structured to appear like a authentic enterprise analytics dashboard. It contained parts for a supposed Enterprise Efficiency Dashboard, together with chart bars and month labels. These parts, nevertheless, had been rendered fully invisible to the consumer by setting their opacity to zero and their fill to clear. This tactic is designed to mislead anybody casually inspecting the file, making it seem as if the SVG’s sole objective is to visualise enterprise information. In actuality, although, it’s a decoy.

Second, the payload’s performance was additionally hidden utilizing a inventive use of enterprise phrases. Throughout the file, the attackers encoded the malicious payload utilizing a protracted sequence of business-related phrases. Phrases like income, operations, danger, or shares had been concatenated right into a hidden data-analytics attribute of an invisible factor inside the SVG.

The phrases on this attribute had been later utilized by embedded JavaScript, which systematically processed the business-related phrases by a number of transformation steps. As a substitute of straight together with malicious code, the attackers encoded the payload by mapping pairs or sequences of those enterprise phrases to particular characters or directions. Because the script runs, it decodes the sequence, reconstructing the hidden performance from what seems to be innocent enterprise metadata. This obfuscated performance included redirecting a consumer’s browser to the preliminary phishing touchdown web page, triggering browser fingerprinting, and initiating session monitoring.

Utilizing AI to investigate the marketing campaign

Given the distinctive strategies used to obfuscate the SVG payload’s performance, we hypothesized that the attacker could have used AI to help them. We requested Safety Copilot to investigate the contents of the SVG file to evaluate whether or not it was generated by AI or an LLM. Safety Copilot’s evaluation indicated that it was extremely probably that the code was artificial and certain generated by an LLM or a instrument utilizing one. Safety Copilot decided that the code exhibited a stage of complexity and verbosity not often seen in manually written scripts, suggesting it was produced by an AI mannequin reasonably than crafted by a human.

Safety Copilot offered 5 key indicators to help its conclusion:

1. Overly descriptive and redundant naming
   • The operate and variable names (e.g., processBusinessMetricsf43e08, parseDataFormatf19e04, convertMetricsDataf98e36, initializeAnalytics4e2250, userIdentifierb8db, securityHash9608) observe a constant sample of descriptive English phrases concatenated with random hexadecimal strings. This naming conference is typical of AI/LLM-generated code, which frequently appends random suffixes to keep away from collisions and improve obfuscation.

2. Modular and over-engineered code construction
   • The code construction is extremely modular, with clear separation of issues and repeated use of comparable logic blocks (e.g., mapping enterprise phrases to character codes, block reversal, offset correction, token-based validation). This systematic method is attribute of AI/LLM output, which tends to over-engineer and generalize options.

3. Generic feedback
   • Feedback are verbose, generic, and use formal enterprise language (“Superior enterprise intelligence information processor”, “Enterprise terminology parser for standardized format conversion”, “Generate safe processing token for information validation”), which is a trademark of AI-generated documentation.

4. Formulaic obfuscation strategies
   • The obfuscation strategies (e.g., encoding enterprise phrases, multi-stage information transformation, dynamic operate creation) are carried out in a manner that’s each thorough and formulaic, matching the fashion of AI/LLM code technology.
5. Uncommon use of CDATA and XML declaration
   • The SVG code consists of each an XML declaration and a CDATA-wrapped script, which is extra typical of LLM-generated code that goals to be “technically right” or to imitate documentation examples, even when such parts are pointless for the assault to operate.

Utilizing AI to detect the marketing campaign

Whereas using AI to obfuscate phishing payloads could seem to be a major leap in attacker sophistication, it’s vital to grasp that AI doesn’t essentially change the core artifacts that safety programs depend on to detect phishing threats. AI-generated code could also be extra advanced or syntactically polished, but it surely nonetheless operates inside the identical behavioral and infrastructural boundaries as human-crafted assaults.

Microsoft Defender for Workplace 365 makes use of AI and machine studying fashions skilled to detect phishing and are designed to determine patterns throughout a number of dimensions—not simply the payload itself. These embrace:

• Assault infrastructure (equivalent to suspicious area traits, internet hosting habits)
• Techniques, strategies, and procedures (TTPs) (equivalent to using redirects, CAPTCHA gates, session monitoring)
• Impersonation methods (equivalent to pretending to share paperwork, mimicking file-sharing notifications)
• Message context and supply patterns (equivalent to self-addressed emails, BCC utilization, mismatched sender/recipient habits)

These alerts are largely unaffected by whether or not the payload was written by a human or an LLM. In reality, AI-generated obfuscation usually introduces artificial artifacts, like verbose naming, redundant logic, or unnatural encoding schemes, that may turn into new detection alerts themselves.

Regardless of using AI to obfuscate the SVG payload, this marketing campaign was blocked by Microsoft Defender for Workplace 365’s detection system by a mix of infrastructure evaluation, behavioral indicators, and message context, none of which had been impacted by way of AI. Indicators used to detect this marketing campaign included the next:

• Use of self-addressed electronic mail with BCCed recipients – This tactic is usually used to aim to bypass fundamental electronic mail heuristics and conceal the true recipient listing.
• Suspicious file sort/identify – SVG recordsdata, usually, have been an rising payload utilized in phishing assaults and the attachments on this marketing campaign had been named to resemble a PDF, which is atypical for authentic doc sharing.
• Redirect to malicious infrastructure – The SVG payload redirected to a website that had beforehand been recognized as being linked to phishing content material.
• Common use of code obfuscation – Whereas the SVG file contained novel obfuscation ways that hadn’t been seen earlier than, the presence of obfuscation alone was an indicator of probably malicious intent.
• Suspicious community habits – Automated evaluation of the phishing website indicated that it employed session monitoring and browser fingerprinting, which can be utilized to selectively serve content material primarily based on geography or surroundings, a habits utilized by some phishing actors.

Suggestions

Whereas this marketing campaign was restricted in scope and successfully blocked, related strategies are more and more being leveraged by a spread of risk actors. Sharing our findings equips organizations to determine and mitigate these rising threats, whatever the particular risk actor behind them. Microsoft Menace Intelligence recommends the next mitigations, that are efficient towards a spread of phishing threats, together with people who could use AI-generated code.

• Evaluate our beneficial settings for Trade On-line Safety and Microsoft Defender for Workplace 365.
• Configure Microsoft Defender for Workplace 365 to recheck hyperlinks on click on. Protected Hyperlinks supplies URL scanning and rewriting of inbound electronic mail messages in mail move, and time-of-click verification of URLs and hyperlinks in electronic mail messages, different Microsoft 365 purposes equivalent to Groups, and different places equivalent to SharePoint On-line. Protected Hyperlinks scanning happens along with the common anti-spam and anti-malware safety in inbound electronic mail messages in Microsoft Trade On-line Safety (EOP). Protected Hyperlinks scanning may help defend your group from malicious hyperlinks utilized in phishing and different assaults.
• Activate Zero-hour auto purge (ZAP) in Defender for Workplace 365 to quarantine despatched mail in response to newly-acquired risk intelligence and retroactively neutralize malicious phishing, spam, or malware messages which have already been delivered to mailboxes.
• Encourage customers to make use of Microsoft Edge and different internet browsers that help Microsoft Defender SmartScreen, which identifies and blocks malicious web sites, together with phishing websites, rip-off websites, and websites that host malware.
• Activate cloud-delivered safety in Microsoft Defender Antivirus or the equal to your antivirus product to cowl quickly evolving assault instruments and strategies. Cloud-based machine studying protections block a majority of latest and unknown variants
• Configure Microsoft Entra with elevated safety.
• Pilot and deploy phishing-resistant authentication strategies for customers.
• Implement Entra ID Conditional Entry authentication energy to require phishing-resistant authentication for workers and exterior customers for vital apps.

Microsoft Defender XDR detections

Microsoft Defender XDR prospects can discuss with the listing of relevant detections beneath. Microsoft Defender XDR coordinates detection, prevention, investigation, and response throughout endpoints, identities, electronic mail, apps to supply built-in safety towards assaults just like the risk mentioned on this weblog.

Clients with provisioned entry may use Microsoft Safety Copilot in Microsoft Defender to research and reply to incidents, hunt for threats, and defend their group with related risk intelligence.

Microsoft Safety Copilot

Safety Copilot prospects can use the standalone expertise to create their very own prompts or run the next prebuilt promptbooks to automate incident response or investigation duties associated to this risk:

• Incident investigation
• Microsoft Person evaluation
• Menace actor profile
• Menace Intelligence 360 report primarily based on MDTI article
• Vulnerability influence evaluation

Notice that some promptbooks require entry to plugins for Microsoft merchandise equivalent to Microsoft Defender XDR or Microsoft Sentinel.

Looking queries

Microsoft Sentinel

Microsoft Sentinel prospects can use the TI Mapping analytics (a collection of analytics all prefixed with ‘TI map’) to mechanically match the malicious area indicators talked about on this weblog publish with information of their workspace. If the TI Map analytics will not be at present deployed, prospects can set up the Menace Intelligence answer from the Microsoft Sentinel Content material Hub to have the analytics rule deployed of their Sentinel workspace.

Under are the queries utilizing Sentinel Superior Safety Data Mannequin (ASIM) capabilities to hunt threats throughout each Microsoft first celebration and third-party information sources. ASIM additionally helps deploying parsers to particular workspaces from GitHub utilizing an ARM template or manually.

Detect community area indicators of compromise utilizing ASIM

The next question checks IP addresses and area IOCs throughout information sources supported by ASIM community session parser:

 //Area list- _Im_NetworkSession let lookback = 30d; let ioc_ip_addr = dynamic([]); let ioc_domains = dynamic(["kmnl.cpfcenters.de"]); _Im_NetworkSession(starttime=todatetime(in the past(lookback)), endtime=now()) | the place DstDomain has_any (ioc_domains) | summarize imNWS_mintime=min(TimeGenerated), imNWS_maxtime=max(TimeGenerated),   EventCount=depend() by SrcIpAddr, DstIpAddr, DstDomain, Dvc, EventProduct, EventVendor 

Detect area and URL indicators of compromise utilizing ASIM

The next question checks area and URL IOCs throughout information sources supported by ASIM internet session parser:

 // Area listing - _Im_WebSession let ioc_domains = dynamic(["kmnl.cpfcenters.de”]);   _Im_WebSession (url_has_any = ioc_domains) 

Indicators of compromise

Study extra

For the most recent safety analysis from the Microsoft Menace Intelligence group, try the Microsoft Menace Intelligence Weblog.

To get notified about new publications and to affix discussions on social media, observe us on LinkedIn, X (previously Twitter), and Bluesky.

To listen to tales and insights from the Microsoft Menace Intelligence group in regards to the ever-evolving risk panorama, take heed to the Microsoft Menace Intelligence podcast.