Each year, a coalition of leading safety solution providers, including Sophos, participates in a comprehensive cyber attack simulation exercise, featuring various scenarios grounded in real-world threat actor tactics, tools, and techniques.
This analysis provides a practical and transparent evaluation of safety options’ effectiveness, focusing on comprehensive assessments that consider end-to-end attack scenarios comprising initial access, persistence, lateral movement, and impact, with publicly accessible outcomes. Within complex test environments, emulators frequently simulate a diverse array of devices, comprising endpoints, servers, domain-joined machines, and managed clients in real-time.
As Sophos celebrated its fourth anniversary in 2024, we wanted to share insights into our latest evaluation, demonstrating the authenticity of the results.
We’ll delve into the realism of the tooling, nuances within our testing methodology, and explore Sophos’ robust safety and detection capabilities. While we cannot cover every scenario, our discussion will focus on a range of examples, showcasing the thoroughness and precision of the emulations in question.
MITRE selected two risk classes for its 2024 analysis: Ransomware and the Democratic People’s Republic of Korea (DPRK). The perennial issue of cybersecurity threats remains one of the most significant concerns in the industry, persistently evolving over time. The latter’s significance can also be quite pronounced, considering the widespread adoption of digital technologies.
Mitre crafted three plausible scenarios, categorizing them as follows: an attack by a North Korea-affiliated threat actor focused on Mac OS, building upon the tactics employed by groups like ; additionally, attacks perpetrated by affiliates of two notable ransomware gangs ().
DPRK
While the North Korean cyber threat is indeed concerning, the initial assessment seemed simplistic, primarily predicated on an adversary exploiting a vulnerability, installing malware, and compromising credentials. North Korean-affiliated hackers have been observed to orchestrate their cyberattacks in a multi-tiered manner, carefully calibrating each stage of their assault.
Preliminary entry
While initial assumptions centred on a supply chain attack, our investigation ultimately focused on a consumer who intentionally downloaded and ran a malicious Ruby script, as corroborated by our findings confirming a direct path to consumer execution through Ruby. During a realistic supply chain attack, pre-loaded software would likely run the script automatically. Notwithstanding this, the tactic of socially engineered persuasion by DPRK-affiliated hackers to trick users into executing malicious code is both credible and substantial.
In the JumpCloud attack, MITRE’s Ruby script, dubbed “Docker-ize”, disguises itself by downloading and executing a first-stage command-and-control (C2) agent in the form of a Mach-O binary, camouflaging it as a legitimate Docker-related component. While attempting to reverse-engineer genuine JumpCloud samples may seem plausible, it’s crucial to acknowledge that these samples are reportedly not publicly accessible according to our knowledge. As with all MITRE ATT&CK Evaluations, the malware used was custom-built for the evaluation.
Persistence
The C2 agent at its initial stage subsequently downloaded a secondary-stage backdoor, commonly referred to as “StratoFear” in the context of the actual JumpCloud breach. This backdoor ensured persistence in a manner analogous to the genuine attack, leveraging LaunchDaemons () for execution.
MITRE crafted a Ruby script for the Preliminary Entry portion, deliberately mirroring the genuine vulnerability. The backdoor was reinstalled at its original site, with a similarly titled model: ‘Arial’ denoted the authentic version, while ‘PingFang’ referred to the analytical replica.
As in the actual JumpCloud attack, the attacker was stealthy and evasive, expunging all signs of the initial implant data from the compromised system within a remarkably short timeframe. Within the emulation, they accomplished this using a single command, consistent with our verified execution path analysis. Although the exact methodology used by the JumpCloud risk actor remains unclear – potentially noisy and more likely to generate logs due to its indirect API approach – we are unable to confirm or deny this theory since genuine world samples remain inaccessible.
The infamous STRATOFEAR malware, a relic of the Cold War era, employed an unprecedented level of sophistication by utilizing encrypted configuration data, complete with a shell-out command and a fixed, hardcoded password for maximum stealthiness. While a direct API-based approach might have been more discreet, it’s unclear whether the JumpCloud threat actor employed this tactic.
To underscore the importance of scrutinizing security: MITRE leverages non-public domains confined to the test environment, yet inaccessible via public DNS resolution for its C2 infrastructure. Regardless, public IP addresses are resolved nonetheless. Because of this, community visitors seem like authentic C2 exercises; however, the domains should remain unreachable outside the test environment.
Influence
Within the context of the JumpCloud breach, the attackers’ primary objective appears to be exfiltration of sensitive information, including system details, login credentials, and confidential data stored locally. Mitre’s STRATOFEAR backdoor demonstrated unparalleled reliability by autonomously downloading and executing additional modules from its command-and-control (C2) server to perpetuate the exfiltration of stolen data. Like modules downloaded by the actual StratoFEAR, these have been written to a file within the listing, each named with a unique, six-character alphanumeric identifier.
Within the analysis, MITRE’s STRATOFEAR was found to have downloaded a module capable of learning MacOS keychain records.
The situation concluded with the backdoor successfully collecting sensitive information; however, the subsequent analysis failed to reveal any concrete evidence of exfiltration. While some critics might view this as a limitation of the approach – having credentials merely serves a purpose if they can be exploited – we could reasonably conclude that it’s a relatively minor issue. Upon detecting credential theft, as an incident responder, I will focus on the likely impact and associated malicious activity.
Cl0p
The second scenario involved a simulated attack by the Cl0p ransomware gang, a. The attackers’ tactics closely mirrored those used by, employing a downloader, a persistent remote access Trojan (RAT), a subtle process injection technique, and an exploitation of a trusted process – ultimately culminating in the deployment of a ransomware payload.
Preliminary entry
While the overall scenario drew heavily from the 2019 real-world marketing campaign, the initial introduction presented some distinct differences. In 2019, threat actors leveraged a Dynamic Link Library (DLL) to silently deploy and persist a Remote Access Trojan (RAT). While the genuine workplace attack involved exploiting malicious Office documents containing an embedded Dynamic Link Library (DLL), which was dynamically loaded into the Office process, the MITRE scenario entailed a user intentionally running and executing the DLL via.
The DLL had already been updated on the host, having been downloaded via a separate command prompt session initiated after initial remote desktop protocol (RDP) connection. It’s worth noting that the prevalence of preliminary entry methodologies among ransomware groups and other threat actors is particularly notable when purchasing stolen credentials or access via Initial Access Brokers (IABs). In one exceptional instance, however, Cl0p uniquely demonstrated.
While it’s plausible that an attacker could gain immediate remote access to the compromised host, the scenario may also involve the initial loading of DLL tools for further thorough simulation.
Persistence
As part of the 2019 marketing campaign, the MITRE ‘risk actor’ successfully deployed a persistent Remote Access Trojan (RAT) by exploiting the trusted course of action, leveraging a malware-infused `.
The SDBbot utilizes encrypted strings and a critical section controlled by a mutex to safeguard its initialization process. The analogy drawn to the DPRK situation utilises a similarly titled yet distinct title for the mutex (“Mutex” in the real-world assault and “Analysis” in the latter), mimicking the pattern employed by MITRE.
In MITRE’s implementation of SDBbot, the crucial element is a series of identical 16-byte increments ranging from 0 to 15, repeating the same material throughout. Although not entirely secure, this arbitrary sequence of characters is sufficient to conceal references to API names and information fields from simple static analysis techniques, effectively obfuscating them. Mitre utilized a string obfuscation methodology throughout both the Cl0p and LockBit situations, as seen below.
Mitre’s pattern was successfully loaded via a reflective loader, effectively superseding any pre-existing picture reminisces within the dot. Since the Address Space Layout Inconsistency (ASLR) randomizes the location of the “picture” memory, making it more challenging for a debugger or other tools to predict where the RAT will be found in the future. While this approach may aim to bypass conventional security measures, its effectiveness is uncertain. Mitre’s methodology posed a challenge in identifying the execution of the installer, specifically the process of installing the SDBbot loader component. The installer deployed the loader to a specific location, establishing a symbolic link within the directory, which in turn enabled the setting of the IFEO registry key at a level mirroring the folder path, effectively introducing an additional layer of indirection between the dropper and the persisting Remote Access Trojan (RAT).
The introduction of the ‘VerifierDLLs’ approach complicated the execution flow, as the loader function was unexpectedly invoked before the method’s entry point due to premature loading by (). The malware exploited this vulnerability to inject and execute embedded shellcode, subsequently making the memory region writeable by clearing its contents before overwriting it with the SDBbot remote-access Trojan (RAT). The reminiscence permissions were subsequently reset to read-execute (RX) as a means of making the code resemble ‘common’ picture reminiscence – much like a dynamically linked library (DLL) would appear when loaded directly from disk.
Here are the results of our detection technique, which focused on several key indicators: the unusual presence of C2 activity stemming from a specific source, and C2 activity itself being a common trigger for memory scans, as we discussed in our 2023 report. A novel instance of malicious code was uncovered through thorough reminiscence scanning procedures.
The suspicious C2 occasion allowed Sophos Detection to capture the exfiltration patterns, revealing that the exfiltration method involved using SDBbot and transmitting data over the C2 channel.
Influence
Mitre’s implementation of the Clop ransomware pattern, downloaded through SBDbot, was meticulously modelled after. While mirroring actual factors, MITRE’s patterns are employed to scrutinize layouts prevalent in Russia, Georgia, and Azerbaijan – a deliberate effort to avoid singling out specific methodologies used within these regions. The system also utilized a comparable approach for its APIs, aiming to achieve the same objective.
The analysis revealed notable similarities in tactics and methodologies, particularly with regard to handling of shadow volumes and attempts to disrupt multiple businesses on infected devices.
Ransomware attackers often attempt to eliminate shadow volumes, a tactic designed to prevent victims from recovering compromised data by resizing the allocated space for these volatile files and thereby preventing new shadows from being generated? Despite the novelty of the 2019 Cl0p ransomware, it opted for a unique strategy, systematically traversing a predetermined range of drives (C-H) with its hardcoded list. Mitre’s pattern effectively replicated these habits with remarkable precision.
In a typical pattern similar to numerous ransomware variants, Cl0p ransomware operates by targeting an extensive list of organisations, including security firms and entities holding sensitive data, and attempts to breach them via.
Mitre’s pattern leveraged an identical checklist employed by the genuine Cl0p ransomware, mirroring its exact sequence, although excluding safety firms presumably to prevent any test-related disruptions.
The MITRE malware employed AES encryption, consistently prefixing a specific identifier (“MITRE_marker”) to each encrypted record’s contents. This was an identical methodology to the actual malware, marked by quotation marks (” “). Although the 2019 datasets leveraged the API for cryptographic algorithm support, the MITRE model relied on an open-source library widely utilized by many ransomware families today.
LockBit
LockBit, akin to Cl0p, is a notorious and prolific ransomware gang that has been wreaking havoc since its inception. Despite this, cybercriminals pose a significant risk. MITRE’s LockBit scenario featured tactics, techniques, and procedures (TTPs) distinct from those seen in the Cl0p situation, with a notable point being that while ransomware binaries typically exhibit consistent habits throughout attacks due to centralized development and distribution, affiliates may have more flexibility in their approaches, resulting in varying playbooks, TTPs, and indicators of compromise. The training tactics, procedures, and strategies (TTPs) encompassed a range of initial infiltration approaches, such as leveraging ThunderShell and PsExec, as well as various means of circumventing detection.
Preliminary entry
MITRE’s “Risk Actor” initiated the attack by successfully authenticating to a vulnerable externally facing TightVNC service, leveraging previously compromised credentials. Ransomware-as-a-Service (RaaS) operators often gain initial access through compromised companies and stolen credentials obtained from Initial Access Brokers (IABs), as seen in the Cl0p case, for instance.
Upon breaching the system, the intruder promptly initiated a series of reconnaissance commands, characteristic of the initial phases in a ransomware-as-a-service (RaaS) attack. This involved tactics commonly observed at the outset, including:
nltest /dclist:<area>
cmdkey /checklist
web group “Area Admins” /area
web group “Enterprise Admins” /area
web localgroup Directors /area
powershell /c "get-wmiobject Win32_Service |where-object { $_.PathName -notmatch "C:Home windows" -and $_.State -eq "Working"} | select-object title, displayname, state, pathname
These instructions are virtually identical to those noticed throughout.
During the remote session, the suspect’s erratic behavior and consistent utilization of TightVNC served as telltale signs of malicious intent, corroborated by the suspicious IP address and logon records.
Persistence
To maintain persistence in the environment, the attacker deployed a PowerShell remote access shell commonly referred to as ThunderShell. This backup mechanism, designed specifically for LockBit affiliates, ensures continued presence on the system in case the initial infiltration technique fails or becomes compromised. Throughout our monitoring efforts, we’ve detected persistent patterns of community interaction that reveal ‘beaconing’ behaviors, allowing us to identify and flag potentially malicious processes and connections.
The MITRE ‘attacker’ established additional persistence by exploiting the Windows computerized logon registry key. In our experience, this motion’s deviation from expected behavior is minimal; risk actors typically list these keys to estimate plaintext credentials.
Influence
Mitre chose to mimic the custom-built LockBit data extraction tool, commonly employed by ransomware-as-a-service (RaaS) operators in double-extortion schemes, allowing them to secretly transfer sensitive data to an external server before encrypting it.
Mitre’s implementation of Stealbit, similarly dubbed, leveraged the ‘BeingDebugged’ flag within the PEB to detect attached debuggers, while also employing dynamic API resolution through and , storing resolved DLLs as XOR-obfuscated file names. This can be a highly comparable approach to the genuine StealBit malware, exhibiting striking similarities in its functionality and modus operandi.
Following exfiltration, the MITRE-designated ‘risk actor’ exploited a simulated instance of the LockBit malware’s core functionality, encrypting data and autonomously propagating itself throughout the compromised environment.
As with the real-world model, MITRE’s LockBit pattern employed multiple evasive tactics, including a dynamic API decision mechanism that leveraged an in-memory API hashing algorithm to conceal API names from static analysis, and anti-debugging techniques via. Although we documented all the strategies in 2022, it’s worth noting that MITRE’s specific implementation utilized. Although distinct from the original LockBit approach, leveraging a ROR-based hashing mechanism with a seed key yields a comparable outcome, thereby preventing the detection of a recognized indicator of compromise (IOC) that our organization and other providers may have previously identified.
Although Sophos detected this exercise using various tools, it’s essential to acknowledge that since the test was running in monitor-only mode, CryptoGuard did not react to the encryption attempt. During a separate, standalone exercise focused on protection measures, a test involving encrypted data successfully restored the original files to their decrypted state, even in emulation scenarios.
2024 marked the fourth 12 months that Sophos has participated in MITRE’s ATT&CK Evaluations: Enterprise. In line with earlier approaches, the focus on seamless attack sequences and realistic simulations has rendered this analysis an invaluable exercise for evaluating our capacities and personnel from various stakeholders. We strongly endorse MITRE’s commitment to transparency, reflecting our shared value of openness in all aspects of the assessment process.
The veracity and plausibility of eventual outcomes in emulation-based evaluations significantly contribute to their overall value. While acknowledging that MITRE’s assessments may have differed from real-world attacks in some minor respects, primarily due to inherent limitations, the overall similarity to known campaigns and threat actors remained strong.
Distributors’ assessments, when shared among industry peers, yield benefits that extend far beyond just individual profit, also enriching customers and the broader community as a whole. We look forward to continuing our participation in these evaluations, ultimately sharing our insights and research whenever possible.
