SUNBURST Backdoor C2 Communication Protocol

SUNBURST Attack Flow

The SUNBURST attack uses the following high-level process:

1. Initial penetration
   The original introduction of malware occurred through a malicious DLL file, which attackers managed to introduce into the SolarWinds update package. This package was delivered to SolarWinds customers, and because it was digitally signed by SolarWinds, it was trusted and deployed in their internal networks.
2. Execution
   After the update is performed, the malicious DLL starts running on endpoints in the target network. Its main features include reconnaissance, the ability to execute files, and running scripts such as PowerShell.
3. Evading detection
   To prevent detection, the malware checks for the existence of specific drivers and processes on the host machine, which are security controls or scanning tools used by researchers to detect threats. If it detects any of these processes, it terminates and does not continue running. In addition, the malware evades detection by:
   1. Attempting to disable security processes by changing registry keys
   2. Evading specific endpoint security measures
   3. Impersonating trusted network entities

6. C2 (command and control)
   The malware is designed to be dormant for two weeks after being deployed. After this waiting period, it communicates with its command and control servers using the SolarWinds application layer protocol. Because this is a legitimate protocol typically used only for application communications, C2 communications did not trigger security alerts.
7. Lateral movement
   Attackers use malware capabilities, in particular PowerShell scripts, to gain access to additional systems in the victim’s network.
8. Exfiltration
   Once attackers gain access to a desired system, they exfiltrate sensitive data, typically using PowerShell email commands, or transmitting compressed files using HTTP requests.

Uncovering the SUNBURST C2 Communication Protocol

The SUNBURST backdoor is not yet fully understood. Spanning almost 3500 lines of code, “obfuscated” with casual naming, trying to evade shallow review, it has many subtleties yet to uncover.

The Cynet research team attempted to gain a better understanding of the command-and-control communication channel, its various stages, and conditions required for execution. 

Our findings explain many of the observations seen so far regarding the SUNBURST backdoor which was implanted in the Orion attack. The main goal of this investigation is to find infected beaconing machines. We will focus mainly on anything that is passed on the wire, while touching upon a few of the evasion techniques employed by the SUNBURST backdoor.

Initial Execution

The backdoor begins its execution in the legitimate SolarWinds.Orion.Core.BusinessLayer.BackgroundInventory.InventoryManager.RefreshInternal function.

Cleverly masqueraded as OrionImprovementBusinessLayer this function call does not draw any attention at first glance.

Furthermore, as it runs, it checks first if it is executed as part of the legitimate “solarwinds.businesslayerhost” process. Following by staying dormant between 12 to 14 days, after first arriving or an update. Afterwards it confirms it is the only instance of the backdoor currently running on the machine using named pipe existence as a synchronization mechanism.

Diving deeper to the communication of the backdoor, we make a distinction between 3 stages in the communication:

• Stage 0: Ends with the internet connectivity check, resolving api.solarwinds.com
• Stage 1: DNS communicating with *.appsync-api.*.avsvmcloud.com
• Stage 2: HTTPS communicating with CNAME from Stage 1

Stage 0

The main function of this stage is OrionImprovementBusinessLayer.DnsHelper.CheckServerConnection. The function receives “api.solarwinds.com” as an input.

All classification of IP is done inside OrionImprovementBusinessLayer.IPAddressesHelper.GetAddressFamily and the most detailed classification is based on a list of ranges called “nList”. The actual logic of this stage can be summarized with the following diagram:

Figure 1: Actual logic behind of stage 0

This logic allows it to stop execution and evade detection on many sandboxes and hostile environments where there is some sort of a “catch-all” domain resolver.

Stage 1 – DNS-Based Communication

The code for this stage resides in the OrionImprovementBusinessLayer.Update function. While we consider the call to UpdateNotification to be part of Stage 0, all the following code, up to the call to HttpHelper.Initialize,is in Stage 1.

Stage 1, Part 1 – FQDN Generation and resolving

In the first part of this stage, a hostname is generated using one of the following 4 functions:

• GetCurrentString
• GetPreviousString
• GetNextStringEx
• GetNextString

At the end of this part, hostName will hold the FQDN to contact and resolve. No matter which method is chosen, the subdomain will be appended with one of the following 4 suffixes generated from the GetStatus function:

• .appsync-api.eu-west-1.avsvmcloud.com
• .appsync-api.us-west-2.avsvmcloud.com
• .appsync-api.us-east-1.avsvmcloud.com
• .appsync-api.us-east-2.avsvmcloud.com

addressFamilyEx, in turn, will hold the resolved IP’s classification mostly based on the mentioned “nList” of ranges.

Stage 1, Part 2 – Acting based on DNS answer for the subdomain

In the second part of this stage, actions will be taken according to the classification of returned IP for this subdomain.

Any IP classification besides AddressFamilyEx.NetBios will ultimately cause failure to move to the next stage and a local config ReportWatcherRetry to be updated, but only if it were set beforehand to any value between 3 and 5.

A classification of AddressFamilyEx.Atm (anything that would have caused stage 0 to fail – meaning IP is a loopback device or in the local network or multicast IP) will cause the backdoor to call the OrionImprovementBusinessLayer.ProcessTracker.SetAutomaticMode function which will try to turn back on services in the OrionImprovementBusinessLayer.svcList list, which were previously disabled by the backdoor in OrionImprovementBusinessLayer.ProcessTracker.SetManualMode. These attempts are to shut down software which might be interfering with clear communication with the C2 DNS server. A list of products attempted to be manipulated can be found in Appendix A.

Decoded nList is shown below for reference:

To move to Stage 2 (HTTPS communication channel) we need 2 conditions:

• appsync-api subdomain IP resolution with ext=true (“ext”/“dnssec”)
  • Also implies NetBios classification
  • Seen as true at the end of line in the screenshot above. Also present in the table below.
  • Without this condition, we will not get NetBios classification when we get CNAME later.

• CNAME response for the appsync-api subdomain (“cname”)
  • Without this condition, the backdoor will not be started

Classification of AddressFamilyEx.NetBios is given to the following ranges:

Furthermore, the last 2 bytes of the IP resolved will have implications on the communication structure and will be discussed at the next stage.

Upon closer inspection of public partial list of passive DNS resolution for subdomains of *.appsync-api.*.avsvmcloud.com among 1460 subdomain, we found only 1 resolution that would truly activate the next stage of the backdoor. This is a testament to the pinpoint precision and OPSEC which the malicious actor was practicing.

Stage 2 – HTTPS-Based Communication

After getting through all the conditions of Stage 0 and Stage 1, we finally arrive to the last stage, Stage 2. This stage starts with the execution of OrionImprovementBusinessLayer.HttpHelper.Initialize function. Here we start to work with a Job-related state machine, in a loop.

Each round in the loop begins with some random delay (IsSynchronized). Then, if a minute passed since last time, some defense evasion checks are implemented (TrackEvent and subsequent ProcessTracker.TrackProcesses). Any intimidating process (from assemblyTimeStamps list or svcList[*].timeStamps lists) will cause this stage to abort and exit.

CreateUploadRequest is responsible for delivering results and errors of previous execution, as well as fetching the next command to execute.

ParseServiceResponse is responsible for parsing the received command from CreateUploadRequest to a Job and arguments.

Finally, ExecuteEngine executes received commands with arguments and prepares any errors and results to be sent back later.

Stage 2 – HTTPs C2 URL

The analysis of this section is based of GetBaseUriImpl as well as relevant contextual information. The URL of the 2nd Stage is largely dependent on the resolved IP of the subdomain at Stage 1. As a rule, the backdoor tries to make these URLs unique, but will give up eventually. Although HTTPS URLs are used, any certificate errors are discarded and connection continues as usual. The URL begins with https://<cname>/ and afterwards there are 3 distinct cases (“pki”, “fonts” and “swip”):

swip/Upload.ashx – last command had big output ( >10,000  bytes after compression )

swip/Events           – last command had small  output ( <= 10,000 bytes after compression )

swip/upd/              – see options in figure below.

Figure 2:  URL Structure for “swip” case, percentage represent probability of each part to appear in the final URL.

Figure  3: URL structure for “fonts” case, no matter if response exists or not.

Figure  4: URL structure for “pki” case, no matter if response exists or not.

Stage 2 – Sending Back Previous Errors and Results

If there is a result (“response”) to be sent back from previous command, it will be concatenated with the error (“err”) and the generated GUID (“customerId”). Afterwards it is compressed and XORed with the checksum. If the response is short, it is further encoded as a JSON object.

Either way, regarding the HTTP method, when there is a result (inData in CreateUploadRequestImpl), a POST method will be used, otherwise a GET method will be used.

Figure  5: First transformation to the result of previous command

Figure  6: Model of result of last command encoded as JSON object

Stage 2 – Command Parsing

The analysis in this part is based on ParseServiceResponse function. First of all, to extract anything meaningful 200 OK status code for the HTTP response required. Parsing of the HTTP response to the above-described HTTP request depends on the case we are working in, which in turn depends on the last number in the IP returned for the subdomain (Stage 1).

If the response is shorter than 4 bytes, no command is attempted to be recovered. For the first part of the parsing (“body”) can be as simple as skipping 48 bytes (in the “fonts” case) or 12 bytes (in the “pki” case). In the remaining “swip” case a more elaborate scheme employed:

• First the following regex is matched, which looks for hex encoded bytes “\{[0-9a-f-]{36}\}”|”[0-9a-f]{32}”|”[0-9a-f]{16}”
• Further cleaned, and hex decoded

After these steps, for all cases the following steps follow (which is almost the opposite of how the previous’ command result was encoded):

Figure 7: Transformation of incoming command

The rest of the command parsing is left as an exercise to the reader.

Stage 2 – Supported Command Execution

The analysis in this part is based on the ExecuteEngine function. Ultimately the back supported the following command:

We have covered many aspects of the SUNBURST backdoor, such as defense evasion mechanisms, but most notably the various stages the backdoor communicates and tries to blend in. 

Future work can be directed towards figuring out all the different subdomain encoding schemes, which in turn will hopefully allow to decode many of the C2 subdomains. We hope the information provided will be helpful to the Cyber Defense Community in their efforts with identifying and remediating any active beacons of this SUNBURST backdoor, from Cozy Bear actors, or whomever it may be.

SUNBURST Protection: How to Minimize the Risk of Future Attacks

The SolarWinds breach still poses a critical threat. To ensure the security of IT environments, organizations need to identify and block all potential points of exposure. Detecting all relevant vulnerabilities can be complex, especially in big IT environments. However, there are several techniques that can help improve your security posture. 

Here are five security measures that can help you detect the SUNBURST backdoor, as well as subsequent activities and other complex threats.

1. Security data lake
   Breaches like the SolarWinds require extensive investigation. To do that, you need to look at log data and analyze past activities. A security data lake can store log data for a long period of time, providing you with the information needed to investigate breaches that occurred long before they were discovered.
2. Visibility
   Security logs are not enough to protect your environment against the SolarWinds breach. For proper coverage, you need to deploy security controls, like EDR, on all network hosts. This should provide you with the visibility needed for quick detection and activity information for logging traffic, user, and device behavior.
3. Asset management
   An organized and updated asset inventory can help you quickly determine whether a breach is relevant to your environment. You should inventory hardware and software components, including software versions, virtual machines, and programs.To ensure efficiency, you can leverage visibility dashboards, which are automatically updated and provide information summaries. You can also set up alerts that notify team members when unexpected changes occur.
4. Proactive threat hunting
   Proactive threat hunting can help you detect threats in a timely manner. A proactive approach might help prevent a long gap between the time of the original breach and the discovery. You can proactively hunt for threats by setting up efficient data analysis and anomaly detection techniques, using manual jobs or automated tools.

5. Connecting security telemetry
   Dispersed solutions deployed across hybrid environments may lead to siloed threat detection. When data sources are not interconnected, your security tools can miss advanced threats and attacks that move laterally across the network. You can solve this by interconnecting and correlating security telemetry with XDR solutions. This can help you reduce blind spots and quickly detect threats across the entire IT stack.

Cynet Protection against SUNBURST backdoor

Rest assured: Cynet 360 customers are fully protected against any attempts of abusing the SUNBURST backdoor implanted in the Orion attack. Customers who utilize the SolarWinds Orion software have been notified by our CyOps team.

Cynet XDR has released relevant detections for the compromised software and the

malicious backdoor, based on multiple detection vectors. We will continue to add additional

IOCs as new ones emerge.

Additional security endpoint protection recommendations:

• Isolate the vulnerable endpoint using the Cynet Dashboard, and ready for triage.
• Reset any passwords used in the SolarWinds platform, treat them as compromised
• passwords.
• Enable all detection and remediation mechanisms at Cynet 360 UI.

Appendix A – List of services attempted to be shut down and reset later

Thanks to the Hashcat Team for cracking the hashes of the malware.

Below are listed service names which are also keys that can be found in the registry at HKLM\SYSTEM\CurrentControlSet\services and the corresponding vendor.

• Windows Defender
  • 917638920165491138 – windefend
• Windows Defender ATP
  • 16335643316870329598 – sense
• Carbon Black
  • 11385275378891906608 – carbonblack
  • 13693525876560827283 – carbonblackk
  • 17849680105131524334 – cbcomms
  • 18246404330670877335 – cbstream
• Crowdstrike
  • 11771945869106552231 – csagent
  • 9234894663364701749 – csdevicecontrol
  • 8698326794961817906 – csfalconservice
• FireEye
  • 15695338751700748390 – xagt
  • 9384605490088500348 – fe_avk
  • 6274014997237900919 – fekern
  • 15092207615430402812 – feelam
  • 3320767229281015341 – fewscservice
• ESET
  • 15587050164583443069 – eamonm
  • 9559632696372799208 – eelam
  • 4931721628717906635 – ehdrv
  • 3200333496547938354 – ekrn
  • 2589926981877829912 – ekrnepfw
  • 17997967489723066537 – epfwwfp
  • 14079676299181301772 – ekbdflt
  • 17939405613729073960 – epfw
• F-Secure
  • 17624147599670377042 – f-secure gatekeeper handler starter
  • 16066651430762394116 – f-secure network request broker
  • 13655261125244647696 – f-secure webui daemon
  • 12445177985737237804 – fsaua
  • 3421213182954201407 – fsma
  • 14243671177281069512 – fsorspclient
  • 16112751343173365533 – f-secure gatekeeper
  • 3425260965299690882 – f-secure hips
  • 9333057603143916814 – fsbts
  • 3413886037471417852 – fsni
  • 7315838824213522000 – fsvista
  • 13783346438774742614 – f-secure filter
  • 2380224015317016190 – f-secure recognizer
  • 3413052607651207697 – fses
  • 3407972863931386250 – fsfw
  • 10393903804869831898 – fsdfw
  • 12445232961318634374 – fsaus
  • 3421197789791424393 – fsms
  • 541172992193764396 – fsdevcon