Analysis Notes on CrowdStrike’s Library Loading and Rapid Upgrade Mechanism

August 03, 2024 By Antiy Security Response -

The original report is in Chinese, and this version is an AI-translated edition.

In response to a large Window host blue screen event caused by CrowdStrike, Antiy Cloud Security Center, Antiy CERT and Antiy Attack and Defense Laboratory released a long analysis report on July 21, “Technical Analysis of CrowdStrike Resulting in Large-scale System Crash: A Meditation on” Falcon Feather “(hereinafter referred to as” Falcon Feather “report). At the same time, it has also promoted the analysis and verification of multiple points, some of which are disclosed in this report.

As for the way to quickly upgrade and delete the problem “channel file,” there is a phenomenon that the problematic C- * 291 * sys file and the latest C- * 291 * 001. sys are allowed to coexist without networking. The problem files are also automatically cleared after the restart. In term of that mechanism, we have put forward two guesses, the first is that when a category in the upgrade system reappears a file with version number 01 (possibly with a timestamp judgment), the category is considered to be reinitialized. Automatically delete old version number files. Second, the handling definition for deleting the corresponding problem file is put in the new file No. 01.

In the first category, there are two guesses: No file with version number 01 can coexist with other versions of files in other categories, and the classification rule is the rule that C- * 291 * should be the main protection type for different scenarios. Document processing should not be carried out directly. The second case is not excluded. the format supports a retention mechanism for self-emergency handling. We’re leaning towards the first. However, our analysis may not be made public in the future. based on the traditional principle in the industry, we can disclose the contents of reverse analysis for the products of other banks, and shall not go beyond the boundary of accident analysis.

– – Antiy Attack and Defense Laboratory

1.Crowdstrike C- * .sys Series Channel File (Rule Upgrade File) Loading Mechanism

1.1 Review of channel file formats

We have analyzed the format of C- *. Sys files in the Falcon Falling Report. It is pointed out that the file that conforms to the relevant file name is actually a CrowdStrike custom format file instead of a Windows PE format file although it is the suffix. “Sys” of the Windows driver file. In that file head structure of this type of file, as shown in the format analysis result in our previous report, the file head content is strongly related to the file name, the program will check the file name according to the file header information, and the file also contains hash check information, For the second check, the method of hash calculation is to call Windows own library function calculation.

Figure 1 C – * .sys Series File Header Format Review

1.2 The channel file verifies how it is loaded

The C- * .sys files are loaded by the CSFalconService.exe process of the CrowdStrike Main Service “CrowdStrike Falcon Sensor Service,” which uses Microsoft’s antivirus product protection service open to antivirus vendors. Start with SERVICE _ LAUNCH _ PROTECTED _ ANTIMALWARE _ LIGHT protection type to realize service protection, and call a series of kernel functions for directory traversal and file reading after service startup to verify the validity of relevant C-00000xxx-00000000-00000xxx files. Policies are then passed to the driver and engine through IPC (inter-process communication) to execute policies.

Use the ZwQueryDirectoryFile function to traverse C:\ Windows\ System32\ drivers\ CrowdStrike directory C-00000xxx-00000000-00000000xxx. Sys file is shown in Figure 2 and Figure 3.  

Figure 2 Traversal of the C-00000xxx-00000000-00000xxx file using the ZwQueryDirectoryFile function (1)

Figure 3 Traversal of the C-00000xxx-00000000-00000xxx file using the ZwQueryDirectoryFile function (2)

Use ZwCreateFile and ZwReadFile to open and read the file content, and parse the file header to verify the file name, as shown in Figure 4 and Figure 5.   

Figure 4 Use ZwCreateFile and ZwReadFile to open and read the file content, parse the file header to verify the file name (1)

Figure 5 Use ZwCreateFile and ZwReadFile to open and read the file content, parse the file header to verify the file name (2)

Call the BCryptFinish Hash function in the Windows encryption primitive library bcrypt.dll to perform hash verification on the file content, which is defined in bcrypt.h, and perform further parsing operations after the verification is passed, as shown in Figure 6. 

Figure 6 Calling the BCryptFinish Hash function to hash verify the contents of the file

1.3 The reason for the blue screen

On the specific analysis of blue screen, Microsoft has given a more detailed analysis. We verify that it is correct.

Figure 7 Reasons for Blue Screen

Microsoft pointed out in the analysis report [FootnoteRef: 1] that since WER (Windows Error Reporting) data only provides a compressed version of the current state, a wider range of disassembly instructions cannot be viewed. By searching for the command feature “45 8B 08,” it is not difficult to locate the key command position of FIG. 8, for further analysis. [1: Https: / / www.microsoft.com / en-us / security / blog / 2024 / 07 / 27 / windows-security-best-practices-for-integrating-and-managing-security-tools /][1] 

Figure 8 Search Instruction Feature “45 8B 08” locates critical instruction locations

As can be seen from FIG. 8, although the program determines whether the r8 register is 0 at the 01400E14E8 offset (avoiding access to the null pointer), it does not further verify the validity of the address pointed to by the r8 register. Then the program accesses the illegal address pointed to by the r8 register at 01400E14ED offset, and finally causes the blue screen to occur. 

1.4 New channel file to remedy blue screen events

In response to the blue screen event, CrowdStrike generated the policy file C-00000291-00000000-00000001 in an emergency manner, and directly deleted the existing C-00000291-*. Sys in the entire upgrade policy to carry out startup processing. There is no C-00000291-00000000-00000001 in the file name of the existing C-00000291-* sys series. the last section of the file name is the library version (number of updates) according to our guess. Judging from the preliminary rules, when the last section of server file returns to reference number 1, it may mean that the rule file of the corresponding category is cleaned based on the deletion policy.

Figure 9 Delete the channel file related to the blue screen time

2.Crowdstrike Agent starts network communication and upgrade mechanism

2.1 Network communication opportunity

For CrowdStrike Falcon Sensor, you can refer back to the “Falcon Falling Fines” report. After the loading phase of the Windows startup kernel and the early phase of the session manager startup phase, CrowdStrike Falcon Sensor establishes a link with the remote server in the background, updates the policy file and returns data. The specific process is to obtain the IP of the server on the cloud through DNS resolution of ts01-gyr-maverick.cloudsink.net and lfodown01-gyr-maverick.cloudsink.net domain names, and establish SSL encrypted communication for policy distribution and information uploading.

Figure 10 CrowdStrike agent starts network communication  

Network communication behavior starts after the Windows kernel loading phase is completed, before the Session Manager startup phase and before the user logs on.

Figure 11 Network communication timing

2.2 Crowdstrike Falcon Sensor Return Address

Except that the above two cloudsink. net subdomains are CrowdStrike Falcon Sensor callback addresses, the callback addresses in different regions are different. the known callback addresses are as follows:

Us-1 environment:

Us-2 environments:

Us-GOV-1 environment:

Us-GOV-2 environments:

Eu-1 environment:

2.3 Network communication channel screening

Crowdstrike Falcon Sensor has built-in network communication channel screening mechanism. on the premise that a global agent is configured in the Windows system, CrowdStrike Falcon Sensor will analyze the domain name to obtain its server IP, and then perform internal logic verification on the system agent. If the internal logic is not satisfied, the Windows Global Agent shall be bypassed directly, and the server shall be connected directly through the Windows network to obtain the policy and upload the data. (In Figure 12, 192.168.43.73 is Windows Global Agent) 

Figure 12 Example of bypassing the Windows system agent and connecting directly to its server through the system network to obtain policies and upload data

The relevant data flow is shown in Figure 13 

Figure 13 Example of bypassing the Windows system agent and directly connecting to its server through the system network to obtain policies and upload data

2.4  Internal certificate verification mechanism

Crowdstrike Falcon Sensor has a built-in certificate verification mechanism, which is independent of the Windows certificate management mechanism and does not trust the Windows credit certificate. When communicating with its server, the server certificate is verified using its internal certificate verification mechanism.

Figure 14 Example of data flow for policy acquisition and upload by directly connecting to its server through the system network, bypassing the Windows system agent

3.Summary

The official website of CrowdStrike gives the following information: “Reboot the host to give it an opportunity to download the diverted channel file.We strongly recommend pushing the host on a wired network (as open to WiFi) Prior to rebooting as the host will acquire internet connectivity reasonably fast via ethernet. Restart the host computer, giving it a chance to download the restored frequency files. We strongly recommend that the host be placed on a wired network (rather than WiFi) before restarting, as the host is much faster to get an internet connection via Ethernet. “From the analysis, CrowdStrike has a mechanism for rapid upgrade, At least after the loading phase of the Windows boot kernel is completed and the session manager is started, there will be an unconditional upgrade mechanism. Crowdstrike may be inclined to think that when the network is fast enough, the mandatory upgrade will be done before the blue screen. However, it is not ruled out that there is a lower level of pre-existing communication mechanism, but we have not been able to reproduce.

/ / BTW:

During the initial event validation, the Antiy Attack and Defense Lab discovered a strange phenomenon in which an earlier version of CrowdStrike was deployed in a public cloud environment, and the blue screen was triggered after the deployment of the problem file C-00000291-00000000-00000xxx. But after its restart, the system can start normally. However, the same verification method is adopted in a single machine, and the screen is repeatedly blue.

We noted that in the discussion of the WeChat group, TK teacher suggested that we should pay attention to the CrowdStrike quick upgrade mechanism, and suggested that we resume the restart of the non-blue screen under the public cloud due to its quick growth mechanism.

Therefore, the relevant analysis work of this article was advanced, and thanks to teacher TK. We are continuing to analyze whether CrowdStrike has a lower-level rapid growth mechanism.

Appendix I: Reference Materials

Windows Security best practices for integrating and managing security tools

Appendix II: Introduction to Antiy Attack and Defense Laboratory

Antiy Attack & Defense Lab, affiliated to Antiy Security Service Center, is responsible for red and blue confrontation, vulnerability excavation and emergency analysis, and enhances the safety capability of Antiy products through security research and operation. Focus on the research of attack and defense techniques and tactics, vulnerability excavation and emergency analysis, output security capability plan, and realize the transformation of attack and defense capability. Currently, the department adopts the totem of the woodpecker from the Antiy Service Center. The woodpecker is known as the “doctor of the forest”, which is very appropriate for the work of the Security Service Center, as it helps to eliminate network pests for customers.

Appendix: About Antiy

Antiy is committed to enhancing the network security defense capabilities of its customers and effectively responding to security threats. Through more than 20 years of independent research and development, Antiy has developed technological leadership in areas such as threat detection engines, advanced threat countermeasures, and large-scale threat automation analysis.

Antiy has developed IEP (Intelligent Endpoint Protection System) security product family for PC, server and other system environments, as well as UWP (Unified Workload Protect) security products for cloud hosts, container and other system environments, providing system security capabilities including endpoint antivirus, endpoint protection (EPP), endpoint detection and response (EDR), and Cloud Workload Protection Platform (CWPP) , etc. Antiy has established a closed-loop product system of threat countermeasures based on its threat intelligence and threat detection capabilities, achieving perception, retardation, blocking and presentation of the advanced threats through products such as the Persistent Threat Detection System (PTD), Persistent Threat Analysis System (PTA), Attack Capture System (ACS), and TDS. For web and business security scenarios, Antiy has launched the PTF Next-generation Web Application and API Protection System (WAAP) and SCS Code Security Detection System to help customers shift their security capabilities to the left in the DevOps process. At the same time, it has developed four major kinds of security service: network attack and defense logic deduction, in-depth threat hunting, security threat inspection, and regular security operations. Through the Threat Confrontation Operation Platform (XDR), multiple security products and services are integrated to effectively support the upgrade of comprehensive threat confrontation capabilities.

Antiy provides comprehensive security solutions for clients with high security requirements, including network and information authorities, military forces, ministries, confidential industries, and critical information infrastructure. Antiy has participated in the security work of major national political and social events since 2005 and has won honors such as the Outstanding Contribution Award and Advanced Security Group. Since 2015, Antiy’s products and services have provided security support for major spaceflight missions including manned spaceflight, lunar exploration, and space station docking, as well as significant missions such as the maiden flight of large aircraft, escort of main force ships, and Antarctic scientific research. We have received several thank-you letters from relevant departments.

Antiy is a core enabler of the global fundamental security supply chain. Nearly a hundred of the world’s leading security and IT enterprises have chosen Antiy as their partner of detection capability. At present, Antiy’s threat detection engine provides security detection capabilities for over 1.3 million network devices and over 3 billion smart terminal devices worldwide, which has become a “national-level” engine. As of now, Antiy has filed 1,877 patents in the field of cybersecurity and obtained 936 patents. It has been awarded the title of National Intellectual Property Advantage Enterprise and the 17th (2015) China Patent Excellence Award.

Antiy is an important enterprise node in China emergency response system and has provided early warning and comprehensive emergency response in major security threats and virus outbreaks such as “Code Red”, “Dvldr”, “Heartbleed”, “Bash Shellcode” and “WannaCry”. Antiy conducts continuous monitoring and in-depth analysis against dozens of advanced cyberspce threat actors (APT groups) such as “Equation”, “White Elephant”, “Lotus” and “Greenspot” and their attack actions, assisting customers to form effective protection when the enemy situation is accurately predicted.