EARLYCROW – Detecting APT Malware Command and Control Over HTTPS

Advanced Persistent Threats (APTs) represent a significant challenge for cybersecurity, targeting critical organizations with stealth and precision.

Traditional Network Intrusion Detection Systems (NIDS) often fail to detect these threats due to their sophisticated tactics.

To address this, researchers Almuthanna Alageel and Sergio Maffeis from Imperial College London have introduced EARLYCROW, an innovative approach to identify APT malware command-and-control (C&C) activities over HTTPS.

The APTs employ tools like Remote Access Trojans (RATs), rootkits, spyware, and keyloggers to maintain long-term access to compromised systems.

These tools communicate with C&C servers using seemingly legitimate HTTPS traffic, making detection difficult.

While the security experts noted that the unlike typical botnets, APTs avoid aggressive behaviors such as spam or DDoS attacks, instead focusing on stealthy data exfiltration and reconnaissance.

EARLYCROW’s Approach

EARLYCROW leverages a novel contextual summary technique based on traffic attributes that are indicative of APT behavior.

It introduces a multipurpose network flow format called PairFlow, which captures key behavioral and statistical features of network traffic.

This format enables the detection of malicious HTTPS communications by analyzing patterns and anomalies in encrypted traffic.

EARLYCROW architecture (Source – Arxiv)

Key features of EARLYCROW include:-

1. Contextual Summaries: By summarizing PCAP data into PairFlow format, EARLYCROW identifies suspicious patterns without decrypting HTTPS traffic.
2. Threat Model: The approach is informed by a threat model that highlights tactics, techniques, and procedures (TTPs) commonly used in APT campaigns.
3. High Accuracy: EARLYCROW achieves an impressive macro-average F1-score of 93.02% with a false-positive rate (FPR) of 0.74%.

EARLYCROW’s detection pipeline involves capturing raw network traffic in PCAP format, converting it into PairFlow summaries, and applying machine learning classifiers trained on PairFlow features to identify anomalies indicative of APT command-and-control (C&C) activity.

Below is an example Python snippet for converting PCAP data into PairFlow format:-

from scapy.all import rdpcap def extract_pairflow(pcap_file): packets = rdpcap(pcap_file) flows = [] for pkt in packets: if pkt.haslayer(‘IP’) and pkt.haslayer(‘TCP’): flow = { ‘src_ip’: pkt[’IP’].src, ‘dst_ip’: pkt[’IP’].dst, ‘src_port’: pkt[’TCP’].sport, ‘dst_port’: pkt[’TCP’].dport, ‘protocol’: ‘TCP’, ‘length’: len(pkt) } flows.append(flow) return flows pairflows = extract_pairflow(‘traffic.pcap’) print(pairflows)

The researchers tested EARLYCROW on real-world datasets containing APT traffic, demonstrating its capability to detect previously unseen threats effectively.

By focusing on early-stage C&C communications, EARLYCROW provides a proactive defense mechanism against APT campaigns.

EARLYCROW represents a significant advancement in APT detection by combining contextual analysis with machine learning.

Its ability to detect malicious HTTPS traffic without decryption makes it a valuable tool for organizations facing sophisticated cyber threats.

Further research could explore extending EARLYCROW’s capabilities to other encrypted protocols and integrating it with existing NIDS solutions for comprehensive threat defense.