There was a time when the web was open. Quite literally—communications taking place on the early web were not masked in any significant fashion. This meant that it was fairly trivial for a bad actor to intercept and read the data being transmitted between networked devices.

This was especially troublesome when it came to sensitive data, such as password authentication or credit card transactions. To address the risks of transmitting such data over the web, traffic encryption was invented, ushering in an era of protected communication.

Today more than half of all websites use HTTPS. In fact, according to data obtained from Cisco Cognitive Intelligence, the cloud-based machine learning engine behind Stealthwatch—Cisco’s network traffic analysis solution—82 percent of HTTP/HTTPS traffic is now encrypted.

The adoption of encrypted traffic has been a boon for security and privacy. By leveraging it, users can trust that sensitive transactions and communications are more secure. The downside to this increase in encrypted traffic is that it’s harder to separate the good from the bad. As adoption of encrypted traffic has grown, masking what’s being sent back and forth, it’s become easier for bad actors to hide their malicious activity in such traffic.

A brief history of encrypted traffic

The concerns around security and privacy in web traffic originally led Netscape to introduce the Secure Sockets Layer (SSL) protocol in 1995. After a few releases, the Internet Engineering Task Force (EITF) took over the protocol, which released future updates under then name “Transport Layer Security” (TLS). While the term SSL is often used informally to refer to both today, the SSL protocol has been depreciated and replaced by TLS.

TLS protocol works directly with existing protocols and encrypts the traffic. This is where protocols like HTTPS come from— the hypertext transfer protocol (HTTP) is transmitted over SSL/TLS. While HTTPS is by far the most common protocol secured by TLS, other popular protocols, such as SFTP and SMTPS can take advantage of the protocol. Even lower-level protocols like TCP and UDP can use TLS.

Threat actors follow suit

Attackers go to great pains to get their threats onto systems and networks. The last thing they want after successfully penetrating an organization is to have their traffic picked up by network-monitoring tools. Many threats are now encrypting their traffic to prevent this from happening.

Where standard network monitoring tools might be able to quickly identify and block unencrypted traffic in the past, TLS provides a mask for the communication threats utilize to operate. In fact, according to data taken from Cognitive Intelligence, 63 percent of all threat incidents discovered by Stealthwatch were discovered in encrypted traffic.

In terms of malicious functionality, there are a number of ways that threats use encryption. From command-and-control (C2) communications, to backdoors, to exfiltrating data, attackers consistently use encryption to hide their malicious traffic.

Botnets

By definition, a botnet is a group of Internet-connected, compromised systems. Generally, the systems in a botnet are connected in a client-server or a peer-to-peer configuration. Either way, the malicious actors usually leverage a C2 system to facilitate the passing of instructions to the compromised systems.

Common botnets such as Sality, Necurs, and Gamarue/Andromeda have all leveraged encryption in their C2 communications to remain hidden. The malicious activity carried out by botnets include downloading additional malicious payloads, spread to other systems, perform distributed-denial-of-service (DDoS) attacks, send spam, and other malicious activities.

Botnets mask C2 traffic with encryption.

RATs

The core purpose of a RAT is to allow an attacker to monitor and control a system remotely. Once a RAT manages to implant itself into a system, it needs to phone home for further instructions. RATs require regular or semi-regular connections to the internet, and often use a C2 infrastructure to perform their malicious activities.

RATs often attempt take administrative control of a computer and/or steal information from it, ranging from passwords, to screenshots, to browser histories. It then sends the stolen data back to the attacker.

Most of today’s RATs use encryption in order to mask what is being sent back and forth. Some examples include Orcus RAT, RevengeRat, and some variants of Gh0st RAT.

RATs use encryption when controlling a computer.

Cryptomining

Cryptocurrency miners establish a TCP connection between the computer it’s running on and a server. In this connection, the computer is regularly receiving work from the server, processing it, then sending it back to the server. Maintaining these connections is critical for cryptomining. Without it the computer would not be able to verify its work.

Given the length of these connections, their importance, and the chance that they can be identified, malicious cryptomining operations often ensure these connections are encrypted.

It’s worth noting that encryption here can apply to any type of cryptomining, both deliberate and malicious in nature. As we covered in our previous Threat of the Month entry on malicious cryptomining, the real difference between these two types of mining is consent.

Miners transfer work back and forth to a server.

Banking trojans

In order for a banking trojan to operate, it has to monitor web traffic on a compromised computer. To do that, some banking trojans siphon web traffic through a malicious proxy or exfiltrate data to a C2 server.

To keep this traffic from being discovered, some banking trojans have taken to encrypting this traffic. For instance, the banking trojan IcedID uses SSL/TLS to send stolen data. Another banking trojan called Vawtrak masks its POST data traffic by using a special encoding scheme that makes it harder to decrypt and identify.

Banking trojans encrypt the data they’re exfiltrating.

Ransomware

The best-known use of encryption in ransomware is obviously when it takes personal files hostage by encrypting them. However, ransomware threats often use encryption in their network communication as well. In particular, some ransomware families encrypt the distribution of decryption keys.

How to spot malicious encrypted traffic

One way to catch malicious encrypted traffic is through a technique called traffic fingerprinting. To leverage this technique, monitor the encrypted packets traveling across your network and look for patterns that match known malicious activity. For instance, the connection to a well-known C2 server can have a distinct pattern, or fingerprint. The same applies to cryptomining traffic or well-known banking trojans.

However, this doesn’t catch all malicious encrypted traffic, since bad actors can simply insert random or dummy packets into their traffic to mask the expected fingerprint. To identify malicious traffic in these cases, other detection techniques are required to identify the traffic, such as machine learning algorithms that can identify more complicated malicious connections. Threats may still manage to evade some machine learning detection methods, so implementing a layered approach, covering a wide variety of techniques, is recommended.

In addition, consider the following:

• Stealthwatch includes Encrypted Traffic Analytics. This technology collects network traffic and uses machine learning and behavioral modeling to detect a wide range of malicious encrypted traffic, without any decryption.
• The DNS protection technologies included in Cisco Umbrella can prevent connections to malicious domains, stopping threats before they’re even able to establish an encrypted connection.
• An effective endpoint protection solution, such as AMP for Endpoints, can also go a long way towards stopping a threat before it starts.

Enjoyed reading this Threat of the Month? Subscribe to the Threat of the Month blog series and get alerted when new blogs are published.