Security Experts:

Do You Know What your DNS Resolver is Doing Right Now?

It could be helping let your most important secrets right out the door and you’d never know it…

Have you ever heard of the domain name ziyouforever.com? As an enlightening experiment, get a hold of one of your recursive DNS server logs and see if there are machines on your network looking up hostnames on that domain right now. You might also check your web server or firewall logs to see if someone from the Internet is trying to resolve a hostname on that domain to locations on your network. Are you even able to get a hold of those logs? Are you even keeping DNS logs? If you said “no” or “not easily” you will need to get those capabilities ASAP—otherwise you’re blind to one of the most insidious ways for hackers to interact with machines on your network unmolested and undetected.

Monitor DNS Requests Network SecurityDon’t worry if you found that domain in your logs somewhere—at least not too much. It is widely believed that this domain name is actually part of a large network of domains and hostnames that provide “DNS tunneling” services to dissidents and other information seekers who live in countries that restrict their citizens Internet access. However, this “positive” use of the DNS to do something it was never intended for—surfing the Internet and sending messages from behind a cordoned off network—is a great example of why most enterprises are wide-open to real attacks via this little-known vector. How much of your secret information is going right out the DNS door right now? How would you even know where to look or how to detect this kind of activity in the future?

In this follow-up to my series on the DNSChanger malware situation, I will take a deeper look into the next generation of DNS manipulating malware. DNSChanger is child’s play compared to malware that actually synthesizes DNS requests and responses to communicate with its command and control server or exfiltrate your organization’s data. While we’ve only seen a handful of these in the wild to date, it is inevitable that we shall see many more, especially with increasing success in combating bots on more traditional protocols like IRC and HTTP. Further, there is little to no protection against them on most networks, while recent upgrades to the DNS infrastructure to support DNSSEC, IPv6 and other new DNS additions make this type of attack far more practical than it used to be.

Two ways to use the DNS to rob you blind

This manipulation of the DNS can be done with techniques called “DNS signaling” and “DNS tunneling.” So how do they work?

With DNS signaling, malware on an infected computer sends a request for a DNS TXT (text) record or generates a request for an encoded hostname (a.k.a. subdomain) that it prepends to a Command and Control (C&C) domain. The C&C domain may be hard-coded into the malware or made from a domain generation algorithm (DGA). For example, it could look something like this:

49203498-randomstuff-12aklsdf23a9809u.evil-c2-domain.com.

The malware can send status updates, requests for commands or even data being exfiltrated from the victim’s network that will appear as such random-looking hostnames in a query. This “evil” DNS request is happily routed by the enterprise’s DNS resolver to the authoritative name server for evil-c2-domain.com.

The authoritative name server for evil-c2-domain.com is controlled by the bad guy, and is programmed to interpret the request for the encoded hostname and respond appropriately. It will receive any stolen data and provide a DNS response code or TXT record that tells the malware what to do next. Of course, the enterprise’s own DNS resolver sits in the middle and facilitates this communication.

To make matters worse, employees (or malware on their computers) can often configure their machines to use an alternative recursive DNS server like Google DNS or OpenDNS, so there’s no chance to even use the corporate resolver to spot this kind of activity. Have we seen this in the wild? Unfortunately yes, and a prime example is called feederbot.

DNS tunneling takes this even further by using DNS queries to encode other protocols (like http, ftp, smtp, or others) over a DNS session. This is done via the same basic techniques you use to encode other transmissions, say like you do to set up a VPN network. Same principles, just a different protocol and one that most people weren’t aware you can do this with. Oh, and you can set up a VPN using this technique too! So by simply adding DNS tunneling as a transmission option, malware authors can run standard software (say rsync, FTP, or RAT tools) in conjunction with their bots. This takes a huge amount of effort off their plates because they don’t have to design ways to avoid IDS/IPS signatures and other behavior-based network monitoring, including even deep packet inspection (DPI). Why watch port 53 (DNS services) anyway? It’s only full of harmless DNS queries you know…

Since you cannot truly block DNS resolution from occurring without taking someone’s computer off the Internet, DNS tunneling is a nearly perfect attack tool. Botnet command and control traffic easily flows back and forth without tripping classic IDS/IPS solutions that look at other ports and protocols. Data isn’t being offloaded by using easily detected file transfers over known risky paths; rather it is being brazenly transmitted as seemingly harmless DNS requests towards legitimate looking DNS servers out there in the wide world. With the advent of IPv6, DNSSEC, DKIM, and other extensions to the basic DNS protocol, DNS traffic has gotten much larger, now regularly flowing over TCP instead of UDP. This means you can use DNS tunneling techniques more easily and efficiently today, since “big packets” are now normal in DNS query streams and networking gear has conveniently been upgraded to support them.

Defense against the dark arts

So is there a magic formula for combating malware and hackers that use DNS traffic to hide their evil deeds? Well on that front there’s some good news. If you’ve read my prior Security Week articles on DNSChanger or DNS Firewalls, you’ve got a good handle on what can be done to prevent, detect, and mitigate these activities by installing DNS firewalls and controlling your port 53 traffic. Beyond those measures, you’re going to have to do some anomaly detection on your DNS traffic, but that’s very doable too.

Step 1: Set up DNS firewalls on all your recursive DNS servers

To review, a DNS firewall is another way of saying a secure DNS resolver. It prevents enterprise employee and system connections to known malicious Internet locations, and can provide immediate feedback to enterprise security teams about potential compromises like botnets and APTs on their networks. All it takes to create one is a list of malicious domains or hostnames, which can then be easily added to the configuration of the DNS resolver server to automatically block access to those locations. This is accomplished by simply loading updated lists of malicious hosts into the resolver along with the destination (or lack thereof) you wish to force users to go to instead of the malicious one. By utilizing this secure DNS gateway, an enterprise can ensure its employees and IT systems are not routed to destinations that could jeopardize communications, proprietary information, customers’ private data and more.

In the case of DNS signaling or tunneling, the list of names to block or redirect needs to include the hostnames and domains that have been identified as part of such a system. This should come standard with any good DNS firewall data feed or system. You can then add other hostnames you detect yourself to your servers as well.

Step 2: Reroute all DNS traffic at your firewalls/gateways

As I covered in my previous SecurityWeek column, control over DNS resolution is of vital importance to make a DNS firewall actually useful. This is as simple as redirecting all port 53 traffic (both UDP and TCP) to your own designated recursive DNS servers. If you’re not familiar with this technique, think about how hotels or network access points in airports often redirect your requests in order to make sure you have to log into their services to utilize the Internet—same concept. Thus a user or virus can attempt to use different DNS servers, but data packets involved in all DNS lookups get redirected to organizationally-approved servers instead. This means your DNS firewall will block known bad names. It also means you can log all your DNS traffic, which leads to the next measure.

Step 3: Log and analyze all DNS traffic, looking for anomalies

While a DNS firewall will block known bad hostnames, you never know when you may be one of the first targets of a new attack, or worse, a targeted attack that only affects you or an industry niche. In such cases, public and even most private data feeds and DNS firewall products aren’t going to know about the domains being used. Thus you need to carefully examine your DNS traffic on a regular basis, looking for the telltale signs of DNS signaling and tunneling from your network.

Monitoring DNS traffic can be done in a couple ways. First, logging can be enabled for your recursive nameservers, and a process set up to regularly download/extract the logs from the server put into place. This has the advantage of not adding anything new to the network, but enabling logging to the necessary level might affect server performance on busy networks, and DNS log files are sometimes difficult to parse. If those downsides are significant, a great alternative is to set up a passive DNS (PDNS) sensor on the network, next to your DNS resolvers. A PDNS sensor can watch traffic in real time and output easily parsed data to other analysis tools. This makes for much easier processing, but does require a new box (or many) on the network.

Once you have your DNS resolution data, how do you look for anomalies that are dangerous without overwhelming your staff? Well, this is often just as much art as science, just as perusing standard firewall logs and other threat indicators from your networks can be. You have to have tools, expertise, services, or some combination of those already in place for handling firewall log analysis that can then be used for other security systems' log analysis, or know you can bring in the resources to do it for you. Here are some rules of thumb to help with simple triage:

1) Look for long, randomized hostname queries sent to the same or small subset of domains. This one is a no-brainer, and you can start by just looking for extremely long hostnames being resolved.

2) Look for TXT requests and of course TXT responses that contain large amounts of gibberish.

3) Watch for “beaconing” behavior—the same hostnames (that aren’t in the Alexa list) being pinged regularly.

4) Look for domains in oddball TLDs and subdomain resellers—especially ones known to harbor many malicious registrations like .su, .ru, and .tk.—particularly if you don’t do business in those countries normally.

5) Look for patterns of recursive DNS usage during odd hours. Most machines don’t need to look up new domain names when people aren’t surfing or doing other “randomizing” types of activities that humans do, so lots of non-cached hostnames shouldn’t be hit late at night.

6) Examine the history and hosting of the nameservers being used by domains queried and the reputation those nameservers have. If they are located in unusual countries, are being supported by fast flux, are brand new domains, or are in hosting facilities with abuse problems, the traffic needs more scrutiny.

Note that these are some of the same techniques you’d use to look for standard malware (non-signaling/tunneling) reaching out to previously undetected C&C’s, so are good practices anyway.

New protection methods needed to fight today’s attacks

Today’s Internet infrastructure was invented decades ago, in an era of much greater trust, and thus wasn’t hardened against the attacks we are seeing now. Cybercriminals can easily exploit cracks in the Internet’s foundation, including using it in ways it was never intended for. Because it is so ubiquitous and resilient, as well as mostly unguarded, DNS presents a tremendous opportunity for stealthy communications. Organizations must expand their online defenses to protect this vital, and insidiously powerful, communication channel.

Rod Rasmussen co-founded Internet Identity and serves as its lead technology development executive. He is widely recognized as a leading expert on the abuse of the domain name system. Rasmussen is co-chair of the Anti-Phishing Working Group’s Internet Policy Committee and serves as the APWG’s Industry Liaison, representing and speaking on behalf of the organization at events around the world and works closely with ICANN. He also is a member of the Online Trust Alliance’s (OTA) Steering Committee and an active member of the Digital PhishNet and is an active participant in the Messaging Anti-Abuse Working Group. Rasmussen earned an MBA from the Haas School of Business at UC-Berkeley and holds two bachelor’s degrees, in Economics and Computer Science, from the University of Rochester.