[Pdns-users] Mitigating / stopping recent Denial of Service Attacks

[okTurtles]

Hi list!

# Background

I am actively developing a NodeJS blockchain-based DNS resolver called DNSChain (which I've recently combined to work with a locally running instance of PowerDNS recursor).

While running DNSChain on its own I noticed that my logs suddenly started showing a large volume of timeouts happening in what was clearly some sort of attack. I temporarily enabled logging of IP addresses (for timeouts only) to see if I could figure out what was going on.

# Attack Details

Here you can see an anonymized excerpt from that log. IPs have been changed but the mapping is consistent to show the pattern:

https://gist.github.com/taoeffect/a74d3e302b06965036bf

Some notes from that:

The first 3 octets were anonymized while the last octet was left intact.
It's clear there was mainly one target in that attack, what I've labelled the "666.666.666" block
They are using randomized subdomains to avoid cached queries
They include other IPs among those attacked (probably random) to try and disguise the attack, but the main focus is on 3 IPs in the "devil's block" ;-).

# PDNS 3.5.3 log

I decided to pair up DNSChain with PowerDNS recursor thinking that maybe since it has been in development for a long time now that it more effectively deal with this problem, however, it seems that it's only marginally doing so.

Here's the log:

May 31 14:58:38 pdns_recursor[666]: stats: 526156 questions, 33494 cache entries, 7006 negative entries, 2% cache hits
May 31 14:58:38 pdns_recursor[666]: stats: throttle map: 1369, ns speeds: 6
May 31 14:58:38 pdns_recursor[666]: stats: outpacket/query ratio 126%, 0% throttled, 0 no-delegation drops
May 31 14:58:38 pdns_recursor[666]: stats: 25 outgoing tcp connections, 19 queries running, 561421 outgoing timeouts
May 31 14:58:38 pdns_recursor[666]: stats: 253422 packet cache entries, 25% packet cache hits
May 31 14:58:38 pdns_recursor[666]: stats: 11 qps (average over 1956 seconds)
May 31 15:01:09 pdns_recursor[666]: [411B blob data]
May 31 15:01:14 pdns_recursor[666]: [411B blob data]
May 31 15:01:19 pdns_recursor[666]: [411B blob data]
May 31 15:05:52 pdns_recursor[666]: [411B blob data]
May 31 15:05:56 pdns_recursor[666]: [411B blob data]
May 31 15:06:01 pdns_recursor[666]: [411B blob data]
May 31 15:10:36 pdns_recursor[666]: [411B blob data]
May 31 15:10:41 pdns_recursor[666]: [411B blob data]

Notice:

0% throttled
2% cache hits
WTF is "[411B blob data]"? That's only started happened recently after it's been running for a while. It did not show this for the first hour of running.

# Questions

I'm currently running PowerDNS recursor 3.5.3-1 on Debian, because that is the most recent version available.

## Question 1

I saw that RC1 of 3.6 was released somewhere: http://mailman.powerdns.com/pipermail/pdns-users/2014-May/010657.html

But it's not available for Debian. How to install for Debian "correctly" when I've already installed the Debian package?

## Question 2

Does 3.6 mitigate this attack by itself?

## Question 3

See WTF is "[411B blob data]"? above.

# Solution

The IPs obviously cannot be relied on for much of anything. Instead we can rely on the following:

The fact that we're getting thousands of timeouts for a particular domain
The fact that a single domain is having hundreds/thousands of its subdomains being enumerated

Therefore this can be fairly trivially detected.

I'd prefer for PDNS recursor to do the detecting and mitigating itself, but I want a solution ASAP and don't want to wait, so if it doesn't already have an answer, then maybe a simple Lua script can be made to mitigate this.

The algorithm can be as simple as:

1. Is a domain having its subdomains enumerated recently? (100+ subdomains within say the past 30 minutes)
2. (Optional) are they returning timeouts too?

If so, silently drop the packets, give no response.

Thoughts?

Can this be done with 3.5.3? What would the Lua code look like? (Sorry, I'm new to PDNS).

Thanks!

Greg Slepak

P.S. This email was not proof-read because I'm short on time. Sorry for inevitable typos and grammar mistakes!

--
Please do not email me anything that you are not comfortable also sharing with the NSA.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20140531/3c72799a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20140531/3c72799a/attachment.sig>