[dnsdist] dnsdist[]: While reading a TCP question: accepting new connection on socket: Too many open files

[Jacob Bunk Nielsen]

Fredrik Pettai via dnsdist <dnsdist at mailman.powerdns.com> writes:

> One dnsdist instance recently got overloaded, and the message (subject + below) appeared a lot in the logs:
>
> 	“dnsdist[]: While reading a TCP question: accepting new connection on socket: Too many open files"
>
> Is this only related to too much DNS-traffic over TCP, or could lots
> of DNS traffic over UDP also potentially lead to slowdown/locking
> issues for dnsdist TCP handling?

It's not just TCP, but also UDP. There's a good chance that you got hit
by a DDOS attack and those tend to often be UDP based because it's much
harder to spoof the source address of a TCP connection.

> I’ve increased the amount of addLocal() + newServer() workers to be able to handle more traffic.

This probably wasn't your problem since you managed to run out of
available file descriptors just fine with the current number of
addLocal() and newServer().

> Dnsdist currently gets 16k fd’s (via systemctl's dnsdist.service configuration)
>
> # grep -E '^Max open files' /proc/$(pidof dnsdist)/limits
> Max open files            16384                16384                files
>
> Would it be okay to increase this 4x or so?

That depends on your specific hardware, but probably, yes.

> What other things could one do to increase dnsdist ability to handle large bursts of DNS traffic better?

Have you checked out dynamic blocks? If not, have a look at https://dnsdist.org/guides/dynblocks.html

Best regards,

Jacob