[dnsdist] dnsdist 1.4 and Debian Buster

[Chris]

Hi all,

I have started replacing my dnsdist servers that are running Debian 
Stretch with the updated Debian Buster release. I came across an issue 
where dnsdist is running for a period of time it will stop answering any 
UDP queries until it is restarted. Sending queries either externally to 
the server or from the server itself results in a timeout.

The server is running on bare metal and is receiving around 900-1200 
queries/second. There is multiple dnsdist instances running for 
different purposes, there is 8 instances total. Each instance is bound 
to a different set of IP's, no IP's are shared between dnsdist 
instances. For better performance I am using reusePort with 4 listeners 
normally, for this issue I tried removing the additional listeners and 
removing the reusePort setting but it still occured.

With reusePort enabled the issue occurs after a short period of time 
usually between 30 and 40 seconds from the period queries start getting 
sent to it. Without reusePort enabled and a single listen thread the 
issue seems to occur after around 4-5 hours, when it occurs the symptom 
seems the same. Once the issue occurs it doesn't seem to recover itself 
(I waited for around 30 hours) until the dnsdist service is restarted.

The dnsdist console and API/web interface continues to work. I can also 
see dnsdist sending out the health checking queries to the backends and 
it correctly marks them as up or down. TCP queries also work fine.

I had a look through the servers logs but couldn't find anything that 
could be related to this problem in syslog/messages/dmesg. I do raise 
the open file limit and the tasksmax limits applied by systemd.

The configuration I am using has been migrated from the older 1.3 
dnsdist release with a few changes to the configuration to make use of 
newer features as the config was built a few years ago.

I also gave the current master release a go but that also had the same 
problem.

I am wondering if anyone else has had the same issue? If the 
configuration I am using would be useful I can upload a copy of that.

Thanks