[Pdns-users] pdns-recursor metrics review and tuning advice request

[Otto Moerbeek]

On Fri, Apr 18, 2025 at 08:28:48AM -0400, Scott Crace via Pdns-users wrote:

Hi,

Please include your config. That said:

You seem to have pretty low cache hit ratio, a high number of outgoing
queries. How is your cache configged?

Also some throttling is going on. I suspect rec has trouble contacting
one or more auths or forwarders. The throttling tables can be viewed
using

	rec_control dump-throttlemap -
	rec_control dump-failedservers -

Also, what happens *during* the trace can be very relevant. If one
auth (or forwarder) does not respond, rec will turn to another one,
but only after the timeout of 1500ms by default.

	-Otto

>  Hello all,
>  Long time lurker on the message list and would like some performance
> and/or tuning advice.
> We've been using pdns-recursor as internal recursive nameservers for quite
> some time now.
> The original implementer of pdns departed and I was recently tasked with
> replacing or upgrading all of the servers with newer RHEL9 versions. I
> opted to build fresh and migrate the configuration to the latest 5.2
> release.
> 
> I'm hearing occasional complaints about odd issues and/or clients cycling
> through their DNS servers rapidly (timeouts?). Manual testing DNS works but
> I am reading through the metrics and performance documentation. I am hoping
> someone with a more experienced eye could take a look at a sampling of the
> periodic statistics report (below) and provide some insight or
> prioritization on any urgent issues I should focus on studying first.
> 
> My observations:
> * I do note that the performance documentation talks about
> firewalld/stateful firewalls impact but the legacy servers were using the
> same basic setup. If the firewall is the problem is there a way to validate
> this (other than stopping firewalld and waiting)?
> * The "worker" threads seem evenly distributed to my novice eye and our qps
> (queries per second) rate is low as I would expect since the name servers
> are internal only resources.
> * I ran a few pcaps and rec_control trace-regex for specific domain items
> being reported as problematic. Everything seemed to be working with the
> trace-regex always showing "Step3 Final resolve: No Error/6 or 8".
> 
> Thank you in advance for your time and consideration.
> 
> Sincerely,
> Scotsie
> 
> ```
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"
> cache-entries="23666" negcache-entries="497" questions="6831695"
> record-cache-acquired="286931329" record-cache-contended="64414"
> record-cache-contended-perc="0.02" record-cache-hitratio-perc="0.87"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"
> packetcache-acquired="16887684" packetcache-contended="1019"
> packetcache-contended-perc="0.01" packetcache-entries="7112"
> packetcache-hitratio-perc="37.75"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"
> edns-entries="38" failed-host-entries="50"
> non-resolving-nameserver-entries="0" nsspeed-entries="968"
> saved-parent-ns-sets-entries="65" throttle-entries="8"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"
> concurrent-queries="1" dot-outqueries="0" idle-tcpout-connections="0"
> outgoing-timeouts="36594" outqueries="14668546"
> outqueries-per-query-perc="214.71" tcp-outqueries="3131"
> throttled-queries-perc="1.90"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"
> taskqueue-expired="0" taskqueue-pushed="540" taskqueue-size="0"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Queries handled by
> thread" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"
> count="3470098" thread="0" tname="worker"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Queries handled by
> thread" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"
> count="3360836" thread="1" tname="worker"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Queries handled by
> thread" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.171"
> count="764" thread="2" tname="tcpworker"
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic QPS report"
> subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.171"
> averagedOver="1800" qps="117"
> ```

> _______________________________________________
> Pdns-users mailing list
> Pdns-users at mailman.powerdns.com
> https://mailman.powerdns.com/mailman/listinfo/pdns-users