<div dir="ltr"><div>Otto,</div><div>Thanks for your assistance.Since these were setup with private IPs I wasn't sure how useful the config would be however, I have included it below.<br></div><div><br></div><div># rec_control dump-throttlemap -<br>; throttle map dump follows<br>; remote IP qname qtype count ttd reason<br>10.0.196.197 0.10.in-addr.arpa A 2 2025-04-18T18:44:22 RCodeRefused<br>10.0.196.197 10.10.in-addr.arpa A 3 2025-04-18T18:44:25 RCodeRefused<br>10.0.196.197 255.10.in-addr.arpa A 1 2025-04-18T18:44:23 RCodeRefused<br>10.0.62.244 0.10.in-addr.arpa A 2 2025-04-18T18:44:22 RCodeRefused<br>10.0.62.244 10.10.in-addr.arpa A 3 2025-04-18T18:44:25 RCodeRefused<br>10.0.62.244 255.10.in-addr.arpa A 2 2025-04-18T18:44:23 RCodeRefused<br>dump-throttlemap: dumped 6 records</div><div><br># rec_control dump-failedservers -<br>I removed any count 1 or 2 for brevity since this email is already a long read.<br>; failed servers dump follows<br>; remote IP count timestamp<br>203.119.25.5 8 2025-04-18T18:43:44<br>203.119.26.5 8 2025-04-18T18:43:42<br>203.119.27.5 8 2025-04-18T18:43:41<br>203.119.28.5 8 2025-04-18T18:43:39<br>203.119.29.5 8 2025-04-18T18:43:45<br>200.189.41.10 7 2025-04-18T18:42:46<br>200.219.148.10 6 2025-04-18T18:39:47<br>200.219.154.10 6 2025-04-18T18:42:43<br>200.219.159.10 7 2025-04-18T18:42:45<br>200.192.233.10 7 2025-04-18T18:42:40<br>200.229.248.10 4 2025-04-18T18:42:42<br>203.119.95.53 3 2025-04-18T18:39:30<br>203.119.86.101 1229 2025-04-18T18:40:03<br>35.173.255.124 4895 2025-04-18T18:36:21<br>dump-failedservers: dumped 43 records</div><div><br></div><div><br></div><div>
<p style="margin:0in;font-family:Calibri;font-size:11pt">Config(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">
</p><div>Please note that one of the zones forwarding is 'split brained' from a legacy setup. The zone consists of a private Active Directory environment and a separately maintained public zone. The configuration forwards to the private AD servers and I believe the lua script drops queries that have no match in that zone. The public zone is being slowly phased out.<br><br></div><div>I noted while reviewing the previous server configs and found a comment about this value but no context for the specific reasoning. This may explain the values you noted but I would like to understand the implications of removing it. It doesn't seem like something that should have been enabled.<br></div><div># <a href="https://github.com/PowerDNS/pdns/issues/6186">https://github.com/PowerDNS/pdns/issues/6186</a><br>max-negative-ttl=0</div><div><br></div><div></div>
<p style="margin:0in;font-family:Calibri;font-size:11pt"> /etc/pdns-recursor/recursor.conf<br>
</p><p style="margin:0in;font-family:Calibri;font-size:11pt"></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">---</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">dnssec:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>validation: validate</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">incoming:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>allow_from:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- <a href="http://127.0.0.1/8">127.0.0.1/8</a></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- <a href="http://10.0.0.0/8">10.0.0.0/8</a></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- <a href="http://172.16.0.0/12">172.16.0.0/12</a></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- <a href="http://192.168.0.0/16">192.168.0.0/16</a></p><p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 'fd00::/8'</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- '2607:B600::/32'</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>listen:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 0.0.0.0</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_tcp_clients: 128</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_tcp_per_client: 0</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_tcp_queries_per_connection: 0</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>port: 53</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>tcp_timeout: 2</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">outgoing:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>dont_query: []</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_qperq: 50</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>network_timeout: 1500</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">packetcache:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_entries: 1000000</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">recordcache:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_entries: 1000000</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_negative_ttl: 0</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_ttl: 86400</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">recursor:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>daemon: false</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>forward_zones:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- zone: <a href="http://momentumbusiness.com">momentumbusiness.com</a></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>recurse: false</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>forwarders:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.255.255.76</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.1.3.228</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- zone: 10.in-addr.arpa</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>recurse: false</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>forwarders:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.0.196.197</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.0.62.244</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- zone: 168.192.in-addr.arpa</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>recurse: false</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>forwarders:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.0.196.197</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.0.62.244</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- zone: 16.172.in-addr.arpa</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>recurse: false</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>forwarders:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.0.196.197</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 10.0.62.244</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>lua_dns_script:
/etc/pdns-recursor/momentumbusiness_com.lua</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_recursion_depth: 40</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>max_total_msec: 7000</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>minimum_ttl_override: 1</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>server_id: <a href="http://nsres01.momentumtelecom.com">nsres01.momentumtelecom.com</a></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>setgid: pdns-recursor</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>setuid: pdns-recursor</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">webservice:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>address: 0.0.0.0</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>allow_from:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 192.168.9.164</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- 192.168.21.134</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>- <a href="http://192.168.20.0/24">192.168.20.0/24</a></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>api_key: <sanitized></p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>port: 8080</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>webserver: true</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">logging:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt"><span> </span>loglevel: 3</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt">...</p>
</div><br>/etc/pdns-recursor/momentumbusiness_com.lua<br>pdnslog("Lua NXDomain filter for <a href="http://momentumbusiness.com">momentumbusiness.com</a> loading...", pdns.loglevels.Notice)<br>nxdomainsuffix=newDN("<a href="http://momentumbusiness.com">momentumbusiness.com</a>")<br>function nxdomain(dq)<br> if dq.qname:isPartOf(nxdomainsuffix)<br> then<br> dq.appliedPolicy.policyKind = pdns.policykinds.Drop<br> return true<br> end<br> return false<br>end<br><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Fri, Apr 18, 2025 at 9:39 AM Otto Moerbeek <<a href="mailto:otto@drijf.net">otto@drijf.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Fri, Apr 18, 2025 at 08:28:48AM -0400, Scott Crace via Pdns-users wrote:<br>
<br>
Hi,<br>
<br>
Please include your config. That said:<br>
<br>
You seem to have pretty low cache hit ratio, a high number of outgoing<br>
queries. How is your cache configged?<br>
<br>
Also some throttling is going on. I suspect rec has trouble contacting<br>
one or more auths or forwarders. The throttling tables can be viewed<br>
using<br>
<br>
rec_control dump-throttlemap -<br>
rec_control dump-failedservers -<br>
<br>
Also, what happens *during* the trace can be very relevant. If one<br>
auth (or forwarder) does not respond, rec will turn to another one,<br>
but only after the timeout of 1500ms by default.<br>
<br>
-Otto<br>
<br>
> Hello all,<br>
> Long time lurker on the message list and would like some performance<br>
> and/or tuning advice.<br>
> We've been using pdns-recursor as internal recursive nameservers for quite<br>
> some time now.<br>
> The original implementer of pdns departed and I was recently tasked with<br>
> replacing or upgrading all of the servers with newer RHEL9 versions. I<br>
> opted to build fresh and migrate the configuration to the latest 5.2<br>
> release.<br>
> <br>
> I'm hearing occasional complaints about odd issues and/or clients cycling<br>
> through their DNS servers rapidly (timeouts?). Manual testing DNS works but<br>
> I am reading through the metrics and performance documentation. I am hoping<br>
> someone with a more experienced eye could take a look at a sampling of the<br>
> periodic statistics report (below) and provide some insight or<br>
> prioritization on any urgent issues I should focus on studying first.<br>
> <br>
> My observations:<br>
> * I do note that the performance documentation talks about<br>
> firewalld/stateful firewalls impact but the legacy servers were using the<br>
> same basic setup. If the firewall is the problem is there a way to validate<br>
> this (other than stopping firewalld and waiting)?<br>
> * The "worker" threads seem evenly distributed to my novice eye and our qps<br>
> (queries per second) rate is low as I would expect since the name servers<br>
> are internal only resources.<br>
> * I ran a few pcaps and rec_control trace-regex for specific domain items<br>
> being reported as problematic. Everything seemed to be working with the<br>
> trace-regex always showing "Step3 Final resolve: No Error/6 or 8".<br>
> <br>
> Thank you in advance for your time and consideration.<br>
> <br>
> Sincerely,<br>
> Scotsie<br>
> <br>
> ```<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics<br>
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"<br>
> cache-entries="23666" negcache-entries="497" questions="6831695"<br>
> record-cache-acquired="286931329" record-cache-contended="64414"<br>
> record-cache-contended-perc="0.02" record-cache-hitratio-perc="0.87"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics<br>
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"<br>
> packetcache-acquired="16887684" packetcache-contended="1019"<br>
> packetcache-contended-perc="0.01" packetcache-entries="7112"<br>
> packetcache-hitratio-perc="37.75"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics<br>
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"<br>
> edns-entries="38" failed-host-entries="50"<br>
> non-resolving-nameserver-entries="0" nsspeed-entries="968"<br>
> saved-parent-ns-sets-entries="65" throttle-entries="8"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics<br>
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"<br>
> concurrent-queries="1" dot-outqueries="0" idle-tcpout-connections="0"<br>
> outgoing-timeouts="36594" outqueries="14668546"<br>
> outqueries-per-query-perc="214.71" tcp-outqueries="3131"<br>
> throttled-queries-perc="1.90"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic statistics<br>
> report" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"<br>
> taskqueue-expired="0" taskqueue-pushed="540" taskqueue-size="0"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Queries handled by<br>
> thread" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"<br>
> count="3470098" thread="0" tname="worker"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Queries handled by<br>
> thread" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.170"<br>
> count="3360836" thread="1" tname="worker"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Queries handled by<br>
> thread" subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.171"<br>
> count="764" thread="2" tname="tcpworker"<br>
> Apr 17 16:07:28 nsrecdns01-1 pdns-recursor[1092]: msg="Periodic QPS report"<br>
> subsystem="stats" level="0" prio="Info" tid="0" ts="1744920448.171"<br>
> averagedOver="1800" qps="117"<br>
> ```<br>
<br>
> _______________________________________________<br>
> Pdns-users mailing list<br>
> <a href="mailto:Pdns-users@mailman.powerdns.com" target="_blank">Pdns-users@mailman.powerdns.com</a><br>
> <a href="https://mailman.powerdns.com/mailman/listinfo/pdns-users" rel="noreferrer" target="_blank">https://mailman.powerdns.com/mailman/listinfo/pdns-users</a><br>
<br>
</blockquote></div></div>