[Pdns-users] Slow query and SERVERFAIL from local pdns_recursor

[Christian Degenkolb]

Hi,

I hope somebody on the ML can help me figure out what I'm doing wrong.
I have a local pdns_recursor (version 4.1.11-1+deb10u1 from debian 10) 
runing and added it at the top of my /etc/resolve.conf as 127.0.0.1.

However I see some strange SERVERFAIL resolves happening and all in all 
a slow DNS system.

For example see the following two consecutive resolves and a direct 
request to the NS.
The first one takes nearly 3 seconds vs 11 ms from the same system if I 
query the NS directly.

$ dig pubs.vmware.com @127.0.0.1

; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> pubs.vmware.com @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 4929
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;pubs.vmware.com.INA

;; ANSWER SECTION:
pubs.vmware.com.30INCNAME   pubs.vmware.com.ds.edgekey.net.
pubs.vmware.com.ds.edgekey.net. 10 IN   CNAME   
e751.dscx.akamaiedge.net.

;; Query time: 3009 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Sep 02 09:19:04 CEST 2020
;; MSG SIZE  rcvd: 123

$ dig pubs.vmware.com @127.0.0.1

; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> pubs.vmware.com @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1345
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;pubs.vmware.com.INA

;; ANSWER SECTION:
pubs.vmware.com.18INCNAME   pubs.vmware.com.ds.edgekey.net.
pubs.vmware.com.ds.edgekey.net. 4 INCNAME   e751.dscx.akamaiedge.net.
e751.dscx.akamaiedge.net. 16INA104.111.214.47

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Sep 02 09:19:08 CEST 2020
;; MSG SIZE  rcvd: 139

$ dig pubs.vmware.com @ns03.vmwdns.com

; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> pubs.vmware.com 
@ns03.vmwdns.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5509
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;pubs.vmware.com.INA

;; ANSWER SECTION:
pubs.vmware.com.30INCNAME   pubs.vmware.com.ds.edgekey.net.

;; Query time: 11 msec
;; SERVER: 45.54.11.129#53(45.54.11.129)
;; WHEN: Wed Sep 02 09:34:42 CEST 2020
;; MSG SIZE  rcvd: 88

Also I have a number SERVFAIL in /var/log/syslog (pdns_recurser is 
currently running with loglevel=6).
For example:

Sep  2 08:45:35 rho pdns_recursor[19311]: Sending SERVFAIL to 127.0.0.1 
during resolve of 'pubs.vmware.com' because: Too much time waiting for 
pubs.vmware.com.ds.edgekey.net|A, timeouts: 5,
throttles: 1, queries: 6, 7991msec

# grep 'Too much time waiting for' /var/log/syslog | wc -l
184

As per 
https://blog.powerdns.com/2014/12/11/powerdns-graphing-as-a-service/ I 
send the metrics to 
https://metronome1.powerdns.com/?server=pdns.rho-test.recursor&beginTime=-172800

Does anybody have an idea whats wrong? This seems way to slow for DNS 
and the SERVFAIL schouldn't happen this often.
The server in question is running in a DC of the german Hoster 
hetzner.de. Besides the strange DNS I don't have any problems with the 
reliability of the network connection.

thanks
Chris