[Pdns-users] recursive server failing

[Charles Sprickman]

Howdy,

Kind of stumped at how to debug this and where the fault lies.  I noticed that we had some issues when customers were noting that emails to anyone at “@dot.nyc.gov” were bouncing.

If I query my local powerdns recursor, I get a SERVFAIL.  If I query a local BIND server, I get a correct response (see both below).

Here’s a few things I’ve tried:

- Verify with DNSVIZ: http://dnsviz.net/d/dot.nyc.gov/dnssec/
- Update PowerDNS to powerdns-recursor-4.0.6
- Remove “scrub” rules from pf configuration
- Change pf rules to be stateless
- Look for denied traffic by running tcpdump against pflog device while performing query
- Checked record by querying BIND on same host
- Checked record elsewhere (successful)

Any ideas where to start with this?  Anyone else seeing the same issue with this record?

Thanks,

Charles

dig @216.220.96.46 -t mx dot.nyc.gov

; <<>> DiG 9.9.5 <<>> @216.220.96.46 -t mx dot.nyc.gov
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 21046
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dot.nyc.gov.			IN	MX

;; Query time: 1448 msec
;; SERVER: 216.220.96.46#53(216.220.96.46)
;; WHEN: Sat Jul 29 00:04:51 EDT 2017
;; MSG SIZE  rcvd: 40

If I query our BIND server (still using that for authoritative and people that forever have those NS IPs configured by hand forever), I get a proper response:

dig @216.220.96.18 -t mx dot.nyc.gov

; <<>> DiG 9.9.5 <<>> @216.220.96.18 -t mx dot.nyc.gov
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31310
;; flags: qr rd ra; QUERY: 1, ANSWER: 8, AUTHORITY: 4, ADDITIONAL: 5

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dot.nyc.gov.			IN	MX

;; ANSWER SECTION:
dot.nyc.gov.		900	IN	MX	10 vwall5.nyc.gov.
dot.nyc.gov.		900	IN	MX	10 vwall8.nyc.gov.
dot.nyc.gov.		900	IN	MX	100 vwall2.nyc.gov.
dot.nyc.gov.		900	IN	MX	100 vwall4.nyc.gov.
dot.nyc.gov.		900	IN	MX	100 vwall1.nyc.gov.
dot.nyc.gov.		900	IN	MX	10 vwall7.nyc.gov.
dot.nyc.gov.		900	IN	MX	10 vwall6.nyc.gov.
dot.nyc.gov.		900	IN	MX	10 vwall3.nyc.gov.

;; AUTHORITY SECTION:
nyc.gov.		85328	IN	NS	vwall2a.nyc.gov.
nyc.gov.		85328	IN	NS	vwall1a.nyc.gov.
nyc.gov.		85328	IN	NS	vwall4a.nyc.gov.
nyc.gov.		85328	IN	NS	vwall3a.nyc.gov.

;; ADDITIONAL SECTION:
vwall1a.nyc.gov.	85328	IN	A	161.185.1.3
vwall2a.nyc.gov.	85328	IN	A	161.185.1.12
vwall3a.nyc.gov.	85328	IN	A	167.153.130.12
vwall4a.nyc.gov.	85328	IN	A	167.153.130.13

;; Query time: 3263 msec
;; SERVER: 216.220.96.18#53(216.220.96.18)
;; WHEN: Sat Jul 29 00:10:08 EDT 2017
;; MSG SIZE  rcvd: 376