[Pdns-users] Recursion issue--SERVFAIL then NOERROR totally at random
Brian Menges
bmenges at gogrid.com
Tue Sep 9 23:29:19 UTC 2014
I'd say google is talking to the one that answers, and caches that.
63.238.139.235 (gslb-ns1.toyota-na.com) definitely has issues
- Brian Menges
Principal Engineer, DevOps @ GoGrid, LLC.
From: pdns-users-bounces at mailman.powerdns.com [mailto:pdns-users-bounces at mailman.powerdns.com] On Behalf Of Todd Smith
Sent: Tuesday, September 09, 2014 10:04 AM
To: 'pdns-users at mailman.powerdns.com'
Subject: Re: [Pdns-users] Recursion issue--SERVFAIL then NOERROR totally at random
Hey Brian,
That would make perfect sense, and I was thinking along similar lines, but if that's the case, why do I get a consistent NOERROR when using Google DNS? Google's cache perhaps?
root at yoshi:/# dig toyotasupplier.com @8.8.8.8
; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> toyotasupplier.com @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35779
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;toyotasupplier.com. IN A
;; ANSWER SECTION:
toyotasupplier.com. 21594 IN A 12.169.52.71
;; Query time: 30 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Sep 9 12:34:43 2014
;; MSG SIZE rcvd: 52
root at yoshi:/# dig toyotasupplier.com @208.88.248.27
; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> toyotasupplier.com @208.88.248.27
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 29841
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;toyotasupplier.com. IN A
;; Query time: 49 msec
;; SERVER: 208.88.248.27#53(208.88.248.27)
;; WHEN: Tue Sep 9 12:35:02 2014
;; MSG SIZE rcvd: 36
-T
From: pdns-users-bounces at mailman.powerdns.com<mailto:pdns-users-bounces at mailman.powerdns.com> [mailto:pdns-users-bounces at mailman.powerdns.com] On Behalf Of Brian Menges
Sent: Tuesday, September 09, 2014 12:56 PM
To: 'pdns-users at mailman.powerdns.com'
Subject: Re: [Pdns-users] Recursion issue--SERVFAIL then NOERROR totally at random
I'd say it's on Toyota's end:
$ dig toyotasupplier.com +short @gslb-ns1.toyota-na.com
<<>> DiG 9.7.3 <<>> toyotasupplier.com +short @gslb-ns1.toyota-na.com
;; global options: +cmd
connection timed out; no servers could be reached
Their other DNS server works fine... several attempts to reach the first one however fails (haven't gotten a success yet).
I'd say it's their problem.
- Brian Menges
Principal Engineer, DevOps @ GoGrid, LLC.
From: pdns-users-bounces at mailman.powerdns.com<mailto:pdns-users-bounces at mailman.powerdns.com> [mailto:pdns-users-bounces at mailman.powerdns.com] On Behalf Of Todd Smith
Sent: Tuesday, September 09, 2014 9:24 AM
To: 'pdns-users at mailman.powerdns.com'
Subject: [Pdns-users] Recursion issue--SERVFAIL then NOERROR totally at random
Hey guys,
I've been having a problem with recursion. For some reason, certain domains seem to throw SERVFAIL errors when dug most of the time, but then NOERROR with a correct response at other random times. For example:
root at yoshi:/# dig toyotasupplier.com
; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> toyotasupplier.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 2636
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;toyotasupplier.com. IN A
;; Query time: 0 msec
;; SERVER: 208.88.248.25#53(208.88.248.25)
;; WHEN: Wed Sep 3 13:36:33 2014
;; MSG SIZE rcvd: 36
And then, a few hours later:
root at yoshi:/# dig toyotasupplier.com
; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> toyotasupplier.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56751
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;toyotasupplier.com. IN A
;; ANSWER SECTION:
toyotasupplier.com. 18296 IN A 12.169.52.71
;; Query time: 1 msec
;; SERVER: 208.88.248.25#53(208.88.248.25)
;; WHEN: Thu Sep 4 10:39:38 2014
;; MSG SIZE rcvd: 52
And then, a few hours later still:
root at yoshi:/# dig toyotasupplier.com
; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> toyotasupplier.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 5171
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;toyotasupplier.com. IN A
;; Query time: 3017 msec
;; SERVER: 208.88.248.25#53(208.88.248.25)
;; WHEN: Fri Sep 5 07:50:25 2014
;; MSG SIZE rcvd: 36
All without making a single change.
I have been working on debugging this for two days now and absolutely cannot pinpoint a source for the issue. I've increased the max query lengths, the recursor's network and client TCP timeouts, restarted the service several times on several of our DNS servers, and nothing I do seems to fix it. It of course doesn't help that the bug is a bit of a gremlin and keeps mischievously disappearing at random (and in fact never, to my knowledge, happened before until about a week ago, when it started to occur for no apparent reason). Any idea on what could be causing this? FWIW, when I run dig toyotasupplier.com ns it consistently works fine:
root at yoshi:/# dig toyotasupplier.com ns
; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> toyotasupplier.com ns
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39522
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;toyotasupplier.com. IN NS
;; ANSWER SECTION:
toyotasupplier.com. 50741 IN NS gslb-ns2.toyota-na.com.
toyotasupplier.com. 50741 IN NS gslb-ns1.toyota-na.com.
;; Query time: 1 msec
;; SERVER: 208.88.248.25#53(208.88.248.25)
;; WHEN: Fri Sep 5 07:49:29 2014
;; MSG SIZE rcvd: 92
Many thanks in advance,
Todd W. Smith
IP Services Technician
2331 East 600 North
Greenfield, IN 46140
(317) 323-2021
tsmith at ninestarconnect.com<mailto:tsmith at ninestarconnect.com>
www.ninestarconnect.com<http://www.ninestarconnect.com/>
________________________________
The information contained in this message, and any attachments, may contain confidential and legally privileged material. It is solely for the use of the person or entity to which it is addressed. Any review, retransmission, dissemination, or action taken in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and delete the material from any computer.
________________________________
The information contained in this message, and any attachments, may contain confidential and legally privileged material. It is solely for the use of the person or entity to which it is addressed. Any review, retransmission, dissemination, or action taken in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and delete the material from any computer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20140909/37d3ca4a/attachment-0001.html>
More information about the Pdns-users
mailing list