[Pdns-users] High latency on recursion without cache

[Ciro Iriarte]

2015-01-27 6:59 GMT-03:00 bert hubert <bert.hubert at powerdns.com>:

> On Sun, Jan 25, 2015 at 10:33:34AM -0300, Ciro Iriarte wrote:
> > Also, the test traffic was stopped, so the trace file should be complete
> > and cleaner!.
>
> Ciro,
>
> I don't see anything that is wrong here. From a cold cache, it takes 11
> queries to resolve 2.centos.pool.ntp.org.
>
> Your network used up the following amounts of time on those queries:
>  in 266ms
>  in 226ms
>  in 184ms
>  in 226ms
>  in 233ms
>  in 267ms
>  in 223ms
>  in 201ms
>  in 224ms
>  in 51ms
>  in 199ms
>
> Which together is around 2 seconds.
>
> If there is a problem, the problem is that your network is pretty far away
> from most servers it appears.
>
> If you redo your query with the latest PowerDNS test version (3.7.0-RC1)
> you'll get slightly better timing output with --trace, which perhaps could
> tell you a little more.
>
> On a high-latency network (and your fastest response to anything in this
> trace was 51ms, even if I look at the other queries too), having a warm
> cache is super important.
>
> Good luck!
>
>         Bert
>

Hi Bert, thanks for the analysis!. I double checked all the configuration
(routing/pdns/linux) and everything seems to be OK. It's obviously not a
PDNS thing as a trace using dig still gives pretty bad times

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.30.rc1.el6 <<>> +trace
2.centos.pool.ntp.org
;; Received 241 bytes from 186.16.16.16#53(186.16.16.16) in 1317 ms <---
this goes to localhost for recursion
;; Received 441 bytes from 199.7.83.42#53(199.7.83.42) in 5628 ms
;; Received 153 bytes from 199.19.57.1#53(199.19.57.1) in 2881 ms
;; Received 189 bytes from 128.175.13.17#53(128.175.13.17) in 8346 ms
;; Received 187 bytes from 94.242.223.210#53(94.242.223.210) in 644 ms

Complete trace:
   http://pastebin.com/tvKqhq2e

What I find weird is that a query to the server 199.7.83.42 takes more than
5 seconds, but a plain ping request gives a RTT of about 177ms. Maybe the
servers are overloaded and I'm over-reacting :P

--- 199.7.83.42 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4187ms
*rtt min/avg/max/mdev = 177.556/178.459/181.044/1.334 ms*


The same goes for the other destinations:

--- 199.19.57.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4183ms
*rtt min/avg/max/mdev = 177.125/177.302/177.644/0.565 ms*

--- 128.175.13.17 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4194ms
*rtt min/avg/max/mdev = 187.530/188.211/189.673/0.931 ms*


--- 94.242.223.210 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4276ms
*rtt min/avg/max/mdev = 265.467/269.252/272.820/2.637 ms*

The only other thing I could think of is some kind of QoS issue and to
blame the carrier. It's time to poke the networking guys...

Regards,

-- 
Ciro Iriarte
http://iriarte.it
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20150127/e506f5d5/attachment-0001.html>