[Pdns-users] Problems with PowerDNS

Wed Nov 11 08:10:17 UTC 2015

Dear Patrick,

I tried to set

gmysql-dnssec="no"
distributor-threads=10
receiver-threads=5

Now:
Mysql 110%
Pdns_server 90 %
Pdns_recursor 25%

But after 10-15 minutes again I got from some domains SERVFAIL..

[root at ns01 ~]# nslookup google.com
Server:         127.0.0.1
Address:        127.0.0.1#53

** server can't find google.com: REFUSED

And logs:
Nov 11 12:08:59 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'ad.bb800.com.' because: Too much time waiting for ad.6gg.cn.|A, timeouts: 5, throttles: 0, queries: 6, 7506msec
Nov 11 12:09:04 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'ad.bb800.com.' because: Too much time waiting for ad.6gg.cn.|A, timeouts: 5, throttles: 5, queries: 6, 7503msec
Nov 11 12:09:09 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'wx.qq.com.' because: Too much time waiting for wx1.qq.com.|A, timeouts: 5, throttles: 0, queries: 8, 8219msec
Nov 11 12:09:34 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'dev.voicecloud.cn.' because: Too much time waiting for dev.voicecloud.cn.|A, timeouts: 4, throttles: 0, queries: 9, 7087msec
Nov 11 12:09:38 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of '79.208.218.41.in-addr.arpa.' because: Too much time waiting for 79.208.218.41.in-addr.arpa.|PTR, timeouts: 4, throttles: 0, queries: 13, 7007msec
Nov 11 12:09:43 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of '61.29.19.113.in-addr.arpa.' because: Too much time waiting for 61.29.19.113.in-addr.arpa.|PTR, timeouts: 4, throttles: 0, queries: 11, 7928msec
Nov 11 12:09:49 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of '50.25.36.204.in-addr.arpa.' because: Too much time waiting for 50.25.36.204.in-addr.arpa.|PTR, timeouts: 5, throttles: 0, queries: 7, 7587msec

-----Original Message-----
From: pdns-users-bounces at mailman.powerdns.com [mailto:pdns-users-bounces at mailman.powerdns.com] On Behalf Of Patrick Domack
Sent: 11 noyabr 2015, çərşənbə 01:08
To: pdns-users at mailman.powerdns.com
Subject: Re: [Pdns-users] Problems with PowerDNS

I suppose sense you have dnssec=yes, you are using dnssec, This will cause a lot of sql queries.

pdns is using 100% cpu of a single core, did you try adjusting receiver-threads >1 probably for that box set it to 4 and test, maybe higher even.

Since I don't know much about what your pdns server is doing (and I haven't had issues on mine), I assume the dnssec dynamic signing is eating your cpu, and it only has one worker thread to do it with, limiting it to a single core.

I could be completely wrong.

Quoting "Nadir M. Aliyev" <admin at bakinter.net>:

> Dear Peter van Dijk, my connection link is 1000Gbps, server hardware 
> from cisco ucs. There is no problem with hardware. But mysql uses huge 
> resources even not zone in db it sends 4-5 queries to the db.
>
> I used percone tools to optimize mysql configuration. But it decreased 
> cpu usage only 10%. I have 10.000 query per second.
>
> Maybe I need do some tuning on TTLs?
>
> -----Original Message-----
> From: pdns-users-bounces at mailman.powerdns.com
> [mailto:pdns-users-bounces at mailman.powerdns.com] On Behalf Of Peter 
> van Dijk
> Sent: 10 noyabr 2015, çərşənbə axşamı 16:58
> To: pdns-users at mailman.powerdns.com
> Subject: Re: [Pdns-users] Problems with PowerDNS
>
> Hello Nadir,
>
> based on the logs, it looks like your powerdns has trouble reaching 
> the Internet at all. Are you on a slow or congested link? Note that in 
> general your machine looks quite busy!
>
> Kind regards,
> --
> Peter van Dijk
> PowerDNS.COM BV - https://www.powerdns.com/
>
> On 10 Nov 2015, at 13:01, Nadir M. Aliyev wrote:
>
>> Hi everyone!
>>
>>
>>
>> I have problems with some domains
>>
>>
>>
>> For ex. When I do google.com sometimes I get ns records but sometimes 
>> I get SERFVAIL also it happens basically with google. When I restrart 
>> pdns it works normally for 5 minutes. Then again SERVFAIL.
>>
>>
>>
>> Strange, some domains works some not works.. Even if cache hits.
>>
>> I increased cache ttls not helped.
>>
>>
>>
>> Server details: 8 core cpu, 8 GB of Ram.
>>
>> Load: pdns 100%, mysql 120%, pdns-recursor 30%, network 40 mbps.
>>
>>
>>
>>
>>
>> Some logs:
>>
>> Nov 10 15:33:08 ns01 pdns_recursor[15237]: Sending SERVFAIL to
>> 127.0.0.1
>> during resolve of 'gm-realm.net.' because: Too much time waiting for 
>> gm-realm.net.|A, timeouts: 5, throttles: 1, queries: 6, 7578msec
>>
>> Nov 10 15:33:09 ns01 pdns_recursor[15237]: Sending SERVFAIL to
>> 127.0.0.1
>> during resolve of 'gm-realm.net.' because: Too much time waiting for 
>> gm-realm.net.|A, timeouts: 5, throttles: 2, queries: 6, 7504msec
>>
>> Nov 10 15:33:12 ns01 pdns_recursor[15237]: Sending SERVFAIL to
>> 127.0.0.1
>> during resolve of 'gm-realm.net.' because: Too much time waiting for 
>> gm-realm.net.|A, timeouts: 5, throttles: 3, queries: 6, 7502msec
>>
>> Nov 10 15:33:13 ns01 pdns_recursor[15237]: Sending SERVFAIL to
>> 127.0.0.1
>> during resolve of 'us.micardapi.micloud.xiaomi.net.' because: Too 
>> much time waiting for us.api.micloud.mi.com.|A, timeouts: 5, 
>> throttles: 0,
>> queries: 7,
>> 7709msec
>>
>> Nov 10 15:33:18 ns01 pdns_recursor[15237]: Sending SERVFAIL to
>> 127.0.0.1
>> during resolve of 'www.coocent.net.' because: Too much time waiting 
>> for s-149179.abc188.com.|A, timeouts: 5, throttles: 0, queries: 8, 
>> 8093msec
>>
>> Nov 10 15:33:18 ns01 pdns_recursor[15237]: Sending SERVFAIL to
>> 127.0.0.1
>> during resolve of 'www.6ud1.com.' because: Too much time waiting for 
>> www.6ud1.com.|A, timeouts: 5, throttles: 0, queries: 6, 7502msec
>>
>> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 1787915 questions,
>> 497334
>> cache entries, 86066 negative entries, 11% cache hits
>>
>> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: throttle map: 6856, 
>> ns
>> speeds: 29645
>>
>> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: outpacket/query 
>> ratio 49%, 11% throttled, 0 no-delegation drops
>>
>> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 211 outgoing tcp 
>> connections, 1 queries running, 50712 outgoing timeouts
>>
>> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 322566 packet cache 
>> entries, 61% packet cache hits
>>
>> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 926 qps (average 
>> over 1930
>> seconds)
>>
>>
>>
>> Config:
>>
>>
>>
>> I have one master server which replicates db to the four slave server.
>>
>>
>>
>> # cat recursor.conf
>>
>> ..
>>
>> hint-file=/etc/pdns/named.root
>>
>> allow-from=127.0.0.0/8
>>
>> local-address=127.0.0.1
>>
>> local-port=5353
>>
>> version-string=Bind Recursor
>>
>> ..
>>
>>
>>
>> # cat /etc/pdns/pdns.conf
>>
>> ..
>>
>> launch=gmysql
>>
>> gmysql-host=127.0.0.1
>>
>> gmysql-port=3306
>>
>> gmysql-user=p_owerdns
>>
>> gmysql-password=verysecretpassword
>>
>> gmysql-dbname=p_ owerdns
>>
>> gmysql-dnssec="yes"
>>
>>
>>
>> #allow to customers
>>
>> allow-recursion=127.0.0.1/8, 172.16.0.0/16, 10.0.0.0/8,
>> xxx.xxx.xxx.xxx/16
>>
>>
>>
>> #master
>>
>> #allow-axfr-ips=172.16.6.30
>>
>>
>>
>> local-address=0.0.0.0
>>
>> local-port=53
>>
>>
>>
>> control-console=no
>>
>>
>>
>> query-cache-ttl=18600
>>
>> cache-ttl=18600
>>
>> default-ttl=7200
>>
>> soa-expire-default=18600
>>
>> soa-minimum-ttl=3600
>>
>> soa-refresh-default=10800
>>
>> soa-retry-default=3600
>>
>>
>>
>> daemon=yes
>>
>>
>>
>> default-soa-name=ns.master.mydomain.net
>>
>>
>>
>> distributor-threads=18
>>
>>
>>
>> guardian=yes
>>
>>
>>
>> #lazy-recursion=yes
>>
>>
>>
>> master=no
>>
>> slave=yes
>>
>> slave-cycle-interval=600
>>
>>
>>
>> max-tcp-connections=100
>>
>> max-queue-length=50000
>>
>>
>>
>> recursor=127.0.0.1:5353
>>
>>
>>
>> out-of-zone-additional-processing=yes
>>
>>
>>
>> webserver=yes
>>
>> webserver-address=172.16.6.34
>>
>> webserver-password=adminadminadmin
>>
>> webserver-port=8081
>>
>> webserver-print-arguments=yes
>>
>>
>>
>> #loglevel=9
>>
>> #log-dns-details=yes
>>
>> #log-dns-queries=yes
>>
>> #query-logging=yes
>>
>>
>>
>> version-string=Bind Resolver
>>
>> ..
>>
>> _______________________________________________
>> Pdns-users mailing list
>> Pdns-users at mailman.powerdns.com
>> http://mailman.powerdns.com/mailman/listinfo/pdns-users
>
> _______________________________________________
> Pdns-users mailing list
> Pdns-users at mailman.powerdns.com
> http://mailman.powerdns.com/mailman/listinfo/pdns-users
>
>
> _______________________________________________
> Pdns-users mailing list
> Pdns-users at mailman.powerdns.com
> http://mailman.powerdns.com/mailman/listinfo/pdns-users

_______________________________________________
Pdns-users mailing list
Pdns-users at mailman.powerdns.com
http://mailman.powerdns.com/mailman/listinfo/pdns-users