[Pdns-users] Problems with PowerDNS

Wed Nov 11 08:21:38 UTC 2015

Does your database have indexes? We have had few cases before where the
user had forgotten to add indexes to the database.

Aki

On Wed, Nov 11, 2015 at 12:10:17PM +0400, Nadir M. Aliyev wrote:
> Dear Patrick,
> 
> I tried to set
> 
> gmysql-dnssec="no"
> distributor-threads=10
> receiver-threads=5
> 
> Now:
> Mysql 110%
> Pdns_server 90 %
> Pdns_recursor 25%
> 
> 
> But after 10-15 minutes again I got from some domains SERVFAIL..
> 
> [root at ns01 ~]# nslookup google.com
> Server:         127.0.0.1
> Address:        127.0.0.1#53
> 
> ** server can't find google.com: REFUSED
> 
> And logs:
> Nov 11 12:08:59 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'ad.bb800.com.' because: Too much time waiting for ad.6gg.cn.|A, timeouts: 5, throttles: 0, queries: 6, 7506msec
> Nov 11 12:09:04 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'ad.bb800.com.' because: Too much time waiting for ad.6gg.cn.|A, timeouts: 5, throttles: 5, queries: 6, 7503msec
> Nov 11 12:09:09 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'wx.qq.com.' because: Too much time waiting for wx1.qq.com.|A, timeouts: 5, throttles: 0, queries: 8, 8219msec
> Nov 11 12:09:34 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of 'dev.voicecloud.cn.' because: Too much time waiting for dev.voicecloud.cn.|A, timeouts: 4, throttles: 0, queries: 9, 7087msec
> Nov 11 12:09:38 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of '79.208.218.41.in-addr.arpa.' because: Too much time waiting for 79.208.218.41.in-addr.arpa.|PTR, timeouts: 4, throttles: 0, queries: 13, 7007msec
> Nov 11 12:09:43 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of '61.29.19.113.in-addr.arpa.' because: Too much time waiting for 61.29.19.113.in-addr.arpa.|PTR, timeouts: 4, throttles: 0, queries: 11, 7928msec
> Nov 11 12:09:49 ns01 pdns_recursor[4559]: Sending SERVFAIL to 127.0.0.1 during resolve of '50.25.36.204.in-addr.arpa.' because: Too much time waiting for 50.25.36.204.in-addr.arpa.|PTR, timeouts: 5, throttles: 0, queries: 7, 7587msec
> 
> 
> -----Original Message-----
> From: pdns-users-bounces at mailman.powerdns.com [mailto:pdns-users-bounces at mailman.powerdns.com] On Behalf Of Patrick Domack
> Sent: 11 noyabr 2015, ÃÂ§ÃÂrÃÂÃÂnbÃÂ 01:08
> To: pdns-users at mailman.powerdns.com
> Subject: Re: [Pdns-users] Problems with PowerDNS
> 
> I suppose sense you have dnssec=yes, you are using dnssec, This will cause a lot of sql queries.
> 
> pdns is using 100% cpu of a single core, did you try adjusting receiver-threads >1 probably for that box set it to 4 and test, maybe higher even.
> 
> Since I don't know much about what your pdns server is doing (and I haven't had issues on mine), I assume the dnssec dynamic signing is eating your cpu, and it only has one worker thread to do it with, limiting it to a single core.
> 
> I could be completely wrong.
> 
> 
> Quoting "Nadir M. Aliyev" <admin at bakinter.net>:
> 
> > Dear Peter van Dijk, my connection link is 1000Gbps, server hardware 
> > from cisco ucs. There is no problem with hardware. But mysql uses huge 
> > resources even not zone in db it sends 4-5 queries to the db.
> >
> > I used percone tools to optimize mysql configuration. But it decreased 
> > cpu usage only 10%. I have 10.000 query per second.
> >
> > Maybe I need do some tuning on TTLs?
> >
> > -----Original Message-----
> > From: pdns-users-bounces at mailman.powerdns.com
> > [mailto:pdns-users-bounces at mailman.powerdns.com] On Behalf Of Peter 
> > van Dijk
> > Sent: 10 noyabr 2015, ÃÂ§ÃÂrÃÂÃÂnbÃÂ axÃÂamÃÂ± 16:58
> > To: pdns-users at mailman.powerdns.com
> > Subject: Re: [Pdns-users] Problems with PowerDNS
> >
> > Hello Nadir,
> >
> > based on the logs, it looks like your powerdns has trouble reaching 
> > the Internet at all. Are you on a slow or congested link? Note that in 
> > general your machine looks quite busy!
> >
> > Kind regards,
> > --
> > Peter van Dijk
> > PowerDNS.COM BV - https://www.powerdns.com/
> >
> > On 10 Nov 2015, at 13:01, Nadir M. Aliyev wrote:
> >
> >> Hi everyone!
> >>
> >>
> >>
> >> I have problems with some domains
> >>
> >>
> >>
> >> For ex. When I do google.com sometimes I get ns records but sometimes 
> >> I get SERFVAIL also it happens basically with google. When I restrart 
> >> pdns it works normally for 5 minutes. Then again SERVFAIL.
> >>
> >>
> >>
> >> Strange, some domains works some not works.. Even if cache hits.
> >>
> >> I increased cache ttls not helped.
> >>
> >>
> >>
> >> Server details: 8 core cpu, 8 GB of Ram.
> >>
> >> Load: pdns 100%, mysql 120%, pdns-recursor 30%, network 40 mbps.
> >>
> >>
> >>
> >>
> >>
> >> Some logs:
> >>
> >> Nov 10 15:33:08 ns01 pdns_recursor[15237]: Sending SERVFAIL to
> >> 127.0.0.1
> >> during resolve of 'gm-realm.net.' because: Too much time waiting for 
> >> gm-realm.net.|A, timeouts: 5, throttles: 1, queries: 6, 7578msec
> >>
> >> Nov 10 15:33:09 ns01 pdns_recursor[15237]: Sending SERVFAIL to
> >> 127.0.0.1
> >> during resolve of 'gm-realm.net.' because: Too much time waiting for 
> >> gm-realm.net.|A, timeouts: 5, throttles: 2, queries: 6, 7504msec
> >>
> >> Nov 10 15:33:12 ns01 pdns_recursor[15237]: Sending SERVFAIL to
> >> 127.0.0.1
> >> during resolve of 'gm-realm.net.' because: Too much time waiting for 
> >> gm-realm.net.|A, timeouts: 5, throttles: 3, queries: 6, 7502msec
> >>
> >> Nov 10 15:33:13 ns01 pdns_recursor[15237]: Sending SERVFAIL to
> >> 127.0.0.1
> >> during resolve of 'us.micardapi.micloud.xiaomi.net.' because: Too 
> >> much time waiting for us.api.micloud.mi.com.|A, timeouts: 5, 
> >> throttles: 0,
> >> queries: 7,
> >> 7709msec
> >>
> >> Nov 10 15:33:18 ns01 pdns_recursor[15237]: Sending SERVFAIL to
> >> 127.0.0.1
> >> during resolve of 'www.coocent.net.' because: Too much time waiting 
> >> for s-149179.abc188.com.|A, timeouts: 5, throttles: 0, queries: 8, 
> >> 8093msec
> >>
> >> Nov 10 15:33:18 ns01 pdns_recursor[15237]: Sending SERVFAIL to
> >> 127.0.0.1
> >> during resolve of 'www.6ud1.com.' because: Too much time waiting for 
> >> www.6ud1.com.|A, timeouts: 5, throttles: 0, queries: 6, 7502msec
> >>
> >> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 1787915 questions,
> >> 497334
> >> cache entries, 86066 negative entries, 11% cache hits
> >>
> >> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: throttle map: 6856, 
> >> ns
> >> speeds: 29645
> >>
> >> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: outpacket/query 
> >> ratio 49%, 11% throttled, 0 no-delegation drops
> >>
> >> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 211 outgoing tcp 
> >> connections, 1 queries running, 50712 outgoing timeouts
> >>
> >> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 322566 packet cache 
> >> entries, 61% packet cache hits
> >>
> >> Nov 10 15:42:52 ns01 pdns_recursor[15237]: stats: 926 qps (average 
> >> over 1930
> >> seconds)
> >>
> >>
> >>
> >> Config:
> >>
> >>
> >>
> >> I have one master server which replicates db to the four slave server.
> >>
> >>
> >>
> >> # cat recursor.conf
> >>
> >> ..
> >>
> >> hint-file=/etc/pdns/named.root
> >>
> >> allow-from=127.0.0.0/8
> >>
> >> local-address=127.0.0.1
> >>
> >> local-port=5353
> >>
> >> version-string=Bind Recursor
> >>
> >> ..
> >>
> >>
> >>
> >> # cat /etc/pdns/pdns.conf
> >>
> >> ..
> >>
> >> launch=gmysql
> >>
> >> gmysql-host=127.0.0.1
> >>
> >> gmysql-port=3306
> >>
> >> gmysql-user=p_owerdns
> >>
> >> gmysql-password=verysecretpassword
> >>
> >> gmysql-dbname=p_ owerdns
> >>
> >> gmysql-dnssec="yes"
> >>
> >>
> >>
> >> #allow to customers
> >>
> >> allow-recursion=127.0.0.1/8, 172.16.0.0/16, 10.0.0.0/8,
> >> xxx.xxx.xxx.xxx/16
> >>
> >>
> >>
> >> #master
> >>
> >> #allow-axfr-ips=172.16.6.30
> >>
> >>
> >>
> >> local-address=0.0.0.0
> >>
> >> local-port=53
> >>
> >>
> >>
> >> control-console=no
> >>
> >>
> >>
> >> query-cache-ttl=18600
> >>
> >> cache-ttl=18600
> >>
> >> default-ttl=7200
> >>
> >> soa-expire-default=18600
> >>
> >> soa-minimum-ttl=3600
> >>
> >> soa-refresh-default=10800
> >>
> >> soa-retry-default=3600
> >>
> >>
> >>
> >> daemon=yes
> >>
> >>
> >>
> >> default-soa-name=ns.master.mydomain.net
> >>
> >>
> >>
> >> distributor-threads=18
> >>
> >>
> >>
> >> guardian=yes
> >>
> >>
> >>
> >> #lazy-recursion=yes
> >>
> >>
> >>
> >> master=no
> >>
> >> slave=yes
> >>
> >> slave-cycle-interval=600
> >>
> >>
> >>
> >> max-tcp-connections=100
> >>
> >> max-queue-length=50000
> >>
> >>
> >>
> >> recursor=127.0.0.1:5353
> >>
> >>
> >>
> >> out-of-zone-additional-processing=yes
> >>
> >>
> >>
> >> webserver=yes
> >>
> >> webserver-address=172.16.6.34
> >>
> >> webserver-password=adminadminadmin
> >>
> >> webserver-port=8081
> >>
> >> webserver-print-arguments=yes
> >>
> >>
> >>
> >> #loglevel=9
> >>
> >> #log-dns-details=yes
> >>
> >> #log-dns-queries=yes
> >>
> >> #query-logging=yes
> >>
> >>
> >>
> >> version-string=Bind Resolver
> >>
> >> ..
> >>
> >> _______________________________________________
> >> Pdns-users mailing list
> >> Pdns-users at mailman.powerdns.com
> >> http://mailman.powerdns.com/mailman/listinfo/pdns-users
> >
> > _______________________________________________
> > Pdns-users mailing list
> > Pdns-users at mailman.powerdns.com
> > http://mailman.powerdns.com/mailman/listinfo/pdns-users
> >
> >
> > _______________________________________________
> > Pdns-users mailing list
> > Pdns-users at mailman.powerdns.com
> > http://mailman.powerdns.com/mailman/listinfo/pdns-users
> 
> 
> 
> 
> _______________________________________________
> Pdns-users mailing list
> Pdns-users at mailman.powerdns.com
> http://mailman.powerdns.com/mailman/listinfo/pdns-users
> 
> 
> _______________________________________________
> Pdns-users mailing list
> Pdns-users at mailman.powerdns.com
> http://mailman.powerdns.com/mailman/listinfo/pdns-users