[Pdns-users] sudden load spikes

[Eric Hendrickson]

Norbert Sendetzky wrote:

> On Thursday 09 December 2004 03:56, Eric Hendrickson wrote:
>
> >I have about 2 million records in a mysql database. Everything's been
> >working beautifully until today. Suddenly pdns jumps to 99.9 percent CPU
> >when the query load gets up to more than 15 or so queries/sec. I've
> >never had this problem before with loads up to 100 or so queries/sec.
> >and a similar number of database entries.
>
> >We add about 500 to 700 A records daily and I'm wondering if we've
> >reached some sort of limit with MySQL/pdns. When q/sec is low, I get
> >response very rapidly but it quickly falls apart as q/sec goes up.
>
> >Right now I'm only answering about 20 of my queries. Ouch.
>
>
> I assume restarting pdns didn't help.
> What version of pdns do you use?
> Is there something suspicious in the logs (maybe after increasing the
> loglevel)?
>
>
> Norbert
>
Hello Norbert. I'm running 2.9.16 on RedHat 9 with MySQL 3.23.58 
patched. I've learned that our web folks deployed a new management gui 
yesterday that allowed the SOA record's 'content' field to be 
overwritten. Once I fixed that record, everything is fine.

It was interesting though, that with the bad SOA record the service 
would answer about 20 percent of its queries. The CPU load would cycle 
from being maxed out and then back down to its normal load of less than 
2 percent utilization. I did learn that PowerDNS does a very good job at 
utilizing multiple CPUs. MySQL was also getting hammered pretty well. It 
would be interesting to see a debug and learn what exactly PowerDNS was 
trying to do when it was maxing out the CPUs.

I need to read more but I assume it's possible to have multiple backends 
running simultaneously. This would allow me to run SOA and other 
critical operation records from a separate database altogether, perhaps 
even a text file.

Eric