[Pdns-users] Dealing with high concurrency and MySQL backend

[Peter van Dijk]

Hello,

On Sep 21, 2011, at 8:35 PM, Mohamed Lrhazi wrote:

> I run a resperf test against a fresh setup of
> pdns-2.9.22-10.el6.x86_64, on CentOS 6, on an ESX VM with 2vCPUs and 2
> GB RAM.
> mysql is: mysql-server-5.1.52-1.el6_0.1.x86_64, all rpms from EPEL,
> with no tweaks of configs....
> 
> The respserf is using a query file made up of all my records, about
> 50,000, duplicated several times to make it a bit over a million
> lines.
> 
> pdns crashes with error:
> 
> 5003 questions waiting for database attention. Limit is 5000, respawning
> 
> Is it possible to tweak MySQL (InnoDB) to handle such load?

It's important to realise what "such load" means. Tools like resperf are basically designed to push a server until it breaks - usually way beyond any real world load. No matter how much tweaking you do, these tools will usually be able to eventually overload a machine. There will always be some limit.

Concurrency between PowerDNS and MySQL is a factor too; http://doc.powerdns.com/performance.html provides some hints in this area. I suggest reading the whole chapter.

> What would be my options, in order to perform well under such high
> amount of concurrent queries? Is MySQL an option at all?

MySQL is a very fine choice for performance.

> Could pdns caching help? Could one warm it up somehow so as to load
> the intire data set into the cache? Would that be serious for real
> world traffic?

All forms of caching should help - MySQL query caching, PDNS query caching, PDNS packet cache. Warming it up could be done by firing resperf at it after startup but there should be no need for real world traffic. Note that some caching is default already - again, see http://doc.powerdns.com/performance-settings.html and http://doc.powerdns.com/all-settings.html to learn more.

Kind regards,
Peter van Dijk