[dnsdist] dnsdist performance

[Nico]

Hi!
We found dnsdist very useful and interesting and using it in a couple of
configurations.
Now we are testing dnsdist to find the best tuning for our setup,
and we find the performance is not what we expected (based on what we saw
in presentations)

We can't get more than 120/150 kqps
We don't think it's network related, and doesn't seem to be interface or
udp stack.

If we test whith only

addAction(AllRule(),DropAction())
We get ~600kqps

with some answer
addAction(AllRule(),RCodeAction(dnsdist.REFUSED))
we get ~450kqps

but when we enable cache, and real queries/answers we get stuck at ~120kqps.
(numbers and test details below.)

The queries are a repeated sequence, so we have 100% cache hit.

Is this a real limit? or there is something else we can tune?

Thanks!

----------------------------------------------------
Our config:
dnsdist-1.1.0 from rpm (also compiled, no difference)
Centos 7 kernel 3.10.0
32 cores  CPU E5-2660/ 32GB ram
1gb card.

we use resperf and drool to test dns.
nuutcp and iperf3 to test network  (reporting 900mbs udp and 900Kpps udp
150bytes pkts)
and netdata and graphite to see usage/errors/qps, etc.

The tests.
Always 12 listeners (addLocal)
===============================================
addAction(AllRule(),DropAction())
drool
sending ~ 900kqps
dnsdist web reports: ~650kqps
udp.inerrrors  small
=================================================
addAction(AllRule(),RCodeAction(dnsdist.REFUSED))
drool
sending ~460Kqps
dnsdist report ~450kqps
udp.inerrrors  ~0 whith sporadic high lost peak

multiple resperf:
Sending 650 Kqps
dnsdist reports ~400kqps
Udp.inerrors ~250 kpps

sending 300 kqps,  reports ~300kqps, lost responses: negligible
udp.inerrors ~0
Recv-Q ~0 Send-Q 0-5000
========================================================
Full config:
12 addLocal, small cache, many backend servers (almost not used)

cache = newPacketCache(1000, 86400, 0, 60, 60)
getPool("dns-dist1"):setCache(cache)
addPoolRule({"0.0.0.0/0"}, "dns-dist1")

setServerPolicy(firstAvailable)

Seding more than 140kqps Recv-Q grows, and queries get lost.
DNSDIST web interface and graphite counters report between 120kqps and
140kqps
never more.

Netstat output
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
udp   14006016   1536 dns-dist1-e:domain    0.0.0.0:*
udp   16487424    768 dns-dist1-e:domain    0.0.0.0:*
udp   16766208   3328 dns-dist1-e:domain    0.0.0.0:*
udp   14918400   9472 dns-dist1-e:domain    0.0.0.0:*
udp   16750848   2048 dns-dist1-e:domain    0.0.0.0:*
udp   13962240    768 dns-dist1-e:domain    0.0.0.0:*
udp   16637184      0 dns-dist1-e:domain    0.0.0.0:*
udp   12026880   2048 dns-dist1-e:domain    0.0.0.0:*
udp   15539712    768 dns-dist1-e:domain    0.0.0.0:*
udp   16772352      0 dns-dist1-e:domain    0.0.0.0:*
udp   10476288    768 dns-dist1-e:domain    0.0.0.0:*
udp   15005184      0 dns-dist1-e:domain    0.0.0.0:*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.powerdns.com/pipermail/dnsdist/attachments/20170403/b0304b5a/attachment.html>