[Pdns-users] An important update on new PowerDNS Products

bert hubert bert.hubert at powerdns.com
Sat May 14 13:10:22 UTC 2016

On Fri, May 13, 2016 at 07:59:57PM -0400, Ciro Iriarte wrote:
> Out of curiosity, how does this part of the platform work?:

Hi Ciro,

In general, I don't want to spam the pdns-users people with information
about the PowerDNS Platform, as outlined on
https://www.powerdns.com/platform.html since it is not part of the open
source stuff, so most of the community won't have any use for the

(the story of our non-open source work is on

But let me explain how this hangs together and what is part of the open
source version. The PowerDNS Recursor 4.0.0 and dnsdist 1.0.0 have the
ability to emit a stream of protobufs over TCP/IP. In case of the Recursor,
this also has the 'policy reason' why a request was intercepted by the RPZ

To receive that stream, use something like xinetd to listen on a TCP/IP port
and store the data to a file. It can then be processed by any tool that can
understand Protobuf. The schema is here:

In the very near future this will also be able to emit standard dnstap

> Long term full query logging & rapid searching
>    - Dimensioned at a trillion queries/day (1000 billion) on commodity
>    hardware with long term retention
>    - For security research, lawful intercept/data retention requirements,
>    customer intelligence, quality assurance/diagnostics

This describes our protobuf receiver 'dstore' which through some clever
programming techniques can store trillions of DNS messages and serve them up
again reasonably quickly. It is not a generic database, but it is really
fast and nearly maintenance free and has no further dependencies (so you
don't need to be a "big data engineer" to benefit from it).

This can be very useful to investigate customer complaints of DNS slowness,
or that a domain was down etc. It is also extremely powerful for finding
infected users. A commandline like:

$ dgrep t=week pr=spamhaus-dbl | jq ".items[].origRequestor"  | sort | uniq -c \
| sort -rn | head -10

.. will find in a few seconds the top-10 IP addresses that over the past week 
had the most queries  blocked by the 'spamhaus-dbl' RPZ. The output of dgrep is JSON, 
easily queried and selected by jq.

But again - I don't want to promote our commercial Platform offering here
too much.  For the open source world, you should be able to bake up a
solution based on elastic search, kibana etc that ingests our protobufs. 


More information about the Pdns-users mailing list