[Pdns-users] Dnstap and kafka

[Brian Candler]

On 13/01/2021 14:58, Michael Chisina via Pdns-users wrote:
> I want to frame stream powerdns  recursor DNS query and response using 
> dnstap to an apache kafka remote server (202.20.20.1).
> # what are the configurations needed on recursor?

dnstap doesn't talk to kafka.

You'll need to run some middleware software which accepts dnstap 
<https://dnstap.info/>-formatted messages, and writes them to kafka - 
which could be in raw dnstap form, or decoded into some other form (e.g. 
JSON).  Google "dnstap kafka" for some options, or write your own.

Then you configure pdns-recursor to send dnstap messages to this 
middleware server.

If you run the middleware on the same server as pdns-recursor, then they 
can communicate over a unix domain socket.  If you want to communicate 
to a remote server over TCP, then you will need a sufficiently new 
version of libfstrm which supports this. In practice this means that if 
you are using Ubuntu then you need 20.04 not 18.04. See this thread:

https://mailman.powerdns.com/pipermail/pdns-users/2020-June/026724.html


> # what is dns message schema(s) format for the database creation?

kafka isn't a database.  If you want to write these messages to a 
database, then that's an additional step.  You'll need some more 
software which reads messages from kafka, decodes them, and writes to a 
database in some schema that you define.  Beware that a busy recursor 
can generate a very large volume of messages, so you might want to 
aggregate them first.

HTH,

Brian.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20210113/742ca880/attachment.htm>