[Pdns-users] TCP Queries stop - can only fix with restart?

[bert hubert]

On Tue, Oct 18, 2005 at 12:55:20AM -0400, Matt Gibson wrote:
> Each box is also acting as a recursive server using PowerDNS's internal 
> recursion server. Powerdns is listening on about 250 ip's on both boxes, 
> TCP and UDP queries.

Matt,

Thanks for the excellent bug report!

Two things stand out - you use the recursor bypass and you have 250 IP
addresses PowerDNS listens on.

Do you have a lot of recursion traffic? Perhaps from mail servers?

> Oct 17 23:02:10 ns1 pdns[5800]: TCP nameserver had error, cycling 
> backend:EOF trying to get length of answer from remote TCP server
> Oct 17 23:02:21 ns1 pdns[5800]: TCP server is without backend 
> connections, launching

Do you get a lot of these messages? Or only one? Does this message come just
befor things fail? Or during?

> At least I think that error has something to do with it.

It seems probable.

> It seems that I'm able to simply restart powerdns and the issue goes 
> away, but that can't be the proper solution for this.

Indeed.

> At the time that the server died out, I ran netstat -an, and in 
> condensed form, this was the result.
> 
> - 1039 total tcp connections at the time

Can you check next time how many of those TCP connections were from the
nameserver to the recursor?

> Has anyone encountered anything like this before? Anyone have any ideas
> on how to fix it? My boss is going nuts and so am I trying to figure 
> this out! :)

This is the first report like this. Most nameservers don't get a lot of
TCP queries, so that part of PowerDNS might hide bugs. I've seen Exchange
servers do a lot of TCP traffic to nameservers, so I'm interested in hearing
of what is using your nameservers as recursor.

Thanks.

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services