[dnsdist] dnsdist using loopback address as source address for queries

[Remi Gacogne]

On 10/29/21 13:04, Adam Bishop via dnsdist wrote:
> On 29 Oct 2021, at 11:33, Remi Gacogne via dnsdist <dnsdist at mailman.powerdns.com> wrote:
>> would you mind getting the output of "lsof -n -p <pid of dnsdist>" while the issue is still happening? A full backtrace with gdb might also be good to have
> 
> Sure that's fine, output follows. I've uploaded the gdb output as it's quite long. We've got three other instances to take the load, so I'm happy to keep it running as is for the moment.

Thanks! So we see from the strace output that dnsdist is calling send() 
(which becomes the sendto() syscall) to send the queries to the backend, 
using the connected UDP sockets:

[pid  1384] sendto(11, "U\251\0 
\0\1\0\0\0\0\0\1\4xxxx\4lbdn\4virt\2ja\3net\0\0\6\0\1\0\0)\20\0\0\0\0\0\0\30\0\10\0\24\0\2\200\0 
\1\0060\0<P\30\0\0\0\0\0\0\1\227", 74, 0, NULL, 0) = 74
[...]
[pid  1382] sendto(11, "V\251\0 
\0\1\0\0\0\0\0\1\4xxxx\4lbdn\4virt\2ja\3net\0\0\6\0\1\0\0)\20\0\0\0\0\0\0\f\0\10\0\10\0\1 
\0\302S\7\305", 62, 0, NULL, 0) = 62

But we then see in the lsof output that these IPv6 sockets (the 11 is 
used above, but the others are the same) are connected using ::1 as 
source, while the IPv4 ones are not:

> dnsdist 1351 dnsdist   11u     IPv6              28637      0t0      UDP [::1]:35903->[2001:630:1:160::195]:domain
> dnsdist 1351 dnsdist   13u     IPv4              28640      0t0      UDP 194.83.7.197:45941->194.83.7.195:domain
> dnsdist 1351 dnsdist   15u     IPv6              28650      0t0      UDP [::1]:51827->[2001:630:1:170::67]:domain
> dnsdist 1351 dnsdist   17u     IPv4              28653      0t0      UDP 194.83.7.197:57454->193.63.72.67:domain

That explains what you are seeing, but I don't understand how we end up 
with these..
Would you mind checking that you still have IPv6 addresses on that 
interface? I see you still have some on the incoming interface, though, 
since we receive a query over IPv6 on file descriptor 18 in your strace 
output. Any events in the system logs that looks like the IP addresses 
might have changed at some point? Anything in the dnsdist logs looking 
like a reconnect (error while writing to the backend, ..)? We should not 
reconnect unless send() failed with EINVAL or ENODEV, which is not 
supposed to happen in your case since you don't set the source interface.

Best regards,

Remi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.powerdns.com/pipermail/dnsdist/attachments/20211029/42d63c42/attachment.sig>