[dnsdist] dnsdist using loopback address as source address for queries

[Remi Gacogne]

On 10/29/21 15:32, Adam Bishop via dnsdist wrote:
> On 29 Oct 2021, at 13:38, Remi Gacogne via dnsdist <dnsdist at mailman.powerdns.com> wrote:
>> Would you mind checking that you still have IPv6 addresses on that interface? I see you still have some on the incoming interface, though, since we receive a query over IPv6 on file descriptor 18 in your strace output.
> 
> Yeah, it's a single interface system so the ingress/egress addresses are the same.
> 
>> Any events in the system logs that looks like the IP addresses might have changed at some point?
> 
> The only thing in dmesg is an SElinux policy update triggered by dnf, but no AVC entries that would indicate that being the cause. NetworkManager hasn't written anything to its own log since boot, which is normal.
> 
>> Anything in the dnsdist logs looking like a reconnect (error while writing to the backend, ..)? We should not reconnect unless send() failed with EINVAL or ENODEV, which is not supposed to happen in your case since you don't set the source interface.
> 
> There's some normal terminations when the backend is reloaded for config changes, e.g.
> 
>    dnsdist[1351]: Timeout while waiting for the health check response from backend [2001:630:1:170::67]:53
>    dnsdist[1351]: Marking downstream [2001:630:1:170::67]:53 as 'down'
>    dnsdist[1351]: Timeout while waiting for the health check response from backend 193.63.72.67:53
>    dnsdist[1351]: Marking downstream 193.63.72.67:53 as 'down'
>    dnsdist[1351]: Marking downstream [2001:630:1:170::67]:53 as 'up'
>    dnsdist[1351]: Marking downstream 193.63.72.67:53 as 'up'
> 
> But that's it - the only other messages are "Packet from:"

Alright, thanks for checking! I have to say I'm running a bit out of 
ideas. We only bind to a specific source address if one is supplied in 
the configuration, and that's not the case here. The field holding that 
source address is not updated at runtime, and I would be surprised to 
see a memory corruption touching only that field. You might actually be 
able to check that via gdb, by attaching to the process, selecting the 
right thread with 'thread 7' (it needs to be one of these in 
responderThread() for the right backend, which is 7 or 9 in your current 
process based on the gdb output) then 'up' three times to reach the 
responderThread frame, and 'print *dss' to see the content of the structure.
Even if that field is corrupted, we would need to reconnect which 
normally does happen only the errors I mentioned earlier, and we don't 
have these in your logs. It also happens on a health-check failure when 
'reconnectOnFailure=true' is set on a backend, but that's not the case 
in your configuration. Actually, I wonder if that would make a 
difference, do you think you could try setting that option on one of the 
IPv6 backends?

Best regards,

Remi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.powerdns.com/pipermail/dnsdist/attachments/20211029/a5db9e8b/attachment.sig>