[Pdns-users] SNAT and notify messages

[Michael Hallager]

Are you using double NAT? If so then its likely to double your issues.

I recommend you fix your underlying issues now by getting all your 
servers onto the same net block or net blocks which can route between 
each other without NAT.

On 2022-11-18 11:37, ch via Pdns-users wrote:
> Hi PDNS users, I need your advice:  (Note: there's a TL;DR: section at
> the bottom)
> 
> We've set up three nameservers (ns1, ns2 and ns3) with two ip
> addresses each: an internal 10.x.x.x and an external public ip.
> 
> Each server runs dnsdist, pdns_recursor, and two copies of
> pdns_server, like this:
> 
>     dnsdist -> pdns_recursor -> pdns_server [internal]
>             \
>               pdns_server [external]
> 
>  	* Dnsdist is listening on 127.0.0.1, 10.x.x.x, and the external
> public ip, all on port 53.
>  	* pdns_recursor is listening on 127.0.0.2
>  	* pdns_server (internal) is listening on 127.0.0.3
>  	* pdns_server (external) is listening on 127.0.0.4
>  	* systemd_resolved is listening on 127.0.0.53 to satisfy local dns
> requests (Ubuntu 20.04 default configuration, with 127.0.0.2 as its
> resolver)
> 
> Requests come in to dnsdist, and based on their source address they're
> forwarded to 127.0.0.2 (internal) or 127.0.0.4 (external).
> Pdns_recursor forwards requests to 127.0.0.3 (pdns_server internal)
> for zones listed in the forward zones file and recursively answers the
> rest.
> 
> So far, so good; requests come in, replies go out, and external
> clients can't abuse our recursor or see our internal dns entries,
> yayyy!
> 
> =====
> 
> Here's where things start go off the rails:
> 
> Our internal network grew "organically", and for reasons lost in the
> mists of time, we've got NAT gateways in our network, one of which is
> between ns3 and the other two name servers.
> 
>  	* When a zone is updated on ns1, it sends a notify to ns3, but the
> source ip is changed by the NAT gateway.
>  	* The notify-acknowledgement is sent back through the NAT gateway
> and ns1 gets it, since the gateway expected the response.
>  	* Ns3 then sends an SOA request to the NAT gateway (as that's where
> the notify came from), but it's lost because it's not related to the
> previous conversation.
>  	* Pdns_recursor seems to eat Notify and AXFR messages, so we've told
> Dnsdist to direct those to 127.0.0.3
> 
> We've temporarily worked around the problem by adding an iptables rule
> to ns3 that says 'redirect packets sent to "port 53 on the gateway" to
> ns1', and it works.
> 
> ======
> 
> Things are now going further off the rails here:
> 
>  	* Now we've added a bunch of independent sub-zones on small name
> servers in different parts of our company, and they're going to be
> using ns1, ns2 and ns3 as their secondary servers, whichever is
> closest.
>  	* Notifies are going to be flying all over, and packets from some of
> them will be going through the same NAT gateway that ns1 uses to get
> to ns3.
>  	* Because of that, the iptables rule is going to mess things up, as
> it assumes outgoing DNS requests to the NAT gateway should really go
> to ns1's internal address.
> 
> Is there a way to get the SOA request from ns3 to go to the right
> place using pdns or dnsdist?
> 
> ======
> 
> Now we're so far off the rails, we're in the middle of a cornfield:
> 
>  	* The internal and external pdns servers have different zone files
> (internal/external ip addresses, some hosts not listed in the external
> zones).
>  	* We're using the internal ips on ns1,2,3 for transferring internal
> zones, and the external ips for external ones.
>  	* We're attempting to use NetmaskGroupRule [1] with src=false to
> have dnsdist direct requests internally/externally based on that.
>  	* The independent sub-zone name servers will be notifying the
> ns1,2,3 on both internal and external ips, and the NAT is going to
> mess up the source ips, iptables can't handle this.
> 
> Can we use DoH/DoT to establish a TCP connection for the NOTIFY and
> reuse it for the SOA and AXFR?  The NAT respects open TCP connections
> much more than UDP conversations.
> 
> Should we manually add entries to the zone metadata to specify where
> zones are really hosted?  I really wanted to use the auto-secondary
> feature, but sometimes we can't have nice things.  :-/
> 
> Oh god, please don't tell me I have to set up a VPN between all the
> name servers.... :-(
> 
> Anyhow, I'd love to hear someone write a happy ending to this story.
> 
> ======
> 
> TL;DR: How do you get notifies + zone transfers to work when the
> source ip addresses of NOTIFY packets are unreliable?
> 
> Thanks!
> 
> --
> CH (ch-and-pdns-users at ch.pkts.ca)
> 
> Links:
> ------
> [1] https://dnsdist.org/rules-actions.html#NetmaskGroupRule
> _______________________________________________
> Pdns-users mailing list
> Pdns-users at mailman.powerdns.com
> https://mailman.powerdns.com/mailman/listinfo/pdns-users

-- 
Net Trust Ltd
Internet Servers and Network Systems Administration
p: (06) 374 0880 | (09) 839 1000
m: 021 963 878
e: michael at nettrust.nz
w: nettrust.nz