[Pdns-users] LUA createForward() records and improvement suggestions
Otto Moerbeek
otto at drijf.net
Tue Jun 23 07:52:46 UTC 2020
On Tue, Jun 23, 2020 at 08:28:38AM +0200, Michael Rommel wrote:
> Hi Otto,
>
> thanks for the pointer! AFAICT it covers my patches as well, looks a lot more complicated, though. I'll take a closer look at it.
>
> Is there any reason, why it hasn't been merged yet? Any cases that would break that needed to be avoided?
I think the patch is more comp;icated becuase it handles more cases.
It needs a final review before merging. And as said, indepedent tests
help as well to get it merged.
-Otto
>
> Thanks,
>
> Michael.
>
> --
> Michael Rommel, Erlangen, Germany
>
> > On 23 Jun 2020, at 08:16, Otto Moerbeek <otto at drijf.net> wrote:
> >
> > On Mon, Jun 22, 2020 at 10:11:30PM +0200, Michael Rommel via Pdns-users wrote:
> >
> >>
> >> Dear all,
> >>
> >> a while ago (2020-03-01) I asked about setting up domains with LUA createForward()
> >> records.
> >>
> >> I suceeded in setting it up and found some peculiarities, which I would like to
> >> discuss here (in parallel I consider to submit PRs for some issues in Github and
> >> would appreciate guidance, whether it makes sense to open them).
> >>
> >> There are four (4) questions in this mail and sorry for the length, but I wanted
> >> to make it explicit with all possible information provided from the get-go.
> >>
> >> The setup for the proof-of-concept is a MASTER/SLAVE setup with sqlite3 as
> >> backend. The used version is 4.3.0-1pdns.bionic from
> >> http://repo.powerdns.com/ubuntu bionic-auth-43.
> >>
> >> The demo setup has essentially these domains and records (taken from the master):
> >>
> >> sqlite> select * from records;
> >> 1|1|example.com|SOA|ns1.example.com ra-dns-admin.example.com 3 10380 3600 604800 3600|86400|||0||1
> >> 2|1|example.com|NS|ns1.example.com|86400|||0||1
> >> 3|1|example.com|NS|ns2.example.com|86400|||0||1
> >> 4|1|ns1.example.com|A|104.41.128.19|86400|||0||1
> >> 5|1|ns2.example.com|A|52.148.215.179|86400|||0||1
> >> 7|1|*.11111111.1001.example.com|LUA|A "createForward()"|60|||0||1
> >> 8|1|*.22222222-2002.example.com|LUA|A "createForward()"|60|||0||1
> >> 9|2|33333333-3003.example.com|SOA|ns1.example.com ra-dns-admin.example.com 2 10380 3600 604800 3600|86400|||0||1
> >> 10|2|*.33333333-3003.example.com|LUA|A "createForward()"|60|||0||1
> >>
> >> sqlite> select * from domains;
> >> 1|example.com|||MASTER|2|
> >> 2|33333333-3003.example.com|||MASTER|2|
> >>
> >> Other tables available on request, I'll try to be as brief as possible.
> >>
> >> The intended use is a DNS resolver for approx. 200.000 devices (more
> >> later), each device shall have one of those wildcard createForward()
> >> records and an accompanying _acme-challenge TXT record to obtain a Let's
> >> Encrypt certificate for that record.
> >>
> >>
> >> Q 1: Structure of the domain/subdomains / current implementation limitations
> >> ====
> >>
> >> Currently the implementation of the LUA createForward() is in a way that
> >> accepts the wildcard only as being directly underneath the domain in
> >> question. In the example setup above, the 4.3 version:
> >>
> >> - will not resolve the record ip10203040.22222222-2002.example.com
> >> - will resolve the record ip10203040.33333333-3003.example.com
> >>
> >> because only the latter one is directly beneath the domain. In my use case
> >> that would mean to create 200.0000 additional entries in the domain table
> >> (the NS records for a proper DNS delegation can be omitted here, because
> >> all live on the same server). Each domain would only have two entries.
> >>
> >> Even with a less aggressive SOA refresh time, that would mean, that pdns
> >> would check all of those 200K domains within one hour. Since they mostly
> >> stay the same, there is no AXFR involved, but the checking imposes a load
> >> on the database and logging (tuneable of course). With PGSQL later this
> >> will certainly bearable, but I think a multi-level structure might be
> >> better suited. Hence the first patch:
> >>
> >> I suggest changing the line 616 in lua-record.cc to
> >>
> >> if(parts.size()<4) {
> >>
> >> This would retain the behaviour of accepting questions like:
> >>
> >> 192.168.1.1.33333333-3003.example.com
> >>
> >> but would enable additionally questions like:
> >>
> >> ip10203040.22222222-2002.example.com
> >> ip10203040.11111111.1001.example.com
> >>
> >> letting me subdivide the domain without the need for separate subdomains
> >> just for the resolution purpose.
> >>
> >> It would be breaking for setups where the top level domain also has a
> >> wildcard record and it is not wished that subdomains are resolved:
> >>
> >> *.example.com|LUA|A "createForward()"
> >>
> >> And ip10203040.test.example.com shall NOT be resolved. With the patch, it
> >> would.
> >>
> >> Shall I submit a PR with this or do you have better ideas for an
> >> implementation.
> >>
> >>
> >> Q 2: Does it make sense to subdivide the domain
> >> ====
> >>
> >> The patch above allows me to structure the domain like the example
> >> 1001.11111111.example.com or vice versa. This would result in
> >>
> >> ~ 850 records like 11111111.example.com, each with
> >> 1 - 10.000 records underneath it like 1001.11111111.example.com each with:
> >> *.1001.11111111.example.com LUA "createForward()" and
> >> _acme-challenge.1001.11111111.example.com TXT "token from LE"
> >> The 850 records would be full domains with their entry in the domains
> >> table, but the 10.000 entries below would not be separate domains.
> >>
> >> This means that once a new device needs a certificate, two records would be
> >> created and in the worst case a domain with 20.000 entries would be needed
> >> to AXFR by the SLAVE (or via native replication later).
> >>
> >> But the refresh would only check the SOA for 850 records between pdns and
> >> its backend db.
> >>
> >> Would you consider a different solution / structure or does that make sense
> >> to you?
> >>
> >>
> >> Q 3: SERVFAIL with special questions
> >> ====
> >>
> >> Currently there is a strange behaviour with createForward(). I would
> >> consider this a bug, but am open to corrections.
> >>
> >> The implementation skips the first two octets, then parses the remainder
> >> with sscanf. This leads to a problem, when someone asks a question like
> >>
> >> 192-168-3-4.33333333-3003.example.com
> >>
> >> which leads to a SERVFAIL, because the string returned from the function is
> >> 2.4294967295.104.4294967293 = 2 . -1 . 0x68 . -3
> >> which then cannot be put into the answer packet.
> >>
> >> ; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> +norecurse @172.24.46.11 192-168-3-4.33333333-3003.example.com
> >> ; (1 server found)
> >> ;; global options: +cmd
> >> ;; Got answer:
> >> ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 10082
> >> ;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
> >>
> >> ;; OPT PSEUDOSECTION:
> >> ; EDNS: version: 0, flags:; udp: 1232
> >> ;; QUESTION SECTION:
> >> ;192-168-3-4.33333333-3003.example.com. IN A
> >>
> >> ;; Query time: 1 msec
> >> ;; SERVER: 172.24.46.11#53(172.24.46.11)
> >> ;; WHEN: Mon Jun 22 19:46:50 UTC 2020
> >> ;; MSG SIZE rcvd: 66
> >>
> >> root:/home/rommel/configuration# tail -100 /var/log/syslog |grep pdns
> >> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Remote 172.24.46.11 wants '192-168-3-4.33333333-3003.example.com|A', do = 0, bufsize = 1232 (4096): packetcache MISS
> >> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Lua record (192-168-3-4.33333333-3003.example.com|A) reported: Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address
> >> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Exception building answer packet for 192-168-3-4.33333333-3003.example.com/A (Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address) sending out servfail
> >>
> >> My patch suggestion would be to add in the check for values above 255, like:
> >> if(sscanf(parts[0].c_str()+2, "%02x%02x%02x%02x", &x1, &x2, &x3, &x4)==4) {
> >> if(x1<=0xff && x2<=0xff && x3<=0xff && x4<=0xff)
> >> return std::to_string(x1)+"."+std::to_string(x2)+"."+std::to_string(x3)+"."+std::to_string(x4);
> >> }
> >>
> >> Would you agree that this might be better to fall through to returning "0.0.0.0"
> >> rather than SERVFAIL?
> >>
> >>
> >> Q 4: Adding the (forgotten) ability to parse the dash delimited decimal questions
> >> ====
> >>
> >> In addition to the hexadecimal notation, I would really like to see the
> >> proper resolving of entries like 192-168-3-4.33333333-3003.example.com.
> >>
> >> These additional lines below the hex portion would allow this:
> >>
> >> if(sscanf(parts[0].c_str(), "%u-%u-%u-%u", &x1, &x2, &x3, &x4)==4) {
> >> if(x1<=0xff && x2<=0xff && x3<=0xff && x4<=0xff)
> >> return std::to_string(x1)+"."+std::to_string(x2)+"."+std::to_string(x3)+"."+std::to_string(x4);
> >> }
> >>
> >> Anyone interested in this PR?
> >>
> >>
> >> Thanks everybody, who has read through this monster to this point. Any
> >> suggestions or corrections or improvements.
> >>
> >> Michael.
> >
> > I must say I did not read your complete post in all detail, but you
> > should take a look at (and preferably test!)
> > https://github.com/PowerDNS/pdns/pull/9101
> >
> > I believe it covers a lot (all?) of your issues.
> >
> > -Otto
> >
> >>
> >>
> >>
> >> --
> >> Michael Rommel, Erlangen, Germany
> >>
> >> _______________________________________________
> >> Pdns-users mailing list
> >> Pdns-users at mailman.powerdns.com
> >> https://mailman.powerdns.com/mailman/listinfo/pdns-users
>
More information about the Pdns-users
mailing list