[Pdns-users] LUA createForward() records and improvement suggestions

Michael Rommel rommel at layer-7.net
Tue Jun 23 06:28:38 UTC 2020


Hi Otto,

thanks for the pointer! AFAICT it covers my patches as well, looks a lot more complicated, though. I'll take a closer look at it.

Is there any reason, why it hasn't been merged yet? Any cases that would break that needed to be avoided?

Thanks,

  Michael.

-- 
Michael Rommel, Erlangen, Germany

> On 23 Jun 2020, at 08:16, Otto Moerbeek <otto at drijf.net> wrote:
> 
> On Mon, Jun 22, 2020 at 10:11:30PM +0200, Michael Rommel via Pdns-users wrote:
> 
>> 
>> Dear all,
>> 
>> a while ago (2020-03-01) I asked about setting up domains with LUA createForward()
>> records.
>> 
>> I suceeded in setting it up and found some peculiarities, which I would like to
>> discuss here (in parallel I consider to submit PRs for some issues in Github and 
>> would appreciate guidance, whether it makes sense to open them).
>> 
>> There are four (4) questions in this mail and sorry for the length, but I wanted 
>> to make it explicit with all possible information provided from the get-go.
>> 
>> The setup for the proof-of-concept is a MASTER/SLAVE setup with sqlite3 as
>> backend. The used version is 4.3.0-1pdns.bionic from
>> http://repo.powerdns.com/ubuntu bionic-auth-43. 
>> 
>> The demo setup has essentially these domains and records (taken from the master):
>> 
>> sqlite> select * from records;
>> 1|1|example.com|SOA|ns1.example.com ra-dns-admin.example.com 3 10380 3600 604800 3600|86400|||0||1
>> 2|1|example.com|NS|ns1.example.com|86400|||0||1
>> 3|1|example.com|NS|ns2.example.com|86400|||0||1
>> 4|1|ns1.example.com|A|104.41.128.19|86400|||0||1
>> 5|1|ns2.example.com|A|52.148.215.179|86400|||0||1
>> 7|1|*.11111111.1001.example.com|LUA|A "createForward()"|60|||0||1
>> 8|1|*.22222222-2002.example.com|LUA|A "createForward()"|60|||0||1
>> 9|2|33333333-3003.example.com|SOA|ns1.example.com ra-dns-admin.example.com 2 10380 3600 604800 3600|86400|||0||1
>> 10|2|*.33333333-3003.example.com|LUA|A "createForward()"|60|||0||1
>> 
>> sqlite> select * from domains;
>> 1|example.com|||MASTER|2|
>> 2|33333333-3003.example.com|||MASTER|2|
>> 
>> Other tables available on request, I'll try to be as brief as possible.
>> 
>> The intended use is a DNS resolver for approx. 200.000 devices (more
>> later), each device shall have one of those wildcard createForward()
>> records and an accompanying _acme-challenge TXT record to obtain a Let's
>> Encrypt certificate for that record.
>> 
>> 
>> Q 1: Structure of the domain/subdomains / current implementation limitations
>> ====
>> 
>> Currently the implementation of the LUA createForward() is in a way that
>> accepts the wildcard only as being directly underneath the domain in
>> question. In the example setup above, the 4.3 version:
>> 
>> - will not resolve the record ip10203040.22222222-2002.example.com
>> - will resolve the record ip10203040.33333333-3003.example.com
>> 
>> because only the latter one is directly beneath the domain. In my use case
>> that would mean to create 200.0000 additional entries in the domain table
>> (the NS records for a proper DNS delegation can be omitted here, because
>> all live on the same server). Each domain would only have two entries.
>> 
>> Even with a less aggressive SOA refresh time, that would mean, that pdns
>> would check all of those 200K domains within one hour. Since they mostly
>> stay the same, there is no AXFR involved, but the checking imposes a load
>> on the database and logging (tuneable of course). With PGSQL later this
>> will certainly bearable, but I think a multi-level structure might be
>> better suited. Hence the first patch:
>> 
>> I suggest changing the line 616 in lua-record.cc to
>> 
>>    if(parts.size()<4) {
>> 
>> This would retain the behaviour of accepting questions like:
>> 
>>  192.168.1.1.33333333-3003.example.com
>> 
>> but would enable additionally questions like:
>> 
>>  ip10203040.22222222-2002.example.com
>>  ip10203040.11111111.1001.example.com
>> 
>> letting me subdivide the domain without the need for separate subdomains
>> just for the resolution purpose.
>> 
>> It would be breaking for setups where the top level domain also has a
>> wildcard record and it is not wished that subdomains are resolved:
>> 
>> *.example.com|LUA|A "createForward()"
>> 
>> And ip10203040.test.example.com shall NOT be resolved. With the patch, it
>> would.
>> 
>> Shall I submit a PR with this or do you have better ideas for an
>> implementation.
>> 
>> 
>> Q 2: Does it make sense to subdivide the domain
>> ====
>> 
>> The patch above allows me to structure the domain like the example
>> 1001.11111111.example.com or vice versa. This would result in
>> 
>> ~ 850 records like 11111111.example.com, each with 
>> 1 - 10.000 records underneath it like 1001.11111111.example.com each with:
>>   *.1001.11111111.example.com LUA "createForward()" and 
>>  _acme-challenge.1001.11111111.example.com TXT "token from LE"
>> The 850 records would be full domains with their entry in the domains
>> table, but the 10.000 entries below would not be separate domains.
>> 
>> This means that once a new device needs a certificate, two records would be
>> created and in the worst case a domain with 20.000 entries would be needed
>> to AXFR by the SLAVE (or via native replication later).
>> 
>> But the refresh would only check the SOA for 850 records between pdns and
>> its backend db.
>> 
>> Would you consider a different solution / structure or does that make sense
>> to you?
>> 
>> 
>> Q 3: SERVFAIL with special questions
>> ==== 
>> 
>> Currently there is a strange behaviour with createForward(). I would
>> consider this a bug, but am open to corrections.
>> 
>> The implementation skips the first two octets, then parses the remainder
>> with sscanf. This leads to a problem, when someone asks a question like
>> 
>> 192-168-3-4.33333333-3003.example.com
>> 
>> which leads to a SERVFAIL, because the string returned from the function is 
>> 2.4294967295.104.4294967293 = 2 . -1 . 0x68 . -3
>> which then cannot be put into the answer packet.
>> 
>> ; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> +norecurse @172.24.46.11 192-168-3-4.33333333-3003.example.com
>> ; (1 server found)
>> ;; global options: +cmd
>> ;; Got answer:
>> ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 10082
>> ;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
>> 
>> ;; OPT PSEUDOSECTION:
>> ; EDNS: version: 0, flags:; udp: 1232
>> ;; QUESTION SECTION:
>> ;192-168-3-4.33333333-3003.example.com. IN A
>> 
>> ;; Query time: 1 msec
>> ;; SERVER: 172.24.46.11#53(172.24.46.11)
>> ;; WHEN: Mon Jun 22 19:46:50 UTC 2020
>> ;; MSG SIZE  rcvd: 66
>> 
>> root:/home/rommel/configuration# tail -100 /var/log/syslog |grep pdns
>> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Remote 172.24.46.11 wants '192-168-3-4.33333333-3003.example.com|A', do = 0, bufsize = 1232 (4096): packetcache MISS
>> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Lua record (192-168-3-4.33333333-3003.example.com|A) reported: Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address
>> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Exception building answer packet for 192-168-3-4.33333333-3003.example.com/A (Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address) sending out servfail
>> 
>> My patch suggestion would be to add in the check for values above 255, like:
>>  if(sscanf(parts[0].c_str()+2, "%02x%02x%02x%02x", &x1, &x2, &x3, &x4)==4) {
>>    if(x1<=0xff && x2<=0xff && x3<=0xff && x4<=0xff)
>>      return std::to_string(x1)+"."+std::to_string(x2)+"."+std::to_string(x3)+"."+std::to_string(x4);
>>  }
>> 
>> Would you agree that this might be better to fall through to returning "0.0.0.0" 
>> rather than SERVFAIL?
>> 
>> 
>> Q 4: Adding the (forgotten) ability to parse the dash delimited decimal questions
>> ====
>> 
>> In addition to the hexadecimal notation, I would really like to see the
>> proper resolving of entries like 192-168-3-4.33333333-3003.example.com.
>> 
>> These additional lines below the hex portion would allow this:
>> 
>>        if(sscanf(parts[0].c_str(), "%u-%u-%u-%u", &x1, &x2, &x3, &x4)==4) {
>>          if(x1<=0xff && x2<=0xff && x3<=0xff && x4<=0xff)
>>            return std::to_string(x1)+"."+std::to_string(x2)+"."+std::to_string(x3)+"."+std::to_string(x4);
>>        }
>> 
>> Anyone interested in this PR?
>> 
>> 
>> Thanks everybody, who has read through this monster to this point. Any
>> suggestions or corrections or improvements.
>> 
>>  Michael.
> 
> I must say I did not read your complete post in all detail, but you
> should take a look at  (and preferably test!) 
> https://github.com/PowerDNS/pdns/pull/9101
> 
> I believe it covers a lot (all?) of your issues.
> 
>    -Otto
> 
>> 
>> 
>> 
>> -- 
>> Michael Rommel, Erlangen, Germany
>> 
>> _______________________________________________
>> Pdns-users mailing list
>> Pdns-users at mailman.powerdns.com
>> https://mailman.powerdns.com/mailman/listinfo/pdns-users



More information about the Pdns-users mailing list