[Pdns-users] API-RECTIFY set to 1 but zone does not rectify
Eric Beck
ericbeck at cadns.ca
Wed Nov 22 16:14:22 UTC 2017
Hello Peter et al,
(I'm not going to bother changing everything to mask the actual domain,
etc., only API key, because it's just a domain we own used for testing
right now, so DNSSEC can actually be verified at
http://dnsviz.net/d/whon.ca/dnssec/).
We have the API listening on the actual IP address of the master
(NATIVE), mariadb gmysql backend, pdns now 4.1.0 rc3
I'm going to list the steps to reproduce this
STEP 1 - secure zone, submit DNSKEY/DS to registry, make sure zone is
rectified and verify
------
# pdnsutil show-zone whon.ca
Nov 22 08:59:46 Reading random entropy from '/dev/urandom'
This is a Native zone
Metadata items:
API-RECTIFY 1
SOA-EDIT-API DEFAULT
Zone has hashed NSEC3 semantics, configuration: 1 0 1 ab
keys:
ID = 2 (CSK), flags = 257, tag = 6782, algo = 13, bits = 256 Active
( ECDSAP256SHA256 )
CSK DNSKEY = whon.ca. IN DNSKEY 257 3 13
2u+IArWXeZtl797oX/9hC6bD+5neR0Ya8IYUqY+zeTTufylT0exzMyZEwmq+R9kU11qFJWvlqoOIA9vYfkN3hw==
; ( ECDSAP256SHA256 )
DS = whon.ca. IN DS 6782 13 1 7244b544ac2bed64083599fd1308bccad310a45d ;
( SHA1 digest )
DS = whon.ca. IN DS 6782 13 2
5bd0c3ca739acc8b84da365284e766cdbf0cae5f8ffc9bc765fb9b6dd01df57b ; (
SHA256 digest )
DS = whon.ca. IN DS 6782 13 4
ef8e414bca900c64cdfc35e24f11102b576381796f5217381612c88ee57354eb12023f8a43f2719608a1979546fd30a4
; ( SHA-384 digest )
and as above: http://dnsviz.net/d/whon.ca/dnssec/
OUTPUT of records for whon.ca after rectification
-------------------------------------------------
https://pastebin.com/raw/5W9UUfnN
As you can see from the above all hashes are present in the ordername
column in this SELECT from the domainmetadata table.
+----+-----------+--------------+----------+
| id | domain_id | kind | content |
+----+-----------+--------------+----------+
| 2 | 6890 | NSEC3PARAM | 1 0 1 ab |
| 7 | 6890 | API-RECTIFY | 1 |
| 8 | 6890 | SOA-EDIT-API | DEFAULT |
+----+-----------+--------------+----------+
STEP 2
-------
>From another server, run the curl command to modify one of the records
in the whon.ca zone. This is done using a json file. We've chosen to
just modify the ttl of the bravo.whon.ca CNAME record. We should see
the zone be rectified afterwards, and of course, the SOA serial updated.
Here is the content of the json file, (called change.whon.ca)
-----------------
{ "rrsets":
[
{
"name": "bravo.whon.ca.",
"type": "CNAME",
"ttl": 50,
"changetype": "REPLACE",
"records":
[
{
"content": "whon.ca.",
"disabled": false
}
]
}
]
}
---------------------
Here is the curl command, (which works perfectly fine without the
API-RECTIFY set to 1) and of course the API is running on 158.69.76.158
( in pdns.conf webserver-address=158.69.76.158 )
/bin/curl -v -X PATCH --data @/home/centos/curl/change.whon.ca -H
'Accept: application/json' -H 'Content-Type: application/json' -H
'X-API-Key: HIDDEN'
http://158.69.76.158:8081/api/v1/servers/localhost/zones/whon.ca |jq .
This produces the following on screen results
----------------------------------------------
* About to connect() to 158.69.76.158 port 8081 (#0)
* Trying 158.69.76.158...
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Connected to 158.69.76.158 (158.69.76.158) port 8081 (#0)
> PATCH /api/v1/servers/localhost/zones/whon.ca HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 158.69.76.158:8081
> Accept: application/json
> Content-Type: application/json
> X-API-Key: HIDDEN
> Content-Length: 254
>
} [data not shown]
* upload completely sent off: 254 out of 254 bytes
100 254 0 0 100 254 0 25 0:00:10 0:00:10 --:--:--
0< HTTP/1.1 500 Internal Server Error
< Connection: close
< Content-Length: 34
< Content-Type: application/json
< Server: PowerDNS/4.1.0-rc3
<
{ [data not shown]
100 288 100 34 100 254 3 25 0:00:11 0:00:10 0:00:01
0
* Closing connection 0
{
"error": "Internal Server Error"
}
--------------------------
In looking at the journalctl on the target server,
Nov 22 09:46:43 HTTP ISE for "/api/v1/servers/localhost/zones/whon.ca":
Exception: GSQLBackend unable to update ordername and auth for domain_id
6890: Could not execute mysql statement: update records set
ordername=?,auth=? where domain_id=? and name=? and disabled=0: Lost
connection to MySQL server during query
Nov 22 09:46:43 ns5.cadns.ca mysqld[4616]: 2017-11-22 9:46:43
140447722108672 [Warning] Aborted connection 1213 to db: 'pdns_db' user:
'pdns_user' host: 'localhost' (Got an error reading communication packets)
-------------------------------------
And, on the target server, nothing is changed at all.
STEP 3 (verify that it works without the API-RECTIFY set to 1 in the
domainmetadata table.) Set the API-RECTIFY to 0
+----+-----------+--------------+----------+
| id | domain_id | kind | content |
+----+-----------+--------------+----------+
| 2 | 6890 | NSEC3PARAM | 1 0 1 ab |
| 7 | 6890 | API-RECTIFY | 0 |
| 8 | 6890 | SOA-EDIT-API | DEFAULT |
+----+-----------+--------------+----------+
STEP 4 run the same curl command again (no errors, and here is the the
onscreen output)
* About to connect() to 158.69.76.158 port 8081 (#0)
* Trying 158.69.76.158...
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Connected to 158.69.76.158 (158.69.76.158) port 8081 (#0)
> PATCH /api/v1/servers/localhost/zones/whon.ca HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 158.69.76.158:8081
> Accept: application/json
> Content-Type: application/json
> X-API-Key: HIDDEN
> Content-Length: 254
>
} [data not shown]
* upload completely sent off: 254 out of 254 bytes
< HTTP/1.1 204 No Content
< Access-Control-Allow-Origin: *
< Connection: close
< Content-Length: 0
< Content-Security-Policy: default-src 'self'; style-src 'self'
'unsafe-inline'
< Server: PowerDNS/4.1.0-rc3
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Pdns-New-Serial: 2017112202
< X-Pdns-Old-Serial: 2017112201
< X-Permitted-Cross-Domain-Policies: none
< X-Xss-Protection: 1; mode=block
<
100 254 0 0 100 254 0 5250 --:--:-- --:--:-- --:--:--
5291
* Closing connection 0
-------------------------------------------------
And looking at the database,
see https://pastebin.com/raw/9kAiKH2t
You will see that the serial number is updated on the SOA record, so
that's all good, and the ttl was modified to 50 on the bravo.whon.ca as
expected, so that's all good. But also what it's done is change the
ordername column to NULL for both the bravo.whon.ca CNAME record, and
the SOA record. So the zone will have to be rectified after this, which
is not the expected behaviour according to the docs. According to the
docs we should be able to run a modification via the API on a zone with
the API-RECTIFY set to 1 with no errors and the zone is rectified
afterwards.
STEP 4
--------
Set API-RECTIFY back to 1
We can run the same curl command again, changing the ttl value in the
change.whon.ca json file once again, AND setting the API-RECTIFY back to
1. We did this to see the behaviour with the zone not fully rectified
(the two previous records, bravo.whon.ca CNAME record and the whon.ca
SOA record have NULL in the ordername column)
Here is the output of the onscreen display, and once again it times out
after 10 seconds, and closes with an error.
--------------------------------------
* About to connect() to 158.69.76.158 port 8081 (#0)
* Trying 158.69.76.158...
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Connected to 158.69.76.158 (158.69.76.158) port 8081 (#0)
> PATCH /api/v1/servers/localhost/zones/whon.ca HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 158.69.76.158:8081
> Accept: application/json
> Content-Type: application/json
> X-API-Key: HIDDEN
> Content-Length: 254
>
} [data not shown]
* upload completely sent off: 254 out of 254 bytes
100 254 0 0 100 254 0 25 0:00:10 0:00:10 --:--:--
0< HTTP/1.1 500 Internal Server Error
< Connection: close
< Content-Length: 34
< Content-Type: application/json
< Server: PowerDNS/4.1.0-rc3
<
{ [data not shown]
100 288 100 34 100 254 3 25 0:00:11 0:00:10 0:00:01
0
* Closing connection 0
{
"error": "Internal Server Error"
}
-------------------------------------------
Result: CNAME record for bravo.whon.ca did not have its TTL updated
(should be 60 not 50)
see: https://pastebin.com/raw/pg6WJUHY
but here we see that the ordername field for the SOA record was updated
(partial rectification) BUT the SOA record itself was not modified. The
serial number should be updated but it's not, because there were no
record changes.
Output of the journalctl
---------------------------
Nov 22 10:17:44 ns5.cadns.ca pdns_server[26781]: Nov 22 10:17:44 HTTP
ISE for "/api/v1/servers/localhost/zones/whon.ca": Exception:
GSQLBackend unable to update ordername and auth for domain_id 6890:
Could not execute mysql statement: update records set ordername=?,auth=?
where domain_id=? and name=? and disabled=0: Lost connection to MySQL
server during query
Nov 22 10:17:44 ns5.cadns.ca mysqld[4616]: 2017-11-22 10:17:44
140447043155712 [Warning] Aborted connection 1236 to db: 'pdns_db' user:
'pdns_user' host: 'localhost' (Got an error reading communication packets)
---------------------------------------
STEP 5
-------
Set the API-RECTIFY back to 0
test again with the curl command (the only ordername field, currently
without a proper value is the CNAME bravo.whon.ca record as in the
pastebin.com link in STEP 4)
run the curl command again
Onscreen output
-----------------
* About to connect() to 158.69.76.158 port 8081 (#0)
* Trying 158.69.76.158...
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Connected to 158.69.76.158 (158.69.76.158) port 8081 (#0)
> PATCH /api/v1/servers/localhost/zones/whon.ca HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 158.69.76.158:8081
> Accept: application/json
> Content-Type: application/json
> X-API-Key: HIDDEN
> Content-Length: 254
>
} [data not shown]
* upload completely sent off: 254 out of 254 bytes
< HTTP/1.1 204 No Content
< Access-Control-Allow-Origin: *
< Connection: close
< Content-Length: 0
< Content-Security-Policy: default-src 'self'; style-src 'self'
'unsafe-inline'
< Server: PowerDNS/4.1.0-rc3
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Pdns-New-Serial: 2017112203
< X-Pdns-Old-Serial: 2017112202
< X-Permitted-Cross-Domain-Policies: none
< X-Xss-Protection: 1; mode=block
<
100 254 0 0 100 254 0 5786 --:--:-- --:--:-- --:--:--
5906
* Closing connection 0
----------------------------
No errors,
Results can be seen here: https://pastebin.com/raw/quXmshL2
As you can see, the ordername column is now NULL (it had a value
before), the serial is properly updated on the SOA record, and the
update is successful also on the TTL change for the CNAME record
bravo.whon.ca (60)
------------------------------
So this is what I've discovered so far. If my memory serves me
correctly, having the webserver-address=127.0.0.1 set and running the
curl command on the same server using 127.0.0.1 as the connect address,
the behaviour is slightly different but still error-prone. I don't have
time to do that whole test right now.
Eric
On 11/13/2017 4:28 AM, Pieter Lexis wrote:
> Hi Eric,
>
> On Sun, 12 Nov 2017 07:44:26 -0500
> Eric Beck <ericbeck at cadns.ca> wrote:
>
> Pushing this back to the mailinglist, please keep it there :).
>
>> sorry, I realized that I hadn't included my version ... feel free to
>> update my post or include this in
>>
>> version 4.1.0-rc2 Centos 7
>>
>> yum list installed | grep pdns
>> pdns.x86_64 4.1.0-0.1.rc2.1pdns.el7
>> @powerdns-auth-41
>> pdns-backend-mysql.x86_64 4.1.0-0.1.rc2.1pdns.el7
>> @powerdns-auth-41
>>
>> .... I should have perhaps submitted this as a bug
>>
>> I also have an update to my pdns-users post .... I was testing this with
>> a zone that was not secured. I thought perhaps it was a bug related to
>> the fact that perhaps the rectification would only work on a domain that
>> was already DNSSEC secured instead of on any zone. So I tried it also
>> with another zone which is DNSSEC secured, (NSEC3PARAM, 1 0 1 ab).
>> There is a further bug in that if you have API-RECTIFY set to 1 in the
>> domainmetadata table for a secured zone with the NSEC3PARM set, there is
>> an error with the API. I'll send it to you here, (I've changed the key
>> and domain name for security).
>>
>>
>> curl -v -X PATCH --data @/home/centos/curl/change.domain.ca -H
>> 'X-API-Key: ............................'
>> http://127.0.0.1:8081/api/v1/servers/localhost/zones/domain.ca |jq .
>>
>> ------------------- copy output from curl API -----------------------
>> * About to connect() to 127.0.0.1 port 8081 (#0)
>> * Trying 127.0.0.1...
>> % Total % Received % Xferd Average Speed Time Time Time
>> Current
>> Dload Upload Total Spent Left
>> Speed
>> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
>> 0* Connected to 127.0.0.1 (127.0.0.1) port 8081 (#0)
>>> PATCH /api/v1/servers/localhost/zones/domain.ca HTTP/1.1
>>> User-Agent: curl/7.29.0
>>> Host: 127.0.0.1:8081
>>> Accept: */*
>>> X-API-Key: ............................
>>> Content-Length: 255
>>> Content-Type: application/x-www-form-urlencoded
>>>
>> } [data not shown]
>> * upload completely sent off: 255 out of 255 bytes
>> 100 255 0 0 100 255 0 25 0:00:10 0:00:10 --:--:--
>> 0< HTTP/1.1 500 Internal Server Error
>> < Connection: close
>> < Content-Length: 21
>> < Content-Type: text/plain; charset=utf-8
>> < Server: PowerDNS/4.1.0-rc2
>> <
>> { [data not shown]
>> 100 276 100 21 100 255 2 25 0:00:10 0:00:10 --:--:--
>> 0
>> * Closing connection 0
>> parse error: Invalid numeric literal at line 1, column 9
>> -------------------- end copy output from curl API ------------------
>
> You send Content-Type: application/x-www-form-urlencoded, but the API only accepts application/json.
>
>> So I too off the API-RECTIFY and then put the curl command in a shell
>> script with the pdnsutil rectify-zone command after it and it was fine.
>> As I said I haven't tried this with a zone that is secured, but doesn't
>> have the NSEC3 set. I also didn't try it with a zone with NSEC3 set,
>> and set to NARROW.
>
> Can you check that the zone actually has keys as well?
>
>> Sorry I didn't put this in as a bug. I should have really. If you want
>> I can do that, but I'm thinking you have all the info now from my
>> testing, so it seems redundant at this point.
>
> If this really is a bug, which I doubt at this moment (but I did not attempt to reproduce this), a step-by-step way to reproduce this would really help.
>
> Best regards,
>
> Pieter
>
More information about the Pdns-users
mailing list