[Pdns-users] Problem with migrating from tinydns to gmysql backend
Łukasz Matys
lukasz.matys at horyzont.net
Tue Jul 2 10:18:34 UTC 2019
Hi.
We are going to move from a tinydns into powerdns.
The pdns setup went well with no problems,
What PowerDNS or dnsdist version you are using
- pdns-4.1.8-1.el7.x86_64
- pdns-backend-tinydns-4.1.8-1.el7.x86_64
- pdns-backend-mysql-4.1.8-1.el7.x86_64
What backends you use (gMySQL, BIND etc)
- tinydns as source backend
- gmysql as destination backend
* both backends work well while querying
If you are using DNSSEC
- no
Operating system it is running on
- CentOS Linux release 7.6.1810 (Core)
Where you got the binary from, i.e. did you compile it yourself (which compiler, including version), from your distribution or ports tree etc
- all from epel repo
What is going wrong
- we have problems with migrating zones from tinydns backend into gmysql
Everything have been done exactly like in the docs:
- 5.3.2 Moving from source to target
- pdnsutil b2b-migrate tinydns gmysql
launch=gmysql,tinydns
gmysql-host=127.0.0.1
gmysql-user=pdns
gmysql-dbname=pdns
gmysql-password=secret_pw
After the above command, I got an information that the process can not move on, because of duplicated records on domain table(before the migration the database was cleared),
Moving zone(s) from tinydns to gmysql
Processing 'wczesna-komunia.com.pl'
Moved 10 record(s), 0 comment(s), 0 metadata(s) and 0 cryptokey(s)
Processing 'wczesna-komunia.com.pl'
Error: Database error trying to insert new domain 'wczesna-komunia.com.pl.': Could not execute mysql statement: insert into domains (type,name,master,account,last_check,notified_serial) values(?,?,?,?,NULL,NULL): Duplicate entry 'wczesna-komunia.com.pl' for key 'name_index'
MariaDB [pdns]> select * from domains;
+----+------------------------+--------+------------+--------+-----------------+---------+
| id | name | master | last_check | type | notified_serial | account |
+----+------------------------+--------+------------+--------+-----------------+---------+
| 8 | wczesna-komunia.com.pl | | NULL | MASTER | 1561964305 | |
+----+------------------------+--------+------------+--------+-----------------+---------+
1 row in set (0.00 sec)
MariaDB [pdns]>
So next, I have removed the uniqe index at domain table for name, and restarted the migration process.
All domains have been imported into gmysql backend, but the problem is that each domain was inserted three times (x3), and all records have been inserted again three times (x3).
MariaDB [pdns]> select * from domains limit 12;
+----+------------------------+--------+------------+--------+-----------------+---------+
| id | name | master | last_check | type | notified_serial | account |
+----+------------------------+--------+------------+--------+-----------------+---------+
| 10 | wczesna-komunia.com.pl | | NULL | MASTER | 1561964305 | |
| 11 | wczesna-komunia.com.pl | | NULL | MASTER | 1561964305 | |
| 12 | wczesna-komunia.com.pl | | NULL | MASTER | 1561964305 | |
| 13 | haake.pl | | NULL | MASTER | 1561964305 | |
| 14 | haake.pl | | NULL | MASTER | 1561964305 | |
| 15 | haake.pl | | NULL | MASTER | 1561964305 | |
| 16 | amicus-kancelaria.pl | | NULL | MASTER | 1561964305 | |
| 17 | amicus-kancelaria.pl | | NULL | MASTER | 1561964305 | |
| 18 | amicus-kancelaria.pl | | NULL | MASTER | 1561964305 | |
| 19 | hydroinstalgaz.com.pl | | NULL | MASTER | 1561964305 | |
| 20 | hydroinstalgaz.com.pl | | NULL | MASTER | 1561964305 | |
| 21 | hydroinstalgaz.com.pl | | NULL | MASTER | 1561964305 | |
+----+------------------------+--------+------------+--------+-----------------+---------+
12 rows in set (0.00 sec)
MariaDB [pdns]> select * from records where domain_id = "13";
+----+-----------+--------------+-------+----------------------------------------------------------------------+------+------+-------------+----------+-----------+------+
| id | domain_id | name | type | content | ttl | prio | change_date | disabled | ordername | auth |
+----+-----------+--------------+-------+----------------------------------------------------------------------+------+------+-------------+----------+-----------+------+
| 62 | 13 | haake.pl | SOA | dns0.hti.pl. dns.horyzont.net. 2019022605 3600 600 604800 3600 | 3600 | 0 | NULL | 0 | NULL | 1 |
| 63 | 13 | haake.pl | SOA | dns0.hti.pl. hostmaster.haake.pl. 1561964305 16384 2048 1048576 2560 | 2560 | 0 | NULL | 0 | NULL | 1 |
| 64 | 13 | haake.pl | NS | dns0.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 65 | 13 | haake.pl | SOA | dns1.hti.pl. hostmaster.haake.pl. 1561964305 16384 2048 1048576 2560 | 2560 | 0 | NULL | 0 | NULL | 1 |
| 66 | 13 | haake.pl | NS | dns1.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 67 | 13 | haake.pl | MX | smtp-proxy-1-1.hti.pl. | 3600 | 10 | NULL | 0 | NULL | 1 |
| 68 | 13 | haake.pl | MX | smtp-proxy-2-1.hti.pl. | 3600 | 10 | NULL | 0 | NULL | 1 |
| 69 | 13 | haake.pl | A | 217.168.143.10 | 3600 | 0 | NULL | 0 | NULL | 1 |
| 70 | 13 | www.haake.pl | CNAME | virt10.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 71 | 13 | haake.pl | TXT | "v=spf1 mx -all" | 3600 | 0 | NULL | 0 | NULL | 1 |
| 72 | 13 | haake.pl | SOA | dns0.hti.pl. dns.horyzont.net. 2019022605 3600 600 604800 3600 | 3600 | 0 | NULL | 0 | NULL | 1 |
| 73 | 13 | haake.pl | SOA | dns0.hti.pl. hostmaster.haake.pl. 1561964305 16384 2048 1048576 2560 | 2560 | 0 | NULL | 0 | NULL | 1 |
| 74 | 13 | haake.pl | NS | dns0.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 75 | 13 | haake.pl | SOA | dns1.hti.pl. hostmaster.haake.pl. 1561964305 16384 2048 1048576 2560 | 2560 | 0 | NULL | 0 | NULL | 1 |
| 76 | 13 | haake.pl | NS | dns1.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 77 | 13 | haake.pl | MX | smtp-proxy-1-1.hti.pl. | 3600 | 10 | NULL | 0 | NULL | 1 |
| 78 | 13 | haake.pl | MX | smtp-proxy-2-1.hti.pl. | 3600 | 10 | NULL | 0 | NULL | 1 |
| 79 | 13 | haake.pl | A | 217.168.143.10 | 3600 | 0 | NULL | 0 | NULL | 1 |
| 80 | 13 | www.haake.pl | CNAME | virt10.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 81 | 13 | haake.pl | TXT | "v=spf1 mx -all" | 3600 | 0 | NULL | 0 | NULL | 1 |
| 82 | 13 | haake.pl | SOA | dns0.hti.pl. dns.horyzont.net. 2019022605 3600 600 604800 3600 | 3600 | 0 | NULL | 0 | NULL | 1 |
| 83 | 13 | haake.pl | SOA | dns0.hti.pl. hostmaster.haake.pl. 1561964305 16384 2048 1048576 2560 | 2560 | 0 | NULL | 0 | NULL | 1 |
| 84 | 13 | haake.pl | NS | dns0.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 85 | 13 | haake.pl | SOA | dns1.hti.pl. hostmaster.haake.pl. 1561964305 16384 2048 1048576 2560 | 2560 | 0 | NULL | 0 | NULL | 1 |
| 86 | 13 | haake.pl | NS | dns1.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 87 | 13 | haake.pl | MX | smtp-proxy-1-1.hti.pl. | 3600 | 10 | NULL | 0 | NULL | 1 |
| 88 | 13 | haake.pl | MX | smtp-proxy-2-1.hti.pl. | 3600 | 10 | NULL | 0 | NULL | 1 |
| 89 | 13 | haake.pl | A | 217.168.143.10 | 3600 | 0 | NULL | 0 | NULL | 1 |
| 90 | 13 | www.haake.pl | CNAME | virt10.hti.pl. | 3600 | 0 | NULL | 0 | NULL | 1 |
| 91 | 13 | haake.pl | TXT | "v=spf1 mx -all" | 3600 | 0 | NULL | 0 | NULL | 1 |
+----+-----------+--------------+-------+----------------------------------------------------------------------+------+------+-------------+----------+-----------+------+
30 rows in set (0.00 sec)
MariaDB [pdns]> select * from records where domain_id = "14";
Empty set (0.00 sec)
MariaDB [pdns]> select * from records where domain_id = "15";
Empty set (0.00 sec)
MariaDB [pdns]>
The next problem is, that all imported hosts (by pdnsutil) records have a '.' on the end of the record, but the example records from the docs do not have a such entry, like below:
MariaDB [pdns]> select * from domains where name = "mytkowski.pl";
+-----+--------------+--------+------------+--------+-----------------+---------+
| id | name | master | last_check | type | notified_serial | account |
+-----+--------------+--------+------------+--------+-----------------+---------+
| 298 | mytkowski.pl | | NULL | MASTER | 1561964305 | |
+-----+--------------+--------+------------+--------+-----------------+---------+
1 row in set (0.00 sec)
MariaDB [pdns]> select * from domains where name = "example.com";
+------+-------------+--------+------------+--------+-----------------+---------+
| id | name | master | last_check | type | notified_serial | account |
+------+-------------+--------+------------+--------+-----------------+---------+
| 1808 | example.com | NULL | NULL | MASTER | 1 | NULL |
+------+-------------+--------+------------+--------+-----------------+---------+
1 row in set (0.00 sec)
MariaDB [pdns]>
MariaDB [pdns]> select name,type,content from records where domain_id = "298";
+------------------+------+--------------------------------------------------------------------------+
| name | type | content |
+------------------+------+--------------------------------------------------------------------------+
| mytkowski.pl | SOA | dns0.hti.pl. dns.horyzont.net. 2019022605 3600 600 604800 3600 |
| mytkowski.pl | SOA | dns0.hti.pl. hostmaster.mytkowski.pl. 1561964305 16384 2048 1048576 2560 |
| mytkowski.pl | NS | dns0.hti.pl. |
| mytkowski.pl | SOA | dns1.hti.pl. hostmaster.mytkowski.pl. 1561964305 16384 2048 1048576 2560 |
| mytkowski.pl | NS | dns1.hti.pl. |
| mytkowski.pl | MX | smtp-proxy-1-1.hti.pl. |
| mytkowski.pl | MX | smtp-proxy-2-1.hti.pl. |
| mytkowski.pl | A | 193.33.111.123 |
| www.mytkowski.pl | A | 193.33.111.123 |
| mytkowski.pl | TXT | "v=spf1 mx -all" |
+------------------+------+--------------------------------------------------------------------------+
10 rows in set (0.00 sec)
MariaDB [pdns]> select name,type,content from records where domain_id = "1808";
+-----------------------+------+------------------------------------------------------+
| name | type | content |
+-----------------------+------+------------------------------------------------------+
| example.com | NS | dns-us1.powerdns.net |
| example.com | SOA | localhost admin.example.com 1 10380 3600 604800 3600 |
| example.com | NS | dns-us1.powerdns.net |
| example.com | NS | dns-eu1.powerdns.net |
| www.example.com | A | 192.0.2.10 |
| mail.example.com | A | 192.0.2.12 |
| localhost.example.com | A | 127.0.0.1 |
| example.com | MX | mail.example.com |
+-----------------------+------+------------------------------------------------------+
8 rows in set (0.00 sec)
MariaDB [pdns]>
What you expect to happen (many problems are in fact wrong expectations, and not bugs)
I expected, to get a proper migration status (not multiplied x3 times). Is it possible, that in the binary tinydns file, the definitions are stored three times? (I have no access to the raw data from which the binary file for the tinydns is being generated).
So what is the proper format of storing hosts records in the gmysql database? With or without dot '.'?
Best regards.
--
Matys Łukasz
Tel. +48 504257944
More information about the Pdns-users
mailing list