<html><head></head><body><div style="color:black;font-size:14px">Hello again.<br></div><div style="color: black;font-size: 14px;">Thanks for your suggestion.<br></div><div style="color: black;font-size: 14px;">I changed the configuration from primary=yes to primary=no and the problem seems to be resolved.<br></div><div style="color: black;font-size: 14px;"><br></div><div style="color: black;font-size: 14px;">I though it was enough to set:<br></div><div style="color: black;font-size: 14px;">only_notify = <br></div><div style="color: black;font-size: 14px;">notify_also =<br></div><div style="color: black;font-size: 14px;"><br></div><div style="color: black;font-size: 14px;">to avoid the notification process.<br></div><div style="color: black;font-size: 14px;"><br></div><div style="color:black;font-size:14px" class="signature">Thank you again for the support.<br></div><div style="color: black;font-size: 14px;">Riccardo</div><div><br></div><div>18/02/2022, 10:28 Klaus Darilion ha scritto:<br></div><blockquote style="border-left: 2px solid #a8a8a8;margin-left: 0px;padding-left: 10px;margin-top: 0.5rem;"><div class="qbox-body" lang="DE-AT"><div class="WordSection1"><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">So, when a zone gets updated, PowerDNS checks if it has to notify slaves and send NOTIFYs to the slaves. Once this is done, it updates
the notified_serial in the domains table. This UPDATE, for whatever reason, causes a deadlock.</span></span></span><br></p><p> <br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">Maybe you can work around that problem. You mentioned that you use DB replication. So, if all the slaves get their zones via DB replication
and NOT via NOTIFY/AXFR, then you can disable the NOTIFYs and "UPDATE notified_serial...." query by setting master=no in pdns.conf.</span></span></span><br></p><p> <br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">So, if Powerdns need not send NOTIFYs, then set master=no.</span></span></span><br></p><p> <br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">regards</span></span></span><br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">Klaus</span></span></span><br></p><p> <br></p><div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span style="" lang="DE"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">Von:</span></span></span></b><span style="" lang="DE"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt"> Pdns-users <<a href="mailto:pdns-users-bounces@mailman.powerdns.com">pdns-users-bounces@mailman.powerdns.com</a>> <b>Im Auftrag von </b>Klaus Darilion via Pdns-users<br> <b>Gesendet:</b> Freitag, 18. Februar 2022 09:58<br> <b>An:</b> Riccardo Brunetti <<a href="mailto:riccardo.brunetti@host.it">riccardo.brunetti@host.it</a>>; <a href="mailto:Pdns-users@mailman.powerdns.com">Pdns-users@mailman.powerdns.com</a><br> <b>Betreff:</b> Re: [Pdns-users] powerdns frequently crashes and restarts</span></span></span></p></div></div><p> <br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">Look in the mysql-server log. Maybe it mentions which other transaction causes the deadlock. That my help to find the cause.</span></span></span><br></p><p> <br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">Is this problem something new? I.e. started it after a PDNs or Mysql Upgrade?</span></span></span><br></p><p> <br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">regards</span></span></span><br></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">Klaus</span></span></span><br></p><p> <br></p><p> <br></p><div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span style="" lang="DE"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">Von:</span></span></span></b><span style="" lang="DE"><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt"> Pdns-users <<a href="mailto:pdns-users-bounces@mailman.powerdns.com">pdns-users-bounces@mailman.powerdns.com</a>> <b>Im Auftrag von </b>Riccardo Brunetti via Pdns-users<br> <b>Gesendet:</b> Freitag, 18. Februar 2022 09:55<br> <b>An:</b> <a href="mailto:pdns-users@mailman.powerdns.com">pdns-users@mailman.powerdns.com</a><br> <b>Betreff:</b> [Pdns-users] powerdns frequently crashes and restarts</span></span></span></p></div></div><p> <br></p><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Hello.</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">We are having quite frequent crashes of our pdns server with the following syslog messages:</span></span><br></p></div><div><p> <br></p></div><blockquote style="border:none;border-left:solid #A8A8A8 1.5pt;padding:0cm 0cm 0cm 8.0pt;margin-top:5.0pt;margin-bottom:5.0pt;margin-top:0.5rem;margin-block-start: 0px;margin-inline-start: 0px"><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:33 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx pdns_server[26672]: Exiting because communicator thread died with error: GSQLBackend unable to refresh domain_id XXXXX: Could not execute mysql statement: update
domains set notified_serial=? where id=?: Deadlock found when trying to get lock; try restarting transaction</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:34 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx systemd[1]: pdns.service: main process exited, code=exited, status=1/FAILURE</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:34 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx systemd[1]: Unit pdns.service entered failed state.</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:34 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx systemd[1]: pdns.service failed.</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:35 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx systemd[1]: pdns.service holdoff time over, scheduling restart.</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:35 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx systemd[1]: Stopped PowerDNS Authoritative Server.</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:35 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx systemd[1]: Starting PowerDNS Authoritative Server...</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:35 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx pdns_server[25899]: Loading '/usr/lib64/pdns/libgmysqlbackend.so'</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:35 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx pdns_server[25899]: This is a standalone pdns</span></span><br></p></div><div><p> <br></p></div></blockquote><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">I don't know if this can be related, but just before those crashes we see messages like:</span></span><br></p></div><div><p> <br></p></div><blockquote style="border:none;border-left:solid #A8A8A8 1.5pt;padding:0cm 0cm 0cm 8.0pt;margin-top:5.0pt;margin-bottom:5.0pt;margin-top:0.5rem;margin-block-start: 0px;margin-inline-start: 0px"><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Feb 17 13:39:32 <a href="http://xxxx.xxxxx.xxx">xxxx.xxxxx.xxx</a>.xxxx pdns_server[26672]: Request to queue notification for domain yyyyy.yy was processed, but no valid nameserver or ALSO-NOTIFYs found. Not notifying!</span></span><br></p></div><div><p> <br></p></div></blockquote><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Just to mention briefly our setup: we are running:</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">PowerDNS Authoritative Server 4.5.2 (C)</span></span><br></p></div><div><p> <br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">in a percona mysql cluster with three nodes, each node having it's own mysql and pdns server (in order to have redundancy of database and APIs).</span></span><br></p></div><div><p> <br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">This cluster acts as a "backend master" for the real authoritative slaves, placed elsewhere, replicating the db.</span></span><br></p></div><div><p> <br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Can you help us to understand why we have those crashes?</span></span><br></p></div><div><p> <br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Thanks</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt">Riccardo</span></span><br></p></div><div><p class="MsoNormal"><span style="color: black;"><span class="size" style="font-size:10.5pt"> </span></span><br></p></div></div></div></div></div></blockquote><div style="color: black;font-size: 14px;"><br></div></body></html>