[Pdns-users] SQLite3 problem during stress

Sat Jul 18 17:10:06 UTC 2009

Very quick reply without thinking, make sure PowerDNS has write
permissions on the *directory* containing the sqlite3 database - this
tends to trip people up 'it can write to the file' is not enough.

I'll read your message more carefully to see what else might be happening.

    Bert

On Sat, Jul 18, 2009 at 6:51 PM, Christian Svensson<blue at cmd.nu> wrote:
> Hello.
>
> We are trying to migrate from bind9 to PowerDNS.
> We have our back-end supermaster successfully constructed using PowerDNS +
> postgresql 8.3 + alsonotify-patch (http://wiki.powerdns.com/trac/ticket/216)
> The slaves have to have a very small footprint and therefore we have chosen
> sqlite3 as our target database.
>
> After combating some weird incompatibility with Debian Lenny 5.0 where PDNSs
> gsqlite3 refused to write anything to the database it begun to crash when we
> did the initial transfer (notify of several hundred domains).
> I have tried with the OpenDBX backend which seems to work much better
> regarding the sqlite3 incompatibility but also crashes during heavy transfer
> load.
> Setting the distributor-threads to 1 helps it somewhat, but I think there
> might be a different thread that messes things up.
>
> Log excerpt:
> Jul 18 16:45:40 Reading random entropy from '/dev/urandom'
> Jul 18 16:45:40  [OpendbxBackend] This is the opendbx module version 2.9.22
> (Jul 18 2009, 14:19:38) reporting
> Jul 18 16:45:40  [OpendbxBackend] This is the opendbx module version 2.9.22
> (Jul 18 2009, 14:19:28) reporting
> Jul 18 16:45:40 This is a standalone pdns
> Jul 18 16:45:40 It is advised to bind to explicit addresses with the
> --local-address option
> Jul 18 16:45:40 UDP server bound to 0.0.0.0:53
> Jul 18 16:45:40 TCP server bound to 0.0.0.0:53
> Jul 18 16:45:40 PowerDNS 2.9.22 (C) 2001-2009 PowerDNS.COM BV (Jul 18 2009,
> 14:23:39, gcc 4.3.2) starting up
> Jul 18 16:45:40 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free
> software, and you are welcome to redistribute it according to the terms of
> the GPL version 2.
> Jul 18 16:45:40 Set effective group id to 104
> Jul 18 16:45:40 Set effective user id to 102
> Jul 18 16:45:40 DNS Proxy launched, local port 27636, remote 127.0.0.1:1053
> Jul 18 16:45:40 Master/slave communicator launching
> Jul 18 16:45:40 Launched webserver on 127.0.0.1:8081
> Jul 18 16:45:40 Creating backend connection for TCP
> % Jul 18 16:45:40 [OpendbxBackend] Database connection (read) to
> '/srv/pdns/' succeeded
> Jul 18 16:45:40 [OpendbxBackend] Database connection (write) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:40 211 slave domains need checking
> Jul 18 16:45:40 [OpendbxBackend] Database connection (read) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:40 [OpendbxBackend] Database connection (write) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:40 About to create 1 backend threads for UDP
> Jul 18 16:45:40 [OpendbxBackend] Database connection (read) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:40 [OpendbxBackend] Database connection (write) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:40 Done launching threads, ready to distribute questions
> Jul 18 16:45:47 Engaging bypass - now operating unthreaded
> Jul 18 16:45:47 [OpendbxBackend] Database connection (read) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:47 [OpendbxBackend] Database connection (write) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:47 Received NOTIFY for stadsbudcentralen.nu from
> 213.132.111.158 for which we are not authoritative
> Jul 18 16:45:47 Created new slave zone 'stadsbudcentralen.nu' from
> supermaster 213.132.111.158, queued axfr
> Jul 18 16:45:47 Received NOTIFY for stadsbudcentralen.se from
> 213.132.111.158 for which we are not authoritative
> Jul 18 16:45:47 Initiating transfer of 'stadsbudcentralen.nu' from remote
> '213.132.111.158'
> Jul 18 16:45:47 [OpendbxBackend] Database connection (read) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:47 [OpendbxBackend] Database connection (write) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:47 Created new slave zone 'stadsbudcentralen.se' from
> supermaster 213.132.111.158, queued axfr
> Jul 18 16:45:47 Received NOTIFY for westberg.info from 213.132.111.158 for
> which we are not authoritative
> Jul 18 16:45:47 AXFR started for 'stadsbudcentralen.nu', transaction started
> Jul 18 16:45:47 AXFR done for 'stadsbudcentralen.nu', zone committed
> Jul 18 16:45:47 Initiating transfer of 'stadsbudcentralen.se' from remote
> '213.132.111.158'
> Jul 18 16:45:47 [OpendbxBackend] Database connection (read) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:47 Created new slave zone 'westberg.info' from supermaster
> 213.132.111.158, queued axfr
> Jul 18 16:45:47 [OpendbxBackend] Database connection (write) to '/srv/pdns/'
> succeeded
> Jul 18 16:45:47 Received NOTIFY for trevlig.nu from 213.132.111.158 for
> which we are not authoritative
> Jul 18 16:45:47 Unable to find backend willing to host trevlig.nu for
> potential supermaster 213.132.111.158
> Jul 18 16:45:47 Received NOTIFY for drproduction.com from 213.132.111.158
> for which we are not authoritative
> Jul 18 16:45:47 Unable to find backend willing to host drproduction.com for
> potential supermaster 213.132.111.158
> Jul 18 16:45:47 Received NOTIFY for floopy.be from 213.132.111.158 for which
> we are not authoritative
> Jul 18 16:45:47 Unable to find backend willing to host floopy.be for
> potential supermaster 213.132.111.158
> Jul 18 16:45:47 Received NOTIFY for gronastuganikalmar.se from
> 213.132.111.158 for which we are not authoritative
> Jul 18 16:45:47 Unable to find backend willing to host gronastuganikalmar.se
> for potential supermaster 213.132.111.158
> Jul 18 16:45:47 Received NOTIFY for kigsrdr.org from 213.132.111.158 for
> which we are not authoritative
> Jul 18 16:45:47 [OpendbxBackend] getRecord: Unable to get next result -
> database is locked
> Jul 18 16:45:47 Communicator thread died because of error: Error:
> odbx_result() failed
> Jul 18 16:45:47 Unable to find backend willing to host kigsrdr.org for
> potential supermaster 213.132.111.158
> Jul 18 16:45:47 Received NOTIFY for kalmargamecenter.com from
> 213.132.111.158 for which we are not authoritative
> Jul 18 16:45:47 Unable to find backend willing to host kalmargamecenter.com
> for potential supermaster 213.132.111.158
> Jul 18 16:45:47 Received NOTIFY for larvig.nu from 213.132.111.158 for which
> we are not authoritative
> Jul 18 16:45:47 Unable to find backend willing to host larvig.nu for
> potential supermaster 213.132.111.158
> Jul 18 16:45:47 Received NOTIFY for lyktan-vilshult.se from 213.132.111.158
> for which we are not authoritative
>
> This is an example where it happens quite early. Sometimes it gets a couple
> of hundred domains before it crashes.
> Tried with both gsqlite3 and opendbx.
>
> Note: I have only been able to test the distributor-threads = 1 with
> opendbx. But I would assume that option and thus the error is backend
> agnostic in that it only requires the backend to handle one write/read
> connection.
>
> Please advice.
>
> Christian "BC" Svensson
> Codelead Systems - http://www.codelead.se
> !DSPAM:4a61fdc1156835958412055!
> _______________________________________________
> Pdns-users mailing list
> Pdns-users at mailman.powerdns.com
> http://mailman.powerdns.com/mailman/listinfo/pdns-users
>
>
> !DSPAM:4a61fdc1156835958412055!
>
>