[Pdns-users] SQLite3 problem during stress

[Christian Svensson]

It is able to write to the database a couple of times - and I know it has
permissions to both write to the file and create it's journal file in the
same directory. Heck, the directory and files are owned by pdns so it can do
whatever it desires.

Christian "BC" Svensson
Codelead Systems - http://www.codelead.se


On Sat, Jul 18, 2009 at 7:10 PM, bert hubert <bert.hubert at netherlabs.nl>wrote:

> Very quick reply without thinking, make sure PowerDNS has write
> permissions on the *directory* containing the sqlite3 database - this
> tends to trip people up 'it can write to the file' is not enough.
>
> I'll read your message more carefully to see what else might be happening.
>
>    Bert
>
> On Sat, Jul 18, 2009 at 6:51 PM, Christian Svensson<blue at cmd.nu> wrote:
> > Hello.
> >
> > We are trying to migrate from bind9 to PowerDNS.
> > We have our back-end supermaster successfully constructed using PowerDNS
> +
> > postgresql 8.3 + alsonotify-patch (
> http://wiki.powerdns.com/trac/ticket/216)
> > The slaves have to have a very small footprint and therefore we have
> chosen
> > sqlite3 as our target database.
> >
> > After combating some weird incompatibility with Debian Lenny 5.0 where
> PDNSs
> > gsqlite3 refused to write anything to the database it begun to crash when
> we
> > did the initial transfer (notify of several hundred domains).
> > I have tried with the OpenDBX backend which seems to work much better
> > regarding the sqlite3 incompatibility but also crashes during heavy
> transfer
> > load.
> > Setting the distributor-threads to 1 helps it somewhat, but I think there
> > might be a different thread that messes things up.
> >
> > Log excerpt:
> > Jul 18 16:45:40 Reading random entropy from '/dev/urandom'
> > Jul 18 16:45:40  [OpendbxBackend] This is the opendbx module version
> 2.9.22
> > (Jul 18 2009, 14:19:38) reporting
> > Jul 18 16:45:40  [OpendbxBackend] This is the opendbx module version
> 2.9.22
> > (Jul 18 2009, 14:19:28) reporting
> > Jul 18 16:45:40 This is a standalone pdns
> > Jul 18 16:45:40 It is advised to bind to explicit addresses with the
> > --local-address option
> > Jul 18 16:45:40 UDP server bound to 0.0.0.0:53
> > Jul 18 16:45:40 TCP server bound to 0.0.0.0:53
> > Jul 18 16:45:40 PowerDNS 2.9.22 (C) 2001-2009 PowerDNS.COM BV (Jul 18
> 2009,
> > 14:23:39, gcc 4.3.2) starting up
> > Jul 18 16:45:40 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free
> > software, and you are welcome to redistribute it according to the terms
> of
> > the GPL version 2.
> > Jul 18 16:45:40 Set effective group id to 104
> > Jul 18 16:45:40 Set effective user id to 102
> > Jul 18 16:45:40 DNS Proxy launched, local port 27636, remote
> 127.0.0.1:1053
> > Jul 18 16:45:40 Master/slave communicator launching
> > Jul 18 16:45:40 Launched webserver on 127.0.0.1:8081
> > Jul 18 16:45:40 Creating backend connection for TCP
> > % Jul 18 16:45:40 [OpendbxBackend] Database connection (read) to
> > '/srv/pdns/' succeeded
> > Jul 18 16:45:40 [OpendbxBackend] Database connection (write) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:40 211 slave domains need checking
> > Jul 18 16:45:40 [OpendbxBackend] Database connection (read) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:40 [OpendbxBackend] Database connection (write) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:40 About to create 1 backend threads for UDP
> > Jul 18 16:45:40 [OpendbxBackend] Database connection (read) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:40 [OpendbxBackend] Database connection (write) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:40 Done launching threads, ready to distribute questions
> > Jul 18 16:45:47 Engaging bypass - now operating unthreaded
> > Jul 18 16:45:47 [OpendbxBackend] Database connection (read) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:47 [OpendbxBackend] Database connection (write) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:47 Received NOTIFY for stadsbudcentralen.nu from
> > 213.132.111.158 for which we are not authoritative
> > Jul 18 16:45:47 Created new slave zone 'stadsbudcentralen.nu' from
> > supermaster 213.132.111.158, queued axfr
> > Jul 18 16:45:47 Received NOTIFY for stadsbudcentralen.se from
> > 213.132.111.158 for which we are not authoritative
> > Jul 18 16:45:47 Initiating transfer of 'stadsbudcentralen.nu' from
> remote
> > '213.132.111.158'
> > Jul 18 16:45:47 [OpendbxBackend] Database connection (read) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:47 [OpendbxBackend] Database connection (write) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:47 Created new slave zone 'stadsbudcentralen.se' from
> > supermaster 213.132.111.158, queued axfr
> > Jul 18 16:45:47 Received NOTIFY for westberg.info from 213.132.111.158
> for
> > which we are not authoritative
> > Jul 18 16:45:47 AXFR started for 'stadsbudcentralen.nu', transaction
> started
> > Jul 18 16:45:47 AXFR done for 'stadsbudcentralen.nu', zone committed
> > Jul 18 16:45:47 Initiating transfer of 'stadsbudcentralen.se' from
> remote
> > '213.132.111.158'
> > Jul 18 16:45:47 [OpendbxBackend] Database connection (read) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:47 Created new slave zone 'westberg.info' from supermaster
> > 213.132.111.158, queued axfr
> > Jul 18 16:45:47 [OpendbxBackend] Database connection (write) to
> '/srv/pdns/'
> > succeeded
> > Jul 18 16:45:47 Received NOTIFY for trevlig.nu from 213.132.111.158 for
> > which we are not authoritative
> > Jul 18 16:45:47 Unable to find backend willing to host trevlig.nu for
> > potential supermaster 213.132.111.158
> > Jul 18 16:45:47 Received NOTIFY for drproduction.com from
> 213.132.111.158
> > for which we are not authoritative
> > Jul 18 16:45:47 Unable to find backend willing to host drproduction.comfor
> > potential supermaster 213.132.111.158
> > Jul 18 16:45:47 Received NOTIFY for floopy.be from 213.132.111.158 for
> which
> > we are not authoritative
> > Jul 18 16:45:47 Unable to find backend willing to host floopy.be for
> > potential supermaster 213.132.111.158
> > Jul 18 16:45:47 Received NOTIFY for gronastuganikalmar.se from
> > 213.132.111.158 for which we are not authoritative
> > Jul 18 16:45:47 Unable to find backend willing to host
> gronastuganikalmar.se
> > for potential supermaster 213.132.111.158
> > Jul 18 16:45:47 Received NOTIFY for kigsrdr.org from 213.132.111.158 for
> > which we are not authoritative
> > Jul 18 16:45:47 [OpendbxBackend] getRecord: Unable to get next result -
> > database is locked
> > Jul 18 16:45:47 Communicator thread died because of error: Error:
> > odbx_result() failed
> > Jul 18 16:45:47 Unable to find backend willing to host kigsrdr.org for
> > potential supermaster 213.132.111.158
> > Jul 18 16:45:47 Received NOTIFY for kalmargamecenter.com from
> > 213.132.111.158 for which we are not authoritative
> > Jul 18 16:45:47 Unable to find backend willing to host
> kalmargamecenter.com
> > for potential supermaster 213.132.111.158
> > Jul 18 16:45:47 Received NOTIFY for larvig.nu from 213.132.111.158 for
> which
> > we are not authoritative
> > Jul 18 16:45:47 Unable to find backend willing to host larvig.nu for
> > potential supermaster 213.132.111.158
> > Jul 18 16:45:47 Received NOTIFY for lyktan-vilshult.se from
> 213.132.111.158
> > for which we are not authoritative
> >
> > This is an example where it happens quite early. Sometimes it gets a
> couple
> > of hundred domains before it crashes.
> > Tried with both gsqlite3 and opendbx.
> >
> > Note: I have only been able to test the distributor-threads = 1 with
> > opendbx. But I would assume that option and thus the error is backend
> > agnostic in that it only requires the backend to handle one write/read
> > connection.
> >
> > Please advice.
> >
> > Christian "BC" Svensson
> > Codelead Systems - http://www.codelead.se
> > !DSPAM:4a61fdc1156835958412055!
> > _______________________________________________
> > Pdns-users mailing list
> > Pdns-users at mailman.powerdns.com
> > http://mailman.powerdns.com/mailman/listinfo/pdns-users
> >
> >
> > !DSPAM:4a61fdc1156835958412055!
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20090719/c4975c91/attachment-0001.html>