[Pdns-users] SQLite3 problem during stress

[Norbert Sendetzky]

On Mon July 20 2009 11:01:05 you wrote:
> Yes, updating only one zone at the time is very much acceptable - that the
> initial transfer takes quite long time does not matter.
>
> We do not want to use an "active" database due to memory / CPU footprint.
> If PowerDNS locks the table and let the other threads just wait for the
> lock like any other mutex it would probably be just fine with
> distributor-threads = 1.

The problem of the SQLite database is that while a zone transfer is in 
progress, all other threads can't serve any records. This is because the 
complete table is locked by SQLite when records are updated and this also 
prevents reading records in other zones. Please keep this in mind for your 
installation.

> I want to note that indeed PowerDNS _crashes_ / _exits_ in the end due to
> the "Database is locked" error. Since I'm currently testing with OpenDBX I
> don't know íf this is something that is regarded as an OpenDBX or PowerDNS
> bug - but gsqlite3 did also crash during heavy load which (as stated
> before) makes me believe that it's backend agnostic.

What you've shown me in your logs is that the PowerDNS opendbx backend gets an 
error from the SQLite library that the database is locked and the opendbx 
backend forces the distributor thread to recreate the connection to the 
database. The problem is that the sqlite3 backend of the OpenDBX library 
returns a fatal error when the database is locked instead of a timeout. This 
is fixed in the SVN trunk and the fix will be also be included in OpenDBX 
1.4.2. The next problem is that the PowerDNS opendbx backend doesn't make use 
of timeouts at the moment and treats them as errors. I've already enhanced the 
PowerDNS opendbx backend but I have to test it as soon as I can and see if the 
problem is gone. I will create a patch and send it to you so you can test 
yourself.

What I haven't seen in your log file excerpt is that the opendbx backend 
crashed with a segfault.


Norbert
-- 
OpenPGP public key
http://www.linuxnetworks.de/norbert.pubkey.asc


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20090721/6241a782/attachment-0001.sig>