[Pdns-users] Weird behaviour of pdns recursor (cache)

[Raphael Thoma]

Hello there

We recently upgraded the pdns-recursor on one of our systems from 3.1.7.2 to 3.3. This system is running on freebsd 8.2 and the powerdns-recursor has been installed via the ports system.

== Configuration ==
Packet-Cache has been disabled in order to ensure that the problem is not related to this cache. However: patch in Ticket is applied (http://wiki.powerdns.com/trac/ticket/333). All problems described below were also seen without the patch!

== Problem Description ==

=== Case 1: negative Caching ===
Environment:
Domain: raphi.es
Records: MX for *.raphi.es, A-Record for raphi.es, NO other A-Records (e.g. for test.raphi.es)

Query: dig @<recursor-IP> test.raphi.es
Expected Reply: NOERROR and empty A-Record
Seen Reply: matches Expectations

Action: Create A-Record for test.raphi.es to 8.8.8.8 and wipe cache: rec_control wipe-cache test.raphi.es raphi.es
Reaction: wiped 4 records, 1 negative records

Query: dig @<recursor-IP> test.raphi.es
Expected Reply: A-Record to 8.8.8.8
Seen Reply: still NOERROR and empty A-Record

--> Reply is incorrect. Further wiping of test.raphi.es shows that there is no such record (wiped 0 records, 0 negative records) even though multiple queries for test.raphi.es have been sent. According to a cache dump (rec_control dump_cache <filename>) there are no entries for test.raphi.es
We did a A-query for test.raphi.es every second and after some time we received to different replies randomly: Sometimes the empty Record was replied and sometimes we received the correct, new A-Record.

=== Case 2: caching of Records ===
Environment:
Domain: raphi.es
Records: A-Record for test2.raphi.es to 8.8.8.2, MX for *.raphi.es, A-Record for raphi.es

Query: dig @<recursor-IP> test2.raphi.es
Expected Reply: A-Record to 8.8.8.2
Seen Reply: matches Expectations

Action: Modify test2.raphi.es A-Record that it points to 8.8.8.1 and wipe cache: rec_control wipe-cache test2.raphi.es raphi.es
Reaction: wiped 4 records, 0 negative records

Query: dig @<recursor-IP> test2.raphi.es
Expected Reply: A-Record to 8.8.8.1
Seen Reply: still pointing to 8.8.8.2

--> Reply is incorrect. Further wiping of test2.raphi.es shows that there is no such record (wiped 0 records, 0 negative records) even though multiple queries for test2.raphi.es have been sent. After some time the replies contain sometimes 8.8.8.1 and sometimes 8.8.8.2. At the end (after some more minutes) it stays with 8.8.8.1

== Observations ==
We did exactly the same stuff on our older version recursors and there it worked as expected. A difference we could see is that less records were wiped - if we wipe test.raphi.es on the new version: 2 records were wiped. In the older Version only 1 record is wiped.

A cache-dump showed that we have thousands of duplicate entries in the cache. Why that? As explained - packet-caching is disabled with disable-packetcache=yes.

It seems that the recursor behaves really weird if the cache is modified. Did you see something similar in the past or is there already a fix available for this problem? Unfortunately I couldn't find any bug reports which may be related to this issue.

Please let me know if you require further information.

Best Regards
Raphael Thoma
-- 
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de