[Pdns-users] Recursor: Black list

[Aki Tuomi]

On Mon, Oct 27, 2014 at 12:58:43PM -0500, ktm at rice.edu wrote:
> On Mon, Oct 27, 2014 at 07:27:15PM +0200, Aki Tuomi wrote:
> > On Mon, Oct 27, 2014 at 01:56:17PM -0300, Ciro Iriarte wrote:
> > > 2014-10-27 3:46 GMT-03:00 Aki Tuomi <cmouse at youzen.ext.b2.fi>:
> > > >
> > > > In a way i'd chosen sqlite3 instead as it is pretty much on par with cdb.
> > > > But, to make it work properly, i'd just add "*.domain.com", and when you lookup,
> > > > you could reduce it like this with get()
> > > >
> > > > www.my.long.name.com => NOT FOUND
> > > > *.my.long.name.com => NOT FOUND
> > > > *.long.name.com => NOT FOUND
> > > > *.name.com => FOUND
> > > >
> > > > (
> > > > of course you could continue with
> > > > *.com
> > > > *
> > > > )
> > > >
> > > > Aki
> > > 
> > > Hi Aki!, I couldn't find a (finished) benchmark that compares directly
> > > sqlite3 vs cdb, but the unfinished tests imply that cdb is faster.
> > > Given it's SQL I assume we can just use a SELECT with LIKE clause to
> > > match an "ending" on the DB with the requested fqdn, would it be
> > > faster than doing multiple cdb queries (one for each part of the
> > > requested fqdn)?
> > > 
> > > Regards,
> > > 
> > > -- 
> > > Ciro Iriarte
> > > http://iriarte.it
> > > --
> > >
> > 
> > The difference, to my eyes, is the diference between
> > 
> >   SELECT name FROM table WHERE name LIKE '%suffix';
> > 
> > and
> > 
> >   SELECT name FROM table WHERE name = 'www.my.long.name.com';
> >   SELECT name FROM table WHERE name = '*.my.long.name.com';
> >   SELECT name FROM table WHERE name = '*.long.name.com';
> >   SELECT name FROM table WHERE name = '*.name.com';
> >   SELECT name FROM table WHERE name = '*.com';
> > 
> > (assuming you'll want to filter out, say, *.xxx)
> > 
> > Obviously using suffix would require you to know what you are
> > doing, since you'd have to know what suffix to look for, otherwise
> > you'll end up with very unpredicable behaviour. 
> > 
> > Consider, you have www.name.com in your blacklist, you'll look for
> > %.name.com. It'll always return match. So it's safer to go with
> > repeated lookups for *.parent. 
> > 
> > Performance-wise you should consider that your most likely usage 
> > patterns are, 
> > 
> > not blacklisted:
> >   SELECT name FROM table WHERE name = 'www.name.com';
> >   SELECT name FROM table WHERE name = '*.name.com';
> >   SELECT name FROM table WHERE name = '*.com';
> > 
> > blacklisted:
> >   SELECT name FROM table WHERE name = 'www.name.com';
> > 
> > or:
> >   SELECT name FROM table WHERE name = 'www.name.com';
> >   SELECT name FROM table WHERE name = '*.name.com';
> > 
> > 
> > to give proper answer whether SQLite3 or CDB is better, you'd have to 
> > run benchmark tests against these use cases as they cover most of your
> > situations.
> > 
> > Also, you might want to consider early-break on any query ending with
> > in-addr.arpa and i6.arpa, unless you are required to filter these too,
> > because you can get pretty long iterations especially with IPv6 reverses. 
> > 
> > All in all, i'd say go with cdb, since you already have the code there
> > and it's not a big mod to make. Just keep this is mind. 
> > 
> > ---
> > Aki 
> > 
> 
> Hi,
> 
> CDB is a very simple key/value store. I would expect it to blow the
> doors off SQLite for simple lookups. In addition, the size of the
> library is much, much smaller for CDB (20k) than for SQLite (400k),
> which means that it should need much fewer resources and produce
> a lighter weight Lua process. Since the logic is mainly in the Lua
> function and the the DB backend, the simple CDB key/value store
> should perform better per amount of resources used.
> 
> Regards,
> Ken
> 

Ken, you are right. Thank you for pointing this out. 

Aki