[Pdns-users] PDNS-Recursor Segfaults

Tue May 20 21:27:58 UTC 2014

http://www.sourceware.org/ml/libc-alpha/2013-02/msg00296.html is almost certainly relevant in many ways.

On May 20, 2014 11:16 PM, Imre Gergely <gimre at narancs.net> wrote:
>
>
> backtrace attached. 
>
> [root at c605 pdns-recursor]# ulimit -a 
> core file size          (blocks, -c) 0 
> data seg size           (kbytes, -d) unlimited 
> scheduling priority             (-e) 0 
> file size               (blocks, -f) unlimited 
> pending signals                 (-i) 3873 
> max locked memory       (kbytes, -l) 64 
> max memory size         (kbytes, -m) unlimited 
> open files                      (-n) 8192 
> pipe size            (512 bytes, -p) 8 
> POSIX message queues     (bytes, -q) 819200 
> real-time priority              (-r) 0 
> stack size              (kbytes, -s) 10240 
> cpu time               (seconds, -t) unlimited 
> max user processes              (-u) 3873 
> virtual memory          (kbytes, -v) unlimited 
> file locks                      (-x) unlimited 
>
>
> On 05/21/2014 12:00 AM, Aki Tuomi wrote: 
> > Can you install the debuginfo package and run it with gdb to get stack trace? Also, can 
> > you give us ulimit -a? 
> > 
> > Aki 
> > 
> > On Tue, May 20, 2014 at 11:41:21PM +0300, Imre Gergely wrote: 
> >> [root at c605 ~]# cat /etc/pdns-recursor/recursor.conf |grep -v "^#" | grep 
> >> -v "^$" 
> >> setuid=pdns-recursor 
> >> setgid=pdns-recursor 
> >> daemon=no 
> >> local-address=127.0.0.1 
> >> threads=1 
> >> 
> >> [root at c605 ~]# strace /usr/sbin/pdns_recursor > 
> >> /tmp/strace-pdns-recursor.txt 2>&1 
> >> Segmentation fault 
> >> [root at c605 ~]# 
> >> 
> >> [root at c605 ~]# ip a |grep inet |wc 
> >>    4576   27451  233679 
> >> 
> >> Attached. If this is not what you had in mind, please let me know. 
> >> 
> >> On 05/20/2014 11:31 PM, bert hubert wrote: 
> >>> Imre, 
> >>> 
> >>> Can you strace the startup with threads=1? 
> >>> 
> >>>     Bert 
> >>> 
> >>> On May 20, 2014 10:25 PM, Imre Gergely <gimre at narancs.net> wrote: 
> >>>> Hi 
> >>>> 
> >>>> I did manage to reproduce this in a VM. Installed a CentOS 6.5, and recursor 3.5.3 from EPEL. Then I did this: 
> >>>> 
> >>>> for i in `seq 1 16`; do for j in `seq 1 254`; do ip a a 10.0.$i.$j/16 dev eth0; done; done 
> >>>> 
> >>>> Then I started the recursor, everything went just fine, did a bunch of digs, no problems. 
> >>>> 
> >>>> Then I added some more IPs: 
> >>>> 
> >>>> for i in `seq 17 32`; do for j in `seq 1 254`; do ip a a 10.0.$i.$j/16 dev eth0; done; done 
> >>>> 
> >>>> And then init.d/pdns-recursor restart: 
> >>>> 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: PowerDNS recursor 3.5.3 (C) 2001-2013 PowerDNS.COM BV (Feb 10 2014, 17:26:52, gcc 4.4.7 20120313 (Red Hat 4.4.7-4)) 
> >>>>  starting up 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2. 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Operating in 32 bits mode 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Reading random entropy from '/dev/urandom' 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Only allowing queries from: 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fe80::/10 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Will not send queries to: 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fe80::/10, 0.0.0.0, :: 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: NOT using IPv6 for outgoing queries - set 'query-local-address6=::' to enable 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Inserting rfc 1918 private space zones 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Listening for UDP queries on 127.0.0.1:53 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Enabled TCP data-ready filter for (slight) DoS protection 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Listening for TCP queries on 127.0.0.1:53 
> >>>> May 20 23:18:24 c605 pdns_recursor[21341]: Calling daemonize, going to background 
> >>>> May 20 23:18:24 c605 pdns_recursor[21342]: Set effective group id to 499 
> >>>> May 20 23:18:24 c605 pdns_recursor[21342]: Set effective user id to 498 
> >>>> May 20 23:18:24 c605 pdns_recursor[21342]: Launching 2 threads 
> >>>> May 20 23:18:24 c605 pdns_recursor[21342]: Done priming cache with root hints 
> >>>> May 20 23:18:24 c605 pdns_recursor[21342]: Enabled 'epoll' multiplexer 
> >>>> May 20 23:18:24 c605 pdns_recursor[21342]: Done priming cache with root hints 
> >>>> May 20 23:18:24 c605 pdns_recursor[21342]: Refreshed . records 
> >>>> May 20 23:18:25 c605 kernel: pdns_recursor[21345]: segfault at ffff01d4 ip 080b1626 sp b6397890 error 4 in pdns_recursor[8048000+112000] 
> >>>> 
> >>>> [root at c605 ~]# ip a |grep inet |wc 
> >>>>    4322   25927  220579 
> >>>> [root at c605 ~]# /etc/init.d/pdns-recursor start 
> >>>> Starting pdns-recursor:                                    [  OK  ]  <-- starts OK 
> >>>> [root at c605 ~]# /etc/init.d/pdns-recursor stop 
> >>>> Stopping pdns-recursor:                                    [  OK  ] 
> >>>> 
> >>>> Adding one more /24: 
> >>>> 
> >>>> [root at c605 ~]# for j in `seq 1 254`; do ip a a 10.0.18.$j/16 dev eth0; done 
> >>>> [root at c605 ~]# /etc/init.d/pdns-recursor start 
> >>>> Starting pdns-recursor:                                    [  OK  ] 
> >>>> [root at c605 ~]# ip a |grep inet |wc 
> >>>>    4576   27451  233679 
> >>>> 
> >>>> It says it starts, but it doesn't, just segfaults. 
> >>>> 
> >>>> [root at c605 ~]# file /bin/bash 
> >>>> /bin/bash: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped 
> >>>> 
> >>>> 
> >>>> On 05/20/2014 10:58 PM, James Baer wrote: 
> >>>>> Hi All - I'm experiencing an issue that I am unsure if it is a bug or just something I need to adjust on my systems to account for. 
> >>>>> 
> >>>>> I have 2 servers, both running pdns_recursor (3.5.3) on Centos 6.5, installed from epel repository. The recursor is only listening on localhost on each system. 
> >>>>> 
> >>>>> I am experiencing somewhat random crashes of the recursor with the following error: 
> >>>>> 
> >>>>> kernel: pdns_recursor[21993]: segfault at 200001fc8 ip 0000000000472780 sp 00007f3f9c03f690 error 4 in pdns_recursor[400000+111000] 
> >>>>> 
> >>>>> Both servers have a large number of ip addresses bound to them, in the range of 3-4k. I was able to replicate the segfaults on one of the servers by adding additional ip addresses. When I got to around 4k ip addresses the recursor simply would not even start, just segafulted right away. I was able to get it to start again, by removing some ip addresses, so i know it has something to do with how many addresses I have bound the server. 
> >>>>> 
> >>>>> Any body have an ideas what I can do to correct this problem? I really don't see a reason why the recursor would care how many ip addresses I have on a system. 
> >>>>> 
> >>>>> thank you 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> _______________________________________________ 
> >>>>> Pdns-users mailing list 
> >>>>> Pdns-users at mailman.powerdns.com 
> >>>>> http://mailman.powerdns.com/mailman/listinfo/pdns-users 
> >>>>> 
> >>>> -- 
> >>>> 
> >>>> Imre Gergely 
> >>>> 
> >>>> http://havaz.net 
> >>>> 
> >>>> gpg --keyserver subkeys.pgp.net --recv-keys 0x34525305 
> >>>> 
> >> -- 
> >> Imre Gergely 
> >> http://havaz.net 
> >> gpg --keyserver subkeys.pgp.net --recv-keys 0x34525305 
> >> 
> > 
> >> _______________________________________________ 
> >> Pdns-users mailing list 
> >> Pdns-users at mailman.powerdns.com 
> >> http://mailman.powerdns.com/mailman/listinfo/pdns-users 
>
> -- 
> Imre Gergely 
> http://havaz.net 
> gpg --keyserver subkeys.pgp.net --recv-keys 0x34525305 
>