eDirectory crashed on start after a weekly cronjob rebooted the server as scheduled. Usually the server comes up with no problems. This time nds failed to start. eDirectory came up after a second reboot.

I used getcore to harvest the core file and I want to know if the single unresolvable stack pointer on line 13 of my dump indicates a problem with stack corruption which will cause eDirectory to crash on the next reboot.

#13 0x0000000000000000 in ?? ()

More details below.


TID 3078409 was a big help.

Jan 25 05:09:25 slesserver su: (to root) sysadmin on /dev/pts/0
Jan 25 05:22:29 slesserver nss: Starting Novell Storage Services (NSS)
Jan 25 05:22:30 slesserver nss: Start up eDir and LUM
Jan 25 05:22:30 slesserver nss: Waiting (60 seconds) for eDirectory(ndsd) to start
Jan 25 05:22:30 slesserver nss: ERROR: required eDirectory(ndsd) is not running. Exiting.
Jan 25 05:23:02 slesserver shutdown[16022]: shutting down for system reboot
Jan 25 05:23:02 slesserver init: Switching to runlevel: 6

I took an eDir crash(core) dump. [Handling ndsd (eDirectory) core files on Linux and Solaris TID 3078409, on how to pull an edir crash dump was helpful.]

slesserver:~ # file core.* /var/opt/novell/eDirectory/data/dib/
core.950: ELF 64-bit LSB core file x86-64, version 1 (SYSV)
/var/opt/novell/eDirectory/data/dib/: directory
slesserver:~ # ls -lh /var/opt/novell/eDirectory/data/dib/core.5062
-rw------- 1 root root 250M Jan 25 04:03 /var/opt/novell/eDirectory/data/dib/core.5062
slesserver:~ # ls -lh /var/opt/novell/eDirectory/data/dib/core.950
/bin/ls: /var/opt/novell/eDirectory/data/dib/core.950: No such file or directory
slesserver:~ # ls -lh /var/opt/novell/eDirectory/data/dib/core.*
-rw------- 1 root root 250M Jan 25 04:03 /var/opt/novell/eDirectory/data/dib/core.5062
slesserver:~ # gdb -version
GNU gdb 6.6
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
slesserver:~ # novell-getcore -b /var/opt/novell/eDirectory/data/dib/core.5062 /opt/novell/eDirectory/sbin/ndsd
Novell GetCore Utility 1.1.38 [Linux]
Copyright (C) 2009 Novell, Inc. All rights reserved.
[*] User specified binary that generated core: /opt/novell/eDirectory/sbin/ndsd[*] Processing '/var/opt/novell/eDirectory/data/dib/core.5062' with GDB...[*] PreProcessing GDB output...[*] Parsing GDB output...[*] Core file /var/opt/novell/eDirectory/data/dib/core.5062 is a valid Linux core[*] Core generated by: /opt/novell/eDirectory/sbin/ndsd[*] Obtaining names of shared libraries listed in core...[*] Counting number of shared libraries listed in core...[*] Total number of shared libraries listed in core: 44[*] Corefile bundle: core_20120125_081946_linux_ndsd_slesserver[*] Generating GDBINIT commands to open core remotely...[*] Generating ./opencore.sh...[*] Gathering package info...[*] Creating core_20120125_081946_linux_ndsd_slesserver.tar...[*] GZipping ./core_20120125_081946_linux_ndsd_slesserver.tar...[*] Done. Corefile bundle is ./core_20120125_081946_linux_ndsd_slesserver.tar.gz

slesserver:~ # ls -lh *gz
-rw-r--r-- 1 root root 17M Jan 25 08:19 core_20120125_081946_linux_ndsd_slesserver.tar.gz
slesserver:~ # mv core_20120125_081946_linux_ndsd_slesserver.tar.gz /home/sysadmin/
slesserver:~ # chown sysadmin:networkers /home/sysadmin/core_20120125_081946_linux_ndsd_slesserver.tar.gz
slesserver:~ #

*less novell-getcore_gdb_command_output.9394


#0 0x00002ac56f315646 in do_lookup_x () from /lib64/ld-linux-x86-64.so.2
#1 0x00002ac56f315a77 in _dl_lookup_symbol_x () from /lib64/ld-linux-x86-64.so.2
#2 0x00002ac56f3190a5 in _dl_fixup () from /lib64/ld-linux-x86-64.so.2
#3 0x00002ac56f31e4c2 in _dl_runtime_resolve () from /lib64/ld-linux-x86-64.so.2
#4 0x00002ac56f896e5f in FormSprintfString () from /opt/novell/eDirectory/lib64/libdsutil.so.1
#5 0x00002ac56f895739 in ParsePrintfArgs () from /opt/novell/eDirectory/lib64/libdsutil.so.1
#6 0x00002ac56f895c50 in DSvsprintf () from /opt/novell/eDirectory/lib64/libdsutil.so.1
#7 0x00002aaaacce5199 in DSHttpSubmit (iEvNum=<value optimized out>, pszFormat=0x54fbc60 "")
from /opt/novell/eDirectory/lib64/nds-modules/libimon.so
#8 0x00002aaaacce5ab9 in Init () from /opt/novell/eDirectory/lib64/nds-modules/libimon.so
#9 0x00002aaaaccdc2d4 in iMonMain (data=0x2aaaad037240) from /opt/novell/eDirectory/lib64/nds-modules/libimon.so
#10 0x0000000000414271 in PoolWorker ()
#11 0x00002ac57025b143 in start_thread () from /lib64/libpthread.so.0
#12 0x00002ac570583bed in clone () from /lib64/libc.so.6
#13 0x0000000000000000 in ?? ()