We can not keep this server up. We were always getting dead screens even though the kernel is pingable. This last time I managed to catch:

Unable to handle kernel paging request at ffffc200035db000 RIP:
<ffffffff8839ad4d>{:nsscomn:LONGNS_utf8ScanNextCom ponent+2731}
PGD db2067 PUD db1067 PMD 4d817067 PTE 0
Oops: 0002 [1] SMP
last sysfs file: /class/net/lo/address
CPU 0
Modules linked in: af_packet zapi nebdrv nsslsa nssmanage nsszlss nsscomn ndpmod nss nsslibrary nsslnxlib linuxmpk libnss admindrv nwraid nls_utf8 cifs apparmor aamatch_pcre loop reiserfs dm_snapshot dm_mod xenblk xennet
Pid: 13993, comm: httpd Tainted: GF X 2.6.16.54-0.2.12-xen #1
RIP: e030:[<ffffffff8839ad4d>] <ffffffff8839ad4d>{:nsscomn:LONGNS_utf8ScanNextCom ponent+2731}
RSP: e02b:ffff8800245d5a78 EFLAGS: 00010202
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000030
RDX: 0000000000000070 RSI: 0000000000000001 RDI: 0000000000000001
RBP: ffff880048db8140 R08: 0000000000000001 R09: ffff88001a500701
R10: ffff8800245d5abe R11: ffffc200035dae00 R12: ffff88001a500700
R13: ffffc200035db000 R14: 0000000000000775 R15: ffff880048db8020
FS: 00002b7258b28040(0000) GS:ffffffff803a2000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000
Process httpd (pid: 13993, threadinfo ffff8800245d4000, task ffff880047b27140)
Stack: 0000000000000001 00000000000000ff 0000000000000001 0000000000000000
0000000000000000 0000000000000001 ffff88001a500701 ffff88001a500701
00700070000280d2 ffff880048db8140
Call Trace: <ffffffff8834e9da>{:nsscomn:ScanNextComponentUTF8N ull+65}
<ffffffff8834f507>{:nsscomn:NMSG_ScanNextComponent +244}
<ffffffff883500a5>{:nsscomn:NMSG_ValidateNameMsg+1 969}
<ffffffff8830a394>{:nsscomn:COMN_Lookup+550} <ffffffff885ae40b>{:nsslsa:userKey+97}
<ffffffff885b0ee0>{:nsslsa:lookup_apply+785} <ffffffff885b0b1f>{:nsslsa:lsa_info+0}
<ffffffff885b10b9>{:nsslsa:LSACOMN_Lookup+35} <ffffffff885aaa02>{:nsslsa:lsa_lookup+104}
<ffffffff885a9ee8>{:nsslsa:lsa_d_compare+77} <ffffffff80177a2e>{__kmalloc+69}
<ffffffff801908d8>{d_alloc+355} <ffffffff80187a56>{do_lookup+214}
<ffffffff80189d57>{__link_path_walk+2478} <ffffffff8018a2ad>{link_path_walk+92}
<ffffffff8017bdd1>{__fput+351} <ffffffff80194ea5>{mntput_no_expire+25}
<ffffffff880f6654>{:linuxmpk:mpkExit+306} <ffffffff8018a711>{do_path_lookup+624}
<ffffffff80189194>{getname+347} <ffffffff8018af84>{__user_walk_fd+55}
<ffffffff80179cfe>{sys_faccessat+228} <ffffffff8017bdd1>{__fput+351}
<ffffffff80194ea5>{mntput_no_expire+25} <ffffffff8017943b>{filp_close+89}
<ffffffff8010b07e>{system_call+134} <ffffffff8010aff8>{system_call+0}

Code: 66 41 89 55 00 49 83 c5 02 48 83 2c 24 01 41 be 00 00 00 00
RIP <ffffffff8839ad4d>{:nsscomn:LONGNS_utf8ScanNextCom ponent+2731} RSP <ffff8800245d5a78>
CR2: ffffc200035db000

So our httpd seemed to be involved in this crash, but it appears tied to NSS. We had tried taking the server out and back into the tree as well as replacing the whole NSS volume with another with no luck.

Really would enjoy hearing a resolution to this one.

thanks,
JES