We have a test OES sp2 server that has a 3T SAN pool containing a single
3T DATA: volume, and were needing to check the procedures for performing
NSS pool rebuilds and verifications. The server and pool have been
available, accessible and working for several weeks.

The server is running on an HP blade with 8G of RAM, accessing 3T of SAN

I deactivated the pool yesterday by entering nsscon and running the
command /PoolDeactive=SANP2.

Attempting to then run "ravsui verify SANP2" or "ravsui rebuild SANP2"
results in the error messages:

> NSS error: Verify aborted prior to producing detailed information
> Status: 20000
> Source: repairZVP.c[1864]

> Press any key to continue....

> NSS error: Rebuild aborted prior to producing detailed information
> Status: 20000
> Source: repairZRP.c[782]

> Press any key to continue....

Attempting to take the pool back *out* of maintanance mode through the
NSS console results in the error message:

> ewsdev01> nss /poolActivate=SANP2
> Activating pool "SANP2"...
> ** Pool layout v43.02

> Aug 30, 2007 4:51:00 pm NSS<COMN>-4.09a-689: /usr/src/packages/BUILD/kernel-bigsmp-2.6.5/modules-2.6.5/nss/comn/comnPool.c[421]
> Could not change pool SANP2 to the ACTIVE state.
> Status=20810 /usr/src/packages/BUILD/kernel-bigsmp-2.6.5/modules-2.6.5/nss/zlss/zfsPool.c[1881].
> Use 'NSS /ErrorCode=20810' to obtain more information.

> ewsdev01> /ErrorCode=20810

The NSS console also displays:

> Aug 30, 2007 4:49:39 pm NSS<NSSLIB>-4.09a-217: /usr/src/packages/BUILD/kernel-bigsmp-2.6.5/modules-2.6.5/nss/library/zalloc.c[82]
> Error allocating 95999776 bytes of memory.
> You may not have enough memory. Either close some other applications or add more memory.

The system was running kernel 2.6.5-7.282 when the problem occurred, has
been updated to kernel 2.6.5-7.286 and rebooted, with no change to the
behaviour when I try and verify, rebuild or re-enable the pool.

According to "top" there is 7.5G of RAM free!

Mem: 8244148k total, 567648k used, 7676500k free, 29728k buffers
Swap: 8388600k total, 0k used, 8388600k free, 226244k cached

Any suggestions on how we can verify or rebuild the NSS pools, or at
least remount them?

As an aside, the mix of nss commandline tools seems very poorly thought
out, swapping back and forth from command-line nss* to an "NSS console"
to something called "ravsui" (a completely non-intuitive name)