A few months ago we purchased what I thought was should be a very
capable server to run our file, print, and other services as our
first OES2-Linux server. Unfortunately so far in my pre-production
tests it has proven to be totally unfit performance-wise, and I just
don't know what to do. Our situation may have some commonalities
with the recent thread "NSS performance with OES2", but I think we
have faster hardware.

Let me start by describing one simple test that shows just how bad
the problem is. I connected my new workstation with a Gb NIC
directly to the server via a crossover cable. I copied 12.6 GB of
data from an NSS volume to my workstation (with my antivirus off).
It took 11 minutes, 30 seconds, an unbearable rate of 18.3 MB/s. I
obviously can't put the server into production like this.

The server is a Dell PowerEdge 2900 III - Quad core Xeon 2.5 GHz,
8 GB of RAM. The hard drives are all 15K RPM SAS connected to a Dell
PERC 6/i RAID controller. I've installed the 64-bit versions of
SLES 10 SP1 and OES 2 onto two of the drives which are configured as
RAID1 with ext3. The other 4 drives are arranged in RAID5 and are
to be dedicated to our NSS volumes. It also has an LTO4 tape drive,
which I chose in part for its speed.

The problem began when backup tests resulted in very slow speeds,
around 12 MB/s. I was eventually referred to a couple of HP test
utilities which seemed to indicate that the problem is with our hard
disk system, not the tape drive. I've done all kinds of benchmark
tests on the drives since then, but I'll just include the most recent
ones here. I've used Bonnie (I couldn't get Bonnie++ to work
correctly on the NSS/RAID5) and IOzone, which is apparently favored
by Novell support.

Dell gave me two suggestions, one to experiment with blockdev, and
to try changing the I/O elevator. Neither really seemed to make
much difference, although I may not have know what I was doing.

Questions: (1) Are these benchmarks valid? Were these good tools to
use and did I seem to use them properly? (2) Is NSS the problem, or
is it something related to our hardware setup or SLES? (3) Is there
anything that I might do to fix the situation and make the server
usable? (4) Failing everything else would Novell support be capable
of helping us on this?

I would appreciate any assistance. Things are getting so dire that
I'm considering putting NetWare on the server.


Jim Wagner


Benchmarks used 8 GB file size to eliminate the effects of caching

Bonnie- System (ext3/RAID1)

oestest4:~ # bonnie -s 8192
Bonnie 1.4: File './Bonnie.26368', size: 8589934592, volumes: 1
Writing with putc()... done: 57898 kB/s 77.7 %CPU
Rewriting... done: 49505 kB/s 6.4 %CPU
Writing intelligently... done: 151523 kB/s 30.6 %CPU
Reading with getc()... done: 58176 kB/s 68.5 %CPU
Reading intelligently... done: 111187 kB/s 6.2 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
oestes 1*8192 57898 77.7151523 30.6 49505 6.4 58176 68.5111187 6.2 1719.8 1.4


Bonnie- Data (NSS/RAID5)

oestest4:/media/nss/VOL1/public/junk # bonnie -d . -s 8192
Bonnie 1.4: File './Bonnie.14854', size: 8589934592, volumes: 1
Writing with putc()... done: 71004 kB/s 99.5 %CPU
Rewriting... done: 28264 kB/s 0.5 %CPU
Writing intelligently... done: 314260 kB/s 37.2 %CPU
Reading with getc()... done: 19187 kB/s 4.8 %CPU
Reading intelligently... done: 31983 kB/s 0.0 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
oestes 1*8192 71004 99.5314260 37.2 28264 0.5 19187 4.8 31983 0.0 1094.0 3.7

IOzone- System

Excel chart generation enabled
Auto Mode
File size set to 8388608 KB
Record Size 512 KB
Command line used: ./iozone -Ra -s 8G -r 512K
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
8388608 512 155817 100328 109573 108636 89959 69436 94781 3623919 105521 151075 144792 110306 111444

IOzone- Data

Excel chart generation enabled
Auto Mode
File size set to 8388608 KB
Record Size 512 KB
Command line used: ./iozone -Ra -s 8G -r 512K
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkwd record stride
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
8388608 512 276143 225427 31951 31984 24585 63248 30555 5745377 28546 264559 255717 31962 31976