Re: Newbie storage questions... (RAID5, SANs, SCSI)

"David Sworder" <[EMAIL PROTECTED]> wrote in message news:<[EMAIL PROTECTED]>...
>     That was a very helpful post. After reading some of your other posts on
> this newsgroup (via Google), it's clear to me that I/O operations/sec are
> more important to me than total bandwidth. What's odd is that the SCSI
> manufacturers such as Adaptec don't list IOs/sec on their spec sheets which
> makes it difficult to compare apples to apples. How would one go about
> finding the IOs/sec for a SCSI RAID card? That fibre channel unit that I
> mentioned in my original post handles 40,000 IOs/sec according to the Dell
> site. I'd like to see how high-end SCSI RAID cards compare.


It's usually a non-issue.  You're grossly limited by the drive
subsystem, which will rarely allow more than a few hundred random I/Os
per second.  Sequential I/O can run significantly faster, but only
rarely do you do long sequential writes on a database.  On sequential
reads (for instance during a table scan), most SCSI (RAID or
otherwise) can pretty well keep up with the disk drives.

Just to put a number on things, let's say you've got a RAID 5 array of
five drives with 5ms typical access time (pretty optimistic).  So each
drive can do about 200 I/Os per second.  So you could sustain
something like 1000 random reads per second (with zero writes), or 250
writes (with zero reads).  Or at something like 80% reads, a total of
625 (mixed) I/Os per second.

With extensive caching at the RAID controller you can get higher
numbers, but on modest sized systems like the one you're discussing,
it's almost always a better idea to put extra cache into the server
instead of the disk subsystem.

FC HBAs are often attached to very large disk arrays (sometimes
thousands of drives), where the number of I/Os per second the *HBA*
can sustain can become a limiting factor.  It's rare to see a SCSI
RAID subsystem that can support more than a couple of dozen drives
which puts a pretty low upper limit on the number of random I/Os per
second.  In any event, vendors quote raw I/Os per second for FC HBAs
because that's all they can measure, as there's no disk involved.  The
vendor of the disk subsystem that you attach to you FC HBA will have a
set of performance numbers, those can be compared to the SCSI RAID
controller performance figures.

On a small system, the FC HBA will simply not be the bottleneck for
random I/Os, the drive array *will* be the bottleneck.  It doesn't
matter that your HBA can do 40,000 I/Os per second if it's only
talking to a drive subsystem with a dozen disks that can hit 1500 IO/s
only with a tailwind.  OTOH, hang 30 of those subsystems on your SAN,
and you're going to be limited by the HBA (assuming you've only got
the one host).


>     The next logical question in this little learning exercise of mine is:
> What exactly defines an IO operation? Maybe an example would clarify my
> question...
> 
>     Let's say I have a machine with one hard drive. SQL Server runs on that
> machine and attempts to read 4,000 bytes of data sequentially


SQL Server (and most other DBMSs) will typically format his datasets
in blocks that are some power-of-2 multiple of 4KB.  He'll then read
or write those blocks as units (or in groups of blocks if he can).


>from the
> drive. This would count as one IO operation correct? Now let's change the
> situation so that instead of one harddrive, I have that cool fibre channel
> device that I mentioned in my previous post. It has 10 drives configured as
> RAID 1+0. In this situation, when SQL Server attempts to read 4,000 bytes,
> Windows still thinks of this as 1 IO operation -- but does the FC device
> consider this to be 1 operation? In order to read 4,000 bytes, the FC device
> is accessing all 10 disks simultaneously. So is this operation considered as
> *ten* IOs, or only one?
> 
>     I'm asking this question because my books states that I should closely
> monitor the IOs/second in 'perfmon' and make sure that it does not exceed
> 85% of the maximum throughput. So if the FC RAID device above supports a
> maximum of 40,000 IOs/second, I'd want to make sure that I don't exceed
> 34,000 IOs/second on a regular basis. Tracking IOs/second is easy using the
> Windows PerfMonitor, but I'm not sure if PerfMon is tracking the correct
> value since Windows has no way of knowing that each IO read/write will
> trigger multiple read/writes across the various drives in the array. Do you
> see what I mean? So what exactly defines an "IO" on a RAID device, be it a
> SCSI RAID or an FC RAID?


Loosely, an I/O is a single read or write operation.  The size is
context dependent, but in the case of a DBMS it's going to typically
be the page size for the table or table space.  If you've got hardware
RAID, the host will see a single I/O for either a read or a write. 
The RAID controller will issue multiple I/Os to the attached devices
as necessary.  For example, let's say you have a FC HBA in the host
connected to a RAID disk subsystem that's got a bunch of disk drives
on an internal SCSI channel.  Your database writes a disk page.  There
will be a single I/O across the fiber from the host to the RAID
controller, and then (assuming no fortuitous caching) *four* I/Os
across the internal SCSI channel from the RAID controller to the disk
drives.