Software RAID vs.

Hardware RAID

Introduction System CPU Load

RAID implementations contain components such As we have already stated, software RAID
as RAID tables defining the configuration of RAID solutions are typically implemented as kernel-
arrays, data structures to store the descriptors for mode components. In fact under Linux it is
cached data, engine(s) for calculating parity and incorporated into the kernel itself. How does
the logic for handling I/Os to and from RAID that impact the CPU? Most kernel mode
arrays. These components may be implemented in components avoid spawning threads to avoid
software – typically in kernel-mode – or embedded the costly overhead of context switching. However,
in the controller for the secondary storage devices kernel mode components are still at the mercy
using which the RAID arrays are created. Which of the scheduler that preempts their operation
alternative is better? This paper answers that as soon as their time quantum expires or a higher
question by presenting an analysis of the issues priority task is scheduled. Thus, even under the
associated with both alternatives and their most hospitable circumstances, a kernel-mode
performance in a real-world environment. RAID engine is compelled to share processor
time with other kernel mode components and
the overlying applications that use them. This
RAID in Software
may not be critical if those applications are
Mainstream system processors continue to evolve
docile with respect to their processing needs.
on a very aggressive curve. We have come a long
However, certain applications (and their under-
way since Intel’s introduction of the first modern
lying drivers) and environmental factors can
microprocessor in 1982. Its 80286 with 134,000
overwhelm the CPU. Let us look at some
transistors achieved speeds of 12 MHz and delivered
of the them.
up to 41K FLOPS with the assistance of its 80287
co-processor. Intel’s flagship (at the time this article
was written), the Pentium 4 with 42,000,000 transis- Network Traffic
tors, achieves a blazing 1.7 GHz, and delivers up Servers by their intrinsic nature are networked
to 900 MFLOPS. This growth – and the implicit to provide services to clients over a network.
assurance of enhanced performance by the use For this reason the effect of network traffic on
of succeeding generation of processors – has servers is of significance. Network interface
enticed developers to place greater loads on cards (NIC) are heavily reliant on the system
system CPUs with a menagerie of applications. CPU for protocol-specific processing and
Software based RAID is one of them. transferring data to and from physical memory.
In fact, they consume a disproportionately large
However, there are some drawbacks to implementing
amount of CPU time in view of this dependency.
RAID in software. First is the issue of portability.
This section presents a picture of how NICs
Since a software implementation will undoubtedly
work and interact with drivers in a system.
have OS-specific components, those components
will have to be re-written for each OS. The second
issue is the one that haunts kernel-mode software
developers. Kernel-mode programs have to be
perfect. Unlike applications, their ability to execute
privileged instructions and manipulate the contents
of any virtual address leaves the system without
any safeguards against programming errors.
The consequence can be a crashed system!
Software RAID vs. Hardware RAID 2

Application • The individual packets must be sequenced.

Transport Driver The transport driver at the receiving system
NIC Driver must re-sequence the packets in the correct
order to reconstruct the original data stream.
• The data content of each received packet must
NIC be copied to system memory at the receiving
system. Note that, DMA is generally not an
option available on NICs; hence operations
to copy data to system memory requires the
To Ethernet system CPU to be interrupted and used to
execute the operation – a process commonly
Figure 1 – Hierarchy of Network Drivers known as Programmed I/O (PIO). Conversely,
data supplied by applications on the transmitting
system has to be copied into network packets
NICs are managed by NIC drivers. Such drivers
constructed appropriately for transmission.
perform functions such as handling interrupts
Furthermore since the size of data packets are
from the NIC, receiving and sending packets to
restricted (though configurable) by each protocol
and from the network, and also providing an
to a size of approximately 1 KByte, it implies
interface to set or query operational characteristics
frequent CPU interruptions when the quantity
of the NIC. An NIC driver typically interfaces with
of data being transmitted is large.
a transport driver above it. A transport driver
implements the stacks for network protocols such
Clearly, these steps provide a good qualitative
as TCP/IP or IPX/SPX. It successively strips and
picture of the burden placed on the system CPU
interprets the network-protocol layers of the packets
by network traffic. Now to get a quantitative picture
handed to it by the NIC driver and transfers the
of this scenario, we recommend a little experiment
data contained in the “stripped” packets to system
to the reader. Log into a NT or Windows 2000
memory. Conversely, it wraps data supplied to it
system that has a network card and is attached to
by the overlying application with suitable layers
your intranet. Fire up Performance Monitor which
required by the network-protocol and hands it
is a standard administrative tool shipped with the
off to the NIC driver for transmission. Figure 1
OS. Within Performance Monitor, switch to the
displays the network driver hierarchy. These
“Chart” view if Performance Monitor does not
drivers handle the bulk of the tasks involved in
already display it. Add the counters % Interrupt
processing network packets, and since these drivers
Time and % DPC Time to this view. These repre-
are executed in the system’s CPU, that CPU bears
sent the percentage of CPU time taken to service
the entire associated processing burden. How severe
hardware interrupts and DPCs. Now select some
is this burden? To answer this question, consider a
files on any server on the network and copy them
client server application built atop sockets using
on to your local hard drive(s). It would be preferable
TCP, and the important processor intensive steps
if the amount of data is large – 100 MByte or more –
that the network drivers must take for such an
so that you can get Performance Monitor to display
application to function correctly.
the values for the aforementioned counters over a
• The use of TCP based sockets implies longer span of time. Note down the approximate
guaranteed delivery of each transmitted median values for the counters. It should not come
packet without loss of integrity. Packets can as a surprise if the approximate values for your %
be easily lost or garbled during transmission. Interrupt Time is (or exceeds) 10%, and that for
Therefore the network drivers at the receiving the % DPC Time is (or exceeds) 25%! In other
system must request re-transmission when words, about a third (or more) of your processors
necessary and the network drivers at the time is spent in being interrupted and completing
transmitting system must have the appropriate I/Os. This experiment should convince the user of
mechanism to comply with such requests. the expense involved in processing network traffic.
Software RAID vs. Hardware RAID 3

Application RAID in Hardware

While the applications driving file and print There are several advantages to implementing
servers have a negligible impact on the CPU, RAID in hardware. Let us first take a look at
application servers tend to impact the CPU severely. embedded processors that are at the heart of
To understand why, let us take a look at the nature hardware RAID solutions. What is their horse-
of application servers. Typically application servers power? Though embedded processors are designed
are the back-end of complex business applications to be application-centric, any mainstream processor
that satisfy the following requirements – high- can be used for embedded development. In fact,
availability, high-performance and redundancy. the cores for embedded processors are usually
related (if not identical) to their mainstream
Consider an application server that envelops a
counterparts. Consequently, the upper bound of
relational database. Anyone familiar with relational
their processing power is no less than that for the
databases is acutely aware of the computational
mainstream ones. However, in practice, embedded
expense of performing many of the standard
processors are generally several orders of magnitude
operations. Operations such as inner joins –
slower than mainstream processors. Why? It is
in mathematical terms – have an order of O(mn)
usually a function of price. Embedded processors
where m and n are the size of the record sets.
are designed to address the needs of specific
Furthermore these results cannot be preprocessed
applications, and are not expected to perform the
since the record sets for most applications are
generalized role of mainstream processors. It is
dynamic, i.e., they change with time. As a
this niche role that usually imposes restrictions
consequence their demand on computing
on their price, and in turn on the horsepower
resources is enormous.
that can be strapped on to them.
Is hardware RAID more efficient than software
OS Architecture and Components
RAID? The answer is yes. First, the RAID firm-
The architecture of the OS can play a role in
ware is executed on a dedicated processor and
affecting CPU load. While a high degree of
therefore does not share the system’s CPU(s)
modularity ensures robustness and facilitates
with other kernel mode components and the
ease of maintenance of OS components, it also
overlying applications that use them. This has
introduces performance latency at inter-module
all the advantages of asymmetric multi-processing.
interfaces. Furthermore, the efficiency of imple-
Second, it is portable across operating systems
mentations for open standards can vary from one
and in the event of a malfunction in the RAID
OS to another. For instance, comparisons of CPU
hardware or firmware, the server can usually
utilization using identical NICs and applications
continue to operate and even inform the user
on Netware and NT often displays disparity in
of the malfunction (assuming that there is a
performance that can be attributed to one or both
watchdog implementation in place). Conversely,
of the following factors – the relative efficiency
if the server crashes due to some unexpected event,
of the NDIS implementations and the relative
hardware RAID generally offers better survivability.
degree of modularity of the operating systems.
Many hardware RAID solutions are armed with
In summary, the load on the system CPU can battery backup modules that allow them to main-
be substantial due to the aforementioned factors tain the coherency of their caches and complete
even when discounting I/O processing to and outstanding operations without loss of integrity.
from secondary storage. Clearly, there is a need to Finally, one of the great advantages offered by
employ auxiliary processors to execute that role and hardware RAID is the fact that, the arena of
relieve the system CPU of the additional burden. embedded development is centered on the principle
Let us now take a look at hardware RAID in detail of specialization for a target application. Consequently,
and illustrate some of the salient aspects of its hardware RAID often incorporates features that
architecture that enhances performance. are specialized for optimizing performance.
Software RAID vs. Hardware RAID 4

Examples of such specialized features include the Two sets of NetBench Disk tests were conducted
following. on RAID 5 arrays, the first set utilizing one array
comprised of six disks and the latter utilizing two
• Use of auxiliary processor(s) dedicated to
arrays comprised of six disks each. The Adaptec
calculating the parity for data blocks that are
SCSI RAID 3210S – a mid-range SCSI controller –
to be written to disk while the main embedded
with 64 MByte of on-board RAM was pitted as a
processor is concurrently fetching or executing
representative of hardware RAID against the native
the next instruction in the RAID (firmware)
software RAID utility provided by Windows 2000
code. This hardware component is not found
server used in conjunction with an Adaptec 39160
on non-RAID HBAs.
SCSI card. Table 1 displays the configuration details
• Use of dedicated cache(s) on the controller and Figure 2 the corresponding cumulative network
for reading or writing data. While the advantage throughput for the first test. Table 2 displays the
offered by the use of a cache for reading is configuration details and Figure 3 the corresponding
rather obvious, the advantage when writing cumulative network throughput for the second test.
may warrant a little explanation. A cache offers
Note that these tests are intended to illustrate the
the host the opportunity to transparently
general superiority of hardware RAID to software
complete “write” commands even while the
RAID and the use of a mid-range controller for hard-
read-write heads on the disk to which the
ware RAID is sufficient for that purpose. Certainly
command is targeted is seeking the appropriate
the use a high-end hardware RAID controller can
sector(s) for writing the associated data. This
be expected to amplify this superiority further.
obviates the need to interrupt the host and
notify it when a desired sector has been sought
by the read-write head permitting it to execute
a write operation. Additionally, it also allows
the controller to coalesce contiguous “dirty”
data blocks that have accumulated over time, Operating System Windows 2000 Server
and write them out in a consolidated chunk. System Memory 1 GByte, PC133
Clearly, this has the advantage of reducing the RAID Type RAID 5
time spent in seeking the appropriate sectors on Number of Drives 6
disks into which to write the individual blocks.
Drive Type Seagate ST318451LC, 15K rpm,
18.35 GByte
Performance Results Number of Arrays 1
To obtain a quantitative picture of the superiority NIC Intel PRO/1000 T Server Adapter,
in performance of hardware RAID to software 1 GBit
RAID consider the following performance test Hardware RAID Software RAID
results obtained from using NetBench Disk test Controller Adaptec Adaptec
(version 7.0). NetBench is an application that 3210S SCSI Card 39160
measures the performance of file servers handling SCSI Interface Ultra160 Ultra160
network file requests from clients running Available
Channels 2 2
Windows® 95/98, Windows NT® or Windows 2000.
Channels Used 1 1

Table 1 – Test Configuration

Software RAID vs. Hardware RAID 5

NetBench Disk Test – using 1 RAID Array NetBench Disk Test – using 2 RAID Arrays
250 300
Total Network Throughput in Mbit/sec

Total Network Throughput in Mbit/sec

250
200

200
150

150
100
100

50
50

0 0
1 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 1 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60
Number of Clients Number of Clients
Software RAID Hardware RAID Software RAID Hardware RAID

Number of Software RAID Harware RAID Number of Software RAID Harware RAID
Clients Mbit/sec Mbit/sec Clients Mbit/sec Mbit/sec
1 5.6 5.8 1 5.4 5.7
4 22.3 23.1 4 21.5 23.1
8 43.3 40.0 8 41.1 46.1
12 63.2 69.0 12 65.4 68.8
16 81.0 91.2 16 86.2 91.6
20 96.1 113.0 20 105.9 113.7
24 103.8 134.3 24 123.9 134.7
28 109.5 154.3 28 140.8 156.4
32 107.4 175.7 32 156.9 175.2
36 98.6 190.3 36 169.5 195.8
40 94.6 204.5 40 175.9 211.4
44 90.2 208.0 44 183.6 228.0
48 85.7 198.1 48 188.4 239.7
52 80.1 180.8 52 190.0 240.9
56 74.0 174.4 56 188.0 245.6
60 73.8 167.1 60 185.4 236.3
Figure 2 – Software vs. Hardware RAID Performance Figure 3 – Software vs. Hardware RAID Performance
Using 1 RAID Array Using 2 RAID Array

Operating System Windows 2000 Server

System Memory 1 GByte, PC133
RAID Type RAID 5
Number of Drives 6 per Array
Conclusion
Drive Type Seagate ST318451LC, 15K rpm,
18.35 GByte Hardware RAID is a superior solution to software
Number of Arrays 2 RAID in a networked environment as is typical for
NIC Intel PRO/1000 T Server Adapter, servers. Its benefits are even more significant when
1 Gbit running applications with high CPU utilization.
Hardware RAID Software RAID
Controller Adaptec Adaptec
3210S SCSI Card 39160
SCSI Interface Ultra160 Ultra160
Available
Channels 2 2
Channels Used 2 2

Table 2 – Test Configuration

Software RAID vs. Hardware RAID 6

Glossary Kernel
The central component of an operating system that is
Application Server
typically responsible for memory, process, security and
An application server is the engine that acts as the
I/O management.
intermediary for data and services between a “thin”
web-enabled client in the front-end and a database
Multi-Processing
or repository of some form in the back-end. This
Multi-processing is the division of labor in computing,
may include web-servers, OLTP servers etc.
with each processor executing a distinct set of tasks.
If the set of tasks being executed by one processor
Asymmetric Multi-Processing
is reasonably independent of the set of tasks being
Multi-processing using two or more processors that
executed by another (or vice-versa) then multi-processing
are not equivalent in their capabilities and their use.
can yield significant performance gains.
Cache
NDIS
A part or whole of a dynamic memory space that is
Network Driver Interface Specification. It is the specifi-
used to store data being written to secondary storage
cation for the interface between device drivers and a
and subsequently read from it.
network. All transport drivers call the NDIS interface
to access and work with NICs.
Context Switch
The action by which the state information for a
O(n)
process whose execution is stopped (by the scheduler)
Pronounced as “order of n”. If an algorithm (or heuristic)
are swapped out and that for a dormant process that
dependent on the variable n has a complexity of O(n),
is to begin execution are swapped in.
then the algorithm (or heuristic) takes time propor-
tional to n to complete execution.
CPU
Central Processing Unit (of which a system may
Outer Join
have one or more).
Simple union of all records from two tables.
Dirty Data
Physical Memory
Data that is residing in cache but has not been written
Dynamic memory or simply random access memory
to its target (such as secondary storage).
(RAM).
DMA
PIO
Direct Memory Access. Methodology by which an
Methodology by which I/O transfers to and from
auxiliary processor transfers data between a peripheral
secondary storage are performed by the system CPU.
device and the system memory without the inter-
vention of the system’s main CPU(s).
RAID
Redundant Array of Inexpensive Disks. Methodology
DPC
using which multiple disks are coalesced to form an
Deferred Procedure Call. It is a software routine that
array that provides redundancy and higher availability
is part of a driver invoked when an I/O is completed.
of data.
I/O completion typically involves checking I/O status,
forwarding I/Os (returned by the underlying drivers)
Relational Database
to overlying drivers in a layered driver model and
Database that employs multiple “related” tables for
executing cleanup actions that may be necessary.
storing data.
Embedded
Scheduler
In conjunction with the terms processor or develop-
Component of the OS kernel that controls the order
ment refers to the area of specialized applications
and time of execution of processes and their associated
that typically run on a single micro-processor board
threads.
with the program residing in flash memory.
Virtual Address
Inner Join
Address that is not necessarily backed up by physical
Combines records from two tables whenever there
memory. Typically the virtual address space is signifi-
are matching values in a common field
cantly larger than the physical memory size, and is
backed up by on-disk space.

Watchdog
An application which “watches” over specified target
component(s). Typically a watchdog performs a set
of diagnostic checks at pre-specified intervals on its
target component(s), and perform suitable action
depending on the status of its target.
Copyright 2002 Adaptec, Inc. All rights reserved. Adaptec and the Adaptec logo are trademarks
of Adaptec, Inc. which may be registered in some jurisdictions. Microsoft, Windows, Windows NT,
Windows 95/98/2000 are trademarks of Microsoft Corporation, used under license. All other
trademarks used are owned by their respective owners.

P/N 666261-011 Printed in USA 2/02