Tweakers.net's findings
Our observations clearly show that the testing methods of AnandTech and Storage Review are too limited to jump to conclusions about the use of striping in desktop environments - let alone that they could be used to sign RAID 0's death sentence. Both AnandTech's and Storage Review's results of the IPEAK largely contradict Tweakers.net's benchmarks, where RAID 0 does show significant improvements in the I/O performance.
In the case of Storage Review, this can be explained by their use of traces based on older software, which isn't representative of present-day 'power user' workloads. Using system benchmarks to assess storage performance is at the very least questionable anyway, because they mainly depend on CPU performance. The same thing goes for the loading times of games - not to mention the fact that a test limited to two games will never give a realistic view of performance in general.
Most striking is the fact that neither AnandTech nor Storage Review even bothered to test RAID in situations where the performance of the storage system really matters - for example when a backup application, anti-virus tool or compression program is active in the background while demanding content creation applications are running in the foreground. These are conditions that quickly give rise to noticeable latencies and in which single drive setups don't suffice anymore.
Tweakers.net bases its testing methods on intense system use of a 'power user'. Such users are, after all, our target audience. Besides traces of Business Winstone 2004, Business Winstone 2004 Multi-tasking Test and Multimedia Content Creation Winstone 2004, we created traces of several games using IPEAK Storage Performance Toolkit and we recorded disk access in I/O-intensive actions, such as copying files, defragmenting partitions, running a virus scanner and installing software. As is to be expected from a 'power user', we often employed multitasking, though not in an unrealistic way. All in all we created 27 tests, together resulting in our Desktop StorageMark 2004, Gaming StorageMark 2004 and Workstation StorageMark 2004 indexes. The result is a broadly oriented test suite, which gives a realistic image of performance in various situations.
Subsuites and tests with 'StorageMark 2004'
On previous pages, we already mentioned differences between benchmarks by AnandTech and Storage Review on the one side, and Tweakers.net on the other side. Testing RAID configurations is not what we would call a new experience. Benchmarking RAID 0 performance is standard procedure when we test new ATA and SATA drives. Since last year, Tweakers.net has tested more than 25 different SATA and SCSI RAID adapters in varying situations and configurations. We know about the possible results that RAID and striping can give us. Anyone who would like to be informed on this subject can have a look at our Benchmark Database, links to a Dutch page.
In this article, we focus even more on RAID 0's possibilities by not only showing IPEAK benchmarks, but also doing some measurements with a stopwatch. To offer a clear view on the differences concerning I/O performance, on the next page we give you the results of desktop-, gaming- and workstation-indexes. Not only configurations with one or two Raptors of the first and second generation were tested, but also a heavily armed LSI Logic MegaRAID SCSI 320-2X with 512MB cache and four Maxtor Atlas 15K SCSI-disks. This MegaRAID SCSI 320-2X is also available as the Intel SRCU42X and is the fastest SCSI RAID adapter at the moment. His job: proving good RAID 5 performance with up to date RAID implementations. This way, we can immediately finish the dated dogma about bad performance of RAID 5. The MegaRAID SCSI 320-2X will show us another advantage of (high-end) RAID adapters, the onboard cache that, together with the striping, is able to offer a remarkable improvement over cache-less RAID controllers.

In the case of Storage Review, this can be explained by their use of traces based on older software, which isn't representative of present-day 'power user' workloads. Using system benchmarks to assess storage performance is at the very least questionable anyway, because they mainly depend on CPU performance. The same thing goes for the loading times of games - not to mention the fact that a test limited to two games will never give a realistic view of performance in general.
Most striking is the fact that neither AnandTech nor Storage Review even bothered to test RAID in situations where the performance of the storage system really matters - for example when a backup application, anti-virus tool or compression program is active in the background while demanding content creation applications are running in the foreground. These are conditions that quickly give rise to noticeable latencies and in which single drive setups don't suffice anymore.
Tweakers.net bases its testing methods on intense system use of a 'power user'. Such users are, after all, our target audience. Besides traces of Business Winstone 2004, Business Winstone 2004 Multi-tasking Test and Multimedia Content Creation Winstone 2004, we created traces of several games using IPEAK Storage Performance Toolkit and we recorded disk access in I/O-intensive actions, such as copying files, defragmenting partitions, running a virus scanner and installing software. As is to be expected from a 'power user', we often employed multitasking, though not in an unrealistic way. All in all we created 27 tests, together resulting in our Desktop StorageMark 2004, Gaming StorageMark 2004 and Workstation StorageMark 2004 indexes. The result is a broadly oriented test suite, which gives a realistic image of performance in various situations.


StorageMark 2004

Desktop StorageMark 2004

Desktop StorageMark 2004 Index (79)

Business Winstone 2004 (84)

Business Winstone 2004 Multi-tasking Test (79)

Filecopy A (83)

Filecopy B (83)

Filecopy C (83)

Winzip compressie (83)

Virusscan (83)

Defragmentatie (83)

Windows XP boot (83)

Explorer zoekopdracht (83)

Windows user switch (83)

Software installatie (83)

Windows Update (83)


Gaming StorageMark 2004

Portable StorageMark 2004

Server StorageMark 2004

Server StorageMark 2004 Index (51)

Disk-to-Disk Backupserver - Low concurrency (58)

Disk-to-Disk Backupserver - High concurrency (62)

Fileserver - Small filesize (62)

Fileserver - Medium filesize (62)

Fileserver - Large filesize (62)

MySQL - Data drive (62)

MySQL - Log drive (62)

MySQL Dump (62)

MySQL Import (62)

MySQL Startup (62)

MySQL Recovery (62)

Streaming media server - Small dataset (59)

Streaming media server - Large dataset (59)

Exchange 2003 - Medium concurrency (51)

Exchange 2003 - High concurrency (51)


Temperatuur- en geluidsdrukmetingen

Workstation StorageMark 2004

Workstation StorageMark 2004 Index (79)

Business Winstone 2004 Multi-tasking Test (79)

Business Winstone 2004 Multi-tasking Torture Test (84)

Multimedia Content Creation Winstone 2004 (84)

Filecopy A (83)

Filecopy B (83)

Filecopy C (83)

Winzip compressie (83)

DVD strip (83)

Photoshop load/save (83)

Audio editing (83)
On previous pages, we already mentioned differences between benchmarks by AnandTech and Storage Review on the one side, and Tweakers.net on the other side. Testing RAID configurations is not what we would call a new experience. Benchmarking RAID 0 performance is standard procedure when we test new ATA and SATA drives. Since last year, Tweakers.net has tested more than 25 different SATA and SCSI RAID adapters in varying situations and configurations. We know about the possible results that RAID and striping can give us. Anyone who would like to be informed on this subject can have a look at our Benchmark Database, links to a Dutch page.
In this article, we focus even more on RAID 0's possibilities by not only showing IPEAK benchmarks, but also doing some measurements with a stopwatch. To offer a clear view on the differences concerning I/O performance, on the next page we give you the results of desktop-, gaming- and workstation-indexes. Not only configurations with one or two Raptors of the first and second generation were tested, but also a heavily armed LSI Logic MegaRAID SCSI 320-2X with 512MB cache and four Maxtor Atlas 15K SCSI-disks. This MegaRAID SCSI 320-2X is also available as the Intel SRCU42X and is the fastest SCSI RAID adapter at the moment. His job: proving good RAID 5 performance with up to date RAID implementations. This way, we can immediately finish the dated dogma about bad performance of RAID 5. The MegaRAID SCSI 320-2X will show us another advantage of (high-end) RAID adapters, the onboard cache that, together with the striping, is able to offer a remarkable improvement over cache-less RAID controllers.

Next page (StorageMark 2004 results - 5/9)
