Before evaluating our own findings and having a look at the benchmarks conducted by AnandTech and Storage Review, we should determine to what type of applications our tests are related. The desktop environment not just implies spreadsheets, word processing, browsing, mail clients and other, rather 'light' applications; on the desktop of the power user p2p-sharing, image- and video editing and backing up all kinds of digital media are frequent activities, too. AnandTech, Storage Review and Tweakers.net all share the same audience: power users and tweakers who want to maximize power, speed and usability of their rig. Unlike the average secretary, our target audience is continuously multi-tasking, which can include downloading everything that can be found on the internet, while taking pictures at the highest possible settings. In short: they demand the highest performance from their desktop.
The testing methods used by all three sites have a few things in common. Storage Review, AnandTech and Tweakers.net all use the famous IPEAK Storage Performance Toolkit from Intel, that records disk access in Windows applications to a trace that can be played back as a benchmark on other disks or RAID arrays. Using the IPEAK toolkit, SR, AT and Tweakers.net developed their own benchmarks, which explains some of the differences in the results obtained. Sadly, AnandTech and StorageReview hardly provide any information regarding their methods, making it difficult to get an insight in the kind of tests that they performed.
AnandTech
Tests performed by AnandTech consist of two IPEAK traces, the application benchmarks Winstone 2004 and SysMark 2004, and stopwatch-measured loading times of popular games like FarCry and Unreal Tournament. IPEAK traces were created with Business Winstone 2004 and Multimedia Content Creation Winstone 2004. Both of these benchmarks are also used by Tweakers.net, so we do have some experience with disk usage of BSW2004 and MCCW2004. On the other hand: Anandtech's traces are not really identical to those of Tweakers.net, since the latter added some load by running WinAMP and eMule. Since disk usage by these two applications was limited to an average of 200KB/s it didn't have any noticeable influence on the trace properties. The Business Winstone 2004 programs Internet Explorer, Outlook, PowerPoint, Norton Anti-Virus and WinZip generated just a little I/O activity, with an average load of eight percent. A safe conclusion would be that a Business Winstone 2004-benchmark alone is not a good starting point when testing RAID 0 performance. On the contrary: to have some reliable tests, we will need to put heavy loads on the array.
That is where Multimedia Content Creation Winstone 2004 comes in. With some powerful applications like Premiere, Photoshop, WaveLab and Windows Media Encoder, this is a much heavier and therefore a more suitable benchmark. After creating our traces, we see an average disk load of 19 percent, most of that concentrated around peaks of up to 90 percent. According to AnandTech's tests, two RAID 0 configured Raptor WD740GD disks are 38,1 percent faster in MCCW2004 and 19,8 percent faster in BSW2004. Results recorded in the Tweakers.net Benchmark Database show two striped WD740GD disks offering a benefit of 36,1 percent in BSW2004 and even 50 percent in MCCW2004. To execute these tests, AnandTech used an Intel ICH5 Southbridge while Tweakers.net had a Promise FastTrak S150 TX2plus controlling the RAID array. Despite the fact that we were forced to use a PCI-slot, which does not offer enough bandwidth to let the Raptors show their full capabilities, instead of a PCI-X-slot, our performance scaling was still better than AnandTech's result show.
 | ||||||
 | | |||||
| ||||||
| Tweakers.net | | Promise S150 TX2plus | |    36,1 | |
| ||||||
| AnandTech | | Intel ICH5 | |    19,8 | |
| ||||||
| ||||||
| | |||||
| ||||||
| Tweakers.net | | Promise S150 TX2plus | | 50,0 | |
| ||||||
| AnandTech | | Intel ICH5 | | 38,1 | |
| ||||||
Apart from Winstone 2004, AnandTech also uses SysMark 2004 and the actual Winstone 2004 application benchmark to compare hard drive performance. With the exception of the SysMark 2004 Communication Office Productivity Performance subtest, all Winstone 2004 and SysMark 2004 benchmarks show a small difference in performance when comparing several hard drives and the Raptors as single disks or in RAID config. This is not a big surprise though, since these system benchmarks rely heavily on which cpu, memory and chipset are used. Because of this, the final results do not reflect differences in responsiveness between various drive configurations. Some power users seem to notice an improvement, though. Data from the Business Winstone 2004 trace has already proven that disk activity with this benchmark is rather low and, although we don't have any details at our disposal, It is reasonable to conclude this applies to most of the SysMark 2004 tests.
Another problem we have to face when using system benchmarks to measure storage performance is that we don't have any clue about how the creators of Winstone and SysMark calculate the results of their benchmarks. A reasonable theory, for example, would be that activities that rely on processing power weigh more heavily in the final score than I/O activities. Although traces created by these benchmarks can be useful to measure disk performance, the system benchmarks themselves are not suitable for this goal. If we do want to make some conclusions based on AnandTech's benchmarks, we could say that the Raptor WD740GD in RAID 0 does not scale badly compared to the (marginal) differences in performance between for example a single Raptor WD740GD and a Western Digital WD1200JB.
| ||||||
| | |||||
| ||||||
| WD Raptor WD740GD | | 2x RAID 0 | | 25,725 | |
| ||||||
| WD Raptor WD740GD | | Single | | 24,825 | |
| ||||||
| WD Raptor WD360GD | | Single | | 24,350 | |
| ||||||
| WD Caviar WD1200JB | | Single | | 23,725 | |
| ||||||
| ||||||
| | |||||
| ||||||
| WD Raptor WD740GD | | 2x RAID 0 | | 33,250 | |
| ||||||
| WD Raptor WD740GD | | Single | | 32,400 | |
| ||||||
| WD Raptor WD360GD | | Single | | 32,125 | |
| ||||||
| WD Caviar WD1200JB | | Single | | 31,450 | |
| ||||||
The third part of AnandTech's benchmark suite, measuring loading times in games, can also be questioned. Just like the previous tests, these tests show a slight improvement in performance with the RAID configurations, with highly diverging performance. Our gaming traces show that most games have very little impact on modern hard drives and generate a rather low load. Even the heavyweight champion Battlefield Vietnam was able to burden a Raptor WD360GD for only 17,1 percent average with a short peak of 70 percent. Other games had even lower averages and didn't rise above 80 percent peak usage. Therefore we conclude that loading game levels is mostly cpu intensive and does not rely on storage devices. This conclusion is again illustrated by the insignificant improvents when loading levels in Far Cry and Unreal Tournament. Also, measurements with Unreal Tournament show some unlikely differences between a Raptor WD740GD and the other disks in the arena. It seems to be 12,9 percent faster than number two, a Seagate Barracuda 7200.7, while the others (one of them is a Raptor 360GD and a rather old IBM Deskstar 75GXP) offer a performance varying not more than 8,4 percent.
| ||||||
| | |||||
| ||||||
| WD Raptor WD740GD | | 2x RAID 0 | | 39,7 | |
| ||||||
| WD Raptor WD740GD | | Single | | 40,6 | |
| ||||||
| WD Raptor WD360GD | | Single | | 41,1 | |
| ||||||
| WD Caviar WD1200JB | | Single | | 42,6 | |
| ||||||
| ||||||
| | |||||
| ||||||
| WD Raptor WD740GD | | 2x RAID 0 | | 29,4 | |
| ||||||
| WD Raptor WD740GD | | Single | | 29,4 | |
| ||||||
| WD Raptor WD360GD | | Single | | 34,1 | |
| ||||||
| WD Caviar WD1200JB | | Single | | 35,8 | |
| ||||||
When we have a look at our own Gaming StorageMark 2004 Index, two Raptors in a RAID 0 array raise performance with 23,5 percent when stripe sizes are set to 128K and even an improvement of 33,3 percent is measured whith stripe sizes of 64K. These results again show that stripe sizes are critical to reach optimal performance, and also show that modern games can, even on a computer with a fast processor, gain benefit from a RAID 0 configuration. Something that is also subjectively noted by users who installed a RAID adapter in their gaming computer.