Brian Hook heeft z'n .plan geupdate met zijn mening over de betrouwbaarheid van de Quake3 benchmarks bij THG:
The numbers that Tom's Hardware has posted make me real uneasy. I'm not sure they're invalid, but I do know that given they are not easy to reproduce on other people's systems and that, more important, the methodology has not been published for review and scrutiny. It is, in fact, leveraging a feature that we have acknowledged is not functioning correctly in Q3TEST. Other sites have dissected that issue in more depth than me, but suffice to say that I think there are enough unknowns and variables with Tom's numbers that I would pretty much just discard them out of hand as being meaningless.
Okay, let me put this less diplomatically: Tom's numbers are not provably correct or valid, and thus we have to assume that they are not correct or valid. I have asked any IHVs that have linked to his numbers or, even worse, used his numbers as marketing material (only his Q3TEST numbers, I have no opinion on any other work he has done in the past), to knock it off. Now. I refuse to stand by idly and let users potentially be misled just because one set of numbers makes one IHV look better than the other.
If someone wants to post numbers to make themselves look good, there is data from a few .plan file postings ago they can base their claims on. Those numbers are good, and if an IHV wants to argue/question those numbers, they are more than welcome to contact me about it.
When the patch is put out (don't know when, but not anytime soon), then new numbers can be posted by anyone just like the good old days. Until then, I'm assuming you trust me, my numbers, and my commentary. If you don't, then I won't notice, but if you do, then it makes all this worth it.
And that's the last I have to say on this matter. Good night.