aFreak@beer.com" rel="external">DaFreak schrijft: "Eindelijk is het raadsel van de CSC keyspace, die inmiddels nabij een onmogelijke 125% is geraakt, opgelost: er werden verkeerde blokjes op onze koetjes losgelaten. Ongeveer 25% van de keyspace moet overgedaan worden, da's genoeg voor een week. Dus gassen met die koetjes! DPC 4ever!".
David McNett heeft een verklaring van de CSC problemen gepost:
Behind the scenes, we've been viewing the rapid approach of 100% CSC keyspace completion with mixed feelings. While it's exciting to see how fast we are all tearing through the keyspace, a bit of pre-100 nervousness is to be expected.
When we passed 90% on the statsbox, we decided to explain the impact of work duplication (as detailed in dbaker's plan the other day) because it looked like we might hit an apparent 100% keyspace completion. Anticipating the chance that our work re-verification system might eventually lead to an "over 100%" state on statsbox, we made the announcement explaining its impact.
At the same time, we stepped up our investigation into computer errors and malicious vandalism. Throughout the project, we have kept watch for anything out of the ordinary that might lead to a problem in our key processing. As the big 100 drew near, our suspicions began to rise...
So did the suspicions of many of our users, but until now we've had nothing concrete to tell you. This has changed. During the the latter part of our search, a clue smacked us in the face and eventually led us to a serious problem, one which affected a section of CSC work performed to date. We received a success packet on the master but were unable to use the reported key to decrypt the ciphertext. By backtracking the reported success, we uncovered an issue which affected the keyspace assignments done by the keymaster. Soon we had isolated it to a specific version of the keymaster code. The bad news: for a period of time, the keymaster was generating blocks with corrupted contest information, making those blocks invalid. Thankfully, the error only affected the CSC contest and not the RC5-64 contest data.
However, the minor error had a major impact on CSC. Roughly 25% of the CSC keyspace was affected by the block corruption. Fortunately, we were able to flag the invalid blocks as "untested", and start dumping them back to clients immediately.
What this means is that a quarter of the CSC work done to date was invalid and needs to be re-tested. Rather than 100% complete, in actuality we're only 75% through the keyspace -- about a week away from true completion, at current rates. It's very fortunate that our user base is so powerful that we can recoup the losses well before this contest comes to a close. It should also be noted that this error actually occured a few weeks ago and was in-advertantly corrected before we knew there was a problem. Because of this, there is no need to dump your buffers; the bad blocks are no longer in the pipeline.
We will be amending the percentage complete reported by the stats server to reflect our true keyspace completion. Participants will still receive full credit for their completed blocks, regardless of their validity. In the near future we will be supplying the stats server with the information it needs to properly discount the effects of any reverification work in the reported percentage.
We realize that a setback of just over a week is quite annoying in a short contest like CSC, but it's entirely possible that the winning key is working its way back to the keymaster right now.
To have passed this problem into the production network is seriously embarrassing. Simply put, we have failed in our responsibility to the users running our client, and we will do everything we can to ensure this never happens again. Despite coordinating some of the most powerful networks on earth, distributed.net is still run by those fallible human being types. We obviously need better testing procedures.
We already have a reliable system in place for client and core coders, to test the proper function of new and beta client code revisions. However, it's now quite apparent that we're lacking an equivalent sandbox (or process) to test changes in the keymaster code and network structure. With recent major revisions to the keymaster code (for OGR, CSC, and others) it's clear that this situation cannot continue.
As a result, distributed.net has committed to building a keyserver testing environment, complimenting our existing client-tests area. For a very long time the keymaster code was static enough that such a provision wasn't really necessary, but as we undertake more projects this will become less and less true.
We'll also take this time to step back and carefully chart a more reliable route through our development cycle, ensuring that errors and mistakes can be caught long before they would enter production.
On behalf of distributed.net, I'd like to apologize to our users -- and also thank you. The magnitude of your support has given us all a good chance at recovery, despite enormous keyspace fallout. We have learned our lesson, and we're very, very sorry. Now, with your help... let's finish this thing.