Many people are aware there was a scoring error on the Einstein field, but not everyone knows the details. I’d like to explain what happened and our plans to minimize the chance of this happening again.
We knew that the automated scoring system in use had a design limitation which could result in disc count errors, so a manual count verification process was put in place at the start of the season to double check counts before scores were finalized. The goals over the driver stations had hinged bottoms to allow easy removal of discs. On Einstein, a field reset person would use a pole to push up on the hinge, allowing the discs to drop in to two separate bins, one on the left side, one on the right. The number of discs in each bin would be counted, written on a scratch piece of paper, totaled, and then the number of discs scored in autonomous would be subtracted from that total count before being transferred to the official score sheet for the match in the ‘teleop’ section. In Semi-Final 2-2 on Einstein, the field reset crew added 20 and 26 on the scratch paper and wrote ‘64’ instead of ‘46’, accidentally reversing the numbers. 10 discs had been scored in autonomous, so the official score sheet showed 10/54 for auto/teleop discs, rather than the correct totals of 10/36.
The alliance captain pointed out the scoring error to the Chief Referee. We reviewed the information, spoke with the field reset crew, and found the scoring error. The field reset crew was very forthcoming about the mistake, and was obviously shaken that it had happened. We told the teams we were correcting the score, and would be going to a third match in that series. Dean came over to talk to the alliance that we erroneously believed had made it to the finals. I went over to the alliance as well, apologized, and showed them the sheets on which the error had been made.
The problem was not with the field reset team, but with the overall system we had in place, which exposed too many opportunities for error. In most cases, the manual disc verification system functioned well and resulted in accurate disc counts, but on Einstein and several other events, we know for certain it did not. At those events during which the errors were caught in time, they were corrected. However, I know there were events at which errors were not caught until it was too late to correct, and to those teams, I apologize. I’m certain this was a painful experience.
Everyone makes mistakes, particularly under high-pressure situations, and I think it’s unlikely that we will be able to fully eliminate human judgment from every possible scoring or refereeing situation. Much of our ability to automate scoring depends on the game, some of which are relatively easy to score automatically, some realistically can’t be scored automatically at all. In any case, we remain committed to making sure that the scoring system in each FRC game is as accurate as possible. We’ll be working to reduce the opportunities for errors by employing more robust system checks and testing the system earlier and even more thoroughly, taking in to account any potential human factors and unlikely ‘corner cases’ that still seem to crop up with the number of matches we run at FRC events - almost 7,000 in the 2013 season. The FRC staff, just like the FRC teams, want matches to be won or lost based on what actually happened on the field.
I’ll blog again soon.