Dave Cook has some great thoughts on judging, and judging the computers of today....
As I recall, he advocated tossing the high and low judges' score per maneuver when more than three were used...
He has some cool ideas, although a bit contrary....
As I recall, he advocated tossing the high and low judges' score per maneuver when more than three were used...
He has some cool ideas, although a bit contrary....
Tossing out high/low is never a good idea, it operates on the presumption that there is a correct score. It will automatically toss out judges that run "high" or "low" normally, whether they are doing a good job or not. Buffalano's method (which involves normalizing the scores first, and is very similar to the methods I used to evaluate some previous results) solves this issue but has the fatal flaw of having to wait until all the scores are available. This method is more-or-less doing the same tracking method used to select judges like we have already done, but looks at an entire round and then excludes them for that round itself, at which point the others are used.
Again, this might work for Top 20 and Top 5, but there aren't going to ever be enough judges per circle to do it for qualifying. And, when I have taken individual sets of results and applied various methods, it never once changed the winner, and it only occasionally changed any of the placements by one or two.
If you process the scores with 3-4-5 different methods, and they all yield the same results, it tells you that all the fiddling or tweaking with format and processing is more-or-less irrelevant, AND, that you should probably choose the simplest version.
Brett
