(clip)
KEITH, IF YOU ARE LURKING...any comments about Mike Belitz and Bob Barons ambitious attempt to improve the skills and methods of judging?
Yes, I am reading this thread. And yes, I am familiar with that Baron/Belitz article, though it has been some time since I have read it.
Now, much of what I say on this subject will be viewed with a jaundiced eye in the fact that there are those who consider me as part of the establishment and as one who wants to thwart anything that would change the status quo.
There is no doubt that somehow, improvements can be made in our system of rules, judging, training, education, and establishing some sort of method to accurately track judge performance (and there are some rather elaborate statistical programs to do so which measure the separate functions of individual judge consistency and individual judge bias).
We can certainly endeavor to encourage such improvements, but it will take a dedicated, open minded core of individuals to accomplish such improvements and there will need to be an equally dedicated core of individuals who volunteer to judge to undergo the extensive training (and perhaps accept a qualification process that could prove cumbersome at best if discouraging for many). I think we already often approach a saturation point where many volunteer to perform the task of judging our major events (the Nats and Team Trials), often at not small personal expense, only to be openly criticised for their efforts by the innuendo and sometimes inappropriate statements by unthinking individuals with some nonproductive agenda in mind. This environment hardly is conducive for individuals to continue to volunteer their time and conscientious effort.
On the other hand, I have seen PAMPA foster the environment to encourage those who have any interest in the event to judge and to want to do a good job with it. Fortunately, we have many enthusiasts who do not have as their primary goal to be a top level competitor, but are willing to contribute in the event in many ways. I know personally, a number of individuals who want to be known as an excellent CLPA judge.
The system that Baron/Belitz proposed is to use separate judges to score intersections, heights, and shapes. Their proposal assigns certain responsibilities for each judge to score different aspects of each maneuver. Each of the judges are positioned to best view those aspects to be judged. These individual scores are then factored in a way to obtain a total score for the flight. A K factor is to be used somewhere in the process. I understand that a contest was organized in Southern California in the late 70's that used this system. Baron/Belitz wrote that the results were good and some suggestions for improvements were made. As far as I know, the system did not go any further than that one contest and the article in Flying Models.
There has since been more suggestions on this theme where different judges score specific aspects of each maneuver, but I am not aware of any contests or organized effort to test seriously the idea, like at even an organized fly-in or a club activity of some sort. I do not think anyone could state that the idea does not have merit nor that it should not be more fully explored. It will be interesting if there is ever going to be another contest organized to try out any system like this. I think that it is certainly worth trying. I am sure that adjustments to whatever system is tried would become apparent. If there is some merit to it, let's find out, and adjust and improve it as appropriate. It will be necessary for all involved to keep an open mind and not try to bulldoze some preconceived agenda.
In the end, I think we will see what we see now. In general, the best flights somehow get scored with the best scores. Admittedly, the current system does allow "errors in judgement" by individual judges. No matter what the system, there will still be individual judges who might make an error or will see something that needs to be scored differently than another judge of casual observer.
Quite frankly, I think much more would be gained by the statistical normalization of individual judge's scoring of each maneuver like is used in full scale aerobatics. This was another system championed by Bob Baron subsequent to his article with Mike Belitz. There is such a system that has been developed by George Buffalano in Maryland that could be used. In fact, Buffalano's program has been used by the FAI to analyze the performance of individual judges following the World Championships for a number of years. There was an article written by Buffalano published in Stunt News several years ago that explained the process in laymen's terms. The only response to that article was the continuation of Bob Baron trying to get it to be used at our major contests (the Nats and the Team Trials). My impression of the lack of interest or response to such a statistical normalization process is that our stunt community was or is not ready to conduct a contest where all of the scores are dumped into a computer, and after the last flight is entered, the computer does its thing and prints out a list that shows some sort of a normalized score and the ranking of each individual flier.
Again, no matter what the system, I think what we will find is that the better flights still get the higher scores. I also doubt that the point spread between the top fliers changes that much on a percentage basis. It still boils down that PA is a subjectively judged event. As such, it is not an activity that that lets a winner be determined with scientific accuracy.
I think a key to any discussion on improving the selection of a top flier at any contest is to improve the education of our judges. The Keith Renecle program is an incredibly usefull tool to do this. Hopefully, his presentation using computer driven graphics and his computer driven simulation of the stunt pattern together his verbal presentation might be available sometime in the future. This would go a long way for judges and pilots to better understand the pattern.
There is still no substitute for any pilot or judge to be totally familiar with the rulebook. I think using separate judges to assign points to specific portions of each maneuver is more of a gimmick, no matter how refined it might become, and will not satisfy the critics of our judges any more than they are now of the current system.
Keith Trostle
Topic: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS (Read 2108 times)