News:


  • July 09, 2025, 10:45:55 AM

Login with username, password and session length

Author Topic: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS  (Read 2108 times)

Offline Shultzie

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 3474
  • Don Shultz "1969 Nats Sting Ray"
Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« on: January 04, 2007, 11:52:06 AM »
One article that I found interesting about the pain and agony of stunt judging practices that recieved a considerable amount of FEEDBACK, that ranged anywhere from who gives a rip to what the hecky darn Mike and Bob have been sniffin?

I can also remember catching a lot of flack from a couple of the seeded flyers....who felt they were being singled out...when I drew some cartoons to add a bit of "farce-n-fodder" to that article.
 However, the more I tried  reading and digesting  all that beautiful K factor stuff of their IHS system of judging, from those years that I tried IN VAIN to become a decent judge...it was glaringly apparent...that I simply didn't have eye-memory-retention skills (but worst of all, I knew so many of my fellow competitiors personal and  flying habits so well...I knew that I could never become subjective enough to become a really good and qualified judge. I loved flying too much and knew during that time I just didn't want to judge any of my colleagues and peers. So with that those that do...do and those that can't FLY? Hummm? whatever???

KEITH, IF YOU ARE LURKING...any comments about Mike Belitz and Bob Barons ambitious attempt to improve the skills and methods of judging?
Don Shultz

Offline Shultzie

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 3474
  • Don Shultz "1969 Nats Sting Ray"
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #1 on: January 04, 2007, 11:57:54 AM »
Sorry Paul...
but the DEVIL MADE ME POST THIS....
Don Shultz

Offline Steve Helmick

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 10272
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #2 on: January 04, 2007, 08:55:46 PM »
 >:D I most certainly DID NOT!  VD~ Steve  :X LL~

PS: I tried to print the article, but you broke my printer!  :-*
"The United States has become a place where professional athletes and entertainers are mistaken for people of importance." - Robert Heinlein

In 1944 18-20 year old's stormed beaches, and parachuted behind enemy lines to almost certain death.  In 2015 18-20 year old's need safe zones so people don't hurt their feelings.

Offline Richard Oliver

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 96
    • RO-Jett Engines
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #3 on: January 04, 2007, 10:15:44 PM »
Hi Don, I'm not Keith but I sure have some opinions! That will really surprise some! ;D

The ideas put forth in this article were right on and really good.  Of late more people have come up with some ideas to add to and or improve on the ones in the article.
It will never happen because the people that make the rules and win the contests and run the contest say they see no problems with the status quo. Maybe the way to look at it is to think of this as a better way to get the job done.
It will be used and in the real near future and, while not by the mainstream, it may be the other groups way of getting it done. There is always room for Chevy's and Fords. It is probably time for Stunt to leave the 50s and come into the new millennium and become progressive. I love classic as much as the next guy but I also like Top of the Line to mean Top of the Line. I can design a pattern that will tax the pilot and his equipment to the point that design and passion will have to be renewed to really call yourself National or World Champion. It is way past due for the classes to not fly the exact same pattern also! But thats just me. HB~>
RO
Richard Oliver

Offline Chris McMillin

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 1917
  • AMA 32529
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #4 on: January 05, 2007, 12:18:20 AM »
Let's see some of that talk really happen, RO. Sounds like fun.
Whatever happened to Ole Brad's contest, double elim, or something like that?
Chris...

Offline Trostle

  • 25 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 3393
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #5 on: January 05, 2007, 02:17:35 AM »

(clip)

KEITH, IF YOU ARE LURKING...any comments about Mike Belitz and Bob Barons ambitious attempt to improve the skills and methods of judging?

Yes, I am reading this thread.  And yes, I am familiar with that Baron/Belitz article, though it has been some time since I have read it.

Now, much of what I say on this subject will be viewed with a jaundiced eye in the fact that there are those who consider me as part of the establishment and as one who wants to thwart anything that would change the status quo.

There is no doubt that somehow, improvements  can be made in our system of rules, judging, training, education, and establishing some sort of method to accurately track judge performance (and there are some rather elaborate statistical programs to do so which measure the separate functions of individual judge consistency and individual judge bias).

We can certainly endeavor to encourage such improvements, but it will take a dedicated, open minded core of individuals to accomplish such improvements and there will need to be an equally dedicated core of individuals who volunteer to judge to undergo the extensive training (and perhaps accept a qualification process that could prove cumbersome at best if discouraging for many).  I think we already often approach a saturation point where many volunteer to perform the task of judging our major events (the Nats and Team Trials), often at not small personal expense, only to be openly criticised for their efforts by the innuendo and sometimes inappropriate statements by unthinking individuals with some nonproductive agenda in mind.  This environment hardly is conducive for individuals to continue to volunteer their time and conscientious effort.

On the other hand, I have seen PAMPA foster the environment to encourage those who have any interest in the event to judge and to want to do a good job with it.  Fortunately, we have many enthusiasts who do not have as their primary goal to be a top level competitor, but are willing to contribute in the event in many ways.  I know personally, a number of individuals who want to be known as an excellent CLPA judge.

The system that Baron/Belitz proposed is to use separate judges to score intersections, heights, and shapes.  Their proposal assigns certain responsibilities for each judge to score different aspects of each maneuver.  Each of the judges are positioned to best view those aspects to be judged.  These individual scores are then factored in a way to obtain a total score for the flight.  A K factor is to be used somewhere in the process.  I understand that a contest was organized in Southern California in the late 70's that used this system.  Baron/Belitz wrote that the results were good and some suggestions for improvements were made.  As far as I know, the system did not go any further than that one contest and the article in Flying Models.

There has since been more suggestions on this theme where different judges score specific aspects of each maneuver, but I am not aware of any contests or organized effort to test seriously the idea, like at even an organized fly-in or a club activity of some sort.  I do not think anyone could state that the idea does not have merit nor that it should not be more fully explored.  It will be interesting if there is ever going to be another contest organized to try out any system like this.  I think that it is certainly worth trying.  I am sure that adjustments to whatever system is tried would become apparent.  If there is some merit to it, let's find out, and adjust and improve it as appropriate.  It will be necessary for all involved to keep an open mind and not try to bulldoze some preconceived agenda.

In the end, I think we will see what we see now.  In general, the best flights somehow get scored with the best scores.  Admittedly, the current system does allow "errors in judgement" by individual judges.  No matter what the system, there will still be individual judges who might make an error or will see something that needs to be scored differently than another judge of casual observer.

Quite frankly, I think much more would be gained by the statistical normalization of individual judge's scoring of each maneuver like is used in full scale aerobatics.  This was another system championed by Bob Baron subsequent to his article with Mike Belitz.   There is such a system that has been developed by George Buffalano in Maryland that could be used.  In fact, Buffalano's program has been used by the FAI to analyze the performance of individual judges following the World Championships for a number of years.  There was an article written by Buffalano published in Stunt News several years ago that explained the process in laymen's terms.  The only response to that article was the continuation of Bob Baron trying to get it to be used at our major contests (the Nats and the Team Trials).  My impression of the lack of interest or response to such a statistical normalization process is that our stunt community was or is not ready to conduct a contest where all of the scores are dumped into a computer, and after the last flight is entered, the computer does its thing and prints out a list that shows some sort of a normalized score and the ranking of each individual flier.

Again, no matter what the system, I think what we will find is that the better flights still get the higher scores.  I also doubt that the point spread between the top fliers changes that much on a percentage basis.  It still boils down that PA is a subjectively judged event.  As such, it is not an activity that that lets a winner be determined with scientific accuracy.

I think a key to any discussion on improving the selection of a top flier at any contest is to improve the education of our judges.  The Keith Renecle program is an incredibly usefull tool to do this.  Hopefully, his presentation using computer driven graphics and his computer driven simulation of the stunt pattern together his verbal presentation might be available sometime in the future.  This would go a long way for judges and pilots to better understand the pattern.

There is still no substitute for any pilot or judge to be totally familiar with the rulebook.  I think using separate judges to assign points to specific portions of each maneuver is more of a gimmick, no matter how refined it might become, and will not satisfy the critics of our judges any more than they are now of the current system.

Keith Trostle

Offline Richard Oliver

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 96
    • RO-Jett Engines
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #6 on: January 05, 2007, 09:45:17 AM »

I knew Keith was out there! Keith is one of the few who I find that has an objective view of this subject. He is also one of the few in the establishment ( I consider you to be anyway ;D) that will listen and not just dump on any opinion outside the 'status quo'. Just my opinion and observation but most of the 'establishment' is not willing to even discuss the topic.  Keith seems, unlike me, to be able to leave out the emotions that come into play when this topic comes up. Let’s face it when you put a few competitors in a room that have a vested interest in a topic it can be colorful to say the least!
Now as to trying a system of somewhat the same as Barons. I put together a group with video to test this system. While as most would surmise the top people were still the top people with the order of finish scrambled.
The system we used was something like what was being put out there by Brad Walker for a while. 4 judges (3 would be minimum) with specific jobs.
Two were shape judges and two are technical judges with one of them being 90° to the flight path. The two shape judges can be only one for local contest where judges are in short supply. The shape judges are exactly as it sounds 'Shapes’.
The tech judges are for bottoms, intersections, size, position (superimposed), turn quality and elevation etc.. When the job is broken up it also becomes considerably less of a drain on the judge through the day.

Big side effect is no ballooning! Consistency in scoring goes way up! Most errors are seen by the judges as well as lack of. Most judges in the system today go with an overall feeling of the pattern rather than judging it's elements. Not a knock on the judges but at the difficulty of the job. When doing my tests here most of the judges after viewing tapes of the flight express that they did not see all of the errors during the flight but sure could see them after watching the tapes. Most of these judges judge their pet peeves and or the area that they deemed most important.
The judges with the new system pretty much got them all. This was done with both types of judging systems at the same time during the flights. The outcome was different in most cases but not all. When a flyer is good he/she is good and will score well with both systems. When a flyer is not as technically correct and has a style that some judges like or has, as some in other threads have mentioned, a flow to his/her pattern their scores really changed. Flow is not a judging element and should not be. If the elements of the pattern are what are judged and you hand a flyer his score sheet he/she knows immediately what to work on. With the system today the scores are all in a bracket and give no indication of where to put your practice efforts. The system also has wild variations from judge to judge on the same maneuver. This does not happen in this other system, I like to call the elements system. It works and if there were no change at all in the order of finish it still has many good effects. The pilot knows which elements that he/she is weak on. The rounds will not balloon. My definition of ballooning is the judges are changing their point values of the individual flyers from round to round. This simply does not happen in the elements system.
The values are the same as the scoring system today, 40 points, but is given by 4 different elements. 10 points max each judge added together.

Another system I tested was to have a side judge which gave a  plus 1 or a minus 1 for each maneuver for 45° and 90° elevation. This was added to the other, regular current type, judge’s scores. This is only one element but it did change the outcome.
The point is there can be better ways of judging if an open mind is used.
I for one was disappointed when I started flying stunt and collecting my score sheets and found no indication of what I needed to work on. I also noticed that the maneuver I knew I blew had a score within 1-2 points of my best maneuver. I also was perplexed when watching flights, and videoing them, when the really bad flight and the really good ones were separated by 10-15 points. This is 1 point per maneuver! The current system fosters bracket judging and we as human beings do not like to hammer our friends with low scores. The element system has none of that. As a judge in the element system you do not even know what the other judge is giving and your score has nothing to do with the other judges scores. This frees the judges up to score what they see. The 10 point system for each judge makes it more simple also. When added together it still is a 40 point system(4 judges  10 points each) so the pilot has not lost his/her reference. Technical’s are not subjective and the pattern is mostly technical’s.
RO
Richard Oliver

Offline Shultzie

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 3474
  • Don Shultz "1969 Nats Sting Ray"
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #7 on: January 05, 2007, 12:23:22 PM »
>:D I most certainly DID NOT!  VD~ Steve  :X LL~

PS: I tried to print the article, but you broke my printer!  :-*
----------------STEVE, STEVE, STEVE,----------------YOUR HUMOR AND RAZZZ, HONESTLY ALWAY CRACK-ME-UP!
Especially after looking at the quality of scans from my CRAPPYAAALDO EPSON ALL IN NONE! printer...It would appear that it is NOT YOUR PRINTER...BUT MINE INDEED THAT IS BUSTED.~! Along with my lame brain.
Send me a private and I would gladly drop that ol rag-mag in the snail mail for you.
However I just  put that into the hanger...to GET THE FLAMES OF ETERNAL STUNT JUDGE RAZZING started again. 
MY CONDOLENCES if any one is TOTALLY P-U'ED OUT with my percieved disrespect for us toy-airplane stunt flying issues...or that equally lack of respect for those sun-windburned-frozen headed brain damaged poor souls that seem to thrive on pain and abuse, that we call STUNT JUDGES...(OR AT TIMES.... LOT WORSE!) 

I still think that we need to give more credit to both (most of the flyers and dedicated stunt judges, that for a fact, take both seriously enough to take the time...to communicate with each other...in an attempt to improve the sport of stunt flying.  Compared to many other judged precision events...our pattern has been pretty much a NON-BRAINER with actually show some pretty EZ looking and few in number list of  maneuvers. It still, after all these years, still boggles my mind and sagging britches (hummm?) is that a spullin error?  Ha! How would I know?  It's EZ to see why I surely fly or judge stunt...as well as I spell and ruuuuun sentences toooogether.
"Blame it on my youth...in sniffin too much Ambroid glue and Aerogloss dope n' glue?"


Don Shultz

Offline john e. holliday

  • 25 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 22992
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #8 on: January 06, 2007, 08:04:06 AM »
I agree that most of the judges do their best to give scores according to what they see.  I know when I have blown a maneuver and wonder why it doesn't show on the score sheet.  But, why does one maneuver at the start of a pattern cause the rest of the pattern to be down graded when in your mind and others that it was one of the best you had done so far.  But, I have to keep having attitude adjustments to keep reminding me that this is supposed to be fun.  Also if I get serious about it, I don't think anyone would like to be around me.  My thanks go to any one who judges, run scores, tabulate and run the pits.  DOC Holliday
John E. "DOC" Holliday
10421 West 56th Terrace
Shawnee, KANSAS  66203
AMA 23530  Have fun as I have and I am still breaking a record.

Offline Steve Fitton

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2278
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #9 on: January 06, 2007, 09:51:14 AM »
Let's see some of that talk really happen, RO. Sounds like fun.
Whatever happened to Ole Brad's contest, double elim, or something like that?
Chris...

Ummmm....Brad's kinda sorta out of stunt, at least for the time being.  Hopefully he will return......


Steve
Steve

Offline phil c

  • 21 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2480
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #10 on: January 07, 2007, 07:17:21 PM »
I agree that most of the judges do their best to give scores according to what they see.  I know when I have blown a maneuver and wonder why it doesn't show on the score sheet.  But, why does one maneuver at the start of a pattern cause the rest of the pattern to be down graded when in your mind and others that it was one of the best you had done so far.  But, I have to keep having attitude adjustments to keep reminding me that this is supposed to be fun.  Also if I get serious about it, I don't think anyone would like to be around me.  My thanks go to any one who judges, run scores, tabulate and run the pits.  DOC Holliday

Just remember Doc,   the fun is in the flying.  Whether or not a couple judges made a few mistakes is the farthest thing from my mind.
phil Cartier

Offline Shultzie

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 3474
  • Don Shultz "1969 Nats Sting Ray"
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #11 on: January 07, 2007, 07:37:18 PM »
Just remember Doc,   the fun is in the flying.  Whether or not a couple judges made a few mistakes is the farthest thing from my mind.
Good attitude...but that ol' saying does have a certain ding-a-ling to it....
"IT MATTERS...NOT IF YOU WIN OR LOSE..that is..."UNTIL YOU LOSE!" The main point to remember is just to practice, practice, practice, with a goal in mind...FLY YOUR BEST and FORGET THE REST but never never eva' forget to honor your peers and give them truly a well meaning congrats. etc etc. <=
Don Shultz

Offline Jim Pollock

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 948
Re: Mike Belitz-Bob Barons STUNT JUDGING CONCEPTS
« Reply #12 on: January 07, 2007, 07:56:30 PM »
Rich,

Judges can be influenced by outside sources.  I know for a fact they can, I saw it happen.  :o

Jim Pollock  %^@

Tags: