News:


  • June 07, 2024, 10:02:16 AM

Login with username, password and session length

Author Topic: Nats Stunt Seeding  (Read 13957 times)

Offline RC Storick

  • Forum owner
  • Administrator
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 12421
  • The finish starts with the first piece of wood cut
    • Stunt Hangar
Re: Nats Stunt Seeding
« Reply #50 on: May 02, 2009, 06:51:28 PM »
"One more reason for 5 judges and high and low score thrown out"

Only three steps to go: 1) You recruit 12 more judges, 2) you write a mathematical proof that throwing out high and low is better than taking the average, and 3) you edit the VBA macro to do it.

Hum I guess all the real sports who use subjective judging must be wrong? You know diving,Ballet,figure skating.This is why I hate voicing my opinion. Because every time I do someone has to come up with stuff like this. Some MATHEMATICAL BS that don't mean squat!
AMA 12366

Offline peabody

  • 23 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2868
Re: Nats Stunt Seeding
« Reply #51 on: May 02, 2009, 06:58:48 PM »
I have judged quite a bit and only awarded one 40, if I recall....Howard made one of the coolest (AND rule book) landings in a gale one year at Muncie....
 

Offline RC Storick

  • Forum owner
  • Administrator
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 12421
  • The finish starts with the first piece of wood cut
    • Stunt Hangar
Re: Nats Stunt Seeding
« Reply #52 on: May 02, 2009, 07:05:17 PM »
I have judged quite a bit and only awarded one 40, if I recall....Howard made one of the coolest (AND rule book) landings in a gale one year at Muncie....
 

Thats cool but last year I will bet dollars to donuts that there were lots of forty's awarded as we had stunt heaven.
AMA 12366

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #53 on: May 02, 2009, 10:44:07 PM »
"Some MATHEMATICAL BS that don't mean squat!"

As an alternative, I suppose that you could construct a proof that it don't mean squat.  There's still the matter of recruiting the judges and changing the program. 
The Jive Combat Team
Making combat and stunt great again

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #54 on: May 03, 2009, 12:03:16 AM »
Thats cool but last year I will bet dollars to donuts that there were lots of forty's awarded as we had stunt heaven.

Hey, I figured out how to put quotes in a box.

I was going to take you up on that bet, because I have the unfair advantage of having the data.  You are correct, however, so I guess I'm morally obligated to give you a donut.  I only looked at the qualifying rounds, because it's kinda tedious to do.  Here are the numbers.  Consider them unofficial: I hand copied them.

Qualification flight 1
Orestes Hernandez     landing  36 40 40
Bill Rich                    landing  36 39 40

Qualification flight 2
Robert Storick           takeoff  40 33 33
Robert Storick           inverted 40 35 35
Keith Trostle             inverted 40 35 35
Dan Banjock              inverted 40 34 32
Louis Rankin              landing   37 35 40

Qualification flight 3
Brett Buck                landing   40 38 36
Steve Moon              landing   37 35 40
Kent Tysor               landing   38 35 40

Qualification flight 4
Brett Buck                landing   40 37 34
Steve Moon              landing   38 35 40
Matt Neumann           takeoff   40 34 34
Matt Neumann           landing   40 38 36
Howard Rush             landing   40 39 37
Bill Rutherford            landing   40 37 31

I was surprised to find myself in this august group.  I don't remember the landing, but it must have been a humdinger.

One interesting tidbit is that almost all of these 40s came from two judges. 


The Jive Combat Team
Making combat and stunt great again

Offline RC Storick

  • Forum owner
  • Administrator
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 12421
  • The finish starts with the first piece of wood cut
    • Stunt Hangar
Re: Nats Stunt Seeding
« Reply #55 on: May 03, 2009, 02:42:31 AM »
Thanks Howard. Flgiht 2 day one I had 2 40s on same flight and a 33 for inverted I think I still have the score sheets. But glad you looked them up. I knew it was heaven cause my pig of a airplane flew fair any wind and I would have been drug all over the circle. Of coarse it was last year and a new day is comming. I'll live.  LL~

PS my day 2 scores were much much better even tho no 40s that day that I can remember.
AMA 12366

Offline john e. holliday

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 22792
Re: Nats Stunt Seeding
« Reply #56 on: May 03, 2009, 07:36:30 AM »
A 40 for one maneuver doesn't mean much if the rest of the maneuvers are the pits.  My part time coach keeps telling me that consistancy pays more.  But, then I watch someone fly 10 to 15 foot bottoms and tracking of maneuvers all over the place and they out score me.  I hit 5 foot once in a while and manage a fair share of tracking.  But, consistency pays. 

But, over all if the judges are consistant themselves from one pilot to the next it is great.  But, there are those that lose concentration during the day.  I also commend the judge that is not afraid to down grade a maneuver that is not what it is supposed to be.  I have score sheets in which it was pretty much consistant numbers all the way down the sheet.  Of course I am surprised that some maneuvers haven't gotten a zero.  Having fun,  DOC Holliday
John E. "DOC" Holliday
10421 West 56th Terrace
Shawnee, KANSAS  66203
AMA 23530  Have fun as I have and I am still breaking a record.

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #57 on: May 03, 2009, 08:53:37 AM »
I was going to take you up on that bet, because I have the unfair advantage of having the data. 


In no way am I trying to be controversial here... but in the spirit of the new openness maybe you could consider the following.

Since it appears you guys have and keep all that scoring data from the Nats (and obviously study it), maybe you should publish that too.   Just to be fair to the rest of the competitors.
« Last Edit: May 03, 2009, 09:42:32 AM by Bradley Walker »
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #58 on: May 03, 2009, 11:37:04 AM »
In no way am I trying to be controversial here... but in the spirit of the new openness maybe you could consider the following.

Since it appears you guys have and keep all that scoring data from the Nats (and obviously study it), maybe you should publish that too.   Just to be fair to the rest of the competitors.

   And, also in the spirit of openness, we should include bullseye's and Laser Target Designators on all the NATs judges shirts!

     I will take you at your word that you aren't trying to start anything, but you can't fail to recognize the problems.   We already when through that back in 1993. Resulting in 15 years of "fun", some world class "gibberish" analysis papers, and a few Federal felonies. Some people proved incapable of acting responsibly with the information.

     There's a reason it's hard to find a lot of NATs judges, and I would argue very vociferously that it would not be in the best interests of the event to open them up to perpetual second-guessing.

     Brett

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #59 on: May 03, 2009, 01:36:24 PM »
Sure.  I know that none of your Tournament of Champions contests has gone by without publishing all the judges' scores for every maneuver.  I suppose that since I am an active Nats contestant, I shouldn't have seen the populated program, but because I wrote and maintain the program, I sorta need to see it.  I'm sure you would have been willing to spend three or four days entering dummy scores so I wouldn't have had to see the real ones, but I didn't see the need for that at the time, because it was absurd to imagine that anybody would accuse me of being unfair by seeing scores after the contest was over.  Ask Paul for a copy.  Be sure to explain to him how "you guys" have an unfair advantage by seeing the results.  OK, OK, I'll give Sparky the donut.   
The Jive Combat Team
Making combat and stunt great again

Offline RC Storick

  • Forum owner
  • Administrator
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 12421
  • The finish starts with the first piece of wood cut
    • Stunt Hangar
Re: Nats Stunt Seeding
« Reply #60 on: May 03, 2009, 06:07:27 PM »
  Be sure to explain to him how "you guys" have an unfair advantage by seeing the results.  OK, OK, I'll give Sparky the donut.   

I'll take a Jelly at the NATS.. LOL and I don't have any problem with anyone seeing the scores but I do agree with Brett it serves no purpose to open them up to the world.
AMA 12366

Offline Paul Smith

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 5816
Re: Nats Stunt Seeding
« Reply #61 on: May 03, 2009, 06:14:01 PM »
You could use Howard's seeding system, then concede a place in the finals to the best flyer in every circle, and make him judge in exchange for the "bye". 

My theory being that the top four seeds would certainly make the "final 20" anyway, so give 'em a pass and put 'em to work.

Just a thought from a spectator. 
Paul Smith

Offline Doug Moon

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2199
Re: Nats Stunt Seeding
« Reply #62 on: May 03, 2009, 07:04:48 PM »
I have an idea for an article. 

You take the scores for the top ten guys in Open on their Top20 qual flights.  Leave their names off and the judges names off.  But specify these are true scores from the nats.

The name of the article would be something like "How do they score so well?"

Showing the data would let fliers from lower classes see just where the top guys are really picking up points as they fly the pattern.  For instance Take Off, Inverted Flight, Loops, and Landing.  These are huge point getters and or givers.  Lots of lower class fliers give it away on these very manuevers.  With some published data, including app score they can really compare where their scores are and the ones form top guys at the nats...

Just an idea...
Doug Moon
AMA 496454
Dougmoon12@yahoo.com

Offline Jim Pollock

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 948
Re: Nats Stunt Seeding
« Reply #63 on: May 03, 2009, 07:12:33 PM »
WOW!

Brett and Robert agree on something!!!!

Jim Pollock   n~

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #64 on: May 03, 2009, 08:06:14 PM »
WOW!

Brett and Robert agree on something!!!!

 As always, most of us agree on most things. We discuss the differences because there's no point in discussing the agreements.

   Brett

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #65 on: May 03, 2009, 08:30:05 PM »
You could use Howard's seeding system, then concede a place in the finals to the best flyer in every circle, and make him judge in exchange for the "bye". 

My theory being that the top four seeds would certainly make the "final 20" anyway, so give 'em a pass and put 'em to work.

  Even though I would benefit from it greatly, I don't think that's a good idea. The sole goal with ALL of this is to try to put everyone on an even footing. Giving a few guys a free pass into the Top 20 is anathema to that idea. It should be based completely and entirely on the performance during the contest - not on past performance, not on reputation, or anything else other than how the 15 maneuvers are executed when it comes time to do it. Doing as you suggest is like giving the Yankees, Red Sox, Dodgers, and Braves free pass into the playoffs and not even making them play the 162 regular season games.

   And I think I can speak for any of the potential beneficiaries and say *we don't want a break*. We ought to be able to make it through qualifying, if not, we don't deserve to win.

    Brett



Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #66 on: May 03, 2009, 08:42:12 PM »
It's Paul Walker's seeding system.  I'm just the programmer.  Brett, I think Paul Smith was suggesting that the qualifying flights be flown, but the winner of each be put in the final, bypassing top-20 day.  I, um, prefer the existing method. 
The Jive Combat Team
Making combat and stunt great again

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #67 on: May 03, 2009, 08:59:21 PM »
I was thinking about Brad's suggestion.  For it, and for Doug's, all you'd need to do is to take the existing program and write a macro to extract the data you want.  It might be good to obfuscate the judges' identities, but you ought to be able to show scores for each maneuver of the contest.  The program only went through the top 20 last year, so you may want to start at this year's Nats.  I'm not particularly interested in the project myself, but most of you have the program, so anybody who wants to write the macro can do so.  Holler if you have questions. 
The Jive Combat Team
Making combat and stunt great again

Offline Paul Smith

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 5816
Re: Nats Stunt Seeding
« Reply #68 on: May 04, 2009, 06:15:21 AM »
It's Paul Walker's seeding system.  I'm just the programmer.  Brett, I think Paul Smith was suggesting that the qualifying flights be flown, but the winner of each be put in the final, bypassing top-20 day.  I, um, prefer the existing method. 

In order to get to the top of the "Walker-Rush" totem pole, a flyer needs to make the top 20 (and do well) several years in row.  So is there really any doubt that he would do it again?  This is sort of like the defending World Champion getting to compete again without going through team trials.  I know you won't do this, but it would provide four more judges.
« Last Edit: May 05, 2009, 01:37:48 PM by Paul Smith »
Paul Smith

Offline peabody

  • 23 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2868
Re: Nats Stunt Seeding
« Reply #69 on: May 04, 2009, 06:54:31 AM »
Paul...
I think that you perpetuate a myth that good fliers make good judges....

Offline john e. holliday

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 22792
Re: Nats Stunt Seeding
« Reply #70 on: May 04, 2009, 08:31:05 AM »
Here I always thought the previous World Champ did get to compete in the World Championship on his own merit.  As I understood it, our past Champs would make the team and then step down to make for anolther member.  When he does that, he does not add to the team score. 

But, I like the idea of seeing the scores to see where the big differences are and not necessarily the top five or top ten.  Just a random draw of some score sheets to see the individual maneuver score.  I know when I used to pick up my score sheets I would wonder why one judge would score a maneuver higher or lower than the other.  I can also tell when a judge is not on the bracket scoring.  Like Int should not score as well as Adv.  I have seen some Int flights that should heve been in expert, but, then you realize the pilot was on that day.  DOC Holliday
John E. "DOC" Holliday
10421 West 56th Terrace
Shawnee, KANSAS  66203
AMA 23530  Have fun as I have and I am still breaking a record.

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #71 on: May 04, 2009, 10:30:26 AM »
Paul...
I think that you perpetuate a myth that good fliers make good judges....

Give me a break. >:(
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #72 on: May 04, 2009, 10:33:02 AM »
   And, also in the spirit of openness, we should include bullseye's and Laser Target Designators on all the NATs judges shirts!

     I will take you at your word that you aren't trying to start anything, but you can't fail to recognize the problems.   We already when through that back in 1993. Resulting in 15 years of "fun", some world class "gibberish" analysis papers, and a few Federal felonies. Some people proved incapable of acting responsibly with the information.

     There's a reason it's hard to find a lot of NATs judges, and I would argue very vociferously that it would not be in the best interests of the event to open them up to perpetual second-guessing.

     Brett

The FAI does it.  No judges killed yet.  Dooooh!!! :P

I also believe it has lead to some very interesting study.
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #73 on: May 04, 2009, 10:44:56 AM »
In order to get to the top pf the "Walker-Rush" totem pole, a flyer needs to make the top 20 (and do well) several years in row.  So is there really any doubt that he would do it again?  This is sort of like the defending World Champion getting to compete again without going through team trials.  I know you won't do this, but it would provide four more judges.

   Well, more judges are a good idea, but I don't think giving supposed hotshots a free ride to Fri..., er, Thursday is the way to do it. If nothing else, you *actually have to fly the flights*, and anything can happen. Do I think David Fitzgerald is going to have much of a problem making the top 5 on his circle in qualifying? Of course not. But he still has to do it. I would consider it grossly unfair to let him (or anybody else) through on reputation. That's exactly what the system is there to PREVENT. And I can tell you right now that on the occasions we had a cancelled round in qualifying, so that you had to make that one good flight or get bombed out, I sweated it out. Anything can go wrong on one flight, no matter how good your reputation is. Ask David about the 98 NATs Top 5.

   A second, far less important issue, is that when David/Paul/Orestes and I are out there judging, we would be WIDE OPEN to accusations of favoring people we think we could beat more easily. Oh, Billy showed up this year? Well, I don't want to go up against him, I think that was a 21 point Square 8 right there, look at all those >5ft radius corners. Of course no one would do that in real life, but I absolutely guarantee that if the wrong person didn't make it, Rich Peabody, et. al. would be writing nasty-grams to everybody in the event over how we jobbed the system AGAIN.

     I would also note that being a top flier doesn't necessarily make them a good judge. Of the current group, I know for certain that David, Paul, and I  have sufficient judging experience to do a good job (David and I have judged at the NATs before, Paul has judged at the Team Trials  and we have all done it extensively at local contests. I would wager we have all conducted judging *clinics*). I don't know about Orestes - as a consummate sportsman I would trust him to be objective but I have no idea if he has any experience, and the NATs is not the place to start.

  So, I don't think so, its an interesting idea, but I don't think it meets the "give everybody an equal sporting chance" principle.

    Brett

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #74 on: May 04, 2009, 10:50:41 AM »
The FAI does it.  No judges killed yet.  Dooooh!!! :P

I also believe it has lead to some very interesting study.

  Interesting is one word. Bullcrap is another word. And there have been numerous FAI judges run off, banned, etc.


       Brett

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #75 on: May 04, 2009, 11:11:29 AM »
Give me a break. >:(

   Agreed. For the most part, very good fliers tend to make good judges. If for no other reason that they also usually coach and judge incessantly all the rest of the year aside from the NATs. It's not necessarily true, but it usually is.

     Brett

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #76 on: May 04, 2009, 11:11:33 AM »
  Interesting is one word. Bullcrap is another word. And there have been numerous FAI judges run off, banned, etc.

Maybe they *needed* to be run off.

Brett, you seem to function under the assumption that judges are immune to doing anything they should not be doing.  When, in fact, history shows that judges *have* made decisions based on factors outside of who flew the best pattern, and people complained and those judges were removed, steps were taken, etc  When it comes to the FAI WC's, some of the biggest complainers have been American pilots.  I cannot remember how many articles I have read from magazines going back 20-30 years written by American pilots basically saying that "we would have won if the judges would have judged right" or basically implying the judges were "in the tank" for a particular country.  In fact, I think you can make the case the judges *were* in the tank for one side or the other.  That is why the FAI started publishing scores from the WC's.

The "that stuff does not happen here" basically denies history, even recent history.

Ultimately, openness never caused more problems than perceived secrecy will ultimately create.  See "seeding".  People who promote secrecy to "protect" people from themselves will ultimately do more harm than good.

"Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience."  CS Lewis
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Randy Powell

  • 21 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 10478
  • TreeTop Flyer
Re: Nats Stunt Seeding
« Reply #77 on: May 04, 2009, 12:37:43 PM »
Brad,

You seem sold. I think you should fly FAI.
Member in good standing of P.I.S.T
(Politically Incorrect Stunt Team)
AMA 67711
 Randy Powell

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #78 on: May 04, 2009, 12:40:53 PM »
Brad,

You seem sold. I think you should fly FAI.

Huh?  Where?
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #79 on: May 04, 2009, 08:59:46 PM »
Maybe they *needed* to be run off.

  Based on WHAT?  Some non-participant's  ex post facto analysis and a few prima donnas out to salve their precious bruised egos? Or even more likely, the prima donna's cheering sections out to salve their precious little bruised egos (because that seems to be the majority of the problem)? 

     Are you actually suggesting that, say, Mary Gebhart, deserved to be defamed behind her back, called every name in the book, and finally get run out of judging, all for the horrible sin of trekking cross country on her own time to give her opinion about a bunch of grown men playing with little toy airplanes in a soybean field?

   Any possible value of critical examination of the scores after the fact has to be weighed with the cost, specifically, judges being hounded, threatened,  and second guessed for perpetuity by people with an axe to grind.

     Past experience has shown us exactly how that worked out. It was of academic interest to most.  To others it was a golden opportunity to display their utter lack of class or sportsmanship and came close to destroying the event.  Given the minuscule purported advantages VS. the demonstrated irresponsible behavior, it not even a close call, it's a no-brainer. In my opinion, of course.

   Does that mean you are being "punished" or discriminated against in your requests because of other people's past actions? Absolutely. If you want to complain about it, might want to take that up with those who messed it up for everyone. I am sure you can figure out where to direct your ire, and it sure isn't me.

     I have seen absolutely nothing to suggest that it would be any different the next time; to the contrary, the pathetic little whispering campaigns and rumor mongering to undermine people is going better now than it was before, aided by any number of electronic communication media.

    Bottom line is that we can get away with 5 less pilots at the NATs. If they think it's rigged, then by all means they can stay away.  We can't get away with 5 less judges.

<snip>

Quote
Ultimately, openness never caused more problems than perceived secrecy will ultimately create.  See "seeding".  People who promote secrecy to "protect" people from themselves will ultimately do more harm than good.

   I don't want protect people from themselves. I want to protect them from you.

     Brett

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #80 on: May 04, 2009, 09:05:16 PM »
"Ultimately, openness never caused more problems than perceived secrecy will ultimately create."

What was your social security number again, Brad?  Actually, I think that the level of openness of the Nats process is a valid topic for discussion, particularly if discussed rationally by informed Nats participants whose purpose is to improve the contest.  
The Jive Combat Team
Making combat and stunt great again

Offline Doug Moon

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2199
Re: Nats Stunt Seeding
« Reply #81 on: May 04, 2009, 09:20:36 PM »
 
<snip>

   I don't want protect people from themselves. I want to protect them from you.

     Brett


I think that is totally uncalled for and way out of line.
Doug Moon
AMA 496454
Dougmoon12@yahoo.com

Offline Doug Moon

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2199
Re: Nats Stunt Seeding
« Reply #82 on: May 04, 2009, 09:25:48 PM »
"Ultimately, openness never caused more problems than perceived secrecy will ultimately create."

What was your social security number again, Brad?  Actually, I think that the level of openness of the Nats process is a valid topic for discussion, particularly if discussed rationally by informed Nats participants whose purpose is to improve the contest.  


I think the level of openness since PW took over is outstanding!  I applaud him on every level and I wasnt even there last time.  But seeing something like seeding published and the draw for flights and so on published is just great in my view.  It wasnt too long ago when people were stirring people up over these very things.  Now there nothing more to get stirred up over about this part.  Just one less thing.  Not that my view really matters all that much to anyone but me.  But, if PW is reading this stuff by chance I wanted him to know, and Howard to know, that I do think going public with these processes is good for the event.  If people gripe or bitch or moan or whatever they do they cant say it was secret so they could screw so and so out of his rightful place.  Plus with it public most wont even give the bitching a moaning a second chance and just move on where as before people could get all whipped over it...

This is a long thread and a lot has been covered and the one thing people seem to be very happy about is "knowing" how it is done.

I am with BB I dont think anyone should get a free ride based on last years score.  If anyone gets a free ride it is the year prior Walker Cup winner and him only!  I bet it would work against him.  Flying the qual rounds helps get one in the mindset and get things moving. 

BUT If the 4 judges were needed in an emergency situation and you did get them a bye to Top 20.  They can only be eligible to judge ADV.  That cures the situation BB described with the judging your future competition stuff. 
« Last Edit: May 04, 2009, 09:42:32 PM by Doug Moon »
Doug Moon
AMA 496454
Dougmoon12@yahoo.com

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #83 on: May 04, 2009, 09:55:50 PM »

I think that is totally uncalled for and way out of line.


   Sorry, but I have learned the hard way. I figure that about a week after you guys get the data, we will have "Anatomy of a Team Trials,  Volume II." (a tome that Brad has in the past expressed admiration for) or some equivalent, and then we will have judges quitting left and right, angry recriminations for anyone who didn't get the "right" answer, and yet another big mess. Even if you and Brad *don't intend it to turn out that way*. The road to hell is paved with good intentions. I for one don't want to risk it, and it won't happen if I can do anything to stop it. 

      Sorry if that's seems unfair,  and maybe it is since it's based partly on past experience and other's irresponsible behavior, but that's what I think is best for the event. Already the "openness" on the seeding is starting another bitching campaign, and that's a tiny fraction of the misinterpretation possible with the judges scores.

   So I am not accusing anyone of wanting it to happen, I have just seen the results and am unwilling to repeat them. Of course it's not really up to me, but I can have my opinion, too.

     Brett


  And just so you know I am not singling out Brad, here's a letter I sent  11 years ago, on the same topic - long before I knew anything about him:



Publishing Judges scores -
B. Buck, 9/24/98

Dear Competition Commitee,

I would like to comment on the concept of publishing individual judges scores at the NATS. While I appreciate the reasoning behind Wynn’s survey, and even agree with points of it in a perfect world, I think that the realities of the situation make such a procedure a very bad idea.

My chief fear is that if the scores get published, everyone with an axe to grind or a pre-conceived notion that they are getting the shaft would be provided with enough raw data that they could plausibly construct whatever conspiracy suited their fancy. The publication of the judges scores of the ‘93 Team Trials initiated the worst and most divisive controversy in the history of PAMPA, with the publication of “Anatomy of a Team Trials” by Mr. Baron and “THE GENIE IS OUT” by Mr. Urtnowski. Without discussing any further the merits of these two documents, the ensuing controversy they spawned needs no further explanation. One can only speculate on the effect the controversy had on recruiting judges but I assume that it was discouraging. I feel compelled to point out the “analysis” of the judges scores was key to the problem, and that this analysis resulted in “proof” of cheating based on scoring differences of  fractions of a percent. It didn’t convince me, but you can prove anything with statistics and a willingness to interpret them to your own advantage. I would have to think that publishing them all at every NATS would results in far more bitching and moaning after the fact by those who are unwilling to accept the official results.

I also disagree with the idea that qualified judges shouldn’t have a problem with having thier scores published. I think that the most qualified judges are the ones who use the entire scoring range and are willing to give a Universal Stunt Hero a 22 if he blows a maneuver and willing to give Joe Bellcrank a 40 if he deserves it. I think that this will become much more difficult if the judges knows every number he writes down will subsequently be utilized in innumerable different ways to prove him/her a cheater. It may not be conscious, but it’s inevitable. One bad score one on one maneuver (deserved or not) will become “proof” that so and so favors pilot X. The temptation will be to narrow down the scoring range so as to avoid having to stand out from the crowd. It won’t be intentional, but it will still happen.  The result - contests where the range of scores even narrower than it is currently. 

The potential for individual judges to become targets of smear campaigns and be threatened with physical violence should be clear. Sad to say, but it’s obvious that this has to be considered. Publishing the scores just gives more potential “justification”.

The vast majority of the competitors wouldn’t care one way or the other whether or not the scores were avaliable, because they accept the would results just as they do now. The only competitors who would really be interested are those who think that they could make something out of them - and it should be clear that those are the ones who would mis-use the data ala “Anatomy of a Team Trials”. Are we willing to accept the potential for abuse in order to potentially placate a few paranoids?

I don't expect this to take, but I would be tempted to propose just the opposite - that all the scores, including the totals, be kept secret. Only post the rankings. That way, no one ever has any “hard data” on which to base cheating accusations.

In summary, I think the potential for abuse far outweighs any possible advantage that could be gained.

  Sincerely

Brett Buck


   
« Last Edit: May 04, 2009, 10:29:34 PM by Brett Buck »

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #84 on: May 05, 2009, 06:12:40 AM »
Already the "openness" on the seeding is starting another bitching campaign

While there will always be people who "bitch", the seeding system that Howard is using is being done in the open, so while people may argue the *flaws* of the system, they cannot argue the intent of openness. 

I believe you will find that many of the people who have complained about seeding in the past were not complaining about the system used, but the secrecy around it.
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #85 on: May 05, 2009, 06:24:01 AM »
I have seen absolutely nothing to suggest that it would be any different the next time; to the contrary, the pathetic little whispering campaigns and rumor mongering to undermine people is going better now than it was before, aided by any number of electronic communication media.

    Bottom line is that we can get away with 5 less pilots at the NATs. If they think it's rigged, then by all means they can stay away.  We can't get away with 5 less judges.

<snip>

   I don't want protect people from themselves. I want to protect them from you.

     Brett

That is OK, Brett, since I am sure you are justified in being suspicious of everyone around you...  especially me.  Since I am huge advocate of openness (which seems to be catching on as of late) and treating my fellow competitors like accountable adults and not little children that need to be watched over by "mommy".

I, of course, said the Nats was "rigged" so please, continue.  The hyperbole is simply off the charts whenever you guys talk about the Nats
Please continue...
« Last Edit: May 06, 2009, 03:57:16 PM by Robert Storick »
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #86 on: May 05, 2009, 08:16:06 AM »
"Ultimately, openness never caused more problems than perceived secrecy will ultimately create."

What was your social security number again, Brad?  Actually, I think that the level of openness of the Nats process is a valid topic for discussion, particularly if discussed rationally by informed Nats participants whose purpose is to improve the contest.  


I am not sure what you are implying…  If you are implying that only current attendees of the Nats should have input into the process, I will comment on that.

I think that this idea that only current competitors of the Nats have an interest in the current and *future* of the Nats is misguided.  In fact, I know it is. 

Yes, it is true, I have not attended a Nats since 2003 (gasp).  This is simply due to the fact that I have been busy trying to make a living and establish myself at my new job(s).  Also, since I have no plans to be divorced any time soon, it is best to consider my wife in my limited vacation plans. 

Unlike some stunt flyers I do not own my own company.  I also do not currently do not have the status of being either retired, an airline pilot or attendant (which means massive amounts of free time) or have 20 plus years in the aerospace industry with most likely 6-8 weeks of vacation every year.  This being the case, I think it is best to consider taking *actual* vacations instead of dragging my wife to Muncie for one week of the two weeks of vacation I get every year.

This does not mean that I do not have plans to attend future Nats when my vacation status improves.  I assume I am not alone in this sentiment, as I would suppose there are many people out there who are struggling right now, and attending model airplane contests might take a back seat to other pressing real world issues.

So, I will operate under the assumption that the current attendees of the Nats do not *OWN* the Nats, and hopefully, the Nats will still be there long after many of the current attendees are gone.  So, if I want to present ideas on an Internet forum or the AMA, I will.

As far as sharing social security numbers...  I expect better of you Howard...  let's please not elevate a model airplane contest to national security code red levels.
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Steve Fitton

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2272
Re: Nats Stunt Seeding
« Reply #87 on: May 05, 2009, 09:54:09 AM »
What about releasing scores with the judges names redacted.  You might still get lots of statistics (which might be interesting in and of themselves), but you wouldn't have as much fodder for flame campaigns against a paticular judge.

Just because data has the potential for being abused seems poor reason to have it supressed.  And, plenty of Pampa wars have started without statistics as the spark.  Shining light onto more aspects of judging/scoring might drive the inevitable arguments into more and more trivial areas as the *big* areas of contention/argument are open for all to see if any attacks are valid or not.  Already it seems the seeding openess has reduced that aspect from the ranks of "conspiracy" into an animated but mostly polite discussion.
Steve

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #88 on: May 05, 2009, 11:50:17 AM »
"What about releasing scores with the judges names redacted.  You might still get lots of statistics (which might be interesting in and of themselves), but you wouldn't have as much fodder for flame campaigns against a paticular judge."

That seems reasonable to me.  I think that's what Doug suggested.  It's merely a matter of writing a VBA macro, although I'm not interested in doing it myself.  I'll send you the Nats tabulating program draft in case you want to take this on.  I'd appreciate any comments you have on the program anyhow.
The Jive Combat Team
Making combat and stunt great again

Offline Randy Powell

  • 21 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 10478
  • TreeTop Flyer
Re: Nats Stunt Seeding
« Reply #89 on: May 05, 2009, 11:56:22 AM »
Brad said "Huh?  Where?""

Well, you could move to Canada or South America. ;D

I don't compete at the Nats and so, don't feel competent to comment on the process. I can only relate it to what I've experienced and it really isn't the same.
Member in good standing of P.I.S.T
(Politically Incorrect Stunt Team)
AMA 67711
 Randy Powell

Offline Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13773
Re: Nats Stunt Seeding
« Reply #90 on: May 05, 2009, 12:09:46 PM »
That is OK, Brett, since I am sure you are justified in being suspicious of everyone around you...  especially me.  Since I am huge advocate of openness (which seems to be catching on as of late) and treating my fellow competitors like accountable adults and not little children that need to be watched over by "mommy".

I, of course, said the Nats was "rigged" so please, continue.  The hyperbole is simply off the charts whenever you guys talk about the Nats (especially when the evil Bob Baron or sociopath Windy card is pulled out). 


    As usual, missed the point utterly and completely. I don't know why I waste my time, but I will try anyway. Actions and decisions have *real world consequences*, it's not some isolated debating society. I for one don't think "destroying stunt to save it" from some narrowly-perceived hypothetical "flaws" makes a lot of sense.

     That's the underlying problem with publishing judge's scores and leaving them wide open (there's that word again) to the same sort of BS whispering campaigns we endured for majority of the 90's. You clearly stated that his was something you could consider acceptable, and that's why I think the judges need protection from you (and people who think similarly).

      There is nothing in this event more important to this event than the judges, and there is nothing in this event more important than protecting them from scurrilous accusations and second guessing. Allow that, and there won't be an event.

    Even if you were prepared to act in a mature and responsible way and were prepared to objectively review the results, there is documented proof that others have not in the past, and there are no indications that they will in the future. We had people committing federal felonies over the release of one set of scores, apparently spurred on by emotionalism. What's going to happen when people get their hands on even more BS analysis used to "prove" the "facts".

   And nice job - now you are engaging in exactly the sort of irresponsible behavior I was worried about. Not a word in my post said a d*mn thing about Baron being "evil" or Windy being a "sociopath". That doesn't appear in my post, the letter I posted, or in the private letter I sent to Bob Baron on the topic, or the commentary I had with Windy on the topic. I said my piece at the time with Bob, he said his piece back, and since he's gone and can't defend himself I will let that stand. I have no current beef with Windy, either, so barking up the wrong tree there, too.

    Those words, chosen BY YOU, Bradley "Godzilla" Walker, were formed solely and exclusively to *start a war*, AGAIN, and get people out of thinking about the rational discussion and into an emotional debate. Now, it will get repeated, "summarized" for the "masses", misquoted and attributed to me, and you will have your mini-firestorm. Well done. Glad you "didn't want to start a controversy". And you didn't even bother to read carefully, I took you at your word, responded entirely to your own current statements. You read a bunch of your own predispositions into it, missing the point entirely, and wow, look how that worked out.

   In this thread alone, you have gone from demurely asking for the scores,  to suggesting that maybe judges "deserved to be run out" by rumor mongering and whispering campaigns, then jumped to falsely attributing statements/attitudes to me, presumably to get more people on your side. Of course that's classic "bomb-throwing" and exactly the reason no one with a shred of common sense would trust you to be able to have a rational debate on the topic or act responsibly with the data. And of course, while it might ""play" with a tiny number of people who still spoil for another war, it essentially proves my point (far better than I would have been willing to articulate it) to everybody else.

  So, to summarize, you aren't a NATs participant (no problem), you expressed the view that it might be good for judges to "be run out" of the event on rumors (big problem), and then falsely attributed me with opinions I didn't express (irritating, but valuable to me as it essentially cinches my point) in an apparent attempt to spin up a new controversy.  And now you want all the scores for "analysis" and we should just trust you to objectively evaluate them?

     I still only have my 2 cents to put in, I don't decide myself, but nothing in this thread has done anything to convince me that releasing the scores (at least with the names intact, or with any method of reconstructing the relationship of judge to score) would go any better this time than before. Quite the contrary, it confirms my expectations pretty much right down the line.   I would be abdicating my responsibility to the event if I didn't make my case. I will not have anyone targeted by "analysis" if I can prevent it.

   I can't see a big problem with releasing the scores "raw" with no way to reconstruct the identities, but that's a lot harder than it first seems. You can't even identify the groups, circles, or the pilots, and without that, there's no sensible analysis that can be done.

     Brett



Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #91 on: May 05, 2009, 01:01:13 PM »
" I can't see a big problem with releasing the scores "raw" with no way to reconstruct the identities, but that's a lot harder than it first seems. You can't even identify the groups, circles, or the pilots, and without that, there's no sensible analysis that can be done."

I'm not sure what you mean.  Data are available partway through the 2008 Nats in the program populated with scores.  They should be available for the entire 2009 Nats, although it's probably not worth the bother to automate the part for the kids.  The program stores scores for each maneuver, who judged it, in what group the contestant is, upon what circle he flew, and-- for 2009-- how much his airplane weighs.  One could write a macro that barfs out any combination of that stuff.  If we could time-stamp the scoresheets, we could correlate it to weather if I get around to hooking up the Anemowimpometer and a couple other sensors to a data logger.  We could also compare sun direction to wind direction to see that effect on scores, which might be interesting. 

Change of subject: Within the program, as you have all seen, is a macro that selects judges for the finals.  This is possibly moot, given the paucity of judges available, but I would appreciate folks looking at that and sending comments on the method to Paul Walker.  You can send me comments on implementation.  The formula was published here and on SSW awhile back. 
The Jive Combat Team
Making combat and stunt great again

Online Mike Ferguson

  • AMA Member
  • Trade Count: (0)
  • Commander
  • *
  • Posts: 282
Re: Nats Stunt Seeding
« Reply #92 on: May 05, 2009, 01:05:12 PM »
Just a thought as to why "drop highest and lowest scores" isn't always a great idea (and to hopefully steer this back to something less incendiary):



In the example (which hopefully is somewhere in this post, Judges 3 & 5 are the high and low judges, so under the "get rid of high and low scores" plan, they get eliminated.

But if you track their scores on the line graph, Judges 1, 3, and 5 are all tracking scores the same way.  They're assigning different numbers to the scores, but they see the pattern the same way.  They all see the Triangles as the low score, and Overhead Eights and Horizontal Eights as the better manuevers.

Judges 2 & 4 - who give the same score to everything - get kept under the "get rid of high and low scores" plan, even though they aren't seeing the pattern the same way it's probably being flown.  These would be the judges you would want to get rid of, except because their scores are closer to the average overall score, they get kept.

I think if you're going to throw out scores - and I haven't been able to come up with a statistical model that shows it makes a meaningful difference - there has to be a more efficient model than "dump highest and lowest score".  As Howard says, though, that may be a moot point this year, if there aren't many judges.

Offline Paul Smith

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 5816
Re: Nats Stunt Seeding
« Reply #93 on: May 05, 2009, 01:39:20 PM »
Paul...
I think that you perpetuate a myth that good fliers make good judges....

I can't judge a judge until I see the number he gives me.
Paul Smith

Offline john e. holliday

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 22792
Re: Nats Stunt Seeding
« Reply #94 on: May 05, 2009, 03:47:26 PM »
Maybe they can draft the Intermediate flyers to judge Advance and Open.   S?P S?P DOC Holliday
John E. "DOC" Holliday
10421 West 56th Terrace
Shawnee, KANSAS  66203
AMA 23530  Have fun as I have and I am still breaking a record.

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #95 on: May 05, 2009, 04:01:49 PM »
    Those words, chosen BY YOU, Bradley "Godzilla" Walker, were formed solely and exclusively to *start a war*, AGAIN, and get people out of thinking about the rational discussion and into an emotional debate. Now, it will get repeated, "summarized" for the "masses", misquoted and attributed to me, and you will have your mini-firestorm. Well done.

Oh my GAWD, Brett...  get a hold of yourself.  Settle down.

I made no mention of *anyone* in particular, but instead commented on the history of why the *FAI* began publishing scores, and I also commented on the fact that in the 1970's and 1980's the biggest complainers about BIAs were the US pilots.  Maybe the FAI started publishing scores because of all of the bitching from the US pilots.  Maybe you missed that.

Even at the very mention of publishing scores, *you* are the first to drag out "Anatomy of Team Trials".  *You* are the first to make the Nats history *personal*.  I made no personal comments whatsoever about the "rigging" of the Nats (you had to make some smart aleck comment about people STAYING HOME if they do not like it).  In fact, I did not say anything about the Nats history at all.

If you do not want to be taken out of context about Windy or Baron, maybe you should stop bringing them up.

I do not want to *start a war*.  The *war is won* as far as I am concerned.  I think what Paul Walker and Bill Rich have been doing with the Nats is just great.  I am just discussing what I would like to see.  This is still America.  I thought we could still do that....
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Bradley Walker

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 1192
    • The Urban Rifleman
Re: Nats Stunt Seeding
« Reply #96 on: May 05, 2009, 04:05:22 PM »
  As Howard says, though, that may be a moot point this year, if there aren't many judges.

I agree with Howard.  What you need is like 20 judges and drop the top 8 and bottom 8.
"The reasonable man adapts himself to his environment. The unreasonable man adapts his environment to himself, therefore all progress is made by unreasonable men."
-George Bernard Shaw

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7818
Re: Nats Stunt Seeding
« Reply #97 on: May 05, 2009, 05:26:01 PM »
"I am just discussing what I would like to see."

I am just discussing what I would like you to do.  There are many opportunities: 1) you could write the data-extraction macro for maneuver scores, 2) you could review the possibly bogus finals judge selection formula, or 3) you could discuss how to use the scores of multiple judges, as Mike did above.  I suggested the latter two because you mentioned being a Six Sigma Black Belt.  The process could use advice from people who know statistics.   

PW at one time suggested maybe throwing out high and low judges for the finals, but the discussion didn't get very far, probably because of what Mike showed above.  Sounds like a bad idea to me, but as Sparky says, other sports do it.  A little research might show why they do.  Then, armed with that information, we could see if it applies to our situation. 
The Jive Combat Team
Making combat and stunt great again

Offline peabody

  • 23 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2868
Re: Nats Stunt Seeding
« Reply #98 on: May 05, 2009, 07:17:43 PM »
Didn't Dave Cook suggest tossing the high and low score PER MANEUVER at one time? I thought that it had merit, reather than tossing the high and low score?

Dave Adamisin

  • Guest
  • Trade Count: (0)
Re: Nats Stunt Seeding
« Reply #99 on: May 05, 2009, 07:47:41 PM »
I can't judge a judge until I see the number he gives me.
Nailed it Paul. But then you were probably thinking of me when you put the good flier/judge thing together......


Advertise Here
Tags:
 


Advertise Here