stunthanger.com

General control line discussion => Open Forum => Topic started by: peabody on November 25, 2007, 05:02:26 PM

Title: Seeding
Post by: peabody on November 25, 2007, 05:02:26 PM
Not your garden.....but flyers and judges at major events like the Nats.

I believe that a judges matrix should be posted at the beginning of the event......at the Nats, for instance, every judge should know his/her circle assignment BEFORE the first flight. Draw straws, or by height, weight or last name....NOT pre-seeded!

I also believe that seeding the flyers places a significant bias on the outcome.....



Title: Re: Seeding
Post by: Trostle on November 25, 2007, 07:29:06 PM

(clip)

I also believe that seeding the flyers places a significant bias on the outcome.....


How does seeding "bias the outcome" other than seeding actually levels the flying field so that every compeititor has more of an equal chance of placing according to his ability rather than have the "luck of the draw" become a significant factor in the outcome of the results?

This is another of your myths you keep trying to perpetuate that some people accept becasue they keep hearing that there is some sort of a conspiracy or fix in how the Nats are run.

Keith Trostle
Title: Re: Seeding
Post by: Dennis Adamisin on November 25, 2007, 08:08:33 PM

I also believe that seeding the flyers places a significant bias on the outcome.....


Sorry Rich, you got history PROVING this wrong.  The purpose of seeding the qualifying circles has always been to try to level the field as much as humanly possible before flying starts.  Sometimes it works better than others but it has ALWAYS - repeat ALWAYS - worked better than prior "random chance" methods...

Guess I do not know where you are going with posting the judges: is that so poeple can "get an edge" by knowing who the judges are before hand?  Similarly the judges should not care WHO is flying before them - just in scoring accuracies and errors the best they can.  I am missing the "fairness" attribute in what pre-posting is supposed to achieve?

Flyers and Judges are BOTH accountable to the rulebook descriptions of manuvers and errors.  Argueably, sometimes this works better than others; but it usually picks the correct semi-finalists and the correct National Champion. 

Only guarantee is at least 1 person will be completely happy with the outcome, while the rest look for... conspiracy theories!
Title: Re: Seeding
Post by: peabody on November 25, 2007, 08:35:40 PM
I don't believe that history proves me wrong....it proves that some/one THINK that they can forecast the outcome of qualifications.

Random should be the game.

The field should be level for all......
Title: Re: Seeding
Post by: L0U CRANE on November 25, 2007, 08:37:17 PM


...ah, never mind...

Title: Re: Seeding
Post by: Howard Rush on November 25, 2007, 08:40:59 PM
I did a crude analysis of contestant seeding awhile back.  My conclusion was that it didn't make a lick of difference.  I offered to show it to people, but nobody was interested in looking at it or doing any calculations himself.  Here you guys are arguing emphatically about how great or how awful it is.  Well, let's see some math.       
Title: Re: Seeding
Post by: Dennis Adamisin on November 25, 2007, 09:26:19 PM
The math has to do with a limited number of qualifiers coming from each pool of qualifying groups.  The NATs were DRIVEN to seeding by virtue that in prior years certain groups had 6-7 people who were capable of winning the NATs vying for 5 qualifying slots in a given circle, and other groups that had 3-4 spots open to anyone warm to the touch.  Seeding was an attempt (proven succesful) to level the field by evening all that out.

PLAN B: forget the mathematical models - try going back in time and RUNNING the NATs like it was prior to seeding (prior to 1974 I think) and I can all but guarantee you will not make it through the first day of qualifying before you realize WHY seeding was tried in the first place!

Remember that seeding has almost nothing to do with determining the Champion - it has MORE to do with what happens below maybe 6th or 7th place.  Howard, you mentioned a crude mathematical analysis - can you elaborate on your method of analsis and how deep into the placings did you go?  It has let more of the correct people into the semi-finals than the random-chance method

As for bias - if you want to eliminate that then go recruit & train some US Navy judges who have never seen a stunt pattern before.  Then prepare for the return of white pants, the "Blue Angels Stunt Team" and other such things!

Seeded fliers do not always prove worthy, unseeded flyers sometimes surprise, but to those of us still around from the last dice-roll NATs, the success and popularity (because of the improved equity in qualifying) is indisputable.
Title: Re: Seeding
Post by: wmiii on November 25, 2007, 10:27:45 PM


...ah, never mind...

My thought also

 Walter
Title: Re: Seeding
Post by: Howard Rush on November 25, 2007, 10:52:53 PM
"PLAN B: forget the mathematical models - try going back in time and RUNNING the NATs like it was prior to seeding (prior to 1974 I think) and I can all but guarantee you will not make it through the first day of qualifying before you realize WHY seeding was tried in the first place!"

I am unimpressed and unconvinced.  Gimme some (n-r)factorial stuff.  

Here's what I posted on SSW in 2004:

I did a simulation. I assumed 40 contestants. They were assigned at random to four circles in the 2003 Nats Wednesday-Thursday format. I ranked the contestants and assumed that a higher-ranked contestant would always place higher than a lower-ranked contestant. I did 10,000 runs with and without seeding the top eight to see how many times the 20th-ranked contestant would make the top 20 and how many times the 21st-ranked contestant would make the top 20. Here is the upchuck:

No seeding:
#20 made the top 20 5079 times.
#21 made the top 20 4009 times.

Seeding the top eight:
#20 made the top 20 4983 times.
#21 made the top 20 3668 times.

That's not quite what I expected. I might do some histograms and add some noise to the ranking to represent wind and judging and flying inconsistencies, but I have an airplane to build, so I probably won't. If somebody else wants to fiddle with this, I'll email him a copy of the Excel file. The macro is written in VBA.

I didn't check to see how fast this converges. I'll do 100,000 runs overnight and report the result tomorrow, maybe.

Edit: In 100,000 trials:

No seeding:
#20 made the top 20 56381 times.
#21 made the top 20 43962 times.

Seeding the top eight:
#20 made the top 20 55564 times.
#21 made the top 20 41222 times.


Lately, they've been using four circles at the Nats and picking the top five from each for the finals.  This may be what they did in the days of yore.  Seeding might have more of an effect in this case.  It ought to be a simple matter to prove it.  I can look for the program.  I never refined it, because nobody was interested.
Title: Re: Seeding
Post by: Arch Adamisin on November 25, 2007, 11:23:48 PM
I don't believe in the seeding process. I'd like to see it eliminated. It has no effect on who is going to win, the only ones affected are those few new faces that might end up in the top 20. Putting some different names on that list will not change who wins or damage the event but it might just inspire some of those first timers to work a little harder. Let's face it gang, second place is the first loser. After first place, nothing else matters. If you didn't fly well enough to win, why would it matter if you made the top 20 or not?

     Arch
Title: Re: Seeding
Post by: Rudy Taube on November 26, 2007, 12:29:49 AM
Well done Mr. Adamisin .... In this crowd, That's like throwing gasoline on a fire! ;-)

I like your winner take all spirit, You must be an NFL coach???  LL~
Title: Re: Seeding
Post by: Howard Rush on November 26, 2007, 12:41:08 AM
Those of us in the lower echelon kinda like to know where we stand in the hierarchy.   If seeding removes some noise, it's probably worth doing.  It is especially worth doing if it would prevent the 15th-best guy from making the top 20 (for every guy encouraged by incorrect placing, at least one guy is bummed), but I doubt if it would. 

 
Title: Re: Seeding
Post by: Trostle on November 26, 2007, 12:55:38 AM
I have two comments based on the posts above from Arch Adamisin and Howard Rush.

1.  As to the comments from Arch and his statements about seeding having no effect on who is going to win.  Basically, I believe the system we have been using at the Nats for a number of years allows for the best flier at the Nats to win.  I think that most would agree that regardless if there is "seeding" or not, the single best flier will win, given the assumption that the judges are "competent" and that they do their job.  From one perspective, the purpose of the CLPA Nats is to determine who the National Stunt Champion is for that year.  But, there are 40 or 50, maybe more, maybe less, that have prepared for the year and then bother to go to the Nats to fly in the Open CLPA competition.  Most of those 40 or 50 know they will not win, but it is important to probably each of these individuals how well they place, not for some incongruous national recognition (when in actuality, very few care what the ranking is beyond the top 5 or 10 or 20), but the individual wants to know how well he performed.  It may be for "bragging rights" at his local flying field, or more importantly and most probably, it is for his own personal knowledge to know how he "ranks" among his peers.  So in this respect, it is important that the flying field is as level as possible so that some random circle assignments do not load up one group with particularly good fliers and another group is filled with comparative "duffers" prior to to the selection process to see who gets into the top twenty semifinals.  At least that was the case years ago when there were separate qualification circles where each group flew on only their respective qualification circle, in front of one set of judges, using the single best of two flights to determine a group to go on to the finals.  So, yes, we are looking to see who is the single best flier, but the Nats is also designed so that each competitor, regardless of his capabilities can have as fullfilling an experience as possible.  That is why there is a two day qualification process which could easily be done in one day if all we were doing is looking for the single best flier.  But that means that we would be back to the Pre-PAMPA formats where 30 to 40 people would get two flights on one day, with the chances of one of those flights being flown in weather, and their Nats experience with the year of preparation leading to it is finished with perhaps one consequential flight of less than 8 minutes.  So, the Nats are run to find the National champion, but it is run to fulfill whatever goals the other 50 or 60 individuals are seeking.  They deserve a level flying field and that is what the PAMPA Nats format that has evolved over the years has tried to accomplish.

Now, to respond to Howard:  In actuality, the format that is currently used at the Nats really makes the process of seeding unnecessary.  (In fact, I think, over the years, that seeding has not always been done during the qualification process.)  The reason that seeding has little effect on the outcome of our Nats is that even though each flier is assigned to a specific group during the qualification period, each flier in each group flies the same number of flights in front of the each group of judges.  So it makes little difference if one group has a preponderance of good fliers while another does not.  Each judge and each set of judges sees each flier the same number of times.  But if seeding is used, that means that each set of judges during each round that is flown in front of them will see something that resembles a more similar range in the quality of flights seen throughout each round than if the fliers are randomly assigned to each qualification group.  I would think that would tend to keep the judges a bit more alert to uniformly/correctly apply the range of their scoring from the particularly poor flights to the particularly good flights if seeding is done for the circle assignments.  Otherwise,  a judge, (not all - but some) might use a fairly narrow range of scoring through the day regardless of the overall quality of flights, and this same narrow range is used throughout each respective round.  (Some may raise their eyebrows on this and ask how can this happen since we are supposed to have really good, experienced and competent judges at the Nats.  Just let me say, it happens which is an argument to track how the judges perform during and after the Nats to determine who the best judges are to use during the finals round, whether it be the Open event or Advanced.)  And regardless of the myths generated by the myth makers, the "poorer" ranked judges in this process are not all relegated to the Advanced circle.  By the time the semifinals come around, some judges are relieved of their duties, but the quality of judges remaining is spread among the two groups.  And then, for the finals round, at least in the past other judges are relieved of their duties and the most competent continue to judge.  What the myth makers and those who subscribe to conspiracy theories do not comprehend (or choose not to comprehend as it has been explained to them on several occasions) is that considerable effort is taken by the Nats Event Director to balance the judges on the circles throughout the qualification process and for the judge assignments are made during the semifinals and finals rounds so that regionalism or the appearance of regionalism is minimized, and the known personal relationships between judges and the fliers - both favorable and unfavorable - and the appearance of such are minimized to the extent possible.  Those efforts are taken to avoid the very charges from the myth makers and conspiracy seekers that have been made about how the Nats are run.

Now, the following comment is not directed at either Arch or Howard.  I have not yet heard of an argument how seeding biases the outcome of our Nats.  Our judges do not know who is seeded.  There is no indication on their score sheets who is seeded.  Yes, the judges know the names of various fliers and may be familiar with the reputation of the flying capability of certain individuals, but the judges are oblivious to whatever seeding process might have taken place.

Now, let the myth makers and conspiracy seekers continue their rant, but they know not of what they speak.

Keith Trostle
Title: Re: Seeding
Post by: Bradley Walker on November 26, 2007, 10:57:40 AM
Sorry Rich, you got history PROVING this wrong. 

I am sorry, Dennis.  *Nothing* that I know of has ever been PROVEN in about how to run a stunt contest.

You are an engineer.  You should know that...

In a subjective event you get subjective results.  Everyone sees a different outcome.

I think seeding sucks.  If for no other reason is *looks* like it *could* influence the outcome, even if it doesn't.

Seeding or more specially "graduating" the JUDGES has more influence on the outcome, in my opinion (it cannot be proven).  I here Paul is doing away with the practice.  Again, THANK YOU PAUL!!!
Title: Re: Seeding
Post by: Randy Powell on November 26, 2007, 11:32:09 AM
>>I think that most would agree that regardless if there is "seeding" or not, the single best flier will win, given the assumption that the judges are "competent" and that they do their job.<<

And not with just the single best flight, but a series of great flights. I think that's important.

Seeding circles would probably keep a schlub like me from "sneaking" into the top twenty. And well it should. On the other hand (and another way to look at it) is, without seeding, I could end up on a circle with other schlubs while the top pilots got grouped together on another circle and so, have a chance to make a small splash. That could be fun. On the gripping hand, the point here is to pick the national champ and rank other pilots by order of accomplishment. Hmmm...
Title: Re: Seeding
Post by: Dave Adamisin on November 26, 2007, 05:15:26 PM
Keith, thank you for the clear and reasoned presentation. It is in my opinion a representation of your commitment and years of experience. You will have to forgive me a dumb question - it's been a while since I last drove a stunter in anger. The last paragraph leaves the impression that even though the method used at the nats obviates seeding that it is still done. Is that true? Thanks
Title: Re: Seeding
Post by: Dennis Adamisin on November 26, 2007, 05:59:04 PM

You are an engineer.  You should know that...


Oh man, Brad you didn't have to get PERSONAL about it.  Yes I'm and engineer but I've spent most of my life RISING above that!  Actually its worse, I'm an engineer with an MB-freaking-A!!!  HB~>  HB~> (meaning I cheese off everybody) VD~  n~

All seriousness aside, I do NOT have all the circle assignments available from 1973-74-75 time frame when NATs CDs were DRIVEN to seeding by events of the day.  I CAN tell you that seeding did not just "happen" because the NATs CD did not have enough to worry about.   It was HARD to explain why the circle with Rabe, Gieseke, "Champoine", Paul, Trostle & Werwage was no different than the circle with Dewey, Cheatum, & Howe. The accidental inequity of the sorting process spawned the idea to seed the qualifying circles.  Please understand no one needed or WANTED that grief!

I just hope that the sorting is done out in the open and not scripted in the pre-planning process before the event starts.  Maybe use the sorting hat from Hogwarts...  LL~

I am also REALLY curious to see how the bias will end.   ???  LL~
Title: Re: Seeding
Post by: Ted Fancher on November 26, 2007, 06:07:39 PM
I don't believe in the seeding process. I'd like to see it eliminated. It has no effect on who is going to win, the only ones affected are those few new faces that might end up in the top 20. Putting some different names on that list will not change who wins or damage the event but it might just inspire some of those first timers to work a little harder. Let's face it gang, second place is the first loser. After first place, nothing else matters. If you didn't fly well enough to win, why would it matter if you made the top 20 or not?

     Arch

HI Archie, Good to see you posting.

If you believe that nobody cares about anything other than who wins you'd almost certainly be right.  No 1 isn't likely to not qualify to finish off the lesser mortals as the rounds advance.  There are a number of guys that take that position whenever the subject comes up.

They are, however, in a distinct minority.  Finishing in the top Twenty or, even more highly prized, the top Five is a "HUGE" deal to the vast majority of those that attend the Nats.  To them it provides a "gauge" of their progress and they are mightily concerned that the "process" by which those who will ultimately be ranked in those "hallowed" slices of competitors are selected. 

The entire process of the multi-day format and the procedures to administer it (including seeding) was intended to provide a means to do the selecting in as fair a way as possible for those "not number 1" flyers.  Let's face it, if no 1 was the only thing the competitors were concerned about we could hold the whole thing in one day, one round, winner take all.

I dare you to try that next July!  ~> ~> ~> ~^ ~^ ~^

Ted Fancher
Title: Re: Seeding
Post by: Ted Fancher on November 26, 2007, 06:27:31 PM
Keith, Denny, et all.

Really, the only time flyer seeding is employed (or necessary) is for the qualifying process.

Once you accept the multi-day format and the whittling down of flyers from day to day the event changes from a "single best flight" scenario to one of continued excellence right down to the best two of three flights on top five day.  This means that a potential winner (a flyer that has the "goods") that has minor problems on days one, two or three can still get "in the race" on day four when the problem has been solved and win the whole magilla.

The value of seeding is to prevent the theoretically possible overpopulation of a single qualifying group (out of the four) that could (and has) resulted from random selection.  It is possible to randomly select the top ten flyers into a single qualilfying group which, by definition means that five of the best flyers will fail to make it to top twenty day.  Seeding of "known quantities" minimizes the chance of such a lop sided circle. 

And, yes, not seeding for the existing format could very well result in the guy with the minor problem (not enough to keep him from qualifying if the "talent" is distributed more equally) who won the imaginary event above from being in the mix come Top Five day.

The random selection scenario above could also result in top twenty places for five less talented flyers because they were in a circle which included none of the "extra" five known quantities in the "hard" circle.

Again, if all you're interested in is finding out who number one is, random selection isn't a big issue (the remote possibililty of the previously mentioned "extreme" scenario notwithstanding).  If, however, you want the final results from No 1 through No 20 to have any validity, come method must be used to balance the qualifying circles.

Ted

p.s. FWIW, I have no big problem with random selection of judges (with the sole exception of a judge clearly and verifiably acting to purposely elevate or crucify certain competitors).  I am, however, something of a minority when I've discussed the subject with event administrators.  Totally different philosophies.
Title: Re: Seeding
Post by: Shultzie on November 26, 2007, 06:30:35 PM
Seeding competiton events has been around years and years and is still alive and well today.
 
  Way too many years ago...I was involved with the Ski Patrol at both Hood Meadows and at Timberline and  sadly, we often used seeding as a way of not only assuring all heats had an equal share of known winning competitors, but more importantly,  this help arrange the skiing order to help assure that the fastest racers were not handicapped with changes in the weather and snow condititons....Was this fair??? Maybe, Maybe not?

TALK ABOUT A HORNETS NEST OF COMPLAINTS.

Then again at Pacific University many of our tennis tournaments were also SEEDED to assure that not only the better players weren't  all grouped together...which risked having the majority of the name players eliminated before the final rounds. Usually in the end...the best players win...but hey!
 WE ARE ALL HUMAN BEINGS...AND EVEN THE BEST HAVE THEIR SO-SO NOT SO HOT MOMENTS and on any given day...(be it the luck of the draw, the wind, stir in equipment failure, brain glitches, etc etc)...
even the NERD-NEAR-DE WELLS can get lucky and  pull off that occational well deserved LUCKY STRIKE WIN. Also this assures more entries...and may increase the chances of an accidental LADY-LUCK DAY for folks with less expertise. WHATEVER???

SEEDING IS A WAY OF LIFE IN SO MANY SPORTS...be it Tennis, Golf, Skiing, car and boat racing...etc etc.
FAIR OR NOT!!! As a near-de-well competitor...in Kite flying I can assure you that SEEDING is often used...for many reasons but mostly to assure that $$$$ keeps coming in from the paying crowds that PLOP DOWN big $$$$ to see their famous and favorite big name players play in the finals...
be it TOY AIRPLANE CONTESTS... US OPEN TENNIS FINALS..OR PGA GOLF EVENTS.
BIG MONEY TALKS!
SEEDING ALSO ASSURES THAT THAT BIG $$$ WILL KEEP ON FLOWING to keep that crowd from bailing out early and goin home before the competitons are over or the finals are reached.

Howard Rush....VERY VERY INTERESTING INFO...AND THANKS FOR TAKING THE TIME TO SHOW THE NUMBERS?
 HB~> HB~> n~ n~ S?P VD~ H^^


Title: Re: Seeding
Post by: phil c on November 26, 2007, 07:02:02 PM
"Otherwise,  a judge, (not all - but some) might use a fairly narrow range of scoring through the day regardless of the overall quality of flights, and this same narrow range is used throughout each respective round.  (Some may raise their eyebrows on this and ask how can this happen since we are supposed to have really good, experienced and competent judges at the Nats.  Just let me say, it happens which is an argument to track how the judges perform during and after the Nats to determine who the best judges are to use during the finals round, whether it be the Open event or Advanced.  Trostle"

Keith,  I'd be interested in seeing exactly how judges performance have been tracked.  The only valid way to do this is to use some form of statistical analysis, perhaps of all the scores for each maneuver, each judge, and each pilot on a circle, taken as a group.  The worst possible way is to peruse the scorecards and try to make some sense of what each judge is doing and why.  Using a narrow scoring range is not per se a reason to question a judge's calls.  The real key is how consistently a group of judges agrees as to which maneuvers were better and which were worse.  Even then, given the vagaries of rules interpretation, a judge could very legitimately downgrade a maneuver that the other judges thought was good.  Even being consistently at odds with the other judges in a group is not necessarily mean that judge is wrong, but perhaps putting more weight in different areas, which is allowed by the rules.
Title: Re: Seeding
Post by: Howard Rush on November 26, 2007, 10:38:37 PM
Come now, Ted, show me how seeding could make any perceptible difference in who crosses the 20-21 divide.   
Title: Re: Seeding
Post by: Bradley Walker on November 27, 2007, 11:08:58 AM
Come now, Ted, show me how seeding could make any perceptible difference in who crosses the 20-21 divide.   

Howard, I think it is more of a "feel" thing.
Title: Re: Seeding
Post by: Marvin Denny on November 27, 2007, 11:47:25 AM
Keith, Denny, et all.

 Ted, I do not know where you pulled up my name from all this argument!!  I have not (nor I EVER have) complained or harped the "seeding".  Please keep my name out of it.

  Marvin Denny
Title: Re: Seeding
Post by: Ted Fancher on November 27, 2007, 12:20:03 PM
Come now, Ted, show me how seeding could make any perceptible difference in who crosses the 20-21 divide.   

Howard,

You're kidding, right?

First scenario.

Random selection to circle one (of four) at the nats: Paul Walker, Orestes Hernandez, David Fitzgerald, Brett Buck, Randy Smith, Windy Urtnowski, Bill Rich, Frank McMillan, Rich Oliver, Doug Moon, Bill Werwage, Joe Bellcrank, Derek Berry and Harry Bellcrank.  Five top scores from Wednesday and Thursday advance to the top twenty.

Random selection to Circle two: Harry Bellcrank, John Bellcrank, Bill Bellcrank, Cynthia Bellcrank, Greg Bellcrank, Melinda Bellcrank, Oliver Bellcrank, Twisted Bellcrank, Junebug Bellcrank, Peter Bellcrank, Brother Darrel, the other brother Darrel,  and Ted Fancher.  Five Top Scores advance to the top twenty.

At least part of the outcome of each circle has been preordained by the seeding.

It is a certainty that at least five of the guys with names other than Bellcrank or Darrell will not make the top twenty in circle one; and an equal certainty that at least four from the family Bellcrank/Brother Darrell consortium will fly in the top twenty  in circle two.  That is math even I can understand.

Second Scenario.

All the Bellcrank family plus the brothers Darrell are split equally between the circles via seeding the entered flyers via performance to the best of the ability of the administrators.

All the flyers with other names are split equally between the two circles; seeded based on substantive and widely known past performance.


The outcome of neither circle is preordained via the seeding.

What could happen is that all or none of the "other named (i.e. seeded via past performance) flyers" could make the top twenty. Ditto, that all or none of the Bellcrank clan and the brothers Darrell could do so as well.  Who makes the top twenty will be based strictly on their performance within the seeded group with whom they compete on the qualifying days.

That doesn't seem hard to understand to me.

Ted

p.s. For those that perceive clandestine subterfuge (cheating) in even the suggestion of "seeding" I would point out that pretty much none of the Clan Bellcrank or brothers Darrell have much of a chance of seeing themselves in the top twenty if they are the randomly selected cannon fodder in the circle with the "famous" guys.  They might well b.i.t.c.h. that the circles weren't equally competitive.  In fact, it seems like I've heard that tale before.
Title: Re: Seeding
Post by: Ted Fancher on November 27, 2007, 12:21:30 PM
Keith, Denny, et all.

 Ted, I do not know where you pulled up my name from all this argument!!  I have not (nor I EVER have) complained or harped the "seeding".  Please keep my name out of it.

  Marvin Denny

Marvin, meet Denny Adamisin.  Denny, meet Marvin Denny.

Sheeeeeesh.

Ted
Title: Re: Seeding
Post by: Trostle on November 27, 2007, 12:26:27 PM
Keith, Denny, et all.

 Ted, I do not know where you pulled up my name from all this argument!!  I have not (nor I EVER have) complained or harped the "seeding".  Please keep my name out of it.

  Marvin Denny

Hi Marvin Denny,

I have not refered to you or your name anywhere in this thread.  If you will notice Dennis Adamisin has posted on this thread.  Ted was addressing his remarks to Dennis Adamisin, not you, Marvin Denny.  Furthermore, Ted would not refer to you by only your last name in this context.  Your name has been kept "out of it" until you interjected it.

Double sheesh

Keith Trostle
Title: Re: Seeding
Post by: Paul Walker on November 27, 2007, 01:16:52 PM
As I am running the 2008  NATs, I can tell you what IS going to happen.

The qualifying rounds WILL be seeded so that all the top fliers are not on circle A and all the Bellcrank family is not on circle B. They will be distributed evenly. After the qualifying rounds are completed, the top 20 pilots will gather to draw circle order for the semi finals. Once the circle assignments are drawn, then the flight order will be drawn. So before the pilots leave the field Wednesday, they will know their assignments for Thursday. Then the judges will draw for their circle assignments as well. Note that the fliers and the judges will be the ones drawing the numbers, not the ED. When Thursday's flying is completed, the process will repeat for the finals, flight order and judging assignments. After the Open finals are done, the process is repeated for the Walker Cup Flyoff.

That's as random as I believe it should be. I fully believe that the top 20 is important enough to many of these fliers that have put up their time and money to get there that we should make sure they have a "fair" chance and not be hurt by a "bad" draw, so seeding the qualifying rounds will eliminate the "bad draw" issue. Bad wind, rain, earthquakes, tornado's, engine runs, etc, don't complain to me, complain upstairs.
Title: Re: Seeding
Post by: Bradley Walker on November 27, 2007, 01:32:24 PM
How are you assigning judges for the later rounds?
Title: Re: Seeding
Post by: Ted Fancher on November 27, 2007, 01:40:25 PM
As I am running the 2008  NATs, I can tell you what IS going to happen.

The qualifying rounds WILL be seeded so that all the top fliers are not on circle A and all the Bellcrank family is not on circle B. They will be distributed evenly. After the qualifying rounds are completed, the top 20 pilots will gather to draw circle order for the semi finals. Once the circle assignments are drawn, then the flight order will be drawn. So before the pilots leave the field Wednesday, they will know their assignments for Thursday. Then the judges will draw for their circle assignments as well. Note that the fliers and the judges will be the ones drawing the numbers, not the ED. When Thursday's flying is completed, the process will repeat for the finals, flight order and judging assignments. After the Open finals are done, the process is repeated for the Walker Cup Flyoff.

That's as random as I believe it should be. I fully believe that the top 20 is important enough to many of these fliers that have put up their time and money to get there that we should make sure they have a "fair" chance and not be hurt by a "bad" draw, so seeding the qualifying rounds will eliminate the "bad draw" issue. Bad wind, rain, earthquakes, tornado's, engine runs, etc, don't complain to me, complain upstairs.

Paul,

Certainly a good start and we agree (obviously) about the flyer seeding in the qualifying rounds.

I would, however, encourage you to reconsider the random selection of judges for later rounds for the following reason.  Can you imagine the hue and cry from predictable vocal sources if one of your volunteer judges (oh, just for an example, Ted Fancher) is randomly assigned to judge the Top Five and/or Walker Flyoff?  Especially if the competitors include names like David Fitzgerald, Brett Buck, Phil Granderson or, in the Walker Flyoff, Paul Ferrell?

What the agitators fail to recognize is that avoiding such assignments to the greatest degree possible is one of the primary functions of the judge seeding process at the Nats. Although I've no problem with flying before a randomly selected group of judges, I would, frankly, prefer "NOT" to be put in such a position as a judge.  The reasons are too obvious to waste time on.

 I would be perfectly happy to accept an assignment to judge other than the top twenty or finals day if doing so would make your life easier.

Ted 

Edited to remove unwarranted editorial comment.
Title: Re: Seeding
Post by: Ted Fancher on November 27, 2007, 01:41:04 PM
How are you assigning judges for the later rounds?

Paul,

I rest my case.

Ted
Title: Re: Seeding
Post by: Marvin Denny on November 27, 2007, 02:13:19 PM
Hi Marvin Denny,

I have not refered to you or your name anywhere in this thread.  If you will notice Dennis Adamisin has posted on this thread.  Ted was addressing his remarks to Dennis Adamisin, not you, Marvin Denny.  Furthermore, Ted would not refer to you by only your last name in this context.  Your name has been kept "out of it" until you interjected it.

Double sheesh

Keith Trostle

  I THOUGHT his name was Dennis???  I am the only Denny posting on this forum that I know of MR  Trostle.  Therefore it is natural that I thought  that I might be blamed for some statements in your petty little thread.   
tripple Sheesh to you too.

  Marvin DENNY
Title: Re: Seeding
Post by: Howard Rush on November 27, 2007, 02:19:44 PM
I wish to point out that snotty comments a few posts above (and snotty comments yet to come) were and will be posted by Howard Rush, not Rush Limbaugh. 
Title: Re: Seeding
Post by: Howard Rush on November 27, 2007, 02:38:35 PM
"You're kidding, right?"

No.  Calculate for me the probability of that skew happening.  Then calculate the effect on the 20-21 divide.  This can be calculated.  The crude analysis above convinced me that, for that case, the effect of seeding is negligible.  You won't convince me otherwise by arm waving. 
Title: Re: Seeding
Post by: Chris McMillin on November 27, 2007, 02:59:29 PM
I don't believe in the seeding process. I'd like to see it eliminated. It has no effect on who is going to win, the only ones affected are those few new faces that might end up in the top 20. Putting some different names on that list will not change who wins or damage the event but it might just inspire some of those first timers to work a little harder. Let's face it gang, second place is the first loser. After first place, nothing else matters. If you didn't fly well enough to win, why would it matter if you made the top 20 or not?

     Arch


That isn't exactly how I remember you feeling at the aborted end of the '93 Nats Qualifications. Both Archie and I were left out in the cold, both unable to recover when Bruce decided it was time for cocktails!
Chris...
Title: Re: Seeding
Post by: Shultzie on November 27, 2007, 04:21:20 PM
OK!
THAT DID IT.....HOWARD! TED!....PAUL!!!!....or any of the rest of you Grunts!!!


TO END ALL THIS "SEEDING CON-TRO-PRO" I have an idea  for a loonie toon' to cap off this SEEDING THREAD or feel free to send me a few ideas for your own SEEDING TOON at:  donolddo@aol.com
 put perhaps all I will get is "phising-spam" but go for it
Title: Re: Seeding
Post by: Paul Walker on November 27, 2007, 04:53:24 PM
How are you assigning judges for the later rounds?

As I stated.....

Then the judges will draw for their circle assignments as well. Note that the fliers and the judges will be the ones drawing the numbers, not the ED. When Thursday's flying is completed, the process will repeat for the finals, flight order and judging assignments. After the Open finals are done, the process is repeated for the Walker Cup Flyoff.
Title: Re: Seeding
Post by: peabody on November 27, 2007, 05:19:28 PM
Bravo on the Judge selection method....!!!!

edit spelling

Title: Re: Seeding
Post by: Wayne Collier on November 27, 2007, 05:30:06 PM
Apples vs Oranges but I learned a lesson as a youth that stuck with me.  As a cub scout I participated in pinewood derby contests.  Some of you guys may have as well.  The "lesson" came after I was too old to compete but went to a regional meet to just watch.  Standard double elimination event.  As I watched not only the heats but the charts I had one of those epiphany moments.  The fastest car, barring some disaster, was going to get first place.  The second fastest car, barring some disaster, would end up in second place losing only one heat to the fastest car.  Placement after that was something of a toss up.  Depending on the line up, the size of the field, and where the cars were slotted, the third, fourth, or fifth fastest cars(etc) could easily have been eliminated from having faced both the fastest and second fastest cars as they progressed across the charts. Isn't seeeding supposed to help offset this kind of skewing?  Isn't it supposed to help insure that the top field of contestants is the top field?
Title: Re: Seeding
Post by: Bradley Walker on November 27, 2007, 06:00:05 PM
As I stated.....

Then the judges will draw for their circle assignments as well. Note that the fliers and the judges will be the ones drawing the numbers, not the ED. When Thursday's flying is completed, the process will repeat for the finals, flight order and judging assignments. After the Open finals are done, the process is repeated for the Walker Cup Flyoff.

I know that it must be something obvious...but you need less judges as you proceed, so I am still confused as to how you will determine who judges the Finals and subsequently the Walker Cup (which requires far less judges).  Are you saying the judges will *draw* to see if they *get* to judge the Walker Cup? 

If so, that's great!!!

No matter what, we APPRECIATE what you are doing Paul.  Bravo, and thanks.
Title: Re: Seeding
Post by: peabody on November 27, 2007, 06:07:55 PM
Brad...
As I remember, ore judges are used as the week progresses:
3 per circle during two-day qualifying....=12
3 per Advanced Circle (the bad ones) and 5 per circle for Open on "Top Twenty Day".......= 16
Generally some have hissy fits because they have been denied judging the Walker Cup Open and are relegated to kiddie stunt....
Title: Re: Seeding
Post by: Dennis Adamisin on November 27, 2007, 07:21:47 PM
  I THOUGHT his name was Dennis???  I am the only Denny posting on this forum that I know of MR  Trostle.  Therefore it is natural that I thought  that I might be blamed for some statements in your petty little thread.   
tripple Sheesh to you too.

  Marvin DENNY

OK, OK, time for us all to get our Sheesh together.   010!

I met Keith, Ted, Shultzie, (and lots of others) when I was routinely called "Denny".   H^^  Heck I met Jack Sheeks when I was still in single digits!

Marv & I met  just last year - uh, I introduced myself as Dennis!  H^^ I post my name here that way too.  Makes me sound like a grown up.


Just sorry I never had a REALLY cool aka like BigIron! 
Title: Re: Seeding
Post by: Howard Rush on November 27, 2007, 07:35:03 PM
Apples vs Oranges but I learned a lesson as a youth that stuck with me.  As a cub scout I participated in pinewood derby contests.  Some of you guys may have as well.  The "lesson" came after I was too old to compete but went to a regional meet to just watch.  Standard double elimination event.  As I watched not only the heats but the charts I had one of those epiphany moments.  The fastest car, barring some disaster, was going to get first place.  The second fastest car, barring some disaster, would end up in second place losing only one heat to the fastest car.  Placement after that was something of a toss up.  Depending on the line up, the size of the field, and where the cars were slotted, the third, fourth, or fifth fastest cars(etc) could easily have been eliminated from having faced both the fastest and second fastest cars as they progressed across the charts. Isn't seeeding supposed to help offset this kind of skewing?  Isn't it supposed to help insure that the top field of contestants is the top field?

Seeding works differently in one-on-one sports such as tennis or control line combat than it does in stunt.  If the best two players in a big single-elimination tennis tournament meet in the first round, one of them will tie for last place in the tournament.  If I understand Paul's plan, the top twenty at the stunt Nats are the best five from each of four groups.  The top 20 flyoff is like a new contest; qualifying scores aren't used.  No Nats champ contender will fail to make the top twenty unless five guys who fly better on qualifying days get dealt to his group of (approximately) ten.  Nats stunt seeding only takes a little randomness out of who among those close to the 20th-best line makes the top 20.  
Title: Re: Seeding
Post by: Dennis Adamisin on November 27, 2007, 07:45:19 PM
Paul

THANKS for posting your intentions for the NATs.  No quarrels with anything you posted.  CLP**

For the competitors, you see the CD's plan, you know what you have to do - just BRING IT.  8)  AP^
Title: Re: Seeding
Post by: peabody on November 27, 2007, 08:13:11 PM
Howard....
To throw the anal retentive a one, but keep some randomness what about seeding, say, the top eight from last year's Nats?

Advanced should be real easy to seed....the Expert flyers ahead of the Advanced bunch.
Title: Re: Seeding
Post by: Arch Adamisin on November 27, 2007, 08:16:26 PM
Chris,
Funny you should mention that fiasco. My only complaint that day is the same today as it was then. There was plenty of time to finish the event and allow everyone to get all their flights. There were no concerns with the seeding process, only the fact that the ED decided that it had been a long day and it was over, even though it was in violation of the entry blank and AMA procedure. After I filed the protest, I was informed that the protest was rejected by the category director, Bev Wisnewski. I accepted this as true and we left. Two days later, I spoke with Bev and she informed me that she had NOT seen the protest and if she had, she would have agreed that the remaining flights should have been flown.

     Arch
Title: Re: Seeding
Post by: Paul Smith on November 27, 2007, 08:26:31 PM
I think I can correctly identify BigIron 100% of the time and could probably score 75% in a name-the-Adamisins event, which is better than a lot of people could do.

This "seeding frenzy" makes interesting reading and I don't personally have a dog in the fight.  

The same issue was kicked around in the Combat world at times.  We had enough data to seed the flyers into quadrants and have a very pure "final four".  But the idea was never, to my knowledge, used.  Reason: the rank-and-file hated it.

In a totally unseeded world, the best man still wins.  However, somebody who is "less than great" still has a hope of getting lucky and placing a bit higher on the totum pole than he deserves.  If you have a bullet-proof seeding system that guarantees a "correct" top twenty, the 21st-35th guys will get the idea pretty soon and just stay home.   Better to allow a break once in a while and let somebody come in 18th place due a lucky draw.



Title: Re: Seeding
Post by: Arch Adamisin on November 27, 2007, 08:34:52 PM
Paul,
Thanks for sharing your plans. If I understand correctly, it sounds like the Walker Cup flyoff will finally be back the way it should be. 3 nat's champs competeing for the trophy, 1 junior, 1 senior, 1open, 2 flights each, high score wins?

     Arch
Title: Re: Seeding
Post by: Chris McMillin on November 27, 2007, 08:38:44 PM
I remember when we got back to California later in the summer Bev asked me about it. I told her I was the "other guy" and she about flipped!
Just reminding you that it wasn't always about just winning first place! When it's our kids, placing well seems to mean a lot more.
Archie and I just wanted to redeem ourselves that day!
I think that is the reason the "balancing" result of seeding the flyers was started, and it probably works to the extent that if flyers known to be able to progress are in multiple circles with a split number of hot rods, the potential to move up is evened out.
I personally think statistical analysis of subjectively judged events is folly, but I have only been a flight engineer. What do I know! ;)
 
Chris...
Title: Re: Seeding
Post by: Chris McMillin on November 27, 2007, 08:41:09 PM
Howard....
To throw the anal retentive a one, but keep some randomness what about seeding, say, the top eight from last year's Nats?

Advanced should be real easy to seed....the Expert flyers ahead of the Advanced bunch.


Whaa, Whaa, Whaa...give it a rest, Rich. You are the guys that want all of this skill level stuff at the Nats, and then you complain about it. That is really "rich".  Z@@ZZZ
Title: Re: Seeding
Post by: RC Storick on November 27, 2007, 08:48:56 PM
Seeding or nor I am getting better than when I first came back. The only thing I would like to see to level the field is Judges who don't know the fliers by name. Perhaps collage students who not only can see better and are more suited to stand all day. Maybe collage students from the aeronautical engineering end of the spectrum.

I also think this years Nats will be a little easier as not as many fliers will attend it. As far as protests go the AMA has put a sure end to it by imposing a $50.00 to do so. What does the money go to? The beer fund?? But no need to worry about that as I doubt anyone will protest ever again.

But what do I know..
Title: Re: Seeding
Post by: Randy Powell on November 27, 2007, 10:29:41 PM
Man, you guys sure make attending a Nats seem like fun.
Title: Re: Seeding
Post by: Paul Walker on November 27, 2007, 10:30:59 PM
I know that it must be something obvious...but you need less judges as you proceed, so I am still confused as to how you will determine who judges the Finals and subsequently the Walker Cup (which requires far less judges).  Are you saying the judges will *draw* to see if they *get* to judge the Walker Cup? 

If so, that's great!!!

No matter what, we APPRECIATE what you are doing Paul.  Bravo, and thanks.

Yes, that is correct. The judges that will judge the OPEN finals will be selected by a draw.  Once that is completed, the judges that didn't judge the OPEN finals will draw for the Walker Cup flyoff. By definition, there will be different judges for the Walker Cup than the OPEN finals!

Paul W
Title: Re: Seeding
Post by: Paul Walker on November 27, 2007, 10:34:49 PM
Paul,
Thanks for sharing your plans. If I understand correctly, it sounds like the Walker Cup flyoff will finally be back the way it should be. 3 nat's champs competeing for the trophy, 1 junior, 1 senior, 1open, 2 flights each, high score wins?

     Arch

Close....The Walker Cup will fly 3 rounds, adding the 2 best to determine the winner.

Good thing that Gruber kid grew up!!!!

Paul
Title: Re: Seeding
Post by: Tom Niebuhr on November 28, 2007, 12:01:21 AM
Robert "Sparky" wrote: "Perhaps collage students who not only can see better and are more suited to stand all day. Maybe collage students from the aeronautical engineering end of the spectrum."


Remember the problems with Navy judging? They were pilots and/or officers as I recall. All with college educations. Let's NEVER go back to that.

We will never satisfy everyone. but what we have now is far better than it was!
Title: Re: Seeding
Post by: Trostle on November 28, 2007, 02:08:10 AM
Seeding or nor I am getting better than when I first came back. The only thing I would like to see to level the field is Judges who don't know the fliers by name. Perhaps collage students who not only can see better and are more suited to stand all day. Maybe collage students from the aeronautical engineering end of the spectrum.

(clip)

But what do I know..

Robert,

This is a neat idea, but there is a problem.  And it is the same problem we have in recruiting judges for our Nats and the Team Trials.  People need to be willing to volunteer their time.  People need to be willing to travel and that will cost either the individuals who volunteer or others who are willing to contribute fairly large sums of money.   People will need to pay for their room and board while doing their volunteer work and that will cost either the volunteers or others who are willing to contribute fairly large sums of money.

In my case, figuring transportations costs (airline travel, rental car), motel room, it costs me about $1,000 to have the privilege to judge the Nats or the Team Trials.  (I have judged at 11 of these events as well as being the ED at the Nats twice and the Team Trials five times.)  Now, not all volunteers need to travel that far and car rental might not be necessary, but there is still a significant out-of-pocket expense in addition to most volunteers have to use vacation time from work to do this job.

Our stunt fliers should feel fortunate that there has been a cadre of people willing to make themselves available to do this job, even though there are often undertones after these events about the quality of judging.  It is for a combination of these reasons that there is not a larger cadre for the Event Directors at these events to choose from.  It does get discouraging at times and I have done my last Nats and Team Trials as a judge for flak that is continually thrown up about how the judging needs to be improved, albeit that some of that nonsense comes from those who do not bother to compete at those levels and really understand little of how the Nats or Team Trials are run and of the efforts made so that the competition field is as level as possible for each individual contestant.

Keith Trostle
Title: Re: Seeding
Post by: Paul Smith on November 28, 2007, 05:36:28 AM
The only thing I would like to see to level the field is Judges who don't know the fliers by name. Perhaps collage students who not only can see better and are more suited to stand all day. Maybe collage students from the aeronautical engineering end of the spectrum.


You have a pretty good concept there.  In a large town like Muncie. there must be people who routinely judge springboard diving, gymnastics, figure skating, and other "stunt-like" competitions.  They are AAU-certified and maybe hungry for a few extra bucks in the summer.  Maybe less expensive than the current scheme.
Title: Re: Seeding
Post by: Bradley Walker on November 28, 2007, 06:34:15 AM
Yes, that is correct. The judges that will judge the OPEN finals will be selected by a draw.  Once that is completed, the judges that didn't judge the OPEN finals will draw for the Walker Cup flyoff. By definition, there will be different judges for the Walker Cup than the OPEN finals!

Paul W

That sounds kind of AWESOME...

No matter what, THANKS for TRYING to get some randomness into the system. 

Something else too.  I think the judges will appreciate it too.  I might be wrong, but I think the method you are describing for selection is much more fair (luck of the draw is fair IMHO) than basing someone's performance on a measurement system that no really knows how it works, and is subjective at best.

Also, the judges will be free to judge how they want without anyone scrutinizing their performance and holding "graduation" over their heads.  I certainly never agree with the previous system that pretty much encouraged bracketing or "staying close to the mean".  I would really like to see judges that penalize "stinko" maneuvers and reward stellar maneuvers.  Let's see a little movement in those scores!!!
Title: Re: Seeding
Post by: Clancy Arnold on November 28, 2007, 07:01:12 AM
Wayne,
Reference your post # 38 above.

I put on over 200 Pine Wood Derby races  for 4000 Cub Scouts in my younger days.  I used a computer scored track that picked the First, Second and Third  car out of each 6 car race.  I used a double elimination system with one additional change.  If the field was less than 72 cars I would take the lower three cars of each first heats and put them into new heats giving a car that was the sixth fastest a chance to still run in the finals if he was against the top cars in his first race.  Never had a complaint in all of the races I officiated.

Would this work at the NATS??

Clancy
Title: Re: Seeding
Post by: Bill Little on November 28, 2007, 08:46:10 AM
I hate to say this, but this forum is starting to *sound* like all the others.  Politics, and such types of discussions are not supposed to be going on.

As we have grown, the "flavor" has slowly, almost imperceptibly, changed. 

What about it Sparky?
Title: Re: Seeding
Post by: peabody on November 28, 2007, 09:04:44 AM
Hi Bill...
I think it's a good thread...lots of information, very little "sniping", but rather several expressing opinions that are almost always squashed when offered in other formats: good stuff here, form people who have varying views and are civil.
Title: Re: Seeding
Post by: Doug Moon on November 28, 2007, 09:17:50 AM
This is not a political nonsense thread.  This is a good dicussion.  Of course there are shots here and there.  I find it interesting. 
Title: Re: Seeding
Post by: Bradley Walker on November 28, 2007, 09:24:55 AM
Why would discussing the format of the next Nats be "political"?

We have the opportunity here to understand the Nats seeding process directly from the mouth of the ED of the next Nats.  I myself am grateful for this thread.  I had a lot of misconceptions about how the Nats would be run.  Now I understand.

You have to be careful that *everything* in stunt could be considered "political".
Title: Re: Seeding
Post by: Ted Fancher on November 28, 2007, 09:26:19 AM
Paul,
Thanks for sharing your plans. If I understand correctly, it sounds like the Walker Cup flyoff will finally be back the way it should be. 3 nat's champs competeing for the trophy, 1 junior, 1 senior, 1open, 2 flights each, high score wins?

     Arch

Arch,

I know you've been away for a while.  The Walker Flyoff between Jr, Sr and Open Champs has been back to the old format for a number of years now.

Ted

Edit:  Just noted your comment on the number of flights.  The current format has the three champs flying three flights, the best two added together to determine the Walker winner.
Title: Re: Seeding
Post by: Bill Little on November 28, 2007, 09:47:26 AM
Doug and Brad,

Yes, this is a good thread.  So far it has been civil.  But, Sparky INTENTIONALLY began this particular internet forum/board to NOT be like any of the others.  A place to discuss stunt models and nothing more.  I know that would get VERY boring to many, but then the NATS discussions are VERY boring To many many more who do not, nor ever will, fly in the NATS.  This also applies to rules discussions, and other things applying to an event and not models. 

What I am saying here is that the original intent of the board was not to discuss these matters, they are cussed and discussed on other sites ad nauseum.  This place was intended to stay away from all the *garbage* out in cyberspace, be NON political and deal only with models. 

I know that as the membership has grown, more and more topics that were, at first not allowed, have become popular.   So it's just time to sit back and re-evaluate the board's position.  Does the admin want this to become just another site?  I don't know, that's why the question to Sparky.

A vast majority of this board's members could live the rest of their lives and never mention the NATS, rules, or anything else of such nature.  Those subjects, like I said, can be hashed out (until deleted) on other sites.  I am just wondering if Sparky is thinking that the wishes of a few are what he wants to run the forum.  I am just the "hired help" who's salary is twice what it was when all this started! ;D  (2 X 0=0 )

I am a member (at least for now) of the PAMPA EC, so all of this is something I can do elsewhere.  Does the majority of the members here want to hear it?
Title: Re: Seeding
Post by: Bradley Walker on November 28, 2007, 10:40:26 AM

What I am saying here is that the original intent of the board was not to discuss these matters, they are cussed and discussed on other sites ad nauseum.  This place was intended to stay away from all the *garbage* out in cyberspace, be NON political and deal only with models. 


I took my board down, and I do not go to SSW.  Why?  Because it is *moderated* on the bias.  They do not run that board from a neutral perspective.  It is a board that is openly hostile to "agitators".  The only guy I see "sniping" or using inflammatory PERSONAL language to describe people who they do not agree on this thread are the same guys who post on SSW and do the same thing there.

I also do not belong to PAMPA because it is basically the same people who populate SSW, so what now?

I think this is a great board, but I dot not understand the entire idea of people protesting what is discussed. 

You have to ask yourself, where does that come from?  It is certainly not the way I was raised.  If you do not like a thread don't friggin read it.  That is the way it is done here *in America*.  Stopping people from discussing things you do not like is more like the USSR or Communist China.

Discussing the Nats format is not POLITICS.  PAMPA is politics.  This is the EVENT for goodness sake!!!
Title: Re: Seeding
Post by: Bill Little on November 28, 2007, 11:26:13 AM
I took my board down, and I do not go to SSW.  I also do not belong to PAMPA, so what now?

I think this is a great board.  I do not understand the entire idea of people protesting what is discussed.  If you do not like a thread don't friggin read it.  That is the way it is done here in America.  Stopping from discussing things you do not like is the USSR or Communist China.

Discussing the Nats format is not POLITICS.  PAMPA is politics.  This is the EVENT for goodness sake!!!

Quote
"I took my board down, and I do not go to SSW.  I also do not belong to PAMPA, so what now?"
 
I cannot answer that question for you.  I know that I have been charged to do a job, and I have done so without complaint from the OWNER (only one that matters).  Maybe you need to start you own board back up if you want to discuss what isn't allowed here.  I would be more than happen to join and contribute.

The EVENT was not the reason Sparky started this board.  Your view of what is or isn't politics, or what is or isn't allowed for discussion actually means no more than Joe Shmoe's opinion, OR MINE!There is no open area of discussion for what is or is not allowed.  if that is "UN_American" as you insinuate (BTW: the USSR has been gone quite a while LOL!!), well then tell that to Sparky, the guy who set up, maintains and pays for *his* board.  This really a Private Club, members are "allowed" to belong and must be approved.  Also, they can be dropped, banned, put on restriction, etc., all by the decisions of the owner. That actually changes things, even in the leagal arena.

I didn't ask you, I asked Sparky.  If I sound harsh here, it is because we have experienced quite a long time of pretty decent harmony here, following the rules that Sparky told ME to follow.  You have ideas that have merit.  I do not contradict that.  I enjoy a good debate with you and Doug or anyone else, but I am remembering what I was told when this all started.  If we open the door in one are, how do we bolt it in another area?

Sparky actually has pulled HIS OWN posts in the past as have I.

This isn't personal, Brad.  I really doubt that you are any more, if as much, patriotic as I am.  I believe in Freedom of Speech.  I just need clarification from the only person who's opinion actually matters in this case.  Since he made me second in command of the entire site, made me the moderator here in this section, and told me to delete what I needed to (with no questions asked), AND I know he has been out of town attending to his Mother who is passing, I am simply asking his guidance on this.

So refrain from making those wild statements about not reading posts, USSR, China, etc., it's not your decision to make.  Otherwise, I will not need his guidance to hit the delete button! ;D


So far, I see this thread as educational in nature and possibly answering questions that are important to a minuscule part of the membership here.  So it is still up.  If Sparky says otherwise, I will "can it" in a heart beat.

You were not here at the start (or more importantly in the planning stages) so you are not privy to WHY the board exists.  It was not begun to just be another SSW or otherwise.  So, believe me, I am not disturbed in the slightest with your posts.  You know full well I would say so if I were.

BTW: How are the handles coming along? ;D

Respectfully,
Bill Little <><
Title: Re: Seeding
Post by: Doug Moon on November 28, 2007, 11:37:20 AM
 
I didn't ask you, I asked Sparky.  If I sound harsh here, it is because we have experienced quite a long time of pretty decent harmony here, following the rules that Sparky told ME to follow.  You have ideas that have merit.  I do not contradict that.  I enjoy a good debate with you and Doug or anyone else, but I am remembering what I was told when this all started.  If we open the door in one are, how do we bolt it in another area?
Respectfully,
Bill Little <><

Well then I respectfully ask you if you didnt want the input o fthe masses why you chose to ask Sparky about pulling a trhead or topic in an open forum?

Why do you keep commenting to me and Brad.  We arent taking this poilitical at any point.  I might attend the nats this year and this is helpful. 

Others have made plenty of little side comments all throghout this thread and the one on number of judges.  Take it up with them.  Leave me out.

I fully understand this is Sparky's site and what he wants on it is his business and he fully has that right.  Remove it or whatever but I made no political posts in this thread and no one else did that I can see.  Just a few comments here and there.  This is the stunt community you know.  Comments are a plenty.  Not always welcomed but they are plenty.
Title: Re: Seeding
Post by: Bradley Walker on November 28, 2007, 11:38:23 AM
Hey, its Sparky's board... obviously, he can do whatever he wants.

It's just my 2 cents.
Title: Re: Seeding
Post by: Bradley Walker on November 28, 2007, 11:41:22 AM
Why do you keep commenting to me and Brad.  We arent taking this poilitical at any point.  I might attend the nats this year and this is helpful. 

Word.

I am thinking of attending the Nats also, *BECAUSE* Paul is changing the format...
Title: Re: Seeding
Post by: Bill Little on November 28, 2007, 11:46:45 AM
Quote
Why do you keep commenting to me and Brad.  We arent taking this poilitical at any point.  I might attend the nats this year and this is helpful.

" keep commenting to me and Brad"  ??

Simply because at the time I went to reply, you and Brad had commented to what I said, that's all.

You are completely right.  It has not really become "political" but that doesn't mean it won't. 

I Put it in the open so that everyone COULD see it.  We did not start this foreum/board to eve discuss these types of topics.   That is fact, so I stated something to remind is a somewhat subtle way.  You and Brad commented, that's all, so I called your name.  I have no quarrel with either of you , as you know.  So, let's leave it that way!  Things have a way of breeding drama for some reason.  Rules are rules.  We can abide by them, or change them, either way suits me!
Title: Re: Seeding
Post by: frank williams on November 28, 2007, 12:14:17 PM
Paul

If we seed ….  then there should be a seeding procedure.

I guess sometimes that I have the feeling that, maybe in an effort to “balance” the circles, seeding takes place to a much lower level than I think it should be, and is done in a somewhat subjective manner.  Suppose it comes down to …. a concionous decision being made on whether to put me and Howard on the same circle …. “Well lets put Frank over on this circle and Howard on that one …. Now lets move Chris over here and John here….. now they all look about even”.  … Wrong.  I’d like to see the procedure.  Do it by the numbers.  Those not seeded would be assigned to the four circles by random draw.  It shouldn’t take more than 30 minutes to accomplish and can be done in the open.  Not only can be accomplished in the open but should be accomplished in the open.

3 pts for being in the top five last year
2pts for being in the top five two years ago.
1 pt for being in the top five three years ago.
3pts for being on current US WC team.
2pts for being on last US WC team .
everyone else 0 pts

Archie Bellcrank 22 pts
Bernie Bellcrank 21 pts
Carl Bellcrank 21 pts
David Bellcrank 19 pts
Earle Bellcrank 18 pts ……..
..
..


Cricle 1 Archie
Circle 2 Carl    ------ tied pts random draw between Bernie and Carl for circle
Circle 3 Bernie
Circle 4 David

Circle 1 Earle
Circle2 ….

This actually might turn out just like what would be accomplished without a procedure … but done by a written procedure it will leave no one with the chance for any complaint.

Title: Re: Seeding
Post by: Bradley Walker on November 28, 2007, 12:41:57 PM
What Frank said... ;)

That Frank Williams is smaaaaaaaaaaart.
Title: Re: Seeding
Post by: Doug Moon on November 28, 2007, 12:43:39 PM
Frank,

It could be even easier.  You could look at last years nats placings 1-20.  

Move down the list splitting out the top 12 from the year prior to each of the four circles in use. 1 to 1, 2 to 2, 3 to 3, 4 to 4, 5 to 1, 6 to 2, 7 to 3, 8 to 4, and so on.  If someone is not present then you move the next person down show is present.  Until you have seeded up the top 12 across four circles.  Then the rest fall where they may.  

That would be pretty fair I think.  Just my 2 cents.

Title: Re: Seeding
Post by: Bradley Walker on November 28, 2007, 12:55:06 PM
Doug and I worked this out once as he described.

I think Frank's point about *whatever* you do to seed, do it in the open.
Title: Re: Seeding
Post by: RC Storick on November 28, 2007, 03:04:30 PM
As I see it ,as long as this thread takes no undertones of mallice it can go on. The Nat's is STUNT!
Title: Re: Seeding
Post by: Dick Fowler on November 29, 2007, 06:07:42 AM
I'm not sure how well a volunteer judging group would do at something as important as the NATS. The rules are numerous and even seasoned judges have a different interpretation of some of the same rules and which rule should apply to a specific situation. I think a good indication of what I'm getting to is the wide ranging opinions of appropriate decisions in some of the judging threads on SSW. If people who fly and are familiar with the pattern can't agree, then people with no experience are definitely going to have a problem. If I were NATS caliber, I wouldn't want some rookie deciding my scores... I don't think they have the background to do a good job. JMHO
Title: Re: Seeding
Post by: Ted Fancher on November 29, 2007, 11:38:56 AM
  I THOUGHT his name was Dennis???  I am the only Denny posting on this forum that I know of MR  Trostle.  Therefore it is natural that I thought  that I might be blamed for some statements in your petty little thread.   
tripple Sheesh to you too.

  Marvin DENNY

Marvin,

I beg you to lighten up on this.  Please go back and check Denny(is) Adamisin's posts on this thread and you'll note that at the bottom of each is his "signature" comment or whatever it's called.  Specifically, it says "aka Denny".  I've known him for decades, always called him Denny and always will because he's just a kid ... well, almost.

I call my friends by either their first name or, if I use their last name, preceded by a title, i.e. Mr., Dr., Captain, etc.  It would never occur to me to address you as "Denny" because I consider you a friend and wouldn't denigrate that relationship with a demeaning form of address.

Ted
Title: Re: Seeding
Post by: Paul Walker on November 29, 2007, 12:53:16 PM

Certainly a good start and we agree (obviously) about the flyer seeding in the qualifying rounds.

I would, however, encourage you to reconsider the random selection of judges for later rounds for the following reason.  Can you imagine the hue and cry from predictable vocal sources if one of your volunteer judges (oh, just for an example, Ted Fancher) is randomly assigned to judge the Top Five and/or Walker Flyoff?  Especially if the competitors include names like David Fitzgerald, Brett Buck, Phil Granderson or, in the Walker Flyoff, Paul Ferrell?

What the agitators fail to recognize is that avoiding such assignments to the greatest degree possible is one of the primary functions of the judge seeding process at the Nats. Although I've no problem with flying before a randomly selected group of judges, I would, frankly, prefer "NOT" to be put in such a position as a judge.  The reasons are too obvious to waste time on.

 I would be perfectly happy to accept an assignment to judge other than the top twenty or finals day if doing so would make your life easier.

Ted 


I have spent a little time thinking about this, in between everything else, and would pose the following option:  PRIOR to the draw for the judges, I would ask if there is any judge who would like to opt OUT of judging the remaining flights, for what ever reason, including having helped that flier work on his pattern in the last 2 months. I believe this would solve your dilemma.

However, flying in front of a judge that is "familiar" to you has been and probably always will continue to be the case. Our event is so small, and this will continue to happen. Having said that, I have no problem with you judging them as I fully believe you will be totally objective with them. I have been there as well. I judged at the 2001 Team Trials and judged Howard, who I fly with regularly.  No one even batted an eye that I am aware of. Yes, the small difference is that the few individuals you are talking about are sure contenders, and Howard was a dark horse at that time (although he showed something more at the last TT's).  I feel that you being there is no problem if it is the pure luck of the draw. The difference is that you would not be accused of being placed there on purpose to better someone's placing.

I think I will give any judge the option of bowing out before the draw if they feel uncomfortable.

Paul Walker
Title: Re: Seeding
Post by: Howard Rush on November 29, 2007, 12:56:49 PM
I would guess that a significant fraction of the population of Muncie is in jail.  A couple dozen trusties would probably be happy to spend a few days in the summer sunshine for just a few ounces of weed.  
Title: Re: Seeding
Post by: Howard Rush on November 29, 2007, 01:04:44 PM
I judged at the 2001 Team Trials and judged Howard, who I fly with regularly.  No one even batted an eye that I am aware of. Yes, the small difference is that the few individuals you are talking about are sure contenders, and Howard was a dark horse at that time (although he showed something more at the last TT's).  I feel that you being there is no problem if it is the pure luck of the draw. The difference is that you would not be accused of being placed there on purpose to better someone's placing.

I think I will give any judge the option of bowing out before the draw if they feel uncomfortable.

Paul Walker

I might note that Mr. Walker did sorta quarantine me before the 2001 TT.  We flew together, but he refrained from giving me flying or trimming advice for a month or two before the event.
Title: Re: Seeding
Post by: Paul Smith on November 30, 2007, 01:18:04 PM
I would guess that a significant fraction of the population of Muncie is in jail.  A couple dozen trusties would probably be happy to spend a few days in the summer sunshine for just a few ounces of weed.  

Howard, if you would dig a little deeper you would see that most of the people doing hard time in Muncie prisons are former sports officials who manipulated high school  basketball, fiddled with ice dancing team trials, and came up with the wrong winners at AMA events.
Title: Re: Seeding
Post by: Dennis Adamisin on November 30, 2007, 02:21:47 PM
Paul & Howard:
Actually some of the inmates choose prison when they heard that their alternative "community service" would be to judge at NATs events!  HB~>  VD~  LL~  LL~  LL~
Title: Re: Seeding
Post by: Dennis Adamisin on November 30, 2007, 06:18:15 PM

...check Denny(is) Adamisin's posts on this thread and you'll note that at the bottom of each is his "signature" comment or whatever it's called.  Specifically, it says "aka Denny".  I've known him for decades, always called him Denny and always will because he's just a kid ... well, almost.

Ted

GOD BLESS YA TED!  BW@

Where else but amongst my birth family and my CLPA family can a 53yo white haired arthritic prone man be called "kid" - NO WONDER I'm hangin with the hanger boyz!  y1

BTW I changed my signature and added the aka AS A RESULT OF THIS THREAD!  :-[

Maybe you guys have heard this story about the old baseball manager Sparky Anderson.  Like any old manager he had his share of "conversations" with umpires.  There was one Ump who insisted on addressing him by his real name of George. Why - "Because I refuse to call a 60 yo man SPARKY!"   LL~  LL~  LL~
Title: Re: Seeding
Post by: Ted Fancher on November 30, 2007, 06:56:39 PM
GOD BLESS YA TED!  BW@

snip

BTW I changed my signature and added the aka AS A RESULT OF THIS THREAD!  :-[

snip

Ooops.

In that case I owe Marvin an apology. 

I just went back and reviewed your posts after seeing Marvin's and saw the "aka" at the end of each.

At any rate, Merry Christmas to both Denny and Marvin.

Ted
Title: Re: Seeding
Post by: john e. holliday on November 30, 2007, 07:08:25 PM
Hey Dennis are we really getting that old?  DOC Holliday
Title: Re: Seeding
Post by: Dave Adamisin on November 30, 2007, 07:46:47 PM
Hi guys. Be nice to my bro Dennis: aka Denny today as he had surgery today. And by the way I had a nice conversation about Foxes (go figure) the other day with the real Denny: aka Marvin. Be nice to him too.
Title: Re: Seeding
Post by: Dennis Adamisin on November 30, 2007, 07:58:18 PM
Ooops.

In that case I owe Marvin an apology. 

I just went back and reviewed your posts after seeing Marvin's and saw the "aka" at the end of each.

At any rate, Merry Christmas to both Denny and Marvin.

Ted

Ted:
I noticed that too.  Apparently when you change your signature here it changes it in ALL the posts you ever made - even the old ones.  It also changes it in the messages stored in the my Inbox & Outbox.  That's a long way of saying you were miss-led into thinking my signature was always that way.  n~

As for Marv, I wrote and told him I was going to call him Mr DENNIS!  H^^  LL~  n1 (but its still cooler to call him BigIron)


Doc
We might be growing old (beats the alternative), but I REFUSE to grow up.  Heck Ted still calls me a KID!  #^
Title: Re: Seeding
Post by: Rudy Taube on November 30, 2007, 11:23:59 PM
Hi Dennis,

Thanks for the great LOL.

There may be a little "sad but true" in that one!  n~
 
Paul & Howard:
Actually some of the inmates choose prison when they heard that their alternative "community service" would be to judge at NATs events!  HB~>  VD~  LL~  LL~  LL~

Title: Re: Seeding
Post by: Bill Little on December 01, 2007, 09:28:41 AM
Quote
Denny Adamisin:
"Doc
We might be growing old (beats the alternative), but I REFUSE to grow up.  Heck Ted still calls me a KID! #^ "

Heck Dennis, compared to Ted, you ARE still a kid! LL~ LL~ LL~ LL~
Title: Re: Seeding
Post by: Paul Smith on December 01, 2007, 10:15:03 AM
Paul & Howard:
Actually some of the inmates choose prison when they heard that their alternative "community service" would be to judge at NATs events!  HB~>  VD~  LL~  LL~  LL~

Gopod point, Dennis.  Maybe those who want to judge need to be investigated in depth.

But seriously, stunt is a rich event to be in a position of needing 8-to-12 good judges and actually be able to discuss selecting them, rather than just finding them.   In all the other events at the Nats, you can walk in, sit down and start scoring without giving your name.

Title: Re: Seeding
Post by: Ted Fancher on December 01, 2007, 11:08:33 AM
"You're kidding, right?"

No.  Calculate for me the probability of that skew happening.  Then calculate the effect on the 20-21 divide.  This can be calculated.  The crude analysis above convinced me that, for that case, the effect of seeding is negligible.  You won't convince me otherwise by arm waving. 


Oh, Howard,

I don't wave my arms, I just respond to multiple decades of exposure to the Nats CLPA process.  If you don't believe there have been years when there were complaints about tough and easy circles, I can't convince you by either saying so or waving my arms.  I think the whole reason for seeding has to do with "appearances" as stressed by Mr. "X" in an earlier post. There will always be a group for whom any work done by those foolish enough to work so they can play will be construed as being done for ulterior purposes. (For instance, the only defensible reason for dropping the high and low scores is the same dang thing, to assuage the appearance of the potential for impropriety, notwithstanding the fact that doing so might very well throw out "better" judges than it leaves in). 

It goes on and on.  I think a large part of my current lack of enthusiasm for the event is the result of whiners who are concerned about the "appearance of impropriety".  I've seen too many people work way too hard and get trashed for that work.  The vast majority of that hard work they do (beyond just the grunt work of standing in the weather all day or pounding out thousands of score sheets) is the result of nothing more than trying to eliminate the perceptions of impropriety.  Clearly, a task for the Gods and not mere mortals as repeated threads such as this demonstrate.

Believe me, the administrators did not undertake the "selection" of flyers and/or judges because they had run short of "stuff" to fill their leisurely days at the Nats/Team Trials.  Again, what most of the conspiracy theorists fail to recognize is that the individuals willing to actually work their butts off to put these events on have no underlying motive to make their jobs even harder and more susceptible to criticism (no matter what decisions they make).  You need no better example than Paul's decision to eliminate judge selection in response to the complaints of a small but vocal minority of individuals who don't even compete in the Nats on a regular basis.  Hopefully, prior to making that decision he discussed the pros and cons with previous administrators who, for some reason, found the selection process to be of value.

The fact is that there have been years when competitors "perceived" that their ability to qualify for the next level of competition was compromised by  "uneven" balance in the circles (often combined with the assumption that judges purposely score certain individuals higher or lower based on who they are  rather than how they fly).  I didn't make this up.  The reality is that the only reason the seeding process exists (which is a lot of extra work for those that work) is because of such complaints.  Of course, no good deed goes unpunished thus, here we are again.

Your "mathematical model" may be 100 percent correct about the probability of a dozen "name" flyers ending up on one of four circles.  The reality is that it "could" happen and if steps aren't taken to avoid that possibility (however mathmatically remote), there will be more complaints -- they'll just be different. Shoot, there already have been on numerous occasions.  Trust me on this one.  (And remember, it doesn't take a dozen "stars" per circle, only five -- or four if you end up no 6)

If, as you declared to have proven, the differences in outcome are minor, with or without seeding, and, whereas, the lack of procedures to avoid the unlikely probability of unbalance historically result in accusations of dishonesty and/or unfairness, why in the world would you not prefer that the process be conducted so as to eliminate to the greatest degree possible the rancor associated with the accusations of unfairness?

Tell you what, I'll stop "waving my arms" if you'll climb down from your Ivory Number Tower into the real world of stunt.  The only thing remotely "number related" in stunt is the scores.  Everything else about it is subjective (including the source of those numbers) ... and by virtue of being so, driven by emotion rather than objectivity.

The thankless task of our administrators has more to do with mollifying emotions than with complex, but ultimately superfluous, ivory tower numbers.

Ted

Edited for grammar and to remove some unwarranted acrimony.
Title: Re: Seeding
Post by: Dennis Adamisin on December 01, 2007, 02:03:24 PM
Thinking this thing through in a historical sense, CLPA at the NATs has evolved to where it has today NOT to become better at choosing the Champion, but rather to make the competition more accessible to more people - These are an abridged history of things I have witnessed, and in a couple cases help cause.  I have likely flossed a few details but the big picture is pretty accurate:

Used to be you showed up at the NATs, put in for a flight then flew, repeated it a second time and took your best score.  Problem is NO ONE wanted to fly first, so nothing happened in the first round until maybe 10am and the second round would cut off around 5PM sometimes with people still waiting to fly.

Logical course was to draw for flight order.  So now everyone got their flights but some "...got the short end of a long ping-pong ball"  And truth is in a long days flying it WAS hard for judges to stay focused too.  Also, your whole years preparation and trip across country came down to 2 flights (or less) in one day.

Next logical progression was to expand to a qualifications/finals format, then a qualifications/semifinals/finals format to give more people a chance at more flights and to NOT toast our now volunteer judges under the sun all day.  In fact it rarely went as late as 2pm.  But some folks noticed "Gee, if I'd only been on that OTHER circle I could made the Semi's  and then...?  Event administrators took on the task to SEED the qualifying circles.  Proponents cited that it made the semifinals (top 20) more accessible to the most people.  Critics claim it causes bias.  Statiticians claim neither exists - probably all three POV's have merit. 

Meanwhile, at local meets, the idea of Skill groups began to take root, and seems to have really taken hold.  Fliers can get their feet wet, then a toe hold as they advance through the ranks to the level their ability and commitment can take them.  The new attitude is "Gee, I'd like to go to the NATs but I don't want to be the champ, I just want to fly."  So now officially and unofficially we got skill classes and OTS & Classic and... etc  all flying at the NATs.

What is significant about many of these stages is that  it has made it easier for the non-champs to participate.  Attendance and interest in the NATs proves that is a GOOD thing.  Now more than ever the NATs is THE CONVENTION for CLPA.  However maybe ENOUGH has been done for the whole, almost NONE of them have been done to improve the selection of the Champion.

Like all PAMPA Presidents before and after me, I believe in promoting the general well-being of CLPA at all levels.  History shows that has been done about as completely as possible - participantion is strong and we are now drawing more model businesses interest in what we do and they are responding with new product.

Paul W has published "THE PLAN" & I think it can select the correct NATs champion - I have no quarrels with it.

Howard R says his  mathematical model "proves" that seeding doesn't matter - history shows that is incorrect based one what HAS HAPPENED in the past; STATISICALLY that lesson may or may not be re-learned in the future.  It is ironic that BOTH the proponents & opponents of seeding can claim "fairness"  to a degree both are correct.  So the question of seeding really boils down to how much do we care about 6th -xx place versus the perception of bias?

Personally I believe the "cream rises to the top" and that seeding has not determined the Champion, probably not the top 3 or 5 either.  Those who claim seeding causes bias would probably disagree.  However, to claim seeding causes bias is to put an asterisk next to every champion's name since... 1974(?).

Title: Re: Seeding
Post by: Howard Rush on December 01, 2007, 11:38:49 PM
I dug up the old Monte Carlo thing I did.  I found a mistake, reran it, and confirmed that seeding makes an insignificant difference in whether the 20th-best guy makes the top 20.  Mind you, this is for the method used in the 2003 Nats and maybe other recent Nats, where contestants were split into two groups.  The top 10 from each of these groups comprised the top 20.  I described the method of my program awhile back.  I ran it with and without the top eight seeded.  The top eight always qualify.  The outcome for the 20th- and 21st-best guys was as follows for 100,000 Nats:

The 20th-best guy made the top 20 56,241 times with no seeding.
The 20th-best guy made the top 20 55,549 times with seeding.
The 21st-best guy made the top 20 44,016 times with no seeding.
The 21st-best guy made the top 20 41,113 times with seeding.

Mean placing in his circle for the 20th-best guy is 10.25 with no seeding, 10.32 with seeding.
Mean placing in his circle for the 21st-best guy is 10.74 with no seeding, 10.80 with seeding.

Standard deviation in placing in his circle for the 20th-best guy is 1.586 places with no seeding, 1.35 with seeding.
Standard deviation in placing in his circle for the 21st-best guy is 1.585 places with no seeding, 1.38 with seeding.

I had to look up the definition of standard deviation.  The above may actually be a normalized standard deviation.  I divided the number I got from the definition by 100,000.

I then changed the program to consider the 2008 Nats method of dividing flyers into four groups, which may be akin to the situation Dennis (53?!) mentioned.  In this scheme, the top five from each circle advance to the finals.  I ran it with and without seeding the top 12 flyers.  Again I looked at the fate of the 20th- and 21st-best flyers:

In 50,000 Nats:

The 20th-best guy made the top 20 26,768 times with no seeding.
The 20th-best guy made the top 20 25,034 times with seeding.
The 21st-best guy made the top 20 23,109 times with no seeding.
The 21st-best guy made the top 20 20,396 times with seeding.

Mean placing in his circle for the 20th-best guy is 5.39 with no seeding, 5.55 with seeding.
Mean placing in his circle for the 21st-best guy is 5.62 with no seeding, 5.78 with seeding.

Standard deviation in placing in his circle for the 20th-best guy is 1.338 places with no seeding, .966 with seeding.
Standard deviation in placing in his circle for the 21st-best guy is 1.333 places with no seeding, 1.003 with seeding.

I'm sorry I don't know how to include these data as tables.  I'll try to attach the file and maybe the histograms.  I'll post this on SSW, too. 

Seeding doesn't do what I thought it would: namely, to affect the placing of the marginal guys.  The four-group scenario is kinda interesting.  Sigma for the 20th-best guy goes down with seeding, but seeding decreases his probability of making the finals.  Looking at the histogram, I figure that this is because without seeding, sometimes the 20th-best guy can place as high as first (just .1% of the time) in his circle, but with seeding, he's limited to only fourth or fifth.  I'm not sure who seeding helps or hurts, let alone whether it helps or hinders fairness. 

Keith Trostle pointed out a different reason for seeding, which may suffice to justify it 


Title: Re: Seeding
Post by: Bradley Walker on December 02, 2007, 06:19:39 AM
It goes on and on.  I think a large part of my current lack of enthusiasm for the event is the result of whiners who are concerned about the "appearance of impropriety".  I've seen too many people work way too hard and get trashed for that work. 

I don't see anyone here whining about what Paul is doing.

It is hard to argue with a man's methods in the case of the "appearance of impropriety" if it done out in the open, in front of God and everybody...
Title: Re: Seeding
Post by: peabody on December 02, 2007, 06:34:13 AM
I think that it still leaves too much to those doing the seeding, especially if they chose to seed beyond the first 5-10....

It might be considered an "honorarium" if those finishing in the top 5 in Open were seeded......but there can be no scientific or even justifiable reason for seeding throughout the field.

I find it laughable, too, that many who are justifying seeding denied that it was taking place at all just a few years ago.
Title: Re: Seeding
Post by: Dick Fowler on December 02, 2007, 06:41:56 AM
From a non-participant's point of view, I believe seeding is appropriate and necessary.

From a "level playing field" perspective, it seems to me that seeding does exactly that... for probably the first twenty best fliers (depending on the number of circles). For Joe average it doesn't affect them much either way and they wouldn't benefit from seeding... as Howard has shown. However there is an exception. The whining would come from both groups if the luck of the draw put much more than five top fliers in any group. Top guy number six is mad and all of the lower ranked guys are mad because they feel that they were effectively blocked from moving on... nobody is happy!

Howard, I wonder how much the outcome of your program is skewed by one of your assumptions. "I did a simulation. I assumed 40 contestants. They were assigned at random to four circles in the 2003 Nats Wednesday-Thursday format. I ranked the contestants and assumed that a higher-ranked contestant would always place higher than a lower-ranked contestant. I did 10,000 runs with and without seeding the top eight to see how many times the 20th-ranked contestant would make the top 20 and how many times the 21st-ranked contestant would make the top 20. Here is the upchuck:"

 As we all know, every now and then somebody pulls one out of their posterior and screws up the whole analysis deal! But even under that scenario, the only guys really affected by a stellar performance from a average guy would be the fifth and lower ranked fliers in that circle if seeded. So in my mind, seeding can help but never hurts... so why not seed?

The real analysis in my mind should be the probability of a circle having more top fliers than the number that will move on, and the bigger question is what method is used to determine the ranking for purposes of seeding.
Title: Re: Seeding
Post by: john e. holliday on December 02, 2007, 07:56:34 AM
I wasn't going to jump on this as I do not fly in the top echelon of NATS stunt competition.  I fly the event as I like the people and it gives me a contest to go to locally.  But, it seems the ones that are doing the complaining about how the NATS  is run have not been to a NATS lately.  I don't hear the top people complaining about how the NATS is run.  Seems the top 5 fly and do the best they can with the conditions of the time.  But, it would be great if one year we had a NATS with the 5 mph breeze blowing away from the sun and a set of judges that could care less about who is on the handle or what the airplane looks like.  Have fun,  DOC Holliday
Title: Re: Seeding
Post by: Alan Hahn on December 02, 2007, 10:00:11 AM
Oh I thought I'd just add to the noise level here! ;D
Title: Re: Seeding
Post by: Bill Little on December 02, 2007, 10:11:41 AM
The last time I was able to actually fly in the NATS, seeding was used.  I thought nothing of it.  It's the way it is.

Maybe it's because I was a coach at a pretty high level for 30 plus years and have become accustomed to playing the game as and where we had to play it.  Might be a wet field, might be below freezing, might be a heat index over 100, etc., you just have to prepare the best you can, perform the best you can, and let the chips fall.

When I return to flying at the NATS, I will not let seeding affect my decision to do so at all.  I do not see (nor have I ever seen) a problem with seeding.
Title: Re: Seeding
Post by: Howard Rush on December 02, 2007, 11:39:44 AM
The last time I was able to actually fly in the NATS, seeding was used.  I thought nothing of it.  It's the way it is.

Maybe it's because I was a coach at a pretty high level for 30 plus years and have become accustomed to playing the game as and where we had to play it.  Might be a wet field, might be below freezing, might be a heat index over 100, etc., you just have to prepare the best you can, perform the best you can, and let the chips fall.

When I return to flying at the NATS, I will not let seeding affect my decision to do so at all.  I do not see (nor have I ever seen) a problem with seeding.

I think that approach to competition will affect your placing a lot more than seeding will.
Title: Re: Seeding
Post by: Howard Rush on December 02, 2007, 11:51:31 AM
I think that it still leaves too much to those doing the seeding, especially if they chose to seed beyond the first 5-10....

It might be considered an "honorarium" if those finishing in the top 5 in Open were seeded......but there can be no scientific or even justifiable reason for seeding throughout the field.

I don't know if it can do any harm.  I doubt it.  Maybe seeding could be done voluntarily: if you want to be seeded, you get seeded. 

I ran the program again to see the effect at the break between the last seed and the best nonseed.  There was no jump in fortune between the two. I posted results on SSW.  For the 2008 Nats four-group format with 12 seeds, seeding helps both the (seeded) 12th-best and (unseeded) 13th-best flyers a little, and hurts both the 20th-best and 21st-best flyers a wee bit. 
Title: Re: Seeding
Post by: Ted Fancher on December 02, 2007, 01:13:04 PM
Howard,

I pulled this quote (attributed to you by the author of the post, Dick Fowler) that, as I understand it, describes the underlying assumptions of your seeding analysis.  If Dick's quote is incorrect (or if I misinterpreted its source), forget all that follows.

The quote: "I did a simulation. I assumed 40 contestants. They were assigned at random to four circles in the 2003 Nats Wednesday-Thursday format. I ranked the contestants and assumed that a higher-ranked contestant would always place higher than a lower-ranked contestant. I did 10,000 runs with and without seeding the top eight to see how many times the 20th-ranked contestant would make the top 20 and how many times the 21st-ranked contestant would make the top 20. Here is the upchuck:"

My concerns are based on the sentence that states: "I ranked the contestants and assumed that a higher ranked contestant would always place higher than a lower ranked contestant".

In my opinion, that assumption is so completely foreign to what we are doing as to render the results of the simulation meaningless.  My experience tells me that our normal spread of nats competitors is very competitive within several sub groupings with increasing overlap of the groups as you descend the historical record of achievment. 

To wit: We probably have between two and five or so competitors that will "always" place better than at least 95% of the total group.  (and, just to short circuit Brad, this is an historical assumption, not a prediction) Within that tiny group, any could (and has) win an event in which they are all entered.  Thus, an assumption that David will always beat Paul who will always beat Billy who will always beat Brett is fundamentally flawed.

Just below that historical performance level you've got perhaps eight to a dozen excellent flyers who historically will usually (but not always) get beaten by the first group but who also will, historically, have beaten about 80% of all the other flyers not in that highest group. (Please note that these % are WAGs based solely on my memory of past performances)

Just below that is a large cohort of very good flyers with track records that indicate they will "likely" beat most of the unknown cohort that might show up in a given year.  They might also put in a particularly strong performance one year and beat out one or many in the second group (but very unlikely to do so to the first group)

The final group will be those that are, for the most part, unknown on a comparative historical basis.  Flyers about which little is known.  The might be very, very good; very, very bad or fit neatly into a predictable peak of the Bell Curve.  That this group cannot be easily catalouged as "also rans" you need go no farther than Doug Moon, Rich Oliver and Konstantin Bajaikin (Sp?).  Certainly unseeded prior to the start of their respective Nats, all three of these guys proved themselves competitive with those seeded at the very highest level.  They did so at their very first attempt flying in Open at the Nats (I may be in error with Konstantin re "first Open".  However, the Rich and Doug instances make my point; even this lowest cohort can't be catalogued as "A will always beat B").

My point in all the above is that your initial assumptions simply don't match the reality of the group you're attempting to simulate.  By " ... ranking the contestants ..." and "... assuming a higher ranked contestant would always place higher than a lower ranked contestant ... " you've essentially determined the outcome before the simulation even begins.

The reality is more like I've described above where within narrow and overlapping groups of flyers a wide ranging number of outcomes between any two is quite likely.  More likely as you descend farther down the historically more predictable outcomes.  In other words, the top rankings are very impervious to any effects of seeding or not while the intermediate ranks are very susceptible to variations that might transpire.  This is true simply because it is not a valid assumption that flyer A will always score higher than flyer B.

I believe this is a good description of why it is valid and meaningful to seed (based on historical performance to the greatest degree possible) as many flyers as possible down to the point that the number to be selected based on performance will encompass those seeded to the extent possible.  Doing so will provide the greatest possible "appearance" of balanced talent in the circles assuming that the administrators have roughly the same knowledge of historical performance as do the competitors who will evaluate the seeds.

If this is incorrect Howard, tell me what I'm missing.

Thanks

Ted
Title: Re: Seeding
Post by: phil c on December 02, 2007, 03:14:20 PM
I think Ted has hit the nail on the head on this one.  Not seeding, and going ahead with the contest when 5-10 of the historically very best flyers all land in the same qualifying circle, is exceeding unfair to them.  Seeding a dozen or so of the historically best flyers, especially using some sort of points system based on recent history suggested elsewhere, is a much better proposition.  The seeding may switch the finish order of a few flyers around the 20th spot for qualifying, but that at best is unfair to one flyer, not 4 or 5, or even possibly more.
Title: Re: Seeding
Post by: Howard Rush on December 02, 2007, 04:51:55 PM
I think Ted has hit the nail on the head on this one.  Not seeding, and going ahead with the contest when 5-10 of the historically very best flyers all land in the same qualifying circle, is exceeding unfair to them.  Seeding a dozen or so of the historically best flyers, especially using some sort of points system based on recent history suggested elsewhere, is a much better proposition.  The seeding may switch the finish order of a few flyers around the 20th spot for qualifying, but that at best is unfair to one flyer, not 4 or 5, or even possibly more.

Balderdash.  Ted is saying, using red letters as the hammer with which to hit the nail, that my analysis is wrong because, although the contestant assignment to groups is random, I assumed deterministic contestant placing.  I take it that he thinks that randomizing the placing sequence enhances the effect of seeding.

There is the case of good flyers piling up on one circle.  Intuitively, you'd think that the effect would be greatest to the 20th-best flyer.  Seeding actually hurts his chances, although very slightly.  Seeding does have an effect a little farther up the ladder.  It helps ensure that a flyer at my level (10th to 13th) makes the top 20, but, curiously, it doesn't seem to matter whether or not such a flyer is among the seeds.

So let's see some math.  I know you got it in you.
Title: Re: Seeding
Post by: Dick Fowler on December 03, 2007, 05:50:12 AM
Hi Howard.  After thinking about your analysis, I think your assumption would be valid for your method. I considered that the ranking would have to be based on some sort of statistical analysis. This ranking then servers to  "predict"  the relative placement of the contestant at the end of the contest. So if a contestant is ranked in the 50th percentile then statistically (assuming a large enough number of data points were used) it would be expected that within some reasonable range, that guy would land on his midpoint most of the time with a bellcurve distribution around his respective finishing position. So if all participants have sufficient data points (contest finishes, NATS placement,competition against the field, etc.) then it should be reasonable to statistically predict the final postion of each contestant... they finish where they are ranked with a normal distribution about their own "ranked" position. I think your analysis  shows this... I don't  know if it really speaks to seeding though.

So in my mind, this analysis works better if all the contestants have  flown against each other and the more times they fly against the field the better the "ranking becomes". Ranking is based on the guys skill level against the field. If he hasn't flown against this group then the errors start to creep in.

Using random circle selection with good ranking would have the 20th guy finishing on average, 20th most of the time, but... statistically he will also finish 1st and also last sometimes with these finishes following the usual bell curve. As I see it, seeding causes a narrowing the curve and increasing the height of this curve. So  with seeding, the number 20 guy will now finish 20th a a much greater frequency than with random selection. Every split of the field increases the chances of random selection stacking some given circle to the point that the flyers ranked lower than the number chosen to advance (seems like five has been the magic number) will never finish according to their ranking.  I think all that happens is that the make up of a seeded circle more reasonably reflects the normal skill levels of the entire field even though it has been split. One circle is the ideal situation and the addition of more circles starts to introduce this error that random selection "can" create. I haven't modeled this... and even if I figure it out, I'm not really driven to do it! JMHO.
Title: Re: Seeding
Post by: SteveMoon on December 03, 2007, 07:54:39 AM
I think Paul's ideas for a mild revamping of the Nats are great. Of course,
it is entirely possible, and most likely probable that some tweaking will
be needed over the next couple of years, but I think this is a great
start. Attempting to eliminate even the *possible appearance* of *any
impropriety* is a positive step. I think having the contestants and judges
draw for their circle assignments and flying order each day is a great idea.

I also agree with Frank Williams' ideas for seeding the first round. It should be
a simple, mathematical, historical system that is easy to understand. It should
not be based on personal, subjective decisions.

Overall, I am excited about the changes to the Nats, and am really looking
forward to another pleasant summer week in Muncie, IN. Texas Roadhouse,
here we come!!

Later, Steve
Title: Re: Seeding
Post by: Ted Fancher on December 03, 2007, 10:35:00 AM
Balderdash.  Ted is saying, using red letters as the hammer with which to hit the nail, that my analysis is wrong because, although the contestant assignment to groups is random, I assumed deterministic contestant placing.  I take it that he thinks that randomizing the placing sequence enhances the effect of seeding.

There is the case of good flyers piling up on one circle.  Intuitively, you'd think that the effect would be greatest to the 20th-best flyer.  Seeding actually hurts his chances, although very slightly.  Seeding does have an effect a little farther up the ladder.  It helps ensure that a flyer at my level (10th to 13th) makes the top 20, but, curiously, it doesn't seem to matter whether or not such a flyer is among the seeds.

So let's see some math.  I know you got it in you.

No need, Howard.

You've stated (highlighted in my post for ease of sourcing) that one of the principles of your analysis is that you know going in who will beat who: A beats B who Beats C all the way through to poor Mr Z. 

I don't think you need much math to determine that A will always place in front of everybody and Z in front of nobody when you've stated as much entering the exercise. 

In the real world A doesn't always beat B and M might beat N as many times as N beats M and even Z will rarely but occasionally beat Y.

Why doesn't that reality have an impact (I hate to keep using the word) on the the validity of your results?

Just asking.  I don't know squat about the math involved, only the apparent disconnect in logic of the assumptions you've taken into the task.

Ted
Title: Re: Seeding
Post by: Howard Rush on December 03, 2007, 12:09:49 PM
I don't know if it has an effect.  I suspect that a statistical whiz could show that randomness in performance and seeding effects are independent. With the Monte Carlo brute-force method, I think randomizing performance would fuzz up the outcome, making the variances larger, and obscuring the seeding effect more.  One could do the experiment by first randomly distributing guys among the four groups as I did previously, then assigning a probability distribution to each guy's placing, narrow at the top where Brett is and wide in the middle where I am, as you suggest, then resorting them for each trial. That's a bunch of work, though.   

One might do the same sort of exercise to see if the benefit in weather and judging uniformity brought by the four-group format warrants the increased tendency for good fliers to get bumped in tough circles.  We are assuming that seeding fixes the tough-circle problem, but maybe it doesn't.  That exercise would be a heck of a lot of work.
Title: Re: Seeding
Post by: Howard Rush on December 03, 2007, 12:15:39 PM
Hi Howard.  After thinking about your analysis, I think your assumption would be valid for your method. I considered that the ranking would have to be based on some sort of statistical analysis. This ranking then servers to  "predict"  the relative placement of the contestant at the end of the contest. So if a contestant is ranked in the 50th percentile then statistically (assuming a large enough number of data points were used) it would be expected that within some reasonable range, that guy would land on his midpoint most of the time with a bellcurve distribution around his respective finishing position. So if all participants have sufficient data points (contest finishes, NATS placement,competition against the field, etc.) then it should be reasonable to statistically predict the final postion of each contestant... they finish where they are ranked with a normal distribution about their own "ranked" position. I think your analysis  shows this... I don't  know if it really speaks to seeding though.

So in my mind, this analysis works better if all the contestants have  flown against each other and the more times they fly against the field the better the "ranking becomes". Ranking is based on the guys skill level against the field. If he hasn't flown against this group then the errors start to creep in.

Using random circle selection with good ranking would have the 20th guy finishing on average, 20th most of the time, but... statistically he will also finish 1st and also last sometimes with these finishes following the usual bell curve. As I see it, seeding causes a narrowing the curve and increasing the height of this curve. So  with seeding, the number 20 guy will now finish 20th a a much greater frequency than with random selection. Every split of the field increases the chances of random selection stacking some given circle to the point that the flyers ranked lower than the number chosen to advance (seems like five has been the magic number) will never finish according to their ranking.  I think all that happens is that the make up of a seeded circle more reasonably reflects the normal skill levels of the entire field even though it has been split. One circle is the ideal situation and the addition of more circles starts to introduce this error that random selection "can" create. I haven't modeled this... and even if I figure it out, I'm not really driven to do it! JMHO.

The analysis I did may be a lot cruder than you give me credit for.  I'll send the program to anybody who's interested.  It's in Excel, with one macro tacked on. 
Title: Re: Seeding
Post by: Paul Walker on December 03, 2007, 01:12:01 PM
Ted said:

  "You need no better example than Paul's decision to eliminate judge selection in response to the complaints of a small but vocal minority of individuals who don't even compete in the Nats on a regular basis. "

This appears to be the crux of the problem here. I didn't consider this because of the group you described, but for the judges. I have been working to get enough judges for the next NATs, and one of the common threads is that if I have NO chance to judge the finals, then I won't come. That led me to what criteria would be applied to selecting judges. This has been in my head for a number of years.  That being trying to define a criteria that I could plug into a computer so that it reviews the data, and then selects the best judges. I can't come up with that, and so far, no one else has either. The criteria I have heard are subjective.

Suppose that a certain judge (judge A) always judges flier A 3% higher on average than all the others.  Should that judge be eliminated if flier A makes the finals?  That's the difference between a 566 and a 550. Is this because the judge is biased? Or, is the judge seeing something in that fliers pattern that fits their perception of what a pattern should look like.  Part of the reason to have multiple judges is to get their different perspectives.  I'm not sure that 3% difference is enough to eliminate that judge. However, if that judge scores the flier 10% higher, 605 vs. 550, then that might be bias. Exactly where do you draw the line. In this case, there is a formula that could be derived and programmen into the computer that could point this out.

How about the judge that scores higher on some fliers and lower on others. These judges have been eliminated as well. The same criteria that I described above applies here as well. I suggest that the amount that difference is significant. Now, suppose that we up front tell the judges the mathematical  formulas in the computer that will "screen" them from the finals. Don't you think they would minimize their scoring ranges to keep from being eliminated. Is this a benefit to the fliers?  Many already complain of the narrow scoring range in the semi and final rounds now.

There are other reasons judges have not been allowed to judge the finals. Proximity to the flier is a frequent one. I know for a fact that there is a certain very qualified judge who was not allowed to judge the finals because this judge lived close to one of the contestants. This judge didn't fly with the finalist, this judge didn't even watch him fly much, other than judge him at local contests, yet this judge was not allowed to judge. Further, he was told that he wasn't judging because he didn't score close enough to the average norm. This wasn't the real reason. I know that Ted has brought this issue forward. Again, I think that if Ted (or anyone else) who is judging feels uncomfortable with this, they can opt out. I personally think that Ted would do his absolute best to be as objective as possible, and I certainly wouldn't question it even if I was flying against Brett or David while he was judging.  I fully believe that the judges are trying to do their absolute best. Period.

One factor that DOES affect this objectivity, is the heat and getting tired. I plan on instituting more breaks for the judges and for longer times as well. I want them as fresh as possible to allow them to do their best job.

The bottom line here is that I set about this path for the judges benefit. My plan will bring back existing qualified judges. It has already! Continuing on the path we have been on will lead us to not having any qualified judges left in the not to distant future.

Paul Walker
Title: Re: Seeding
Post by: Paul Walker on December 03, 2007, 01:15:51 PM
Ted said:
"Why doesn't that reality have an impact (I hate to keep using the word) "

That's OK, it's a good word!
Title: Re: Seeding
Post by: frank carlisle on December 03, 2007, 05:00:21 PM
I've just finished reading this entire thread, which is no small feat. I haven't anything to add to the conversation. I am however very impressed with the depth and width of your discussion.
It's a good thing that there are people that care enough and are involved enough to go to such great lengths to work out the details in judging and seeding. Keep it up.
Title: Re: Seeding
Post by: Trostle on December 03, 2007, 08:44:28 PM

(clip)

This has been in my head for a number of years.  That being trying to define a criteria that I could plug into a computer so that it reviews the data, and then selects the best judges. I can't come up with that, and so far, no one else has either.

(clip)

Paul Walker

Paul,

There really is a quite sophisticated statistical process that has been developed to do the very thing I think you are looking for.  I have described this before on several previous threads.  Charles Buffalano, a longtime PAMPA member, developed a normalization process for our CL Aerobatic contests that accomplishes a number of things.  It is based on the same system that the full scale aerobatic competitions use, at least at the international and I believe our National level competition.  Not only does it use normalized scores to determine the ranking of individual competitors, but it can also be used to measure judges' performance in terms of accuracy and bias.  Buffalano has also analyzed performance of individual judges for the FAI CIAM organizers after a number of World CL Championships.  Bob Baron championed this process while he was still active and was campaigning hard for change.  Charles wrote a fairly elaborate explanation of the process in the Mar/Apr 96 issue of Stunt News, pages 81-82.  Again, that was during the time that Bob Baron was really stirring the pot for change.  I wrote an introduction to that article.  As I recall, there was absolutely no, like zero, nothing, that appeared in response to that article.  In the several times I have brought this up in these forums, there has been absolutely no response to using any kind of a sophisticated normalization process to determine scores or judge performance.   In my opinion, and based on the zero response of that article and my explanation of it several times since, our CLPA event has not been ready for such a process to be used.

Buffalano's tool could be used in real time at the Nats to measure judge performance.  Regardless of what is said on these forums, some judges do not perform very well.  There have been extreme examples of individual judges ballooning in the later rounds or late in the day of those long days in the sun.  There are numerous examples of judges who do not even come close to placing the fliers they see in the same order as the average scores that we accept as the official placings - either after various rounds of qualifications, or semi finals or the finals.  With your experiment to randomly select judges for the various segments of competition, you are inviting a random selection of various judging  capabilities, and quite possibly end up with  a finals that are satisfying ONLY to the 1 or 2 or 3 fliers on top that the entire field recognize as undeserving.

However, you have, by that time, at least bowed to the vocal malcontents who subscribe to various conspiracies of how inappropriate the Nats are run and claims of improprieties.  According to these people, and now your acceptance of adopting some random process for judge selection, a shadow is officially cast on every Nats and Team Trials for the past 30 plus years.  Do we now need to put an asterisk behind the name of every National Stunt Champion because of some yet to be defined inappropriate operations or out-and-out improprieties occurred?  What makes this so frustrating is that many of those vocal malcontents and most who subscribe to the myths and conspiracy theories do not even bother to compete at the Nats or Team Trials level.= and choose to absolutely not accept that the Nats are run so that each individual has an equal opportunity to demonstrate his skills and to be as fairly judged as possible.

You might want to contact Buffalano to develop his process for use at the Nats to grade judges and to even consider employing a normalized process to determine the placings of each individual competitor, unless you are completely locked in to your random judge selection process.

One thing about the normalized determinination of placings, it requires a computer program, input of scores from each judge of each maneuver, and is not run until all of the scores are in to determine final placings.  In other words, there is no indication of how the competition is going until each segment of the competition (qualifications, semifinals and finals) is completed.  However, the program could be used to analyze individual judge performance at the end of each day or round of competition.

Keith Trostle
Title: Re: Seeding
Post by: Doug Moon on December 03, 2007, 10:59:45 PM
Keith,

Can you explain something in your post please. 

It is obvious you read PW's post.  Due to the fact you gave some very good information about a normalization process that has been discussed in the past.

Then you went on to totally discount PW's actions for running the nats the way he has decided to do so by saying he is just bowing to the wishes of some malcontents who dont even go to the nats.  You didnt even give PW's reason for running the nats this way any merit what so ever for why he might be doing so.  Even after he fully explained the one running theme he has received from many of the judges he has talked with whle putting together the next judging corps.   

Why did you dismiss that reason and then tell us a different reason PW is running the nats this way?  Did he tell you he wanted to quiet down some internet fodder from some people who dont even compete?

I dont really see the point in trying to do that.  Do you?

I for one surely dont see a shadow or a cloud cast over the past 30 years of nats.  Making a change to a future event has nothing to do with the past events.  I have flown in the nats 8 times, 7 of them in row.  I surely felt I got a fair shake at the ones I attended.  To say PW is placing a cloud or shadow over those events is not fair to him or his efforts to run the nats.  He is volunteering his time and vacation to do such a task as are many others.  I for one appreciate the work they do even when I dont go to the nats I still appreciate because it keeps the event alive.

Title: Re: Seeding
Post by: Trostle on December 04, 2007, 01:31:22 AM
Keith,

Can you explain something in your post please. 

(clip)

I for one surely dont see a shadow or a cloud cast over the past 30 years of nats.  Making a change to a future event has nothing to do with the past events.  I have flown in the nats 8 times, 7 of them in row.  I surely felt I got a fair shake at the ones I attended.  To say PW is placing a cloud or shadow over those events is not fair to him or his efforts to run the nats.  He is volunteering his time and vacation to do such a task as are many others.  I for one appreciate the work they do even when I dont go to the nats I still appreciate because it keeps the event alive.


Doug,

I do not doubt for a minute that you do not "see a shadow or a cloud cast over" how the Nats have been run for a long period of time.  However, there are the myth makers and the conspiracy seekers that have been carrying their perceived misconceptions and they have been carping on these for so long, they and many who do not bother to even try to learn the facts nor have ever participated in our major events start to believe that something is wrong with the way the Nats are run.  For example, in post #106 above, the writer stated "Attempting to eliminate even the possible appearance of any impropriety is a positive step."  To me, the writer is suggesting that there have been improprieties in the way our Nats are run.  If the writer did not say that, then he used a poor choice of words.  If that writer did not directly so state, the innuendo is there and others have more directly implied so.  To me, to adopt a totally random selection of judges for the various sessions at our Nats is courting disaster simply because there needs to be a system to minimize skewed regionalism represented among the combinations of judges as well as minimizing the known close affiliations between individual judges and fliers plus the simple fact that by the time the semi-finals and finals are reached, there can be and has been some comparatively poor performance demonstrated by individual judges (as in ballooning and definite bias or even less than competent scoring).  The poorer performing judges should not be judging the final rounds in either the Open or Advanced categories.  To randomly assign judges even prior to the competition can, and I admit, not necessarily result in a combination of judges that may not be able to accurately assess the flying abilities of our top 20 pilots and particularly our to 5 pilots as well as a group of judges whose performance during the Nats is reviewed prior to the semi-finals and finals.  Again, I can foresee a situation developing where only the top 2 or 3 placed individuals at the Nats would be satisfied by the results when the rest of the people there (including competitors and observers) will believe that they have witnessed a travesty. 

How can a random assignment of judges minimize regionalism which is one of the problems seen by the myth makers?  Those myth makers are always anxious to point out the West Coast pattern that they claim is emphasized at the Nats when in reality, the Rulebook Pattern is what is emphasized at the Nats.  Those same myth makers try to rationalize that judges favor certain fliers to ensure certain fliers win and more specifically, to ensure that certain fliers or even a certain flier does not win.  Those same myth makers/conspiracy seekers have stated that it is statistically impossible for the National Stunt Champion title (the Walker Cup) to have been won by only an handfull of individuals over the past 15 or so years.  They are implying that somehow the system is rigged and for some reason, they cannot accept that demonstrated skill/ability has something to do with selecting the National Stunt Champion, not some statistical process.  And now, there has been a clamor to support this "random assignment of judges" as if this somehow will resolve the conspiracies and improprieties that have evidently been prevalent over all of these years.  So Paul comes along and in a sweeping change is going to adopt a system that is being hailed as a brilliant ray of sunshine that eliminates these conspiracies and improprieties.  He is giving credence to the myth makers and the conspiracy seekers and has essentially acknowledged what the myth makers hve been saying about how our Nats have been run for the past 30 plus years.

Yes, Paul is to be commended that he has volunteered his time and energy to run the Nats.  I will even applaud that he is willing to venture into different ways of running the Nats.    Unfortunately, I do not doubt that there will be complaints on the results of his efforts, no matter what the results are or what he does or how well he pulls it off.  Experience only shows that there will be some undertone about the quality of the judges, or something was done by or through the ED that some malcontent feels was somehow not to his understanding or comprehension and resulted in somebody not winning that should have.

Paul is not casting a shadow over the way previous Nats have been run.  That shadow has been cast for many years by a handful of people who generate myths and subscribe to conspiracies.  To adopt a random judge selection process, to me, gives credence to those myth makers and conspiracy seekers and can possibly result in less than optimal assignment of judges which will be grounds for more open complaints about how the Nats are run.

I will agree that in the past, some judges are not satisfied by the way they are assigned to the different circles throughout the Nats event.  Not everybody is going to get to judge the Open Finals, though I think most individuals realize that when they volunteer for the Nats.  The reality of not judging the finals is just not pleasant.  However, I wonder if any random selection of judges for the different venues at this Nats will be any more satisfying to each individual judge in whatever group of judges that do volunteer this year.  In short, a random assignment of judges will likely prove to be no panacea to the judges or to those who study the results of the competition.

Keith Trostle 
Title: Re: Seeding
Post by: SteveMoon on December 04, 2007, 06:36:46 AM
Keith: As usual, I manage to find your latest response rather offensive. I did not say that there have been any improprieties in past Nats. I was saying that there may have been an appearance that an impropriety could occur. Of course, you have to read more into this than is there. I firmly believe that having the contestants and judges assigned to their circles by random drawing and in public is the better way to do this. You can continue to tell me I'm wrong for the rest of your life, and it will not change what I believe in this matter.

I am also wondering why you keep refering to those of us who have wanted to see a change in the way the Nats are conducted as "vocal malcontents"? And why we are assumed to be the minority? If somebody doesn't agree with you, does that automatically make them a malcontent or minority? I tend to doubt that.

As far as being a "malcontent" I, like Doug, have been generally satisfied with my results at the Nats. I have competed in the last 12 Nats and feel that for the most part I have been judged quite fairly and finished about where I expected to. I thought that I flew the best I have ever flown at a Nats this past year, and I had my highest finish ever. I am quite satisfied with that. I also hope to improve on that this year. If I was such a "malcontent" would I spend a boat load of money to go to Muncie flippin' IN every summer to compete in the Nats?

Now to the question of being the "vocal minority". I don't really know the feelings within your circle of friends concerning this subject. So, I will not make a comment about that. But, within my circle of friends (this includes some Nats contestants and judges) we are applauding heartily Paul's ideas for changes to the Nats. Are there more of us that are happy about this, than not? I don't know. Do you? Personally, I feel that the "vocal minority" has been miscast, and may actually be the majority. I have no concrete proof of that, It's just what I think may be true.

As I've said before I applaud Paul's efforts and am really looking forward to this years Nats.

Steve Moon
Title: Re: Seeding
Post by: Dick Fowler on December 04, 2007, 09:00:40 AM
Keith, could you explain the process used in previous years to select judges for the finals. Was it free from personal bias by those who did the selection?

I would think random selection has significant merit and would preclude any sort of "conspiracy nonsense". To those who express discontent you simply say..."Hey, that's the luck of the draw!" A rather simple yet elegant solution.
Title: Re: Seeding
Post by: Trostle on December 04, 2007, 11:04:23 AM
Keith, could you explain the process used in previous years to select judges for the finals. Was it free from personal bias by those who did the selection?

I would think random selection has significant merit and would preclude any sort of "conspiracy nonsense". To those who express discontent you simply say..."Hey, that's the luck of the draw!" A rather simple yet elegant solution.

Dick,

There is an Event Director's Guide that has been developed by the Event Directors over the years.  There are basically three topics that are used to assign judges.  One of those is to remove the appearance of any particular region being overly represented by any set of judges assigned to a specific circle.   In other words, having a preponderance of judges from any particular region in a group is to be avoided.  This is not always easy to do because sometimes a significant portion of the judges come from the midwest (or more specifically, they come from an area "convenient to the Nats site") though this is not always the case.  At some Nats, there have been a fair number of judges from the West Coast.  As far as I know, for the Nats that I judged for Warren Tiahrt, there has never been more than one West Coast judge used in the finals or the semifinals rounds of the Nats.  In spite of the extraordinary effort by Warren to eliminate even the appearance of regionalism in the judge assignments, there have been accusations, always unwarranted, to the contrary.  Is the elimination of regionalism a good thing?  Maybe not, because when this procedure is followed, sometimes the most competent group of judges is not selected.  (From my own personal experience, I have not been selected to judge the Open semifinals and finals because there was another West Coast judge for each semifinals circle as well as for the finals circle.  I did not mind judging the Advanced Finals as that was part of what was volunteered for.

Another factor considered in judge assignments is to reduce/eliminate the assignment of any judge that has a standing relationship with one or several of the fliers in those semifinals and finals rounds.  This is not to say that the judges cannot know any of the individual competitors.  It would be impossible to assign any group of judges at the Nats where no judge knew any of those being judged.  However, where known personal relationships, where individual judges often fly with the competitors or even have coached any of the competitors, these combinations are reduced to some practical limit.  Here again, Warren has gone to great lengths to minimize the appearance of allowing personal friends/coaches of individual competitors to be assigned to those groups of judges who will be judging the semifinals and finals.

Even during the qualification rounds, efforts are made to make sure that each judge grouping is not overly represented by any single region and to separate those judges who are know to have particular affiliations with any individual or group of competitors.

The third area is (horrors) the rating of individual judge performance as the Nationals proceeds from day to day.  Ballooning has been a problem and probably always will be.  Some judges appear to be more prone to this.  Ballooning is fairly easy to detect when all of the scores are compiled at the end of the round.  Those judges who show ballooning tendencies are simply not assigned to judge during the semifinals and finals rounds.  There is another process that has been used over the years to evaluate judge performance.  It is probably not a statically sound as what a normalization process would provide.  The process is based on the assumption that the average score for each competitor for each of his flights determined by the combination of each judge evaluating that flight is a valid indication of how well that individual flew.  This assumption essentially needs to be made and apperas to be valid as it is those average scores that eventually determine the placing of each individual after the completion of the Nats, the placings are recorded and most seem content to accept those placings.  Now, after a round is completed,  the individual competitors are ranked from the best average score to the lowest average score.  Compared to that "waterfall" chart that shows the highest to lowest average scores, individual judges scores are also plotted.  There is normally some variance in the ranking of individual competitors by each judge.  But using the assumption that the average score is a fair assessment of the individual's flight, there will be judges who are way off of the scale of even being close to assessing individual flight scores and/or individual placings.  Ballooning will show it ugly head here also because it can be shown what time of day (or portion of the round) higher scores that appear out of sync start to appear.  There have been cases where there have been 5 or more judges on a circle.  Most judges will generally have a similar order of how the competitors are placed.  Yet, in one situation, one judge had the order completely reversed.  Now, do you want to have already randomly assigned that judge with that kind of performance to judge the finals?  Or to have already assigned a judge who is prone to ballooning?  I think not.

This evaluation process is probably against any pure statistician's thinking.  (Obviously, I am not a statistician.)  Generally, by the time the semifinals and finals are reached, several of the corps of judge volunteers are not needed and some will not be assigned to the finals round.  The process used over the years is a tool, in my opinion as accurate as the scoring system we use to determine placings of competitors, to help assign a group of judges that will yield the best possible assessment of each individual competitor.

I do not think selection of our National Stunt Champion and the placing at our Nats should be written off as simply the luck of the draw.  Yes, weather can play a part and that can become a significant fact involving the luck of the draw.  However, the weather is something the ED cannot control.  The ED can control various aspects of the competition that does minimize to some extent how the luck of the draw might impact the final results.  Of those things the ED can control is the seeding process and how the judges are assigned to various circles.  Warren Tiahart and those preceding him should all be applauded for trying to make the competition field as level as possible while minimizing the luck of the draw to some acceptable level.  (Let the naysayers howl, but they are the ones who chose either not to understand or are incapable of understanding the efforts taken over long periods of time to make the Nats as fair as possible for every individual competitor.)

I have judged at 11 Nats and Team Trials.  Based on my knowledge of our stunt event and experience with being the ED at previous Nats and Team Trials as well as being a judge at 3 World Championships, I have had the privilege to be asked by Warren to be involved with sorting through the judge assignment process.  Admittedly, it is not a perfect process, but I can say that there is no doubt in my mind that the process used to select and assign the judges thoughout each entire Nats and Team Trials program is absolutely free from any personal bias that could in any way be a negative impact on the placing of any individual competitor.  Contrary to statements from the myth makers, the people involved with these processes are well above the actions of those myth makers.  Many understand that the myth makers have no credibility.  Unfortunately, there those who do not have the experience or knowledge to doubt the subterfuge that is generated by the continued harangue that comes from that small but destructive group.

This has all been explained before.  But, thanks for asking.

Keith Trostle
Title: Re: Seeding
Post by: Dick Fowler on December 04, 2007, 11:28:10 AM
Keith, thanks for the explaination. I appreciate you taking the time to cover the subject again.

Title: Re: Seeding
Post by: Paul Walker on December 04, 2007, 01:05:18 PM
Keith said:  "However, you have, by that time, at least bowed to the vocal malcontents who subscribe to various conspiracies of how inappropriate the Nats are run and claims of improprieties.  According to these people, and now your acceptance of adopting some random process for judge selection, a shadow is officially cast on every Nats and Team Trials for the past 30 plus years.  Do we now need to put an asterisk behind the name of every National Stunt Champion because of some yet to be defined inappropriate operations or out-and-out improprieties occurred?  What makes this so frustrating is that many of those vocal malcontents and most who subscribe to the myths and conspiracy theories do not even bother to compete at the Nats or Team Trials level.= and choose to absolutely not accept that the Nats are run so that each individual has an equal opportunity to demonstrate his skills and to be as fairly judged as possible."

Maybe in your eyes I have bowed. That is not the reason for what I have done. In trying to get judges to volunteer for this NATs, I found that "some" were not interested because they have not been allowed to judge in the finals. I have reviewed the provided information to them, and in one case it was for what I considered a subjective assessment, that I didn't see the same way. The other case was really due to proximity, and that person was told their score didn't measure up to the rest.  Another wanted a guarantee that they would judge in the finals (which I couldn't provide either way) due to the cost in getting there.

I will be working with Shareen to develop a plan for quantifying the process to be put on a computer. The judges will know up front what the criteria is that potentially weeds them out. From the ones that are left, there will be a draw.  Yes, this is slightly different than what I had described earlier. There will be some consideration of the issues you described, but the filter will be a bit coarse to start off with. Maybe we learn that the criteria used is too coarse, and if so, it will be modified accordingly for the next year. As I mentioned earlier, there will be more breaks for the judges, and longer oner as well, to do what we can to avoid the burnout and then ballooning that comes from that. The worse day for this is going to be the semifinals day. More breaks should help this. The finals will only be 15 flights, with long breaks between rounds. Then different judges will be used for the Walker Cup. No burnout as a result of this!

The one issue that does concern me is the proximity issue. Guys, it's hard to get judges to do this for a week. You are going to get "groups" of regions represented in the judging as a result.  However, I don't see this as any different than the World Champs. If we were to carry the same criteria that is being used at our NAT's, the US would never see a US judge at the World Champs. In fact, who WOULD judge there. Your criteria would consider any judge from country A "too close" to any flier from country A.  In fact, Gary McClellan judged the last WC's and he saw me fly in Seattle only a month prior to being in Spain.  Maybe I should surrender my medal as a result!!! We have gone past this issue at the WC's. Why does it have to create so many problems here in the US? As I discussed in a previous post, I intend to filter out an obvious bias in a judge. If some are concerned that someone from a nearby region will judge their nearby flier higher, this bias filter will find that obvious bias, before the finals. I know that TED is concerned about judging Brett, and that is for appearances only, however, I believe Ted will do a fine job no matter. As I also said before, I would give that judge a chance to opt out if they felt uncomfortable doing that. Again, I know there is a perception issue, but if they are randomly drawn, I don't see the issue (as they were not placed there on purpose).


You might want to contact Buffalano to develop his process for use at the Nats to grade judges and to even consider employing a normalized process to determine the placings of each individual competitor, unless you are completely locked in to your random judge selection process.

I will contact Charles. 

Paul W

Title: Re: Seeding
Post by: john e. holliday on December 05, 2007, 10:12:40 AM
This is getting to be ridiculous,  I have been fortunate enough to go compete in some meets outside of my area.  It seems the judges were more leniant than the local judges I have flown in front of.  If someone can prove that our NATS judges aren't doing the best job possible then that judge should be pointed out.  I myself have not judged that many contests because of the people I know.  But, when I did judge I could care less about who was in the circle except to see I had the right score sheet.  The pilot and plane are supposed to be doing the pattern the way I think the book says it should be done.  As stated some judges are afraid to give low scores for a bad maneuver and sometimes when an exceptional maneuver is done does not get the proper score.  Anyway this coming year is Pauls show lets give him some support.  DOC Holliday
Title: Re: Seeding
Post by: Bill Little on December 05, 2007, 10:19:44 AM
Thanks, Doc.  A point most should remember is that CLPA is not a mandatory activity!  No one is forced to ever compete, much less attend a NATS as a competitor!   The process is spelled out, and it is the same for all.  If an individual does not agree with the process, then they can opt to exercise their personal right of not participating!  There will, in all probability, still be a NATS where CLPA is flown and a NATS Champ is crowned.  Harsh?  not in any way.  A true competitor accepts the rules/procedures in place at the time and participates to the best of their abilities.  Or they choose another endeavor to participate in!
Title: Re: Seeding
Post by: Shultzie on December 05, 2007, 11:45:38 AM
A point most should remember is that CLPA is not a mandatory activity!  No one is forced to ever compete, much less attend a NATS as a competitor!   The process is spelled out, and it is the same for all.  If an individual does not agree with the process, then they can opt to exercise their personal right of not participating!

HOOOOOOOOOOOOOOLLLLLLLLLLLD ON A MINUTE!!!

I am usually wrong...but somewhere in my demented memory banks...when one PLUNKS DOWN PRETTY USELESS $$$ FOR THEIR LITTLE AMA CARD...I thought that came with VOTING RIGHTS WITH THE ABILITY TO IMPROVE AND CHANGE THINGS.
OK! EZ....I know pretty much how much our insurance program covers...and yes, a pretty good magazine comes with it...and most importantly....allows the worthy, to compete in toy airplane competitons.

Still I am under the impression that the membership that is saddled with all this AMA extra $$$ spent making a Castle in the sky to themselves...IT SHOULD RIGHTFULLY BE THE MEMBERSHIPS DUTY TO USE THAT SAME VOTING RIGHT...IN ORDER TO IMPROVE ITSELF for the masses of folks who just wanna fly toy airplanes and have a great time doin it?
After reading my crap....I would like to make an apolo.....for those who do their HECKITY DARNEST OF TIRELESS GIVING OF THEIR TIME, TALENT, EFFORT AND YES....DOWN RIGHT LOVE OF THE "SPORT!"
I have NO RIGHT TO EVEN RESPOND TO THIS THREAD...since I am NOT AN AMA MEMBER so deservingly so...
I SHOULD AND HAVE NO VOICE!  n1
Title: Re: Seeding
Post by: Bill Little on December 05, 2007, 12:06:11 PM
Brother Shultzie,

I am not in any way saying that the process of voting should not be used.  BUT, we are talking about a NATS event that is handed over to the SIG (in this case PAMPA) for administration at the NATS.  PAMPA has basically worked out the procedures for administration of the CLPA event at the NATS. So, I am not talking about changing things now.  That (AMA Rules Change) would take a two year process and the cycle is over for now.  I am not even sure that the AMA sets the procedure, but instead leaves that up to the ED/CD in charge with their  (AMA) approval.

So just HOW would someone go about changing this procedure?? AND, can they even change it?  ???
Title: Re: Seeding
Post by: Shultzie on December 05, 2007, 01:41:14 PM
My apologies to you...Bill for "runnin' off at the lip again!
Dumb-n' dumber don (no wonder I am exiled here in Gig Haaaba?) ~> VD~
Title: Seeding Frenzy
Post by: Paul Smith on December 05, 2007, 01:50:16 PM
This is quite the HOT topic, radioactive to say the least !!!

167 comments and 2071 viewing on two forums. If all these people flew, they'd need eight circles and 24 judges.

It seems like Mr. Walker's seeding plan has been accepted without much argument and the discussion has morphed into some questions about the selection and performance of the judges......
Title: Re: Seeding
Post by: Dick Fowler on December 05, 2007, 02:35:29 PM
The selection of judges and circle appointments was also a part of the initial post.
Title: Re: Seeding Frenzy
Post by: Marvin Denny on December 05, 2007, 05:34:25 PM


167 comments and 2071 viewing on two forums. If all these people flew, they'd need eight circles and 24 judges.

    HMMM  How so ?  Out of viewings and postings, I fine only 40  individuals  posting, and of that number some are not even AMA members and others do not attend the nationals I would say four judges and two circles could handle everything and still have time for several breaks.  The viewers is even more skewed as most viewers have viewed MANY MANY times, ans most of those are the posters themselves.

  Bigiron
Title: Re: Seeding
Post by: Dennis Adamisin on December 05, 2007, 07:59:34 PM
Ya know, while I've been away, I have watched my kids particiapate in Dance competitons, Cheerleading, Poms, Marching Band, Jazz band, & Concert band competitions - all with PROFESSIONAL judges.  My observation is that by comparison our volunteer judges do a pretty good job!  Doesn't mean it could not be better, but I wonder if we are using all the tools in the arsenal to train and evaluate our judges?  To wit:

I am sure we have all experienced the frustration of a judge(s) that seem to fall in love with a number and write it down for every manuver.  Looking at the scoresheets you cannot see any differential between something you did well and something you did poorly.  Yet we seem to want judges that "see the same thing".  I think by seeking and "rewarding" that behavior we run the risk of handcuffing judges - after all those who got too high or too low will either have their scores thrown out or they won't "make the cut" at the next round's judge selections.  By the time we get to the finals we got judges "trained" not to stray...

Put a different way; I have always learned more from judges that use as much of the point spread as they (and my flying errors) can stand: If I made four mistakes they'll catch them all with a big fat downgrade, if I did four things right they'll reward that too!  Unfortunately these are the judges who will tend to sway from the mean, and who might be weeded out for doing so.  Instead, somehow I wish there was a way to instead weed out the judges who are stuck on a number and not comfortable using a broader range of scores.

I find that I really care less about whether the judges are only 10 points apart instead of 70 points apart.  What I REALLY hope is that when I compare their scores on a manuver to manuver basis, it looks like they are mapping the same flight!

At the sailplane NATs we "normalized" scores for each flight group.  This overcame the "luck of the draw" with respect to groups flying in high lift versus high sink conditions - Its tough to fly a 12 minute task in 5 minute air!  Of course the TD sailplane task is dirt simple: timed duration and spot landing.  Still, it begs the question of HOW to use some kind of normalization scheme in something as intricate as CLPA judging.

Keith mentioned Charles Buffalano's system: is this that silver bullet we're looking for?
* Could it find a judge who was sitting on a score?
* If it can document when a judge is ballooning, is their a method to adjust that out of the scores?
* What else can it do?

Title: Re: Seeding
Post by: Bill Little on December 05, 2007, 09:46:41 PM
My apologies to you...Bill for "runnin' off at the lip again!
Dumb-n' dumber don (no wonder I am exiled here in Gig Haaaba?) ~> VD~

Brother Donaldo,

No apologies here, my friend.  I didn't really mean what it sounded like (I think!).  What I was saying is, how DO we go about changing any of this?  it appears to be up to the ED/CD and committee to decide how things go.  I am not sure myself.  Just observing what has been written!  Paul has pointed out how HE is going to do it. (not that I have the slightest problem with that!)  Just that I don't know that it is done any other way.  It seems that the majority of the decisions are left up to the ED, with guidance from the Guidelines in place.

Title: Re: Seeding
Post by: Bradley Walker on December 06, 2007, 06:40:56 AM

Keith mentioned Charles Buffalano's system: is this that silver bullet we're looking for?
* Could it find a judge who was sitting on a score?
* If it can document when a judge is ballooning, is their a method to adjust that out of the scores?
* What else can it do?


Some years ago, when I presented my ideas for using Minitab (statistical analysis software) to study judge's scores in a Nats type environment, Brett Buck said on SSW that *no* statistical method could be used to pinpoint bias in judging.  Brett is pretty smart, so I figured that was the end of it. 

If statistical methods (no matter how elementary) cannot be used to find *bad* judges, it cannot be used to find *good* judges.

From what I have seen, the methods used in the past to find "good" judges has never stood up to direct "academic" scrutiny.  The method used by Gary McClellan that he presented at a DMAA club meeting in 2002 was was a simple line graph in Excel.  It was certainly not complex.  It was also completely not conclusive in any way, and was completely open to personal interpretation (which is where the problem lies---a "good" judge to you may not be a "good" judge to me depending on what fliers' scores we are looking at on this little line graph).  All this being said it was being promoted  it as a method to "graduate" judges at the Nats.

I would rather see a coin toss, at least it is in God's hands.  At least if you had a gripe, you could take it up with God.

If "analytical" methods are to be used, at a minimum, whatever method one uses, a detailed description should be made available to the judges and pilots.  The only problem is that once published the method will be open to criticism and scrutiny.  So, it is better to just stick to the coin toss...
Title: Re: Seeding
Post by: Howard Rush on December 06, 2007, 06:38:23 PM
Please show me your Minitab idea.  Some of us were discussing judge-picking programs earlier this year, but it was close to the Nats, and nobody had time to work on a program.

Title: Re: Seeding
Post by: phil c on December 07, 2007, 12:57:05 PM
here is  a web site that is set up to do the basic ANOVA needed to assess a batch of stunt scores.
http://www.physics.csbsju.edu/stats/anova.html
Title: Re: Seeding
Post by: Bradley Walker on December 07, 2007, 02:20:49 PM
here is  a web site that is set up to do the basic ANOVA needed to assess a batch of stunt scores.
http://www.physics.csbsju.edu/stats/anova.html

Yah, Phil and I have talked a lot about statistical methods to study scores both online and on the phone. 

Phil knows more about statistic study of variation and bias than anyone I have ever spoken too.  Though there are some pretty sharp Master Black Belts in Six Sigma here at Textron.

I do not get the double standard.  One guy's statistical analysis is bull and another guy's is unquestionable.  Phil was there for the online lynching on SSW, when I suggested using Minitab to study scores.  Now we can't live without analysis?  What gives?


Title: Re: Seeding
Post by: Howard Rush on December 07, 2007, 07:27:55 PM
I looked at the Web site.  Can you elaborate on how you would apply this to stunt?