Nats Stunt Seeding

[Howard Rush]

Paul Walker just sent me the official seeding spreadsheet for stunt at the 2009 Nats. The seeding formula is based on Nats placing for the last 10 years. US team members are included, ranked the same as Nats winners. I'll put the seeding into the Nats stunt tabulation program. I'll forward copies of the seeding spreadsheet to anybody who wants it. Likewise, I'll send the tabulation program to anybody who wants it. Here is the seeding:

1 David Fitzgerald
2 Paul Walker
3 Brett Buck
4 Orestes Hernandez
5 Terry Fancher
6 Frank McMillan
7 Richard Oliver
8 Derek Barry
9 Windy Urtnowski
10 Randy Smith
11 Bill Rich
12 Howard Rush
13 Bill Werwage
14 Matt Neumann
15 Dan Banjock
16 Alberto Haber
17 Mike Palko
18 Bob Hunt
19 Josias Delgado
20 Kenny Stevens
21 Kent Tysor
22 Steve Moon
23 Bob Gieseke
24 M Hiki
25 Todd Lee
26 Kaz Minato
27 Allen Goff
28 Bene Rodrigues
29 Gordan Delaney
30 Curt Contrata
31 Steve Millet
32 Bill Suarez
33 Doug Moon
34 Frank Williams
35 Bob McDonald
36 Konstatine Bajaikine
37 Bob Whitely
38 Paulo Gomes
39 Yurii Yatsenko
40 Andrii Yatsenko
41 Alex Schrek
42 Bob Dixon
43 Dale Barry
44 Igor Burger
45 Bill Rutherford
46 Uwe Degner
47 Chris Cox
48 Jim Aron
49 John Sunderland
50 Bruce Perry
51 Henk DeJong
52 Matsuro Yokoyama
53 Dee Rice
54 Steve Starr
55 Don Melanson
56 Phil Granderson
57 A Tozim
58 Jim Lee
59 Dave Hemstrought
60 Bill Byles
61 Keith Trostle
62 Kirk Mullinex
63 Allen Brickhaus
64 John Simpson
65 Gerry Phelps
66 Gene Martine

[peabody]

Interesting that "Mr. Fifth Place" is seeded ninth....one would imagine that he might be seeded higher that fifth, even.

I applaud your work, but believe that seeding is wrong.

Maybe seed through eight, avoid a complete random imbalance...

Random fliers and random judges...

[RC Storick]

Paul Walker just sent me the official seeding spreadsheet for stunt at the 2009 Nats. The seeding formula is based on Nats placing for the last 10 years. US team members are included, ranked the same as Nats winners. I'll put the seeding into the Nats stunt tabulation program. I'll forward copies of the seeding spreadsheet to anybody who wants it. Likewise, I'll send the tabulation program to anybody who wants it. Here is the seeding:

1 David Fitzgerald
2 Paul Walker
3 Brett Buck
4 Orestes Hernandez
5 Terry Fancher
6 Frank McMillan
7 Richard Oliver
8 Derek Barry
9 Windy Urtnowski
10 Randy Smith
11 Bill Rich
12 Howard Rush
13 Bill Werwage
14 Matt Neumann
15 Dan Banjock
16 Alberto Haber
17 Mike Palko
18 Bob Hunt
19 Josias Delgado
20 Kenny Stevens
21 Kent Tysor
22 Steve Moon
23 Bob Gieseke
24 M Hiki
25 Todd Lee
26 Kaz Minato
27 Allen Goff
28 Bene Rodrigues
29 Gordan Delaney
30 Curt Contrata
31 Steve Millet
32 Bill Suarez
33 Doug Moon
34 Frank Williams
35 Bob McDonald
36 Konstatine Bajaikine
37 Bob Whitely
38 Paulo Gomes
39 Yurii Yatsenko
40 Andrii Yatsenko
41 Alex Schrek
42 Bob Dixon
43 Dale Barry
44 Igor Burger
45 Bill Rutherford
46 Uwe Degner
47 Chris Cox
48 Jim Aron
49 John Sunderland
50 Bruce Perry
51 Henk DeJong
52 Matsuro Yokoyama
53 Dee Rice
54 Steve Starr
55 Don Melanson
56 Phil Granderson
57 A Tozim
58 Jim Lee
59 Dave Hemstrought
60 Bill Byles
61 Keith Trostle
62 Kirk Mullinex
63 Allen Brickhaus
64 John Simpson
65 Gerry Phelps
66 Gene Martine

I guess I dont get to fly?

[Paul Smith]

Nice work, Howard.

I notice you rated the JCT pretty low.  Very modest of you.  Do doubt, another analyst would have put you and PTG much higher.

This listing might "flush out" some dormant flyers, in a effort to move up on next year's totem pole.

[Howard Rush]

I didn't do the seeding.  Paul came up with the formula.  I think it's pretty good.  I'll send the spreadsheet to anybody who's interested. It's the official word.  Here is my unofficial understanding of how it works:

For the top 20 in Open for the last ten years, Paul assigns 20 points for first place, 19 for second, and so on.  US Team members who were out of town for the WC get 20 points each.  Scores get multiplied by 10 for 2008, 9 for 2007, and so on.    Advanced scores are then multiplied by .5.  Orestes's score includes both his Open and Advanced placings, for example.  Top score is seeded #1.  Guys who haven't placed in the top 20 in either Advanced or Open are unseeded.  Their assignment to one of the four groups for qualifying rounds is done by random draw. 

Windy didn't get seeded higher because he hasn't placed high in recent years, which are weighted more heavily.  Not making the top 20 in 2007 set him back.

I don't think seeding matters much, either.  Rich, try out that program I sent you.  The way it folds the seeds, includes Advanced, and evens up the circles is pretty cool, even though it was probably a waste of my time.   

[john e. holliday]

Don't give up Sparky,  it'll come around.  Just go blow them away,  flying that is.  I flew several contests in carrier one year and never even got mentioned.  The NATS was one of them flying Sportsman and Nostalgia.  Have fun,  DOC Holliday

PS:I can just imagine the turmoil if they went back to doing it the Navy way.  Draw your card when ready to fly.  At 1700 hours flying stops.  jeh

[Clayton Smith]

How is this seeding used to select the flight circle groups?

[Paul Smith]

Does this mean that 66 people have placed in the top 20 in the last 10 years, or were there more than 66 and you just published the "top 66".

Whether it's 66 or more than 66, this still indicates very good participation.

[Howard Rush]

"How is this seeding used to select the flight circle groups? "

There are four groups: A, B, C, D.  Top Open seed entered goes to A, the next seed to B, then C, D, D, C, B, A, A, B, C...  Then the randomly ranked unseeded Open guys, then the seeded Advanced  guys, then the randomly ranked unseeded Advanced guys.  When everybody is assigned to the four letter groups, the groups are assigned to circles by random draw.  The whole thing happens before your eyes at the pilots' meeting during appearance judging.  The folding was a nuisance for the amateur programmer.

"Does this mean that 66 people have placed in the top 20 in the last 10 years, or were there more than 66 and you just published the "top 66"."

I assume that was a question.  It includes Advanced.  If PW entered the data correctly, 66 people have placed in the top 20 in the last 10 years in either Open or Advanced. 

[peabody]

Howard....I have zero problem with the numbers or rankings, and even think I understand the methodology of assigning the scores.
My gripe is that seeding tends to create a bias by pre-prejudicing the contest.

[Brett Buck]

Howard....I have zero problem with the numbers or rankings, and even think I understand the methodology of assigning the scores.
My gripe is that seeding tends to create a bias by pre-prejudicing the contest.

   So, your theory is that it would be OK as long as the results weren't published?  Sort of like it was a few years ago?

     Brett

[Matt Colan]

Windy didn't get seeded higher because he hasn't placed high in recent years, which are weighted more heavily.  Not making the top 20 in 2007 set him back.

Windy came in 5th in 2006 and 5th last year.  In 2007 he crashed his gorgeous Novanta, but got it repaired as I'm sure many of you know.  That is probably why he isn't seeded higher.
I would say 5th is a pretty high placing.

[Howard Rush]

Coming in fifth in that crowd is quite an achievement.  It's a long way from fifth to sixth, as I learned last year.

[Howard Rush]

"My gripe is that seeding tends to create a bias by pre-prejudicing the contest."

I presume by tipping off the judges as to who did well at previous Nats.  Judges could see the seeding list here, I guess, but they could also be prejudiced by the covers of Stunt News or the press releases of the guys who think they're good.  There won't be evidence of seeding in the actual contest.  The flight order will be randomly drawn after the seeding, so it ain't like the guys appear before the judges in order of reputation.   It would be pretty hard to deduce what the seeding was by looking at the makeup of the four groups.  Remember also that the seeding is just for the qualifying rounds: the shootout is between the fifth- and sixth-best guy on each circle.  Seeding doesn't affect the top guys.  These rounds are a mere formality for the likes of Brett and Windy.  That said, you have a point.  Seeding doesn't do much.  It doesn't do much good, so it doesn't take much harm for it to do more harm than good. 

[Matt Colan]

Seeding doesn't do much.  It doesn't do much good, so it doesn't take much harm for it to do more harm than good. 

I think it does more good than harm.  With seeding, it could avoid one circle being stacked with a bunch of the top flyers.  Everybody is evenly distributed.

[proparc]

I guess I dont get to fly?

Sparky,just head down to the Nats hook up them lines,and make them understand the terrible, terrrible mistake they made. Or, you can just suspend posting priviledges and make them give you a higher seeding-much easier!!!

[RC Storick]

I paid my 90 bucks today. I will just take what I get.

[Howard Rush]

If you have a better idea, let's hear it. 

[RC Storick]

Who me? I dont have a better idea and I like this one.

[Bradley Walker]

If this is some kind of a "US ranking", I do not understand how Brett would be above Orestes.

In the PGA golf rankings, the points for a win has a rate of decay.  In other words recent wins count a great deal more than old wins.  I assume Paul did something like that here.  I just do not see how the guy who won the Nats the last two times (and won the Team Trials and competed in the WC's) is behind Brett who never competed in a WC's and won the Nats some years back.  A win counts drastically higher than a second, third, etc  A guy could have 20 seconds and it would be eclipsed by one win.

Using the rate of decay, even Paul's wins would start to fall off.  A Nats win 10 years ago would not count the same as one last year...  but like I said, it appears Paul did something like this.  All in all, it looks like Paul put a lot of thought and work into the ranking system.

All that being said, I thought Howard proved seeding did not matter with his simulation?

[RC Storick]

It has nothing to do with ranking it has to do with circle placings. If the top 20 guys on this list flew against each other it would look something like this.
1 David Fitzgerald         Circle 1
5 Terry Fancher
9 Windy Urtnowski
17 Mike Palko         
13 Bill Werwage   
 
2 Paul Walker               Circle 2
6 Frank McMillan 
10 Randy Smith 
14 Matt Neumann
18 Bob Hunt 
             
3 Brett Buck                 Circle 3
7 Richard Oliver
11 Bill Rich 
15 Dan Banjock     
19 Josias Delgado
               
4 Orestes Hernandez      Circle 4
8 Derek Barry   
12 Howard Rush     
16 Alberto Haber   
20 Kenny Stevens   

[Bradley Walker]

Is this "seeding" list not a ranking from highest to lowest rank based on points awarded by finishes?

[RC Storick]

NATS Seeding list. Of coarse it would be better for the lower ranking guys if the top 20 seeds flew on 2 circles (hence the masters class)and the rest of the lower ranking guys flew on the other two circles. It would change top 20 results but it would probably not change the outcome of who really is the best flier. But this system is fair.

[phil c]

Fair is fair.  With a complete random draw it would be really nuts if the top ten on that list all got put on one circle!  And if more than one of the judges hasn't heard of most of the guys on this list, just where have they been judging?  Judge quality, lack of bias, ranking, and seeding are not real problems conducting large contests like the NATS.

[Mike Ferguson]

It has nothing to do with ranking it has to do with circle placings. If the top 20 guys on this list flew against each other it would look something like this.
1 David Fitzgerald         Circle 1
5 Terry Fancher
9 Windy Urtnowski
17 Mike Palko         
13 Bill Werwage   
 
2 Paul Walker               Circle 2
6 Frank McMillan 
10 Randy Smith 
14 Matt Neumann
18 Bob Hunt 
             
3 Brett Buck                 Circle 3
7 Richard Oliver
11 Bill Rich 
15 Dan Banjock     
19 Josias Delgado
               
4 Orestes Hernandez      Circle 4
8 Derek Barry   
12 Howard Rush     
16 Alberto Haber   
20 Kenny Stevens   

Really?  I guess I thought it would look like this:

1 David Fitzgerald         Circle 1
8 Derek Barry   
9 Windy Urtnowski
16 Alberto Haber   
17 Mike Palko         
2 Paul Walker               Circle 2
7 Richard Oliver
10 Randy Smith 
15 Dan Banjock     
18 Bob Hunt 
             
3 Brett Buck                 Circle 3
6 Frank McMillan 
11 Bill Rich 
14 Matt Neumann
19 Josias Delgado
           
4 Orestes Hernandez      Circle 4
5 Terry Fancher
12 Howard Rush     
13 Bill Werwage   
20 Kenny Stevens 
 

I could be wrong as well, though.

Howard's program/Excel spreadsheet is really quite good.  I've been tinkering around with putting it in Access, but I haven't had a ton of time to devote to it lately.

I just like the fact that it eliminates any questions about how flyers are seeded.  People may not agree with the seedings, but there's an objective way to reach them.

[RC Storick]

It does not matter how it looks to who and the order. I WAS GIVING A EXAMPLE!

Why do people on these forums try to be so critical on stuff that don't matter?

[Bradley Walker]

Why do people on these forums try to be so critical on stuff that don't matter?

 

[Bradley Walker]

I just like the fact that it eliminates any questions about how flyers are seeded. 

That is very true.  Very, very true.

[Mike Ferguson]

Easy.     I wasn't trying to be critical.

I said I didn't know who was right or not.  I was just curious about how the seedings would actually look.

[RC Storick]

  Judge quality, lack of bias, ranking, and seeding are not real problems conducting large contests like the NATS.

I would argue this statement as I have a few examples.

[Howard Rush]

Mike's right.  If we'd done it the way Robert shows, it would have all the guys in the first group a notch higher seeded than all the guys in the second group. I originally had it assign guys as Robert assumed, but Paul made me do the folding.   

Phil, show us the probability of the top ten or even ten of the top 20 getting on one circle in a random draw.  Let's see some (n-r)!

I think I have sent copies of the program to all the participants in this conversation, and I showed the seeding process above, so there should be no reason for people to make erroneous statements about how either works.  I'll hose Paul's seeding spreadsheet around when I get back from flying. 

[Howard Rush]

"If this is some kind of a "US ranking", I do not understand how Brett would be above Orestes.

In the PGA golf rankings, the points for a win has a rate of decay.  In other words recent wins count a great deal more than old wins.  I assume Paul did something like that here.  I just do not see how the guy who won the Nats the last two times (and won the Team Trials and competed in the WC's) is behind Brett who never competed in a WC's and won the Nats some years back.  A win counts drastically higher than a second, third, etc  A guy could have 20 seconds and it would be eclipsed by one win.

Using the rate of decay, even Paul's wins would start to fall off.  A Nats win 10 years ago would not count the same as one last year...  but like I said, it appears Paul did something like this.  All in all, it looks like Paul put a lot of thought and work into the ranking system."


OK, Brad, I'll reveal the secret formula (my unofficial interpretation of the official spreadsheet):  For the top 20 in Open for the last ten years, Paul assigns 20 points for first place, 19 for second, and so on.  US Team members who were out of town for the WC get 20 points each.  Scores get multiplied by 10 for 2008, 9 for 2007, and so on.    Advanced scores are then multiplied by .5.  Orestes's score includes both his Open and Advanced placings, for example.  Top score is seeded #1.  Guys who haven't placed in the top 20 in either Advanced or Open are unseeded.  Their assignment to one of the four groups for qualifying rounds is done by random draw.

As for the relative placings of Brett and Orestes, you should first check to see if Paul did the calculation as he intended.  The method might be improved, but if I have to do the work, it's good enough as is.  Leonard had an idea for a better method .  I sent him the spreadsheet to work on.  Perhaps you can collaborate.   

[Bradley Walker]

I think as a seeding process goes, this is the best I have ever seen.  I was just making comparisons to golf rankings.  It is just a matter of the weighting differences (which give way more credit for wins), but this looks fine for what it does.

I think the main thing is that the method is published.  Just doing that is a major leap forward, in my opinion.

[Jim Pollock]

I just know that I sure wouldn't want to be in Kenny Stevens shoes on circle 4 with that seeding.

Jim Pollock   

[Brett Buck]

If this is some kind of a "US ranking", I do not understand how Brett would be above Orestes.

  Of course, it isn't a ranking, but the reason would be that I have made the flyoff a lot more times and my record goes back further. The system only gives one point per place, so winning doesn't have a disproportionate effect, he only got one more point than I did for last years win. The 10 years includes some years Orestes wasn't even entered, and a goodly number of Advanced places.

    If you want to have an all time ranking, I suggest using Dick Byron's list and updating it. I think I am nearly unassailable in first place, and I want my 590 from last weekend corrected to 615 to compensate for the elimination of pattern points. David gets a 617 for this weekend, but I am still well ahead. Therefore I conclude I am the greatest flier of all recorded history. The rest of you guys should just quit...

   Of idle curiosity, depending on how one defines a tie for third, but I have finished in every place from 1-7, and in 5 flyoffs I have finished in each of the 5 places.

     Brett

[Chuck Feldman]

Perhaps this is none of my business but it is interesting to read.

I do not like seeding. Seems to me it benefits the established folks and puts the others at longer odds. Of course this is not a race?

I think the way they fly at the VSC eliminates all this. The seeding would not be needed with the VSC format.

[Bradley Walker]

    If you want to have an all time ranking, I suggest using Dick Byron's list and updating it. I think I am nearly unassailable in first place, and I want my 590 from last weekend corrected to 615 to compensate for the elimination of pattern points. David gets a 617 for this weekend, but I am still well ahead. Therefore I conclude I am the greatest flier of all recorded history. The rest of you guys should just quit...

I agree.

[Mark Scarborough]

Chuck,
the ony way I can see that seeding is a negative is for the guy who really isnt a legitimate contender, or has NO history, he may have a better chance if by some stroke all the big guns got loaded onto one or two circles and he was on a circle with nothing but lower scoring pilots. The downside is that then there are some people that are on the stacked circles that legitimatly belong in the fly offs that wont get there

[Howard Rush]

"I do not like seeding. Seems to me it benefits the established folks and puts the others at longer odds. "

How does it go about seeming? Let's see some math.  I did a very crude simulation of it and found that: a) it makes very little difference, b) it doesn't matter for the top guys, who will qualify anyhow, c) it favors skill over luck a tiny bit more for marginal guys, allowing those about 10th-best a little better chance of qualifying at the expense of those about 20th-best.  I haven't seen anybody do any analysis beyond what I did.  By "established folks" I presume you mean guys who have placed high at past Nats.  Does seeding favor them, or does it favor people who fly better at the Nats where seeding is used?  Beats me.  If you can show mathematically that it unfairly benefits "established folks", you might be able to talk PW out of using seeding, but not if it merely improves the chance of a substandard flyer to bump a better one through luck of the draw, as Mark notes.

[RC Storick]

Well if the top 20 guys fly on two circles and us lowly guys fly on the other two it gives us a 50% better chance of make the fly off. However it will not change the outcome.

[peabody]

Howard...
Are seeds going to "move up" if a higher seed doesn't show?

[Howard Rush]

Yes.  Try out the program.  It's so cool.

[Doug Moon]

NATS Seeding list. Of coarse it would be better for the lower ranking guys if the top 20 seeds flew on 2 circles (hence the masters class)and the rest of the lower ranking guys flew on the other two circles. It would change top 20 results but it would probably not change the outcome of who really is the best flier. But this system is fair.

I dont think you can do that.  It would not be a fair way to get to top 5 and then the champ.    If you have a lower class why would they get to compete for the top honors?  If one wants to compete for top honors they have to go through the best to get there.  Entering open then putting in with the top pilots and cutting through to the top.


Seeding is very fair the way Howard has come up with it especially using the folding.  Without the folding you will get some lop sided circles.

[RC Storick]

I hope you noticed I said ,I think this system is fair. I also don't give a hoot how they do it. I just like going and probably have no chance anyway.

Almost made it last year but 1 judge on 1 maneuver kept me out. 40 40 and 33 for inverted level flight. I wonder how perfect it would have to be? I did not know there was degrees of perfect. One more reason for 5 judges and high and low score thrown out

[Paul Smith]

Easy.     I wasn't trying to be critical.


I said I didn't know who was right or not.  I was just curious about how the seedings would actually look.

Mike,

Your seeding was the more correct of the two.

The other version clearly stacked the talent heavy in #1 and progressily lighter toward #4.

[Gene Martine]

Howard:
This doesn't seem to make much sense to me. There are flyers on the list that as far as I can remember haven't flown expert at the NATS. How did Paul arrive at the listing?   
Gene

[Howard Rush]

OK, Gene, I'll reveal the secret formula (my unofficial interpretation of the official spreadsheet):  For the top 20 in Open for the last ten years, Paul assigns 20 points for first place, 19 for second, and so on.  US Team members who were out of town for the WC get 20 points each.  Scores get multiplied by 10 for 2008, 9 for 2007, and so on.    Advanced scores are then multiplied by .5.  Orestes's score includes both his Open and Advanced placings, for example.  Top score is seeded #1.  Guys who haven't placed in the top 20 in either Advanced or Open are unseeded.  Their assignment to one of the four groups for qualifying rounds is done by random draw.

[Howard Rush]

"One more reason for 5 judges and high and low score thrown out"

Only three steps to go: 1) You recruit 12 more judges, 2) you write a mathematical proof that throwing out high and low is better than taking the average, and 3) you edit the VBA macro to do it.

[Brett Buck]

Almost made it last year but 1 judge on 1 maneuver kept me out. 40 40 and 33 for inverted level flight. I wonder how perfect it would have to be? I did not know there was degrees of perfect. One more reason for 5 judges and high and low score thrown out

   How do you come to this conclusion? Based on the fact that it would have benefitted you personally? Why do you think the 40's were right, and the 33 was wrong -  couldn't it have been just the converse?

     Brett

[RC Storick]

Because it was a 39 or 40 and that judge had scored right along with the rest before that easy maneuver. It either is or it ain't. If it was on a square 8 or triangle or one of the harder maneuvers I would have understood. Of coarse all the arguing in the world won't ever change your mind or mine. I know I got the shaft but oh well. I also Have a idea why. On to the next contest.