stunthanger.com
General control line discussion => Open Forum => Topic started by: Howard Rush on April 29, 2009, 09:46:12 PM
-
Paul Walker just sent me the official seeding spreadsheet for stunt at the 2009 Nats. The seeding formula is based on Nats placing for the last 10 years. US team members are included, ranked the same as Nats winners. I'll put the seeding into the Nats stunt tabulation program. I'll forward copies of the seeding spreadsheet to anybody who wants it. Likewise, I'll send the tabulation program to anybody who wants it. Here is the seeding:
1 David Fitzgerald
2 Paul Walker
3 Brett Buck
4 Orestes Hernandez
5 Terry Fancher
6 Frank McMillan
7 Richard Oliver
8 Derek Barry
9 Windy Urtnowski
10 Randy Smith
11 Bill Rich
12 Howard Rush
13 Bill Werwage
14 Matt Neumann
15 Dan Banjock
16 Alberto Haber
17 Mike Palko
18 Bob Hunt
19 Josias Delgado
20 Kenny Stevens
21 Kent Tysor
22 Steve Moon
23 Bob Gieseke
24 M Hiki
25 Todd Lee
26 Kaz Minato
27 Allen Goff
28 Bene Rodrigues
29 Gordan Delaney
30 Curt Contrata
31 Steve Millet
32 Bill Suarez
33 Doug Moon
34 Frank Williams
35 Bob McDonald
36 Konstatine Bajaikine
37 Bob Whitely
38 Paulo Gomes
39 Yurii Yatsenko
40 Andrii Yatsenko
41 Alex Schrek
42 Bob Dixon
43 Dale Barry
44 Igor Burger
45 Bill Rutherford
46 Uwe Degner
47 Chris Cox
48 Jim Aron
49 John Sunderland
50 Bruce Perry
51 Henk DeJong
52 Matsuro Yokoyama
53 Dee Rice
54 Steve Starr
55 Don Melanson
56 Phil Granderson
57 A Tozim
58 Jim Lee
59 Dave Hemstrought
60 Bill Byles
61 Keith Trostle
62 Kirk Mullinex
63 Allen Brickhaus
64 John Simpson
65 Gerry Phelps
66 Gene Martine
-
Interesting that "Mr. Fifth Place" is seeded ninth....one would imagine that he might be seeded higher that fifth, even.
I applaud your work, but believe that seeding is wrong.
Maybe seed through eight, avoid a complete random imbalance...
Random fliers and random judges...
-
Paul Walker just sent me the official seeding spreadsheet for stunt at the 2009 Nats. The seeding formula is based on Nats placing for the last 10 years. US team members are included, ranked the same as Nats winners. I'll put the seeding into the Nats stunt tabulation program. I'll forward copies of the seeding spreadsheet to anybody who wants it. Likewise, I'll send the tabulation program to anybody who wants it. Here is the seeding:
1 David Fitzgerald
2 Paul Walker
3 Brett Buck
4 Orestes Hernandez
5 Terry Fancher
6 Frank McMillan
7 Richard Oliver
8 Derek Barry
9 Windy Urtnowski
10 Randy Smith
11 Bill Rich
12 Howard Rush
13 Bill Werwage
14 Matt Neumann
15 Dan Banjock
16 Alberto Haber
17 Mike Palko
18 Bob Hunt
19 Josias Delgado
20 Kenny Stevens
21 Kent Tysor
22 Steve Moon
23 Bob Gieseke
24 M Hiki
25 Todd Lee
26 Kaz Minato
27 Allen Goff
28 Bene Rodrigues
29 Gordan Delaney
30 Curt Contrata
31 Steve Millet
32 Bill Suarez
33 Doug Moon
34 Frank Williams
35 Bob McDonald
36 Konstatine Bajaikine
37 Bob Whitely
38 Paulo Gomes
39 Yurii Yatsenko
40 Andrii Yatsenko
41 Alex Schrek
42 Bob Dixon
43 Dale Barry
44 Igor Burger
45 Bill Rutherford
46 Uwe Degner
47 Chris Cox
48 Jim Aron
49 John Sunderland
50 Bruce Perry
51 Henk DeJong
52 Matsuro Yokoyama
53 Dee Rice
54 Steve Starr
55 Don Melanson
56 Phil Granderson
57 A Tozim
58 Jim Lee
59 Dave Hemstrought
60 Bill Byles
61 Keith Trostle
62 Kirk Mullinex
63 Allen Brickhaus
64 John Simpson
65 Gerry Phelps
66 Gene Martine
I guess I dont get to fly?
-
Nice work, Howard.
I notice you rated the JCT pretty low. Very modest of you. Do doubt, another analyst would have put you and PTG much higher.
This listing might "flush out" some dormant flyers, in a effort to move up on next year's totem pole.
-
I didn't do the seeding. Paul came up with the formula. I think it's pretty good. I'll send the spreadsheet to anybody who's interested. It's the official word. Here is my unofficial understanding of how it works:
For the top 20 in Open for the last ten years, Paul assigns 20 points for first place, 19 for second, and so on. US Team members who were out of town for the WC get 20 points each. Scores get multiplied by 10 for 2008, 9 for 2007, and so on. Advanced scores are then multiplied by .5. Orestes's score includes both his Open and Advanced placings, for example. Top score is seeded #1. Guys who haven't placed in the top 20 in either Advanced or Open are unseeded. Their assignment to one of the four groups for qualifying rounds is done by random draw.
Windy didn't get seeded higher because he hasn't placed high in recent years, which are weighted more heavily. Not making the top 20 in 2007 set him back.
I don't think seeding matters much, either. Rich, try out that program I sent you. The way it folds the seeds, includes Advanced, and evens up the circles is pretty cool, even though it was probably a waste of my time.
-
Don't give up Sparky, it'll come around. Just go blow them away, flying that is. I flew several contests in carrier one year and never even got mentioned. The NATS was one of them flying Sportsman and Nostalgia. Have fun, DOC Holliday
PS:I can just imagine the turmoil if they went back to doing it the Navy way. Draw your card when ready to fly. At 1700 hours flying stops. jeh
-
How is this seeding used to select the flight circle groups?
-
Does this mean that 66 people have placed in the top 20 in the last 10 years, or were there more than 66 and you just published the "top 66".
Whether it's 66 or more than 66, this still indicates very good participation.
-
"How is this seeding used to select the flight circle groups? "
There are four groups: A, B, C, D. Top Open seed entered goes to A, the next seed to B, then C, D, D, C, B, A, A, B, C... Then the randomly ranked unseeded Open guys, then the seeded Advanced guys, then the randomly ranked unseeded Advanced guys. When everybody is assigned to the four letter groups, the groups are assigned to circles by random draw. The whole thing happens before your eyes at the pilots' meeting during appearance judging. The folding was a nuisance for the amateur programmer.
"Does this mean that 66 people have placed in the top 20 in the last 10 years, or were there more than 66 and you just published the "top 66"."
I assume that was a question. It includes Advanced. If PW entered the data correctly, 66 people have placed in the top 20 in the last 10 years in either Open or Advanced.
-
Howard....I have zero problem with the numbers or rankings, and even think I understand the methodology of assigning the scores.
My gripe is that seeding tends to create a bias by pre-prejudicing the contest.
-
Howard....I have zero problem with the numbers or rankings, and even think I understand the methodology of assigning the scores.
My gripe is that seeding tends to create a bias by pre-prejudicing the contest.
So, your theory is that it would be OK as long as the results weren't published? Sort of like it was a few years ago?
Brett
-
Windy didn't get seeded higher because he hasn't placed high in recent years, which are weighted more heavily. Not making the top 20 in 2007 set him back.
Windy came in 5th in 2006 and 5th last year. In 2007 he crashed his gorgeous Novanta, but got it repaired as I'm sure many of you know. That is probably why he isn't seeded higher.
I would say 5th is a pretty high placing.
-
Coming in fifth in that crowd is quite an achievement. It's a long way from fifth to sixth, as I learned last year.
-
"My gripe is that seeding tends to create a bias by pre-prejudicing the contest."
I presume by tipping off the judges as to who did well at previous Nats. Judges could see the seeding list here, I guess, but they could also be prejudiced by the covers of Stunt News or the press releases of the guys who think they're good. There won't be evidence of seeding in the actual contest. The flight order will be randomly drawn after the seeding, so it ain't like the guys appear before the judges in order of reputation. It would be pretty hard to deduce what the seeding was by looking at the makeup of the four groups. Remember also that the seeding is just for the qualifying rounds: the shootout is between the fifth- and sixth-best guy on each circle. Seeding doesn't affect the top guys. These rounds are a mere formality for the likes of Brett and Windy. That said, you have a point. Seeding doesn't do much. It doesn't do much good, so it doesn't take much harm for it to do more harm than good.
-
Seeding doesn't do much. It doesn't do much good, so it doesn't take much harm for it to do more harm than good.
I think it does more good than harm. With seeding, it could avoid one circle being stacked with a bunch of the top flyers. Everybody is evenly distributed.
-
I guess I dont get to fly?
Sparky,just head down to the Nats hook up them lines,and make them understand the terrible, terrrible mistake they made. Or, you can just suspend posting priviledges and make them give you a higher seeding-much easier!!!
-
I paid my 90 bucks today. I will just take what I get.
-
If you have a better idea, let's hear it.
-
Who me? I dont have a better idea and I like this one.
-
If this is some kind of a "US ranking", I do not understand how Brett would be above Orestes.
In the PGA golf rankings, the points for a win has a rate of decay. In other words recent wins count a great deal more than old wins. I assume Paul did something like that here. I just do not see how the guy who won the Nats the last two times (and won the Team Trials and competed in the WC's) is behind Brett who never competed in a WC's and won the Nats some years back. A win counts drastically higher than a second, third, etc A guy could have 20 seconds and it would be eclipsed by one win.
Using the rate of decay, even Paul's wins would start to fall off. A Nats win 10 years ago would not count the same as one last year... but like I said, it appears Paul did something like this. All in all, it looks like Paul put a lot of thought and work into the ranking system.
All that being said, I thought Howard proved seeding did not matter with his simulation?
-
It has nothing to do with ranking it has to do with circle placings. If the top 20 guys on this list flew against each other it would look something like this.
1 David Fitzgerald Circle 1
5 Terry Fancher
9 Windy Urtnowski
17 Mike Palko
13 Bill Werwage
2 Paul Walker Circle 2
6 Frank McMillan
10 Randy Smith
14 Matt Neumann
18 Bob Hunt
3 Brett Buck Circle 3
7 Richard Oliver
11 Bill Rich
15 Dan Banjock
19 Josias Delgado
4 Orestes Hernandez Circle 4
8 Derek Barry
12 Howard Rush
16 Alberto Haber
20 Kenny Stevens
-
Is this "seeding" list not a ranking from highest to lowest rank based on points awarded by finishes?
-
NATS Seeding list. Of coarse it would be better for the lower ranking guys if the top 20 seeds flew on 2 circles (hence the masters class)and the rest of the lower ranking guys flew on the other two circles. It would change top 20 results but it would probably not change the outcome of who really is the best flier. But this system is fair.
-
Fair is fair. With a complete random draw it would be really nuts if the top ten on that list all got put on one circle! And if more than one of the judges hasn't heard of most of the guys on this list, just where have they been judging? Judge quality, lack of bias, ranking, and seeding are not real problems conducting large contests like the NATS.
-
It has nothing to do with ranking it has to do with circle placings. If the top 20 guys on this list flew against each other it would look something like this.
1 David Fitzgerald Circle 1
5 Terry Fancher
9 Windy Urtnowski
17 Mike Palko
13 Bill Werwage
2 Paul Walker Circle 2
6 Frank McMillan
10 Randy Smith
14 Matt Neumann
18 Bob Hunt
3 Brett Buck Circle 3
7 Richard Oliver
11 Bill Rich
15 Dan Banjock
19 Josias Delgado
4 Orestes Hernandez Circle 4
8 Derek Barry
12 Howard Rush
16 Alberto Haber
20 Kenny Stevens
Really? I guess I thought it would look like this:
1 David Fitzgerald Circle 1
8 Derek Barry
9 Windy Urtnowski
16 Alberto Haber
17 Mike Palko
2 Paul Walker Circle 2
7 Richard Oliver
10 Randy Smith
15 Dan Banjock
18 Bob Hunt
3 Brett Buck Circle 3
6 Frank McMillan
11 Bill Rich
14 Matt Neumann
19 Josias Delgado
4 Orestes Hernandez Circle 4
5 Terry Fancher
12 Howard Rush
13 Bill Werwage
20 Kenny Stevens
I could be wrong as well, though.
Howard's program/Excel spreadsheet is really quite good. I've been tinkering around with putting it in Access, but I haven't had a ton of time to devote to it lately.
I just like the fact that it eliminates any questions about how flyers are seeded. People may not agree with the seedings, but there's an objective way to reach them.
-
It does not matter how it looks to who and the order. I WAS GIVING A EXAMPLE!
Why do people on these forums try to be so critical on stuff that don't matter?
-
Why do people on these forums try to be so critical on stuff that don't matter?
:o
LL~
-
I just like the fact that it eliminates any questions about how flyers are seeded.
That is very true. Very, very true.
-
Easy. :) I wasn't trying to be critical.
I said I didn't know who was right or not. I was just curious about how the seedings would actually look.
-
Judge quality, lack of bias, ranking, and seeding are not real problems conducting large contests like the NATS.
I would argue this statement as I have a few examples.
-
Mike's right. If we'd done it the way Robert shows, it would have all the guys in the first group a notch higher seeded than all the guys in the second group. I originally had it assign guys as Robert assumed, but Paul made me do the folding.
Phil, show us the probability of the top ten or even ten of the top 20 getting on one circle in a random draw. Let's see some (n-r)!
I think I have sent copies of the program to all the participants in this conversation, and I showed the seeding process above, so there should be no reason for people to make erroneous statements about how either works. I'll hose Paul's seeding spreadsheet around when I get back from flying.
-
"If this is some kind of a "US ranking", I do not understand how Brett would be above Orestes.
In the PGA golf rankings, the points for a win has a rate of decay. In other words recent wins count a great deal more than old wins. I assume Paul did something like that here. I just do not see how the guy who won the Nats the last two times (and won the Team Trials and competed in the WC's) is behind Brett who never competed in a WC's and won the Nats some years back. A win counts drastically higher than a second, third, etc A guy could have 20 seconds and it would be eclipsed by one win.
Using the rate of decay, even Paul's wins would start to fall off. A Nats win 10 years ago would not count the same as one last year... but like I said, it appears Paul did something like this. All in all, it looks like Paul put a lot of thought and work into the ranking system."
OK, Brad, I'll reveal the secret formula (my unofficial interpretation of the official spreadsheet): For the top 20 in Open for the last ten years, Paul assigns 20 points for first place, 19 for second, and so on. US Team members who were out of town for the WC get 20 points each. Scores get multiplied by 10 for 2008, 9 for 2007, and so on. Advanced scores are then multiplied by .5. Orestes's score includes both his Open and Advanced placings, for example. Top score is seeded #1. Guys who haven't placed in the top 20 in either Advanced or Open are unseeded. Their assignment to one of the four groups for qualifying rounds is done by random draw.
As for the relative placings of Brett and Orestes, you should first check to see if Paul did the calculation as he intended. The method might be improved, but if I have to do the work, it's good enough as is. Leonard had an idea for a better method . I sent him the spreadsheet to work on. Perhaps you can collaborate.
-
I think as a seeding process goes, this is the best I have ever seen. I was just making comparisons to golf rankings. It is just a matter of the weighting differences (which give way more credit for wins), but this looks fine for what it does.
I think the main thing is that the method is published. Just doing that is a major leap forward, in my opinion.
-
I just know that I sure wouldn't want to be in Kenny Stevens shoes on circle 4 with that seeding.
Jim Pollock n~
-
If this is some kind of a "US ranking", I do not understand how Brett would be above Orestes.
Of course, it isn't a ranking, but the reason would be that I have made the flyoff a lot more times and my record goes back further. The system only gives one point per place, so winning doesn't have a disproportionate effect, he only got one more point than I did for last years win. The 10 years includes some years Orestes wasn't even entered, and a goodly number of Advanced places.
If you want to have an all time ranking, I suggest using Dick Byron's list and updating it. I think I am nearly unassailable in first place, and I want my 590 from last weekend corrected to 615 to compensate for the elimination of pattern points. David gets a 617 for this weekend, but I am still well ahead. Therefore I conclude I am the greatest flier of all recorded history. The rest of you guys should just quit...
Of idle curiosity, depending on how one defines a tie for third, but I have finished in every place from 1-7, and in 5 flyoffs I have finished in each of the 5 places.
Brett
-
Perhaps this is none of my business but it is interesting to read.
I do not like seeding. Seems to me it benefits the established folks and puts the others at longer odds. Of course this is not a race?
I think the way they fly at the VSC eliminates all this. The seeding would not be needed with the VSC format.
-
If you want to have an all time ranking, I suggest using Dick Byron's list and updating it. I think I am nearly unassailable in first place, and I want my 590 from last weekend corrected to 615 to compensate for the elimination of pattern points. David gets a 617 for this weekend, but I am still well ahead. Therefore I conclude I am the greatest flier of all recorded history. The rest of you guys should just quit...
I agree.
-
Chuck,
the ony way I can see that seeding is a negative is for the guy who really isnt a legitimate contender, or has NO history, he may have a better chance if by some stroke all the big guns got loaded onto one or two circles and he was on a circle with nothing but lower scoring pilots. The downside is that then there are some people that are on the stacked circles that legitimatly belong in the fly offs that wont get there
-
"I do not like seeding. Seems to me it benefits the established folks and puts the others at longer odds. "
How does it go about seeming? Let's see some math. I did a very crude simulation of it and found that: a) it makes very little difference, b) it doesn't matter for the top guys, who will qualify anyhow, c) it favors skill over luck a tiny bit more for marginal guys, allowing those about 10th-best a little better chance of qualifying at the expense of those about 20th-best. I haven't seen anybody do any analysis beyond what I did. By "established folks" I presume you mean guys who have placed high at past Nats. Does seeding favor them, or does it favor people who fly better at the Nats where seeding is used? Beats me. If you can show mathematically that it unfairly benefits "established folks", you might be able to talk PW out of using seeding, but not if it merely improves the chance of a substandard flyer to bump a better one through luck of the draw, as Mark notes.
-
Well if the top 20 guys fly on two circles and us lowly guys fly on the other two it gives us a 50% better chance of make the fly off. However it will not change the outcome.
-
Howard...
Are seeds going to "move up" if a higher seed doesn't show?
-
Yes. Try out the program. It's so cool.
-
NATS Seeding list. Of coarse it would be better for the lower ranking guys if the top 20 seeds flew on 2 circles (hence the masters class)and the rest of the lower ranking guys flew on the other two circles. It would change top 20 results but it would probably not change the outcome of who really is the best flier. But this system is fair.
I dont think you can do that. It would not be a fair way to get to top 5 and then the champ. If you have a lower class why would they get to compete for the top honors? If one wants to compete for top honors they have to go through the best to get there. Entering open then putting in with the top pilots and cutting through to the top.
Seeding is very fair the way Howard has come up with it especially using the folding. Without the folding you will get some lop sided circles.
-
I hope you noticed I said ,I think this system is fair. I also don't give a hoot how they do it. I just like going and probably have no chance anyway.
Almost made it last year but 1 judge on 1 maneuver kept me out. 40 40 and 33 for inverted level flight. I wonder how perfect it would have to be? I did not know there was degrees of perfect. One more reason for 5 judges and high and low score thrown out
-
Easy. :) I wasn't trying to be critical.
I said I didn't know who was right or not. I was just curious about how the seedings would actually look.
Mike,
Your seeding was the more correct of the two.
The other version clearly stacked the talent heavy in #1 and progressily lighter toward #4.
-
Howard:
This doesn't seem to make much sense to me. There are flyers on the list that as far as I can remember haven't flown expert at the NATS. How did Paul arrive at the listing???? ???:!
Gene
-
OK, Gene, I'll reveal the secret formula (my unofficial interpretation of the official spreadsheet): For the top 20 in Open for the last ten years, Paul assigns 20 points for first place, 19 for second, and so on. US Team members who were out of town for the WC get 20 points each. Scores get multiplied by 10 for 2008, 9 for 2007, and so on. Advanced scores are then multiplied by .5. Orestes's score includes both his Open and Advanced placings, for example. Top score is seeded #1. Guys who haven't placed in the top 20 in either Advanced or Open are unseeded. Their assignment to one of the four groups for qualifying rounds is done by random draw.
-
"One more reason for 5 judges and high and low score thrown out"
Only three steps to go: 1) You recruit 12 more judges, 2) you write a mathematical proof that throwing out high and low is better than taking the average, and 3) you edit the VBA macro to do it.
-
Almost made it last year but 1 judge on 1 maneuver kept me out. 40 40 and 33 for inverted level flight. I wonder how perfect it would have to be? I did not know there was degrees of perfect. One more reason for 5 judges and high and low score thrown out
How do you come to this conclusion? Based on the fact that it would have benefitted you personally? Why do you think the 40's were right, and the 33 was wrong - couldn't it have been just the converse?
Brett
-
Because it was a 39 or 40 and that judge had scored right along with the rest before that easy maneuver. It either is or it ain't. If it was on a square 8 or triangle or one of the harder maneuvers I would have understood. Of coarse all the arguing in the world won't ever change your mind or mine. I know I got the shaft but oh well. I also Have a idea why. On to the next contest.
-
"One more reason for 5 judges and high and low score thrown out"
Only three steps to go: 1) You recruit 12 more judges, 2) you write a mathematical proof that throwing out high and low is better than taking the average, and 3) you edit the VBA macro to do it.
Hum I guess all the real sports who use subjective judging must be wrong? You know diving,Ballet,figure skating.This is why I hate voicing my opinion. Because every time I do someone has to come up with stuff like this. Some MATHEMATICAL BS that don't mean squat!
-
I have judged quite a bit and only awarded one 40, if I recall....Howard made one of the coolest (AND rule book) landings in a gale one year at Muncie....
-
I have judged quite a bit and only awarded one 40, if I recall....Howard made one of the coolest (AND rule book) landings in a gale one year at Muncie....
Thats cool but last year I will bet dollars to donuts that there were lots of forty's awarded as we had stunt heaven.
-
"Some MATHEMATICAL BS that don't mean squat!"
As an alternative, I suppose that you could construct a proof that it don't mean squat. There's still the matter of recruiting the judges and changing the program.
-
Thats cool but last year I will bet dollars to donuts that there were lots of forty's awarded as we had stunt heaven.
Hey, I figured out how to put quotes in a box.
I was going to take you up on that bet, because I have the unfair advantage of having the data. You are correct, however, so I guess I'm morally obligated to give you a donut. I only looked at the qualifying rounds, because it's kinda tedious to do. Here are the numbers. Consider them unofficial: I hand copied them.
Qualification flight 1
Orestes Hernandez landing 36 40 40
Bill Rich landing 36 39 40
Qualification flight 2
Robert Storick takeoff 40 33 33
Robert Storick inverted 40 35 35
Keith Trostle inverted 40 35 35
Dan Banjock inverted 40 34 32
Louis Rankin landing 37 35 40
Qualification flight 3
Brett Buck landing 40 38 36
Steve Moon landing 37 35 40
Kent Tysor landing 38 35 40
Qualification flight 4
Brett Buck landing 40 37 34
Steve Moon landing 38 35 40
Matt Neumann takeoff 40 34 34
Matt Neumann landing 40 38 36
Howard Rush landing 40 39 37
Bill Rutherford landing 40 37 31
I was surprised to find myself in this august group. I don't remember the landing, but it must have been a humdinger.
One interesting tidbit is that almost all of these 40s came from two judges.
-
Thanks Howard. Flgiht 2 day one I had 2 40s on same flight and a 33 for inverted I think I still have the score sheets. But glad you looked them up. I knew it was heaven cause my pig of a airplane flew fair any wind and I would have been drug all over the circle. Of coarse it was last year and a new day is comming. I'll live. LL~
PS my day 2 scores were much much better even tho no 40s that day that I can remember.
-
A 40 for one maneuver doesn't mean much if the rest of the maneuvers are the pits. My part time coach keeps telling me that consistancy pays more. But, then I watch someone fly 10 to 15 foot bottoms and tracking of maneuvers all over the place and they out score me. I hit 5 foot once in a while and manage a fair share of tracking. But, consistency pays.
But, over all if the judges are consistant themselves from one pilot to the next it is great. But, there are those that lose concentration during the day. I also commend the judge that is not afraid to down grade a maneuver that is not what it is supposed to be. I have score sheets in which it was pretty much consistant numbers all the way down the sheet. Of course I am surprised that some maneuvers haven't gotten a zero. Having fun, DOC Holliday
-
I was going to take you up on that bet, because I have the unfair advantage of having the data.
In no way am I trying to be controversial here... but in the spirit of the new openness maybe you could consider the following.
Since it appears you guys have and keep all that scoring data from the Nats (and obviously study it), maybe you should publish that too. Just to be fair to the rest of the competitors.
-
In no way am I trying to be controversial here... but in the spirit of the new openness maybe you could consider the following.
Since it appears you guys have and keep all that scoring data from the Nats (and obviously study it), maybe you should publish that too. Just to be fair to the rest of the competitors.
And, also in the spirit of openness, we should include bullseye's and Laser Target Designators on all the NATs judges shirts!
I will take you at your word that you aren't trying to start anything, but you can't fail to recognize the problems. We already when through that back in 1993. Resulting in 15 years of "fun", some world class "gibberish" analysis papers, and a few Federal felonies. Some people proved incapable of acting responsibly with the information.
There's a reason it's hard to find a lot of NATs judges, and I would argue very vociferously that it would not be in the best interests of the event to open them up to perpetual second-guessing.
Brett
-
Sure. I know that none of your Tournament of Champions contests has gone by without publishing all the judges' scores for every maneuver. I suppose that since I am an active Nats contestant, I shouldn't have seen the populated program, but because I wrote and maintain the program, I sorta need to see it. I'm sure you would have been willing to spend three or four days entering dummy scores so I wouldn't have had to see the real ones, but I didn't see the need for that at the time, because it was absurd to imagine that anybody would accuse me of being unfair by seeing scores after the contest was over. Ask Paul for a copy. Be sure to explain to him how "you guys" have an unfair advantage by seeing the results. OK, OK, I'll give Sparky the donut.
-
Be sure to explain to him how "you guys" have an unfair advantage by seeing the results. OK, OK, I'll give Sparky the donut.
I'll take a Jelly at the NATS.. LOL and I don't have any problem with anyone seeing the scores but I do agree with Brett it serves no purpose to open them up to the world.
-
You could use Howard's seeding system, then concede a place in the finals to the best flyer in every circle, and make him judge in exchange for the "bye".
My theory being that the top four seeds would certainly make the "final 20" anyway, so give 'em a pass and put 'em to work.
Just a thought from a spectator.
-
I have an idea for an article.
You take the scores for the top ten guys in Open on their Top20 qual flights. Leave their names off and the judges names off. But specify these are true scores from the nats.
The name of the article would be something like "How do they score so well?"
Showing the data would let fliers from lower classes see just where the top guys are really picking up points as they fly the pattern. For instance Take Off, Inverted Flight, Loops, and Landing. These are huge point getters and or givers. Lots of lower class fliers give it away on these very manuevers. With some published data, including app score they can really compare where their scores are and the ones form top guys at the nats...
Just an idea...
-
WOW!
Brett and Robert agree on something!!!!
Jim Pollock n~
-
WOW!
Brett and Robert agree on something!!!!
As always, most of us agree on most things. We discuss the differences because there's no point in discussing the agreements.
Brett
-
You could use Howard's seeding system, then concede a place in the finals to the best flyer in every circle, and make him judge in exchange for the "bye".
My theory being that the top four seeds would certainly make the "final 20" anyway, so give 'em a pass and put 'em to work.
Even though I would benefit from it greatly, I don't think that's a good idea. The sole goal with ALL of this is to try to put everyone on an even footing. Giving a few guys a free pass into the Top 20 is anathema to that idea. It should be based completely and entirely on the performance during the contest - not on past performance, not on reputation, or anything else other than how the 15 maneuvers are executed when it comes time to do it. Doing as you suggest is like giving the Yankees, Red Sox, Dodgers, and Braves free pass into the playoffs and not even making them play the 162 regular season games.
And I think I can speak for any of the potential beneficiaries and say *we don't want a break*. We ought to be able to make it through qualifying, if not, we don't deserve to win.
Brett
-
It's Paul Walker's seeding system. I'm just the programmer. Brett, I think Paul Smith was suggesting that the qualifying flights be flown, but the winner of each be put in the final, bypassing top-20 day. I, um, prefer the existing method.
-
I was thinking about Brad's suggestion. For it, and for Doug's, all you'd need to do is to take the existing program and write a macro to extract the data you want. It might be good to obfuscate the judges' identities, but you ought to be able to show scores for each maneuver of the contest. The program only went through the top 20 last year, so you may want to start at this year's Nats. I'm not particularly interested in the project myself, but most of you have the program, so anybody who wants to write the macro can do so. Holler if you have questions.
-
It's Paul Walker's seeding system. I'm just the programmer. Brett, I think Paul Smith was suggesting that the qualifying flights be flown, but the winner of each be put in the final, bypassing top-20 day. I, um, prefer the existing method.
In order to get to the top of the "Walker-Rush" totem pole, a flyer needs to make the top 20 (and do well) several years in row. So is there really any doubt that he would do it again? This is sort of like the defending World Champion getting to compete again without going through team trials. I know you won't do this, but it would provide four more judges.
-
Paul...
I think that you perpetuate a myth that good fliers make good judges....
-
Here I always thought the previous World Champ did get to compete in the World Championship on his own merit. As I understood it, our past Champs would make the team and then step down to make for anolther member. When he does that, he does not add to the team score.
But, I like the idea of seeing the scores to see where the big differences are and not necessarily the top five or top ten. Just a random draw of some score sheets to see the individual maneuver score. I know when I used to pick up my score sheets I would wonder why one judge would score a maneuver higher or lower than the other. I can also tell when a judge is not on the bracket scoring. Like Int should not score as well as Adv. I have seen some Int flights that should heve been in expert, but, then you realize the pilot was on that day. DOC Holliday
-
Paul...
I think that you perpetuate a myth that good fliers make good judges....
Give me a break. >:(
-
And, also in the spirit of openness, we should include bullseye's and Laser Target Designators on all the NATs judges shirts!
I will take you at your word that you aren't trying to start anything, but you can't fail to recognize the problems. We already when through that back in 1993. Resulting in 15 years of "fun", some world class "gibberish" analysis papers, and a few Federal felonies. Some people proved incapable of acting responsibly with the information.
There's a reason it's hard to find a lot of NATs judges, and I would argue very vociferously that it would not be in the best interests of the event to open them up to perpetual second-guessing.
Brett
The FAI does it. No judges killed yet. Dooooh!!! :P
I also believe it has lead to some very interesting study.
-
In order to get to the top pf the "Walker-Rush" totem pole, a flyer needs to make the top 20 (and do well) several years in row. So is there really any doubt that he would do it again? This is sort of like the defending World Champion getting to compete again without going through team trials. I know you won't do this, but it would provide four more judges.
Well, more judges are a good idea, but I don't think giving supposed hotshots a free ride to Fri..., er, Thursday is the way to do it. If nothing else, you *actually have to fly the flights*, and anything can happen. Do I think David Fitzgerald is going to have much of a problem making the top 5 on his circle in qualifying? Of course not. But he still has to do it. I would consider it grossly unfair to let him (or anybody else) through on reputation. That's exactly what the system is there to PREVENT. And I can tell you right now that on the occasions we had a cancelled round in qualifying, so that you had to make that one good flight or get bombed out, I sweated it out. Anything can go wrong on one flight, no matter how good your reputation is. Ask David about the 98 NATs Top 5.
A second, far less important issue, is that when David/Paul/Orestes and I are out there judging, we would be WIDE OPEN to accusations of favoring people we think we could beat more easily. Oh, Billy showed up this year? Well, I don't want to go up against him, I think that was a 21 point Square 8 right there, look at all those >5ft radius corners. Of course no one would do that in real life, but I absolutely guarantee that if the wrong person didn't make it, Rich Peabody, et. al. would be writing nasty-grams to everybody in the event over how we jobbed the system AGAIN.
I would also note that being a top flier doesn't necessarily make them a good judge. Of the current group, I know for certain that David, Paul, and I have sufficient judging experience to do a good job (David and I have judged at the NATs before, Paul has judged at the Team Trials and we have all done it extensively at local contests. I would wager we have all conducted judging *clinics*). I don't know about Orestes - as a consummate sportsman I would trust him to be objective but I have no idea if he has any experience, and the NATs is not the place to start.
So, I don't think so, its an interesting idea, but I don't think it meets the "give everybody an equal sporting chance" principle.
Brett
-
The FAI does it. No judges killed yet. Dooooh!!! :P
I also believe it has lead to some very interesting study.
Interesting is one word. Bullcrap is another word. And there have been numerous FAI judges run off, banned, etc.
Brett
-
Give me a break. >:(
Agreed. For the most part, very good fliers tend to make good judges. If for no other reason that they also usually coach and judge incessantly all the rest of the year aside from the NATs. It's not necessarily true, but it usually is.
Brett
-
Interesting is one word. Bullcrap is another word. And there have been numerous FAI judges run off, banned, etc.
Maybe they *needed* to be run off.
Brett, you seem to function under the assumption that judges are immune to doing anything they should not be doing. When, in fact, history shows that judges *have* made decisions based on factors outside of who flew the best pattern, and people complained and those judges were removed, steps were taken, etc When it comes to the FAI WC's, some of the biggest complainers have been American pilots. I cannot remember how many articles I have read from magazines going back 20-30 years written by American pilots basically saying that "we would have won if the judges would have judged right" or basically implying the judges were "in the tank" for a particular country. In fact, I think you can make the case the judges *were* in the tank for one side or the other. That is why the FAI started publishing scores from the WC's.
The "that stuff does not happen here" basically denies history, even recent history.
Ultimately, openness never caused more problems than perceived secrecy will ultimately create. See "seeding". People who promote secrecy to "protect" people from themselves will ultimately do more harm than good.
"Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience." CS Lewis
-
Brad,
You seem sold. I think you should fly FAI.
-
Brad,
You seem sold. I think you should fly FAI.
Huh? Where?
-
Maybe they *needed* to be run off.
Based on WHAT? Some non-participant's ex post facto analysis and a few prima donnas out to salve their precious bruised egos? Or even more likely, the prima donna's cheering sections out to salve their precious little bruised egos (because that seems to be the majority of the problem)?
Are you actually suggesting that, say, Mary Gebhart, deserved to be defamed behind her back, called every name in the book, and finally get run out of judging, all for the horrible sin of trekking cross country on her own time to give her opinion about a bunch of grown men playing with little toy airplanes in a soybean field?
Any possible value of critical examination of the scores after the fact has to be weighed with the cost, specifically, judges being hounded, threatened, and second guessed for perpetuity by people with an axe to grind.
Past experience has shown us exactly how that worked out. It was of academic interest to most. To others it was a golden opportunity to display their utter lack of class or sportsmanship and came close to destroying the event. Given the minuscule purported advantages VS. the demonstrated irresponsible behavior, it not even a close call, it's a no-brainer. In my opinion, of course.
Does that mean you are being "punished" or discriminated against in your requests because of other people's past actions? Absolutely. If you want to complain about it, might want to take that up with those who messed it up for everyone. I am sure you can figure out where to direct your ire, and it sure isn't me.
I have seen absolutely nothing to suggest that it would be any different the next time; to the contrary, the pathetic little whispering campaigns and rumor mongering to undermine people is going better now than it was before, aided by any number of electronic communication media.
Bottom line is that we can get away with 5 less pilots at the NATs. If they think it's rigged, then by all means they can stay away. We can't get away with 5 less judges.
<snip>
Ultimately, openness never caused more problems than perceived secrecy will ultimately create. See "seeding". People who promote secrecy to "protect" people from themselves will ultimately do more harm than good.
I don't want protect people from themselves. I want to protect them from you.
Brett
-
"Ultimately, openness never caused more problems than perceived secrecy will ultimately create."
What was your social security number again, Brad? Actually, I think that the level of openness of the Nats process is a valid topic for discussion, particularly if discussed rationally by informed Nats participants whose purpose is to improve the contest.
-
<snip>
I don't want protect people from themselves. I want to protect them from you.
Brett
I think that is totally uncalled for and way out of line.
-
"Ultimately, openness never caused more problems than perceived secrecy will ultimately create."
What was your social security number again, Brad? Actually, I think that the level of openness of the Nats process is a valid topic for discussion, particularly if discussed rationally by informed Nats participants whose purpose is to improve the contest.
I think the level of openness since PW took over is outstanding! I applaud him on every level and I wasnt even there last time. But seeing something like seeding published and the draw for flights and so on published is just great in my view. It wasnt too long ago when people were stirring people up over these very things. Now there nothing more to get stirred up over about this part. Just one less thing. Not that my view really matters all that much to anyone but me. But, if PW is reading this stuff by chance I wanted him to know, and Howard to know, that I do think going public with these processes is good for the event. If people gripe or bitch or moan or whatever they do they cant say it was secret so they could screw so and so out of his rightful place. Plus with it public most wont even give the bitching a moaning a second chance and just move on where as before people could get all whipped over it...
This is a long thread and a lot has been covered and the one thing people seem to be very happy about is "knowing" how it is done.
I am with BB I dont think anyone should get a free ride based on last years score. If anyone gets a free ride it is the year prior Walker Cup winner and him only! I bet it would work against him. Flying the qual rounds helps get one in the mindset and get things moving.
BUT If the 4 judges were needed in an emergency situation and you did get them a bye to Top 20. They can only be eligible to judge ADV. That cures the situation BB described with the judging your future competition stuff.
-
I think that is totally uncalled for and way out of line.
Sorry, but I have learned the hard way. I figure that about a week after you guys get the data, we will have "Anatomy of a Team Trials, Volume II." (a tome that Brad has in the past expressed admiration for) or some equivalent, and then we will have judges quitting left and right, angry recriminations for anyone who didn't get the "right" answer, and yet another big mess. Even if you and Brad *don't intend it to turn out that way*. The road to hell is paved with good intentions. I for one don't want to risk it, and it won't happen if I can do anything to stop it.
Sorry if that's seems unfair, and maybe it is since it's based partly on past experience and other's irresponsible behavior, but that's what I think is best for the event. Already the "openness" on the seeding is starting another bitching campaign, and that's a tiny fraction of the misinterpretation possible with the judges scores.
So I am not accusing anyone of wanting it to happen, I have just seen the results and am unwilling to repeat them. Of course it's not really up to me, but I can have my opinion, too.
Brett
And just so you know I am not singling out Brad, here's a letter I sent 11 years ago, on the same topic - long before I knew anything about him:
Publishing Judges scores -
B. Buck, 9/24/98
Dear Competition Commitee,
I would like to comment on the concept of publishing individual judges scores at the NATS. While I appreciate the reasoning behind Wynn’s survey, and even agree with points of it in a perfect world, I think that the realities of the situation make such a procedure a very bad idea.
My chief fear is that if the scores get published, everyone with an axe to grind or a pre-conceived notion that they are getting the shaft would be provided with enough raw data that they could plausibly construct whatever conspiracy suited their fancy. The publication of the judges scores of the ‘93 Team Trials initiated the worst and most divisive controversy in the history of PAMPA, with the publication of “Anatomy of a Team Trials” by Mr. Baron and “THE GENIE IS OUT” by Mr. Urtnowski. Without discussing any further the merits of these two documents, the ensuing controversy they spawned needs no further explanation. One can only speculate on the effect the controversy had on recruiting judges but I assume that it was discouraging. I feel compelled to point out the “analysis” of the judges scores was key to the problem, and that this analysis resulted in “proof” of cheating based on scoring differences of fractions of a percent. It didn’t convince me, but you can prove anything with statistics and a willingness to interpret them to your own advantage. I would have to think that publishing them all at every NATS would results in far more bitching and moaning after the fact by those who are unwilling to accept the official results.
I also disagree with the idea that qualified judges shouldn’t have a problem with having thier scores published. I think that the most qualified judges are the ones who use the entire scoring range and are willing to give a Universal Stunt Hero a 22 if he blows a maneuver and willing to give Joe Bellcrank a 40 if he deserves it. I think that this will become much more difficult if the judges knows every number he writes down will subsequently be utilized in innumerable different ways to prove him/her a cheater. It may not be conscious, but it’s inevitable. One bad score one on one maneuver (deserved or not) will become “proof” that so and so favors pilot X. The temptation will be to narrow down the scoring range so as to avoid having to stand out from the crowd. It won’t be intentional, but it will still happen. The result - contests where the range of scores even narrower than it is currently.
The potential for individual judges to become targets of smear campaigns and be threatened with physical violence should be clear. Sad to say, but it’s obvious that this has to be considered. Publishing the scores just gives more potential “justification”.
The vast majority of the competitors wouldn’t care one way or the other whether or not the scores were avaliable, because they accept the would results just as they do now. The only competitors who would really be interested are those who think that they could make something out of them - and it should be clear that those are the ones who would mis-use the data ala “Anatomy of a Team Trials”. Are we willing to accept the potential for abuse in order to potentially placate a few paranoids?
I don't expect this to take, but I would be tempted to propose just the opposite - that all the scores, including the totals, be kept secret. Only post the rankings. That way, no one ever has any “hard data” on which to base cheating accusations.
In summary, I think the potential for abuse far outweighs any possible advantage that could be gained.
Sincerely
Brett Buck
-
Already the "openness" on the seeding is starting another bitching campaign
While there will always be people who "bitch", the seeding system that Howard is using is being done in the open, so while people may argue the *flaws* of the system, they cannot argue the intent of openness.
I believe you will find that many of the people who have complained about seeding in the past were not complaining about the system used, but the secrecy around it.
-
I have seen absolutely nothing to suggest that it would be any different the next time; to the contrary, the pathetic little whispering campaigns and rumor mongering to undermine people is going better now than it was before, aided by any number of electronic communication media.
Bottom line is that we can get away with 5 less pilots at the NATs. If they think it's rigged, then by all means they can stay away. We can't get away with 5 less judges.
<snip>
I don't want protect people from themselves. I want to protect them from you.
Brett
That is OK, Brett, since I am sure you are justified in being suspicious of everyone around you... especially me. Since I am huge advocate of openness (which seems to be catching on as of late) and treating my fellow competitors like accountable adults and not little children that need to be watched over by "mommy".
I, of course, said the Nats was "rigged" so please, continue. The hyperbole is simply off the charts whenever you guys talk about the Nats
Please continue...
-
"Ultimately, openness never caused more problems than perceived secrecy will ultimately create."
What was your social security number again, Brad? Actually, I think that the level of openness of the Nats process is a valid topic for discussion, particularly if discussed rationally by informed Nats participants whose purpose is to improve the contest.
I am not sure what you are implying… If you are implying that only current attendees of the Nats should have input into the process, I will comment on that.
I think that this idea that only current competitors of the Nats have an interest in the current and *future* of the Nats is misguided. In fact, I know it is.
Yes, it is true, I have not attended a Nats since 2003 (gasp). This is simply due to the fact that I have been busy trying to make a living and establish myself at my new job(s). Also, since I have no plans to be divorced any time soon, it is best to consider my wife in my limited vacation plans.
Unlike some stunt flyers I do not own my own company. I also do not currently do not have the status of being either retired, an airline pilot or attendant (which means massive amounts of free time) or have 20 plus years in the aerospace industry with most likely 6-8 weeks of vacation every year. This being the case, I think it is best to consider taking *actual* vacations instead of dragging my wife to Muncie for one week of the two weeks of vacation I get every year.
This does not mean that I do not have plans to attend future Nats when my vacation status improves. I assume I am not alone in this sentiment, as I would suppose there are many people out there who are struggling right now, and attending model airplane contests might take a back seat to other pressing real world issues.
So, I will operate under the assumption that the current attendees of the Nats do not *OWN* the Nats, and hopefully, the Nats will still be there long after many of the current attendees are gone. So, if I want to present ideas on an Internet forum or the AMA, I will.
As far as sharing social security numbers... I expect better of you Howard... let's please not elevate a model airplane contest to national security code red levels.
-
What about releasing scores with the judges names redacted. You might still get lots of statistics (which might be interesting in and of themselves), but you wouldn't have as much fodder for flame campaigns against a paticular judge.
Just because data has the potential for being abused seems poor reason to have it supressed. And, plenty of Pampa wars have started without statistics as the spark. Shining light onto more aspects of judging/scoring might drive the inevitable arguments into more and more trivial areas as the *big* areas of contention/argument are open for all to see if any attacks are valid or not. Already it seems the seeding openess has reduced that aspect from the ranks of "conspiracy" into an animated but mostly polite discussion.
-
"What about releasing scores with the judges names redacted. You might still get lots of statistics (which might be interesting in and of themselves), but you wouldn't have as much fodder for flame campaigns against a paticular judge."
That seems reasonable to me. I think that's what Doug suggested. It's merely a matter of writing a VBA macro, although I'm not interested in doing it myself. I'll send you the Nats tabulating program draft in case you want to take this on. I'd appreciate any comments you have on the program anyhow.
-
Brad said "Huh? Where?""
Well, you could move to Canada or South America. ;D
I don't compete at the Nats and so, don't feel competent to comment on the process. I can only relate it to what I've experienced and it really isn't the same.
-
That is OK, Brett, since I am sure you are justified in being suspicious of everyone around you... especially me. Since I am huge advocate of openness (which seems to be catching on as of late) and treating my fellow competitors like accountable adults and not little children that need to be watched over by "mommy".
I, of course, said the Nats was "rigged" so please, continue. The hyperbole is simply off the charts whenever you guys talk about the Nats (especially when the evil Bob Baron or sociopath Windy card is pulled out).
As usual, missed the point utterly and completely. I don't know why I waste my time, but I will try anyway. Actions and decisions have *real world consequences*, it's not some isolated debating society. I for one don't think "destroying stunt to save it" from some narrowly-perceived hypothetical "flaws" makes a lot of sense.
That's the underlying problem with publishing judge's scores and leaving them wide open (there's that word again) to the same sort of BS whispering campaigns we endured for majority of the 90's. You clearly stated that his was something you could consider acceptable, and that's why I think the judges need protection from you (and people who think similarly).
There is nothing in this event more important to this event than the judges, and there is nothing in this event more important than protecting them from scurrilous accusations and second guessing. Allow that, and there won't be an event.
Even if you were prepared to act in a mature and responsible way and were prepared to objectively review the results, there is documented proof that others have not in the past, and there are no indications that they will in the future. We had people committing federal felonies over the release of one set of scores, apparently spurred on by emotionalism. What's going to happen when people get their hands on even more BS analysis used to "prove" the "facts".
And nice job - now you are engaging in exactly the sort of irresponsible behavior I was worried about. Not a word in my post said a d*mn thing about Baron being "evil" or Windy being a "sociopath". That doesn't appear in my post, the letter I posted, or in the private letter I sent to Bob Baron on the topic, or the commentary I had with Windy on the topic. I said my piece at the time with Bob, he said his piece back, and since he's gone and can't defend himself I will let that stand. I have no current beef with Windy, either, so barking up the wrong tree there, too.
Those words, chosen BY YOU, Bradley "Godzilla" Walker, were formed solely and exclusively to *start a war*, AGAIN, and get people out of thinking about the rational discussion and into an emotional debate. Now, it will get repeated, "summarized" for the "masses", misquoted and attributed to me, and you will have your mini-firestorm. Well done. Glad you "didn't want to start a controversy". And you didn't even bother to read carefully, I took you at your word, responded entirely to your own current statements. You read a bunch of your own predispositions into it, missing the point entirely, and wow, look how that worked out.
In this thread alone, you have gone from demurely asking for the scores, to suggesting that maybe judges "deserved to be run out" by rumor mongering and whispering campaigns, then jumped to falsely attributing statements/attitudes to me, presumably to get more people on your side. Of course that's classic "bomb-throwing" and exactly the reason no one with a shred of common sense would trust you to be able to have a rational debate on the topic or act responsibly with the data. And of course, while it might ""play" with a tiny number of people who still spoil for another war, it essentially proves my point (far better than I would have been willing to articulate it) to everybody else.
So, to summarize, you aren't a NATs participant (no problem), you expressed the view that it might be good for judges to "be run out" of the event on rumors (big problem), and then falsely attributed me with opinions I didn't express (irritating, but valuable to me as it essentially cinches my point) in an apparent attempt to spin up a new controversy. And now you want all the scores for "analysis" and we should just trust you to objectively evaluate them?
I still only have my 2 cents to put in, I don't decide myself, but nothing in this thread has done anything to convince me that releasing the scores (at least with the names intact, or with any method of reconstructing the relationship of judge to score) would go any better this time than before. Quite the contrary, it confirms my expectations pretty much right down the line. I would be abdicating my responsibility to the event if I didn't make my case. I will not have anyone targeted by "analysis" if I can prevent it.
I can't see a big problem with releasing the scores "raw" with no way to reconstruct the identities, but that's a lot harder than it first seems. You can't even identify the groups, circles, or the pilots, and without that, there's no sensible analysis that can be done.
Brett
-
" I can't see a big problem with releasing the scores "raw" with no way to reconstruct the identities, but that's a lot harder than it first seems. You can't even identify the groups, circles, or the pilots, and without that, there's no sensible analysis that can be done."
I'm not sure what you mean. Data are available partway through the 2008 Nats in the program populated with scores. They should be available for the entire 2009 Nats, although it's probably not worth the bother to automate the part for the kids. The program stores scores for each maneuver, who judged it, in what group the contestant is, upon what circle he flew, and-- for 2009-- how much his airplane weighs. One could write a macro that barfs out any combination of that stuff. If we could time-stamp the scoresheets, we could correlate it to weather if I get around to hooking up the Anemowimpometer and a couple other sensors to a data logger. We could also compare sun direction to wind direction to see that effect on scores, which might be interesting.
Change of subject: Within the program, as you have all seen, is a macro that selects judges for the finals. This is possibly moot, given the paucity of judges available, but I would appreciate folks looking at that and sending comments on the method to Paul Walker. You can send me comments on implementation. The formula was published here and on SSW awhile back.
-
Just a thought as to why "drop highest and lowest scores" isn't always a great idea (and to hopefully steer this back to something less incendiary):
(http://stunthanger.com/smf/index.php?action=dlattach;topic=12406.0;attach=41997;image)
In the example (which hopefully is somewhere in this post, Judges 3 & 5 are the high and low judges, so under the "get rid of high and low scores" plan, they get eliminated.
But if you track their scores on the line graph, Judges 1, 3, and 5 are all tracking scores the same way. They're assigning different numbers to the scores, but they see the pattern the same way. They all see the Triangles as the low score, and Overhead Eights and Horizontal Eights as the better manuevers.
Judges 2 & 4 - who give the same score to everything - get kept under the "get rid of high and low scores" plan, even though they aren't seeing the pattern the same way it's probably being flown. These would be the judges you would want to get rid of, except because their scores are closer to the average overall score, they get kept.
I think if you're going to throw out scores - and I haven't been able to come up with a statistical model that shows it makes a meaningful difference - there has to be a more efficient model than "dump highest and lowest score". As Howard says, though, that may be a moot point this year, if there aren't many judges.
-
Paul...
I think that you perpetuate a myth that good fliers make good judges....
I can't judge a judge until I see the number he gives me.
-
Maybe they can draft the Intermediate flyers to judge Advance and Open. S?P S?P DOC Holliday
-
Those words, chosen BY YOU, Bradley "Godzilla" Walker, were formed solely and exclusively to *start a war*, AGAIN, and get people out of thinking about the rational discussion and into an emotional debate. Now, it will get repeated, "summarized" for the "masses", misquoted and attributed to me, and you will have your mini-firestorm. Well done.
Oh my GAWD, Brett... get a hold of yourself. Settle down.
I made no mention of *anyone* in particular, but instead commented on the history of why the *FAI* began publishing scores, and I also commented on the fact that in the 1970's and 1980's the biggest complainers about BIAs were the US pilots. Maybe the FAI started publishing scores because of all of the bitching from the US pilots. Maybe you missed that.
Even at the very mention of publishing scores, *you* are the first to drag out "Anatomy of Team Trials". *You* are the first to make the Nats history *personal*. I made no personal comments whatsoever about the "rigging" of the Nats (you had to make some smart aleck comment about people STAYING HOME if they do not like it). In fact, I did not say anything about the Nats history at all.
If you do not want to be taken out of context about Windy or Baron, maybe you should stop bringing them up.
I do not want to *start a war*. The *war is won* as far as I am concerned. I think what Paul Walker and Bill Rich have been doing with the Nats is just great. I am just discussing what I would like to see. This is still America. I thought we could still do that....
-
As Howard says, though, that may be a moot point this year, if there aren't many judges.
I agree with Howard. What you need is like 20 judges and drop the top 8 and bottom 8.
-
"I am just discussing what I would like to see."
I am just discussing what I would like you to do. There are many opportunities: 1) you could write the data-extraction macro for maneuver scores, 2) you could review the possibly bogus finals judge selection formula, or 3) you could discuss how to use the scores of multiple judges, as Mike did above. I suggested the latter two because you mentioned being a Six Sigma Black Belt. The process could use advice from people who know statistics.
PW at one time suggested maybe throwing out high and low judges for the finals, but the discussion didn't get very far, probably because of what Mike showed above. Sounds like a bad idea to me, but as Sparky says, other sports do it. A little research might show why they do. Then, armed with that information, we could see if it applies to our situation.
-
Didn't Dave Cook suggest tossing the high and low score PER MANEUVER at one time? I thought that it had merit, reather than tossing the high and low score?
-
I can't judge a judge until I see the number he gives me.
Nailed it Paul. But then you were probably thinking of me when you put the good flier/judge thing together......
-
"I am just discussing what I would like to see."
I am just discussing what I would like you to do. There are many opportunities: 1) you could write the data-extraction macro for maneuver scores, 2) you could review the possibly bogus finals judge selection formula, or 3) you could discuss how to use the scores of multiple judges, as Mike did above. I suggested the latter two because you mentioned being a Six Sigma Black Belt. The process could use advice from people who know statistics.
PW at one time suggested maybe throwing out high and low judges for the finals, but the discussion didn't get very far, probably because of what Mike showed above. Sounds like a bad idea to me, but as Sparky says, other sports do it. A little research might show why they do. Then, armed with that information, we could see if it applies to our situation.
I think it is done on a per maneuver attempt in large sporting events but I am not sure. If you did it that way I am sure it would not be the same judges score thrown out all the time. Makes me no difference how it is decided to be done. That said with the more I monitor this thread I think the idea of showing the scores by each judge for each flight is a even worse Idea than I had originally thought.
-
Didn't Dave Cook suggest tossing the high and low score PER MANEUVER at one time? I thought that it had merit, reather than tossing the high and low score?
Based on what Mike shows above, I can't see that it has merit, but lets see some analysis.
-
Oh my GAWD, Brett... get a hold of yourself. Settle down.
I do not want to *start a war*. The *war is won* as far as I am concerned. I think what Paul Walker and Bill Rich have been doing with the Nats is just great. I am just discussing what I would like to see. This is still America. I thought we could still do that....
Sure you can, and you can expect someone else to discuss what THEY want to see, too. Who said you couldn't say it? I found it quite illuminating.
And once again, you said the magic words yourself - "maybe those judges *should have been run off*" in response to my comments about NATs judges in the 90's. Anybody who thinks that was in any way justified has taken themselves out of the "rational analysis" realm and into the "harassment is just fine" realm. If that's not what you meant, it was what you typed, and there is an "edit" function.
The fact remains that there was a campaign of harassment against judges and the organizers based on the results of one analysis of one contest that resulted in several people getting fed up and refusing to judge anymore, and several others being threatened with physical violence. That's not a point of debate, it's not an opinion, that actually happened. It's still happening, on a low level and on occasion. I for one find that appalling, I think any reasonable person would, and one of the primary responsibilities I think we have as a group (stunt, PAMPA, however you want to do it) is to prevent that from happening again.
It's my opinion, strongly reinforced by this thread and the small amount of other information I have, the same thing would happen again if the data got released, at least without obscuring the mapping to pilot and judge to the point that the data became completely meaningless.
It's also my opinion that any such analysis is patently pointless by it's nature. As I mentioned to several in 1994, there are no "facts" associated with judges scores. It's someone's opinion. Any attempt to glean "fact" by analysis is therefore doomed to failure. Any "outlier" could be the result of intentional bias, unintentional bias, bad but honest judging, or a genuinely different opinion. The methodology Paul and Howard came up with (essentially an automated version of what Warren/Shareen did for years, and were regularly pilloried for) will probably detect grossly incompetent judges in some cases, assuming there were any. And now that it is completely automated nobody can plausibly claim that it is being done to favor or harm anyone. There's absolutely no way to distinguish the other three things afterwards, there is no math that can tell the difference, it's inherently not knowable. So any attempt to figure it out is doomed by definition.
So, once again, I come to the conclusion that there is virtually no good that can come from publishing sufficient data to do an analysis, and plenty of documented evidence that such data was grossly misused to the severe detriment of the event in the past, and no reason to think it might be different this time around. Like I said, a no-brainer, not even a close call.
I don't expect you to agree, but that's my case. Tear it apart if you want/can. I think you need to show that the potential advantages outweigh the potential for abuse and I think that bar is going to be VERY high to convince me or anyone else.
Brett
-
" I think what Paul Walker and Bill Rich have been doing with the Nats is just great. "
What has Bill Rich done?
-
Based on what Mike shows above, I can't see that it has merit, but lets see some analysis.
I did that back in the day. What it does is select for a narrow scoring range, even if you normalize it first. If you don't normalize it first, it selects for narrow scoring range AND for how close the average scores are to each other. It's like codifying "bracket scoring".
I think it's quite clear that ANY scheme that you might come up with would first have to normalize the scores, meaning it has to happen after all the scores were available, meaning you couldn't post the scores until the round was complete. That's not necessarily a show-stopper but something that would have to be considered.
Brett
-
All of these thoughts have some merit...but would not sway the results...
The real issue is that different regions produce different interpretations of the rule book, and fliers from each region exemplify the interpretation of that region..I would believe that the early results (when Navy judges were used) might reflect a more realistic view of who flew the best...
-
An Old Rancher's Advice:
* Your fences need to be horse-high, pig-tight, and bull-strong.
* Keep skunks, bankers (and generally lawyers) at a distance.
* Life is simpler when you plow around the stump.
* A bumble bee is considerably faster than a John Deere tractor.
* Words that soak into your ears are whispered...not yelled.
* Meanness don't jes' happen overnight.
* Forgive your enemies. It messes up their heads.
* Do not corner something that you know is meaner than you.
* It don't take a very big person to carry a grudge.
* You cannot unsay a cruel word.
* Every path has a few puddles.
* When you wallow with pigs, expect to get dirty.
* The best sermons are lived, not preached.
* Most of the stuff people worry about ain't never gonna happen anyway.
* Don't judge folks by their relatives.
* Remember that silence is sometimes the best answer.
* Live a good, honorable life. Then when you get older and think back, you'll enjoy it a second time.
* Don't interfere with somethin' that ain't botherin' you none.
* Timing has a lot to do with the outcome of a rain dance.
* If you find yourself in a hole, the first thing to do is stop diggin'.
* Sometimes you get, and sometimes you get got.
* The biggest troublemaker you'll ever have to deal with, watches you from the mirror every mornin'.
* Always drink upstream from the herd.
* Good judgment comes from experience, and a lotta that comes from bad judgment.
* Lettin' the cat outta the bag is a whole lot easier than puttin' it back in.
* If you get to thinkin' you're a person of some influence, try orderin' somebody else's dog around.
* Live simply. Love generously. Care deeply. Speak kindly.
-
" I think what Paul Walker and Bill Rich have been doing with the Nats is just great. "
What has Bill Rich done?
Nothing yet, I suppose.
-
I don't expect you to agree, but that's my case. Tear it apart if you want/can. I think you need to show that the potential advantages outweigh the potential for abuse and I think that bar is going to be VERY high to convince me or anyone else.
Fine. That is your opinion, I am not going to fault you for it.
Stop making everything into a "war".
-
Didn't Dave Cook suggest tossing the high and low score PER MANEUVER at one time? I thought that it had merit, reather than tossing the high and low score?
But if you use the scores I posted earlier, that still means you keep the guys who score every single manuever a 33 or 34 for most of the scoring. "Per manuever" doesn't necessarily mean you're keeping more accurate scoring, either.
Brett said it best, I think - you'd really need to normalize the scores, which couldn't be done until all the flights for a given round on a given circle were completed. That'd certainly be different. And I'd really need to run an analysis of many rounds of competition to see what difference (if any) it would make.
-
OK,We'll try this just one more time. If you have something intelligent to contribute to Howards thread, feel free to proceed in an civil manner. If personal attacks are on your agenda,and you still think you are intelligent, do not post. It is unfair to Howard and everyone else who are truly interested.
-
Just to be sure I understand this correctly, regardless of whatever computer information is or isn't released, the individual pilot can still get the copies of his scores, right?
-
Yes. I specifically asked the event director. He said, "Yes, the individual score sheets will be available as we discussed. Yes, the average score for each flight will go on the board. Has to!" He doesn't want to release individual judge's information for all the fliers, though.
Thanks for letting this continue, Richard.
-
Just to be sure I understand this correctly, regardless of whatever computer information is or isn't released, the individual pilot can still get the copies of his scores, right?
Hi Steve
The scores for the individual flyer has alway been given to the flyer, and will also be there for the flyers to pickup this year also.
Regards
Randy
-
In the end and in reality isn't this all supposed be covered at the judges straining sessions at the NATS? At the end of the session isn't the faults of the training flight brought to light? Also the people that do the training flights are they trying for the perfect flight or their best flight? In my opinion the training pilot should throw in a not so par maneuver to see if the judges are scoring accordingly or not. Also do the training pilots come from the Intermediate class? I hope they don't come from the group that is going to be judged.
In my opinion the current and past Stunt Event Directors have done their best with what they had. How many whiners and gripers have stepped forward to help out?
Now you know why I don't do the NATS event directing anymore. Of course I have never tried running stunt, combat or speed. R%%%% DOC Holliday ???
-
Maybe the seeding list *could* be made into a national ranking, and then it can be used on an ongoing basis.
The problem with this is that the only people on the list are those who enter the AMA Nats. No doubt, there are at least a few good flyers who do not chose the spend a week at Muncie and pay a rather high entry fee ten years in a row to move up on the totem pole.
It's a valid Nats seeding list, but that's all it is.
-
The problem with this is that the only people on the list are those who enter the AMA Nats. No doubt, there are at least a few good flyers who do not chose the spend a week at Muncie and pay a rather high entry fee ten years in a row to move up on the totem pole.
It's a valid Nats seeding list, but that's all it is.
The question is, how would you use a National Ranking list, if one existed. Even if it was perfect - what would be the end goal or use for such a thing? Other than trying to figure out who went into Advanced or Intermediate (which, for the most part, is already taken care of), it doesn't leap out at me how it would matter. It's not like we are "seeding" anything in the sense that the higher-ranked fliers get an easier path, local or National, and I can't imagine that we would ever do that. Until you answer that question, I don't see how or even why you would bother with such a thing, aside from simply feeding one's ego.
I want #800 to get the same chance and have jump through the same hoops as #1. If that's the plan, who cares where you rank.
Alternately, if you aren't using it for anything, then it doesn't matter whether the ranking is right or not, so use whatever criteria you want.
Brett
-
In the end and in reality isn't this all supposed be covered at the judges straining sessions at the NATS? At the end of the session isn't the faults of the training flight brought to light? Also the people that do the training flights are they trying for the perfect flight or their best flight? In my opinion the training pilot should throw in a not so par maneuver to see if the judges are scoring accordingly or not. Also do the training pilots come from the Intermediate class? I hope they don't come from the group that is going to be judged.
The two Nats training sessions I attended (2006 & 2008) seemed to focus mostly on rules and interpretations. The real value of judging at the Nats is the total immersion into the process. When you are out there judging 6-8 hours (and more sometimes!) for 7-8 days with a variety of other judges and a broad spectrum of flyers and abilities, it is like graduate school. As Randi Gifford told me, "You judge... you eat and you sleep... and you judge...!!! The practical side of judging a practice flight was generally limited to a warm-up session each day. Don't worry about throwing the judges a ringer, Doc. There are enough unintendeded "not so par" maneuvers to keep the judges on their toes.
Most of the judges I have worked with seemed honest and were trying to do a good job. In local contests, I am often paired with a flyer who has been drafted to judge. I can't think of one who did not give it his or her best shot. Some are hard and unforgiving. Some are generous. So are "Nats-Trained" judges. Contrary to popular belief, most judges are pulling for you. We have been standing out there in the sun (or rain!) all day. One of the things that makes it worthwhile is to see a beautifully executed set of triangles or a square eight with real corners and no bounce in between. I love to see 40 point maneuvers!
I have been judging only four years locally, regionally and nationally. I have a long way to go before I will be very good at it. I know how to be objective and I believe I am. I know how to be honest and I believe I am. I am learning more and more about what a maneuver should look like. I was taught from the beginning that 40 was the maximum score and zero was the bottom. Judges can and should use any number in between to describe the maneuver he or she has just seen. I think I have used most of the numbers. Like every judge (and flyer!), I have seen some really gorgeous maneuvers! And, like every judge (and flyer!), I have struggled to recognise enough elements to define the intended stunt.
I don't know if Paul Walker's Seeding or Judge's Tracking works but I applaud him and Howard for trying to find a structure.
bill marvel