2013 Nats coverage!

[Ted Fancher]

One thing I forgot to mention earlier was the fact that it was my birthday last Saturday. Nice present wasn't it?

I'm surprised Ted didn't mention it!

Shoot, Paul.  Sorry you mentioned it.  I wasn't going to tell everyone I gave you one extra point per candle on the cake.


JUUUUUUUST KIDDING!!!!!!!!!!!! 

Happy Birthday, Paul.  Mine was on the next day and I celebrated it waiting for delayed flights to get us home.

Congratulations, trophy hog!    Well done.

Ted

[Crist Rigotti]

snip<Mine was on the next day and I celebrated it waiting for delayed flights to get us home.>
Ted

You know what they say "Time to spare, go by air"!   

[MarcusCordeiro]

Oh how right you are my friend, how right you are......

I "third" that...

Marcus

[Howard Rush]

I was disappointed to hear that on Monday before the the NATS even began, you were already upset about the prejudice of circle distribution system and the you felt that the outcome was predetermined to not include you.

Any circle led by the 7th-best guy at the contest can't be very tough.

[Randy Powell]

I don't know how many flights I've had that after looking at the score sheets, I wondered if they were watching the same flight I was. Best advise ever given to me (by Ted Fancher) was my job was to fly, the judges job was to judge. Just go fly the best pattern you can. You can ask guys like Ted or Paul to watch and critique. I do that every chance I get because they are both a lot better than me and usually have good comments to make. The judges judge. If you don't like it, try asking what they saw that they didn't like and where you can improve. They are the ones making the judgment. As a pilot, it's my job to figure out how to give them what they want, not what I want.

[Robert Redmon]

Everyone contributing to this discussion of judging is guilty of oversimplifying the relationship between perception and scoring. This applies to both a flyer's self perception of performance and the judges' perception of observed phenomena. Both are influenced by expectation and various forms of pre-conception.....and this cannot be avoided (as long as real people are doing the perceiving). Subjective adjudication of performance will always be colored by an infinite number of variables not directly tied to what is observed. It is what it is. Even if we devoted months or years to professional level judge training (debriefing/deconstructing perceptions, judge leveling, etc), scores would still reflect these nudges and often still defy logic. FWIW

Bob

[Doug Moon]

Robert,

I think you should listen to folk that are genuinely wishing you(or any other fellow pilot) to improve. I was disappointed to hear that on Monday before the the NATS even began, you were already upset about the prejudice of circle distribution system and the you felt that the outcome was predetermined to not include you. You are better off putting blinders on during that time and focusing on improving and putting up your personal best. Such pessimistic attitude may have prevented you from flying your best(unless you feed off of that emotion to help you compete).

Derek was spot on on his comments about your flying. I did not have an opportunity to judge your flight but I did steal a few glimpses of your flying in between flights. You looked high and big(as in above 60 degrees big).

You mentioned that you have only recently built your model and the rains prevented you from being able to practice. Do you think that the pilots that have flown their models for more than a few months do not have a potential advantage over you!? Of course they do! Do yo think the folks that had good practice weather this spring do not have a potential advantage over you? Of course they do! How do you expect to beat so many good pilots when they have so many things working in their favor.

If you think the pilots are being bracketed, my judging record should COMPLETELY disprove that(feel free to request ALL of my score sheets from the NATS, I dare you!). I came to the NATS having never seen 90% of pilots that entered. The only ones I've seen fly before are Buddy Weider, Rich Giacabone and Jose Modesto and even those guys I have not seen fly in two years( I wish I got to hang out with you guys more often: you're a hoot and a half!)! I was as "virgin" and as "unbiased" NATS judge as they come. The difference between me and the "college students" is the fact that I dedicated the last 3 years of my CL stunt flying career to becoming a better judge. I did not fly in a single contest and judged every contest I could get to. I must've read judges guide a gazillion times. I listened to David Cook: one of the best coaches and judges around(and he flies one MEAN pattern albeit the wrong way ).

Jose,

You told me when we saw each other at the AMA headquarters on Monday, that you were flying your new model and that the model was not finished with trimming. That would also indicate that you did not have enough practice(how can you prepare for the NATS yourself if the model is not ready). Now compare it with the rest of the top 20 guys. Many of them have been flying their models for more than one season(Frank McMillan was flying a 5 year old model!) which means they are a lot more familiar with the airframe.


On Judging:

Most of you don't realize that judges at the NATS are all genuinely interested in providing the best judging possible.
Have you seen any of us mingling with pilots in the pagoda or look at the scoreboards? Ever wonder why? Remove a possibility of subconscious bias or "bracketing" by seeing who scored how.
Have you EVER wondered why judges almost ALWAYS ate by themselves? We were trying to avoid being influence by pilots comments!
Have you ever wondered why judges are required to watch the warmup flights and then explain their scores to others? This is to make sure that EVERY JUDGE is judging to the rules. It turns out, EVERY judge was judging based on the same criteria. The difference was in how much each one of us was deducting for errors.
Judges are pilots too! We know it sucks when you are scored based on a bias towards you or your competitor. It's the sinking feeling of unfairness that keeps us from NEVER wanting to be biased towards ANY pilot.


On Low scoring:

By now everyone has figured out who the low scoring judge was. Before getting upset about how low I scored your flight, I suggest you take a look at other pilots score sheets. Jose, take a look at Paul Walker's Friday's first round flight scoresheet and compare it to yours! You'll stop talking about "halo" effect or predisposition of scoring "elites" higher than the rest.
Jose, I consider you a friend and I would feel like sh$*t if I was not fair to you in scoring. I was genuinely rooting for you but you gave up points on every maneuver. Your second flight was and order of magnitude better than the first one! You looked like Paul Walker: instead of getting upset about the bad flight, you regrouped and were kicking butt. It was not your ability to step up and put up a great flight that kept you from the top 20. It was the equipment.

Good post!

FWIW, I flew in front of you and you scored me lower than the other judges.  I wondered what was going on.  That was an early round qualifier.  Later I looked over the scores again and began to remember what was going on and sure enough there was some stuff in there I could work on.  I should have known by now it was me.  I tend to get better as the week goes along.  Sure enough on Friday I wanted to get a real smooth clean flight right out of the gun.  You scored me well for it. It was a much better flight in my opinion too.  We were seeing the same things.  Friday was a tough one with the steady winds.  I knew if I could keep it clean and smooth in the junk all would work out.  And so it did.

Thank you and Ted and all the judges for working the contest!!  Please come back!!

[billbyles]

Everyone contributing to this discussion of judging is guilty of oversimplifying the relationship between perception and scoring. This applies to both a flyer's self perception of performance and the judges' perception of observed phenomena. Both are influenced by expectation and various forms of pre-conception.....and this cannot be avoided (as long as real people are doing the perceiving). Subjective adjudication of performance will always be colored by an infinite number of variables not directly tied to what is observed. It is what it is. Even if we devoted months or years to professional level judge training (debriefing/deconstructing perceptions, judge leveling, etc), scores would still reflect these nudges and often still defy logic. FWIW

Bob

So Bob, is there a suggestion as to how to improve this judging/self-perception thing buried in your post somewhere?  Are you including yourself in the "Everyone contributing to this discussion of judging..." sentence of yours?

[Robert Redmon]

Bill,

What I am saying is that we (as a community) are already doing all that can reasonably be done. We flyers (usually) read the rules and (sometimes) benefit from the feedback we receive from judges and fellows. We practice and refine our equipment to satisfy what we believe to be necessary to create the impression that the patterns we fly are congruent with the rules and the expectations of others. Judges already do their best to score these patterns in ways consistent with these rules (as they understand them) and what they believe to be the expectations of the community they serve. My point is that both the complaining of the losers and indignation of the winners reflected in the latter portion of this thread are counter productive and simply amplify the confusion and rancor. Nothing in this discussion is unique to CLPA; it is present in any form of judged competition (at any level). As to whether I am including myself in this discussion of judging, the answer is rather obvious. Can we "fix" this system? Only if we can eliminate people from the equation. Can we improve it? Only by redoubling our already considerable effort to do the best we know how.

Bob

[Doug Moon]

.....Low judge even wrote comments on my score sheet, a first for me,15 national 6 team trials.
I did not include the judges names
Jose Modesto

I was thinking about this some today.  Just my thoughts but I dont really think this is a best practice at the nats. If there are notes on some scheets and not others it could be seen as helping some but not others.  Just thinking out loud....

[steven yampolsky]

I was thinking about this some today.  Just my thoughts but I dont really think this is a best practice at the nats. If there are notes on some scheets and not others it could be seen as helping some but not others.  Just thinking out loud....

I put comments on pretty much EVERY pilot's sheets but only in round 1 of qualifying and only if they had VERY obvious errors that propagated the entire pattern. My apologies but it was VERY hard to find anything wrong with your pattern as whole.

If you feel that comments should not be put on score sheets, that's ok too. Just talk to ED and they will instruct the head judge. At the Judges meeting, we discussed putting comments on forms and the general consensus was that it was up to each judge to decide if they want to put comments on forms or not.

[Doug Moon]

I put comments on pretty much EVERY pilot's sheets but only in round 1 of qualifying and only if they had VERY obvious errors that propagated the entire pattern. My apologies but it was VERY hard to find anything wrong with your pattern as whole.

Thanks!! 

If you feel that comments should not be put on score sheets, that's ok too. Just talk to ED and they will instruct the head judge. At the Judges meeting, we discussed putting comments on forms and the general consensus was that it was up to each judge to decide if they want to put comments on forms or not.

Now that I know that information I will no longer think it is a problem.  Thanks for letting us know it was discussed and how the decision made. 

[Ted Fancher]

snip
If you feel that comments should not be put on score sheets, that's ok too. Just talk to ED and they will instruct the head judge. At the Judges meeting, we discussed putting comments on forms and the general consensus was that it was up to each judge to decide if they want to put comments on forms or not.

Steve,

I agree the subject should have been discussed at the judges' sessions but don't recall that it was this year.  It is my strong feeling (and I'm reasonably sure that of Mark Overmeier) that if it had been I would have vehemently opposed it for just the reasons Derek (ooops, Doug!) mentioned.  Not fair to all the competitors plus it takes the judge's attention off judging and directs it to critiquing.

It "Might' be justifiable on the last flight of a contest but certainly not prior to that point.

Ted

[steven yampolsky]

I agree the subject should have been discussed at the judges' sessions but don't recall that it was this year.  It is my strong feeling (and I'm reasonably sure that of Mark Overmeier) that if it had been I would have vehemently opposed it for just the reasons Derek (ooops, Doug!) mentioned.  Not fair to all the competitors plus it takes the judge's attention off judging and directs it to critiquing.

The context was: judges should put down notes on why they scored the pattern the way they did in case a pilots challenges the score.

[Brett Buck]

I put comments on pretty much EVERY pilot's sheets but only in round 1 of qualifying and only if they had VERY obvious errors that propagated the entire pattern. My apologies but it was VERY hard to find anything wrong with your pattern as whole.

If you feel that comments should not be put on score sheets, that's ok too. Just talk to ED and they will instruct the head judge. At the Judges meeting, we discussed putting comments on forms and the general consensus was that it was up to each judge to decide if they want to put comments on forms or not.

   I understand the temptation and it's certainly open to debate, but think it's not a good idea to put notes on the score sheet in this context. I do it in the second round of local contest (since you don't want to coach people in a way that they can take advantage of during the contest). but here, you never know until Saturday afternoon  when anyone's last flight will be, and they aren't going to need any coaching.

    Brett

[Jorge de Azevedo]

Everyone contributing to this discussion of judging is guilty of oversimplifying the relationship between perception and scoring. This applies to both a flyer's self perception of performance and the judges' perception of observed phenomena. Both are influenced by expectation and various forms of pre-conception.....and this cannot be avoided (as long as real people are doing the perceiving). Subjective adjudication of performance will always be colored by an infinite number of variables not directly tied to what is observed. It is what it is. Even if we devoted months or years to professional level judge training (debriefing/deconstructing perceptions, judge leveling, etc), scores would still reflect these nudges and often still defy logic. FWIW

Bob

Researchers use statistical resources to minimize the "observer bias in subjective evaluations", such biases are intentional or not. For this it is necessary to fully understand these biases and statistical resources available, like the best choice of measure of central tendency to determine the notes of maneuvers based on the score from the judges.

[jose modesto]

On Low scoring:

By now everyone has figured out who the low scoring judge was. Before getting upset about how low I scored your flight, I suggest you take a look at other pilots score sheets. Jose, take a look at Paul Walker's Friday's first round flight scoresheet and compare it to yours! You'll stop talking about "halo" effect or predisposition of scoring "elites" higher than the rest.
Jose, I consider you a friend and I would feel like sh$*t if I was not fair to you in scoring. I was genuinely rooting for you but you gave up points on every maneuver. Your second flight was and order of magnitude better than the first one! You looked like Paul Walker: instead of getting upset about the bad flight, you regrouped and were kicking butt. It was not your ability to step up and put up a great flight that kept you from the top 20. It was the equipment
    Re: 2013 Nat's coverage!
« Reply #498 on: July 29, 2013, 11:27:38 AM »    Quote  Modify  Remove

________________________________________
Posting my scores from first round qualifying at 2013 Nat's
the score sheets clearly show the disparity in scoring and illustrate with actual numbers the discussion above.
After first round flight I had Doug's response WTF to do now. Fly for the judges that scored me high or modify to please low judge.
i was down 10 points after one flight. the spread are 96 and 70 Pt's between high,mid and low judge.
Low judge even wrote comments on my score sheet, a first for me,15 national 6 team trials.
I did not include the judges names
Jose Modesto
Re: 2013 Nats coverage!
« Reply #500 on: July 29, 2013, 12:07:53 PM »    Quote  Modify  Remove

________________________________________
The comments were on my first flight of first day. Second flight has a story behind it. Look for it in electric section soon. Others flyers received notes for both flights.
Hey missed you at Nat's this year. Hope to c u soon. The Texas boys did a great job filling in for you.
Jose Modesto
2013 Supporter
Trade Count: (0)
Captain
 

Rate User 17
 Online

Posts: 591


          Re: 2013 Nats coverage!
« Reply #504 on: July 29, 2013, 02:22:20 PM »    Quote  Modify  Remove

________________________________________
Posted to show numerically the discussion started by Ted F. Doug posted a letter grade scoring system were my scores can be converted to letter grade.
I know that the low score were not a personal bias but a pattern  interpretation difference. Another flyer in my group scored 5 points lower from low judge yet average 5 points higher than my average score. His disparity in points were even larger than my score.
Jose Modesto

Steve. You have attributed to me words in quote that were never said by me.Above are ALL  MY POSTING ON THIS SUBJECT.   NONE OF THE WORDS THAT YOU ASSIGN TO ME IN QUOTES ARE IN MY POSTINGS. Please correct your post and retract your false assertions of my views by your use of words in quote.
I must also correct two other FALSE assertions.
1)You posted that I had never made the top 20(electric forum)  false 2010 Even Led a round as an UN ranked circle fill in.
2) That my motor overheated on second flight FALSE I set timer to 2 min. thus when the motor gave me the shut off warning I waved off the rest of the Flight. Then I landed. The chill that I felt when I heard the cutoff warning and realizing that I dint reset the timer. (there is a story behind this) MY GRAVE ERROR, your first round low score would be meaningless with out my blunder. Your assertion that my equipment prevented me from reaching the qualifying round is NOT CORRECT.
After first round and comparing scores with guys in my group,I understood that you were really low scoring across the board. (all flyers) See My underlined comments above

Your scoring methods came into question by fellow judges. Mr. Fancher also in this thread questions not YOUR ABILITY TO SEE ERRORS BUT THE VALUE YOU ASSIGHNED TO THE ERRORS. Your scoring was overly harsh to minor errors.
One Area that you mentioned can't be questioned,NATS PREPARATION.  You can't go to Nat's with hopes of winning without extreme preparation. Flew model for fist time morning of appearance judging.
I WATCHED Every flight of the top Five and I cant imaging how to separate the #1 and 2 pilots. The styles are diametrically opposed Doug slow with the stop and go corner beautiful ,aggressive,small, smooth ballet.
Then Paul Aggressive ,Faster ,Harder with blinding corners MMA at 70 feet.
The finals judges also had a hard time separating those two by the final score.
I would like to express my thanks to all those guys and gals that VOLANTEER so that I can compete in the Best US contest at the best site to fly CLPA.
With all the respect in the world I Thank you guys.
Back to Steve you must at some level ask your self  was I overly aggressive in my scoring in Comparison to extremely season NATS level judges. Only you can answer.

You and I must Meet at the end of  70 of wire and resume this discussion while doing the tricks that make this so enjoyable.
Jose Modesto

[Brett Buck]

Researchers use statistical resources to minimize the "observer bias in subjective evaluations", such biases are intentional or not. For this it is necessary to fully understand these biases and statistical resources available, like the best choice of measure of central tendency to determine the notes of maneuvers based on the score from the judges.

   There is no valid mathematical method for determining whether a bias is due to improper, intentional attempts to rig the scores, or the bias is merely a difference of error weighting or preference for a particular flight. The former is cheating, the latter is exactly what we are asking the judges to do. There are some good example cases in the SSW archives.

    We ask the judges for their opinions, and they give them.

    Brett

[Bob Whitely]

If one is not prepared for a contest, especially the Nat's, one should not cry
because one did not win. As for the one that had a problem with his takeoff
score  not as high as was expected he would do well listening to those that get
good scores and really, really get a good coach. I have flown against him thought
that with a little more practice and coaching should make the top 20 eventually.
Also ever notice that when all is said and done that the judges almost always get
the winners and losers in the correct order? As they say, "This ain't rocket science!"
RJ

[Howard Rush]

Speaking of takeoffs, Mr. Whitely, thanks for the takeoff and landing advice you gave me in Madera.  It works.

[Bob Whitely]

I don't remember what I said but you're welcome.  RJ

[Jorge de Azevedo]

   There is no valid mathematical method for determining whether a bias is due to improper, intentional attempts to rig the scores, or the bias is merely a difference of error weighting or preference for a particular flight. The former is cheating, the latter is exactly what we are asking the judges to do. There are some good example cases in the SSW archives.

    We ask the judges for their opinions, and they give them.

    Brett

With respect to the calculation of the final score for each maneuver, taking into account the scores of all the judges:
It is not necessary to find out whether or not a NAT judge is giving wrong note. Certainly there is no problem with the quality of the judges nor the pilots. Resources mathematical / statistical to which I refer is not concerned with these details. The problem is to calculate the scores for each maneuver, the “average = arithmetic mean” is being used and this mathematical procedure makes room for assumptions (most of the time on unfair behavior of judges), as each note of the particular judge has great influence on average final value. If you were using the “median” to calculate the final score, when there extreme values (the statistical point of view, does not mean they are wrong values) have little or no influence on the final score of the maneuver. Would be a protective procedure of the activity of judges and the inter-personal relationships judges / pilots. It would be advisable to study more deeply the virtues and limitations and applications of the "average = mean" and the "median" to better understand what I'm trying to share. The "mean" is a mathematical procedure / statistical vastly used when you want to summarize in just a number (final score of maneuver), a series of other numbers (value from each judge), but not everyone knows fully the strengths and limitations for using this procedure mathematician / statistician.
Doing trials comparing the use of mean and median, will see that when there is no outlier in the series (series of notes for the same maneuver) the value of the mean and the median is exactly the same. However, if any extreme value, the median value is different.
To evaluate the performance of judges there are other procedures required.
PS: I'm Brazilian and never attended NATS, but I'm interested in sharing my personal opinions and my notions of descriptive statistics in this forum (I think that's what the Control Line FĂłrum is) because North Americans are trendsetters in the F2B on the American continent  and elsewhere.

[Brett Buck]

Doing trials comparing the use of mean and median, will see that when there is no outlier in the series (series of notes for the same maneuver) the value of the mean and the median is exactly the same. However, if any extreme value, the median value is different.
To evaluate the performance of judges there are other procedures required.
PS: I'm Brazilian and never attended NATS, but I'm interested in sharing my personal opinions and my notions of descriptive statistics in this forum (I think that's what the Control Line FĂłrum is) because North Americans are trendsetters in the F2B on the American continent  and elsewhere.

   I think we ought to start a different thread for this, but this is not a statistical problem in the first place. We aren't sampling a random distribution around a "true" mean, we are taking the entire population of data, all available information is used (if you do it right, anyway). The statistics approach is treating it like political exit polling, but in fact we are directly counting all the votes. You don't need statistics to count the vote. See this thread:
http://www.clstunt.com/htdocs/dc/dcboard.php?az=show_topic&forum=103&topic_id=114598&mesg_id=114598&listing_type=search#114611


     Brett

[steven yampolsky]

You and I must Meet at the end of  70 of wire and resume this discussion while doing the tricks that make this so enjoyable.

Absolutely! I am trying to find lodging for the Bill Hummel's wonderful contest. Hope to see you there!

[Howard Rush]

Pardon me if I steer this thread back to the subject of the Nats.  I just received some pictures taken by a guy I worked with who lives in Dayton.  He came up to Muncie to visit.  Pretty good pictures for a guy completely unfamiliar with stunt:  https://www.dropbox.com/l/9H8xtuWXDLQMREDg2L5VKb

[jose modesto]

Steve please correct your post and remove the quoted words I did not use. By using the erroneous words you distort  my view of the subject discussed.
No contest for me in august other than team trials. Come to team trials it's only the weekend. With your plane (real)I can lend you a  complete take apart Yatsenko model. They are easy to transport. Would you want a gas or electric model. Or a hybrid that can use either mode of power.
Made deal with wife that I would go sailing the month of august in exchange for the 2 weeks for NATS
Please correct the miss applied quoted words that you assigned to me.
"Halo" and " elite" were never used by me.
I copied my post on this subject and at no time did I take the tact that you have applied to me.
And just to repeat. MY EQUIPMENT WORKED FLAWLESSLY for all rounds.
Please correct your posting.that is a greater problem than your low scoring. At least the score was based on facts.
Steve there is Hunterville in October that is also a great contest with numerous top 20 pilots in attendance.
Jose Modesto

[Brett Buck]

Pardon me if I steer this thread back to the subject of the Nats.  I just received some pictures taken by a guy I worked with who lives in Dayton.  He came up to Muncie to visit.  Pretty good pictures for a guy completely unfamiliar with stunt:  https://www.dropbox.com/l/9H8xtuWXDLQMREDg2L5VKb

     Pretty good indeed. Interesting to see what people see when they don't know anything about it ahead of time, or have pre-conceived notions.

    On the topic of drift, if ever there was a thread that should be split, this is it. The NATs reporting is the real thread, the rest of it should be split off (and IMHO, not locked, although I understand the temptation).

     Brett

[Dave_Trible]

Agreed.......nice to leave it as a historical reference.  (Clean)

Dave

[john e. holliday]

Tell the guy he did a great job on the pictures.   I know most of those people he took shots of.  Also Howard you shouldn't smile so much as it is rubbing off on the young lad.

[steven yampolsky]

Steve please correct your post and remove the quoted words I did not use. By using the erroneous words you distort  my view of the subject discussed.
No contest for me in august other than team trials. Come to team trials it's only the weekend. With your plane (real)I can lend you a  complete take apart Yatsenko model. They are easy to transport. Would you want a gas or electric model. Or a hybrid that can use either mode of power.
Made deal with wife that I would go sailing the month of august in exchange for the 2 weeks for NATS
Please correct the miss applied quoted words that you assigned to me.
"Halo" and " elite" were never used by me.
I copied my post on this subject and at no time did I take the tact that you have applied to me.
And just to repeat. MY EQUIPMENT WORKED FLAWLESSLY for all rounds.
Please correct your posting.that is a greater problem than your low scoring. At least the score was based on facts.
Steve there is Hunterville in October that is also a great contest with numerous top 20 pilots in attendance.
Jose Modesto

Done!