News:


  • March 29, 2024, 12:34:37 AM

Login with username, password and session length

Author Topic: Scoring Machine  (Read 2804 times)

Offline frank williams

  • 24 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 823
Scoring Machine
« on: July 17, 2018, 06:54:38 AM »
These pictures were from the WC site.  I assume they are for F2B.  Whether the official scoring or not I don't know.
Interesting.

Offline James Holford

  • 2020 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 1515
Re: Scoring Machine
« Reply #1 on: July 17, 2018, 07:07:30 AM »
Hmmm thats interesting....

Sent from my SAMSUNG-SM-G891A using Tapatalk

Jamie Holford
Baton Rouge Bi-Liners
Lafayette, La
AMA #1126767

Offline dave siegler

  • 2016 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 1128
  • sport flier
    • Circlemasters Flying club
Re: Scoring Machine
« Reply #2 on: July 17, 2018, 08:13:21 AM »
yeah I used that when I was over at the Gee- Bee cup. 

The ones i used were in a different package and used an RF link to communicate to the judges tools. 

they had a computer that feed all the judges tablets (excel) with the name of the competitor and the maneuver that was next. 


it worked very well unless the system was out of sequence. 

The scores were sent at the end of each flight back to the computer. 
before you sent it back, you could check your scores before you sent them. 

Igor Burger probably has the details. 
Dave Siegler
NE9N extra class
AMA 720731
EAA 1231299 UAS Certificate Number FA39HY9ML7  Member of the Milwaukee Circlemasters. A Gold Leader Club for over 25 years!  http://www.circlemasters.com/

Offline jose modesto

  • 22 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 842
Re: Scoring Machine
« Reply #3 on: July 17, 2018, 10:27:08 AM »
Great system you land before you roll up lines you have your score.
I’m serious
Jose modesto
The scores come with a print out for you to check
You see all judges by maneuver and the percentage of the mean score per judge
You can see who loves you or NOT
THIS system can detect bias by flyer or country.
Really nice
The best part it is instant. The bad part no board to look at as all scores are given to individuals
Daily placing can be looked at online or the next day
And has I’m here at the worlds

Online Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13717
Re: Scoring Machine
« Reply #4 on: July 17, 2018, 10:59:22 AM »
Great system you land before you roll up lines you have your score.
I’m seriousn

   You mean like happened at the NATS for decades without any machine aside from a calculator? I will tell Shareen how much you appreciated her efforts.

You can see who loves you or NOT
THIS system can detect bias by flyer or country.

 How does it do that? How does the machine know how well a maneuver was executed? I think what you really mean is that it allows the mob and cheering sections to know which judge  try to intimidate and hound off the circle to get their desired result. It's certainly easier to sic a mob on an individual than it is to, say, learn to trim and set up engines.

    Don't say it wouldn't happen, because that is EXACTLY what happened when we did that  - just once. VERY FORTUNATELY, it didn't work and the practice was discontinued. the aftermath went on furiously for 20+ years and still lingers.

     Brett

Offline Derek Barry

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 2830
Re: Scoring Machine
« Reply #5 on: July 18, 2018, 03:07:56 PM »
   You mean like happened at the NATS for decades without any machine aside from a calculator?

     Brett

I love Shareen, but that's a bit of a stretch. You would be lucky if the runner got the scores to the tabulators by the time you got there to see the scoreboard, and they would be processing 4 circles worth every 8 minutes.

Derek

Offline Doug Moon

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2188
Re: Scoring Machine
« Reply #6 on: July 18, 2018, 03:13:43 PM »
   You mean like happened at the NATS for decades without any machine aside from a calculator? I will tell Shareen how much you appreciated her efforts.

 How does it do that? How does the machine know how well a maneuver was executed? I think what you really mean is that it allows the mob and cheering sections to know which judge  try to intimidate and hound off the circle to get their desired result. It's certainly easier to sic a mob on an individual than it is to, say, learn to trim and set up engines.

    Don't say it wouldn't happen, because that is EXACTLY what happened when we did that  - just once. VERY FORTUNATELY, it didn't work and the practice was discontinued. the aftermath went on furiously for 20+ years and still lingers.

     Brett

Wow! That esculated fast. Why? I saw absolutely nothing in his post to say he had a problem with past nats tabulation. He was simply saying what he was writnessing first hand at the contest he is attending right now....

I had no thought of past nats either about our tabulation until you brought it up... 

Plus this type of system is sorely needed at the worlds as I have read often later in the day competitors didn’t know their scores until the next day. Sounds like a vast improvement. But somehow mention of it is a slight to our system and those who run it. Really??
Doug Moon
AMA 496454
Dougmoon12@yahoo.com

Offline peabody

  • 23 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2866
Re: Scoring Machine
« Reply #7 on: July 18, 2018, 05:40:25 PM »
Around 1990 Scott Smith and the NY gang had a contest where the experts were flying with FAI scoring.
Dave Cook dragged his Apple computer, a monitor and generator and used a program that did the math and gave us results.


For an old guy, Dave was pretty prophetic: we talked at the 1999 Nats about each judge having a Blackberry like device and beaming scores to a central machine and then back out to a printer at the flying site.


Who would have thought?


I miss Dave

Online Brett Buck

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 13717
Re: Scoring Machine
« Reply #8 on: July 18, 2018, 05:58:59 PM »

Plus this type of system is sorely needed at the worlds as I have read often later in the day competitors didn’t know their scores until the next day. Sounds like a vast improvement. But somehow mention of it is a slight to our system and those who run it. Really??

   No, the problem is that he suggested:

Quote
You can see who loves you or NOT
THIS system can detect bias by flyer or country.

   which it most certainly can not. But I can see people chomping at the bit to get all the score sheets and all the judges scores, so they can know who to target. Which is EXACTLY WHAT HAPPENED IN 1993, leading to near disaster when it got in the hands of the wrong people.

    You weren't around when, say, Mary Gebhardt, was hounded out of NATs judging by whispering campaigns and hate mail accusing her of being a homer for Ted, and the endless nasty back-biting when people got the results of a single contest. What are they going to do with 6 days of NATS results? Run their absurd "bias algorithm" on it and conclude Judge X is cheating flier Y?

   Maybe it wouldn't happen again, with the current set of participants. But getting this information leave people WIDE OPEN to the same nonsense we just got rid of after 20+ years.  I expect some of the "wrong" people will shortly pop up again because they smell blood in the water.

   You might think it wouldn't happen. I hope it wouldn't. But I am not sure enough to once again declare open season on the judges.

   We can use the technology if available to speed up tabulating -  but "who loves you or not" and "detect bias"? That is EXACTLY what we cannot permit. We already did that and it caused decades of (entirely unjustified) strife. 

    Brett

Offline Ted Fancher

  • 23 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 2323
Re: Scoring Machine
« Reply #9 on: July 18, 2018, 06:28:32 PM »
Great system you land before you roll up lines you have your score.
I’m serious
Jose modesto
The scores come with a print out for you to check
You see all judges by maneuver and the percentage of the mean score per judge
You can see who loves you or NOT
THIS system can detect bias by flyer or country.
Really nice
The best part it is instant. The bad part no board to look at as all scores are given to individuals
Daily placing can be looked at online or the next day
And has I’m here at the worlds

Hi Jose,

thanks for the info and the pix.  I wasn't at all aware of the scoring machines.  Interesting development for sure.

Re your comments about the system.  Are all score sheets available for viewing by all pilots?  Or only given to the individual pilot for his/her flights?

Is the ability to detect bias limited to event officials who can review all of the scores by pilots and judges or is that data available to all?

Are the officials authorized and/or required to note and act on perceived bias?  Was that part of the reasons for implementing the system?

If so, how are they directed how to determine whether differences in scores are the result of national bias or simply a different interpretation of the quality of maneuvers?

I would really miss having the scoreboard to follow during the competition.  Can't imagine a stunt event where there aren't pilots clustered around the board calculating who, how much, when, etc.  I did, however, fly at WCs where the scoreboard postings were so tardy as to be disregarded as "planning" tools for your next flight!  Somehow, they managed to finish the competitions. :) :)

Re your last,  are you actually there at the WCs?  It wasn't entirely clear in your post.  If so, have a good time.  Your comments re the airplanes and flights would be welcomed by those of us who aren't there.

Ted Fancher

Offline Curare

  • 2014 Supporters
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 779
Re: Scoring Machine
« Reply #10 on: July 18, 2018, 06:47:08 PM »
Greg Kowalski
AUS 36694

Offline frank williams

  • 24 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 823
Re: Scoring Machine
« Reply #11 on: July 18, 2018, 07:10:42 PM »
Looks like about $200 each.  Doable for AMA.

Offline jose modesto

  • 22 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 842
Re: Scoring Machine
« Reply #12 on: July 18, 2018, 09:44:03 PM »
Brett I see the black helicopters are still flying over your house

I should of put a smiley face after who loves you NOT  I was joking

What’s missing here is a score board.
Ted only the individual flyer gets his score. Most flyers are kind to me and allow me to shoot photo of their scores. I send to my friends in NY. This allows them to get up to the minute scores
Hey Brett my point was that it was nice to land go to pits and a wonderful lady would walk up to you and hand you your score
1993 team trials was 25 years ago. All the guys that were involved in the controversy are DEAD or no longer flying
Just One Remains.
The remaining one is possibly the Best AMA flyer on the planet
You have to stop living in the past.

Your phones are not taped no black helicopters are tracking you

And yes I competed in the 1993 team trials
Lots of negativity accursed and a faction did dominate for a long time after ( you will take faction and run with it I know) not a negative word

What changed for us
First ping pong balls.  Next random circle and flying order
NOT NEGATIVE TO ANY WHO VOLUNTERED JUST AND OPEN WINDOW TO ALL

Jose modesto
« Last Edit: July 18, 2018, 10:13:59 PM by jose modesto »

Offline jose modesto

  • 22 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 842
Re: Scoring Machine
« Reply #13 on: July 18, 2018, 09:55:44 PM »
One additional piece of info on the score sheet
It shown what percentage an individual judge is above or below the mean average
One one circle a judge was 11+ % below mean on Orestes. The next 5 flyers allowed me to look at their score sheets and see if he was consistently below the mean. Yes he was. Not as large a percentage as Orestes but he was consitlently the low judge
At some Nat’s we wait up to one hour to get scores. Sometimes we fly our second flight with out mowing what we scored on our first
Again not negative but factual
NO DISRESPECT TO OUR VOLUNTEERS BRETT
We can learn something
I don’t know who sees the overall scores. What the administrators do with info  how it is used
I don’t know if a judge is evaluated per flying day

I’m not part of a vast conspiracie
Jose modesto

Offline jose modesto

  • 22 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 842
Re: Scoring Machine
« Reply #14 on: July 18, 2018, 10:08:30 PM »
One more piece of info
The entire Chinese team is flying Electric Yatsenko ready to fly models
Also the entire team made the finals
The Japanese team is a mix of home built and Yatsenko models. Their entire team made the finals
The Yatsenko brothers are here but not competing. Country politics.
The Chinese junior may be the next great FAI flyer in the world
Our team had a steep leaning curb. Plus 36 hour delay in arriving. I hope they share their story. Of perseverance through many Hurdels
My only advice to future teams take apart ELECTRIC MODELS   Gas has many obstacles to overcome
First it’s not your gas from home second investigate the site prior. Ie: elevation air density  field conditions
Richard was not able to really practice due to fuel issues. Can you imaging having to flush your system after every flight. Impossible to do. Yet he missed the cut by 37 points. Valiant effort. No quit  on his last flight his motor held its setting and allowed him to finally compete

Offline jose modesto

  • 22 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 842
Re: Scoring Machine
« Reply #15 on: July 18, 2018, 10:24:09 PM »
Ted I’m like you I miss the board. Here no one knows were they stand until the entire round is finished and is posted on line   
At the field they are posted the following day.
All flyers are blind to their placing and what their competition has scores
I have made friends with most flyers so I knew the placing as it accured
And an advantage of being multy lingual
The American team is at a disadvantage do to lack of language skills  specially ordering food,doing laundry. The basics  lol
BRETT AGIN NOT NEGATIVE TUNG IN CHEEK
Jose modesto


Offline Steve Helmick

  • AMA Member and supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 9920
Re: Scoring Machine
« Reply #16 on: July 19, 2018, 06:33:15 PM »
If this scoring machine system is so hot, how come they didn't get the results right the first try?  :o Steve
"The United States has become a place where professional athletes and entertainers are mistaken for people of importance." - Robert Heinlein

In 1944 18-20 year old's stormed beaches, and parachuted behind enemy lines to almost certain death.  In 2015 18-20 year old's need safe zones so people don't hurt their feelings.

Offline Curare

  • 2014 Supporters
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 779
Re: Scoring Machine
« Reply #17 on: July 19, 2018, 07:34:52 PM »
Has anyone got any idea what they ACTUAL problem was?
Greg Kowalski
AUS 36694

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7805
Re: Scoring Machine
« Reply #18 on: July 20, 2018, 05:20:06 PM »
We can learn something

One thing I learned, but haven't yet included in the program used at our team trials, is that we should print out for each flight the scores given for each maneuver by each judge.  The purpose of this is to allow a contestant to check that the tabulators entered his scores correctly without the contestant having to do all the "k"-factor multiplication.  The guys at the world champs this year and in 2014 did that.

I agree with Brett that printing the percentage that a judge is above or below the mean is a really bad idea.  It tells you nothing and can intimidate judges into scoring wrong.  If anybody wants to discuss judge evaluation formulae, I'm interested.  We have a criterion we use at the Nats-- it's available to use at the Nats, anyhow-- but I'm not convinced of its statistical validity. 
The Jive Combat Team
Making combat and stunt great again

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7805
Re: Scoring Machine
« Reply #19 on: July 20, 2018, 05:31:01 PM »
What changed for us
First ping pong balls.  Next random circle and flying order

And circle seeding by objective formula, and next-round judge selection by objective formula, and optimization of number of circles and how many contestants advance to the Friday round by objective formula, and now finals second and third flight order by mapping to ensure uniformity.   It shut up the chronic complainers.   
The Jive Combat Team
Making combat and stunt great again

Offline Paul Walker

  • 23 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 1626
Re: Scoring Machine
« Reply #20 on: July 20, 2018, 09:20:35 PM »
Has anyone got any idea what they ACTUAL problem was?

Chris Rudd told me they had somehow use qualification scores in the averaging calculation of the finals. Don't know how that happened.

Offline Igor Burger

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 2165
Re: Scoring Machine
« Reply #21 on: July 22, 2018, 01:06:41 AM »
yeah I used that when I was over at the Gee- Bee cup. 

The ones i used were in a different package and used an RF link to communicate to the judges tools. 

they had a computer that feed all the judges tablets (excel) with the name of the competitor and the maneuver that was next. 


it worked very well unless the system was out of sequence. 

The scores were sent at the end of each flight back to the computer. 
before you sent it back, you could check your scores before you sent them. 

Igor Burger probably has the details.

Hi Dave, that system which you used on Gbcup was simplified version of larger system used in Hradec Kralove already 2 or 3 years. That larger system shows ONLINE all points given by judges. REALLY online, you can see every click, also if he comes back to some maneuver after flight finished, before he “sends” point to PC. It is on large table and visible during whole contest.

This system on picture used now in France dos not show anything and it is pity that organizers keep points secret, I do not understand it too much. Points should be public, we pay for that in fees, so I do not know why not to show them. I fully agree with Jose.

The advantage to that from Hradec Kralove is printer, so the judge has paper which he can sign.

For Brett: the paper shows % deviation for every judge. So in case that you can see (as Jose wrote only in case that someone will show you as it is kept secret) you can see if the same judge is still 10% “under” or not “accidentally” 10% on “his” flyer. However since top 3 are from countries not having their judges, I cannot tell I saw such tricks.

And if there is a tool to see judger bias – well yes, there are usually after-champ analysis showing how judger performed. The best tool is results by single judges compared to final results. It can tell lot. I also remember one judge pushed “his” country to top 3. I do not recall who it was and which country. However there still can be case when only one judge can really see and there are 5 blind, but I do not think such case has some significant probability :- ))))

The problem of posting results after finals was program mistake in summing. The program summed all 3 flights and did not discard lowest score. Looks like noone tested it before :- ))))))))))))))))))

Offline BillLee

  • AMA Member
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 1292
Re: Scoring Machine
« Reply #22 on: July 22, 2018, 02:05:01 AM »
Does anyone remember 2016 where the results were available to the WORLD within milliseconds of tabulation? For ALL events, not just F2B?

The introduction of the electronic tablets for F2B is a step forward, but without the underlying management and tabulation system, it is like buying a new steering wheel for a 1973 Yugo.
Bill Lee
AMA 20018

Offline Igor Burger

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 2165
Re: Scoring Machine
« Reply #23 on: July 22, 2018, 02:08:28 AM »
Does anyone remember 2016 where the results were available to the WORLD within milliseconds of tabulation? For ALL events, not just F2B?

And public points from every judge. For me the best WCh organization so far (from those which I remember).

Offline Howard Rush

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 7805
Re: Scoring Machine
« Reply #24 on: July 22, 2018, 01:51:13 PM »
Igor, can you send me information on the Hradec Kralove system?  People would like to automate US Nationals scoring. Thanks.
The Jive Combat Team
Making combat and stunt great again

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: Scoring Machine
« Reply #25 on: July 22, 2018, 02:18:46 PM »
One stupid question here on the subject. What happens if someone looses his or her head and due to pressure and misses a maneuver? Unlikely to happen at the worlds but it can.
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline dave siegler

  • 2016 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 1128
  • sport flier
    • Circlemasters Flying club
Re: Scoring Machine
« Reply #26 on: July 22, 2018, 07:52:33 PM »
One stupid question here on the subject. What happens if someone looses his or her head and due to pressure and misses a maneuver? Unlikely to happen at the worlds but it can.

the ones I used allowed you to step thought the maneuvers to correct an error before you send it.   
« Last Edit: July 23, 2018, 11:43:34 AM by dave siegler »
Dave Siegler
NE9N extra class
AMA 720731
EAA 1231299 UAS Certificate Number FA39HY9ML7  Member of the Milwaukee Circlemasters. A Gold Leader Club for over 25 years!  http://www.circlemasters.com/

Offline Randy Cuberly

  • 21 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 3674
Re: Scoring Machine
« Reply #27 on: July 22, 2018, 09:10:30 PM »
the ones I used allowed you to step thought the manures to correct an error before you send it.

"Manures"...Is that a typo or a "Freudian Slip"?   LL~ LL~ LL~ LL~ LL~ LL~

Randy Cuberly









Randy Cuberly
Tucson, AZ

Offline dave siegler

  • 2016 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 1128
  • sport flier
    • Circlemasters Flying club
Re: Scoring Machine
« Reply #28 on: July 23, 2018, 11:43:56 AM »
opps, 

Dave Siegler
NE9N extra class
AMA 720731
EAA 1231299 UAS Certificate Number FA39HY9ML7  Member of the Milwaukee Circlemasters. A Gold Leader Club for over 25 years!  http://www.circlemasters.com/

Offline Igor Burger

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 2165
Re: Scoring Machine
« Reply #29 on: July 23, 2018, 11:46:21 AM »
Igor, can you send me information on the Hradec Kralove system?  People would like to automate US Nationals scoring. Thanks.

I will contact those guys for more info, but if you have specific questions, ask, I will forward them.

Offline jose modesto

  • 22 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 842
Re: Scoring Machine
« Reply #30 on: July 23, 2018, 12:08:37 PM »
Each judge has head phones that will instruct what menu ever is being flown and scored
Jose modesto

Offline jose modesto

  • 22 supporter
  • Trade Count: (0)
  • Captain
  • *
  • Posts: 842
Re: Scoring Machine
« Reply #31 on: July 23, 2018, 12:17:18 PM »
Sina had and unusuall accurence. He flew the entire pattern but his triangles were not scored
Sina’s friend video the flight. The triangles were flown.
Keith T. Instructed Sina on how to submit a protest. He presented the video showing that he performed the triangle to the organization.
The case went to the chief judge and he was awarded points for the triangles
I believe it was and everage of flight points
Jose modesto 


Advertise Here
Tags:
 


Advertise Here