News:



  • March 28, 2024, 09:54:05 AM

Login with username, password and session length

Author Topic: VideoF2B  (Read 10098 times)

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
VideoF2B
« on: November 09, 2018, 03:56:54 AM »
Hello,

I will post here future updates about VideoF2B.

There is a new version in the blog, hope you like it!

https://videof2b.blogspot.com/2018/11/release-v04-augmented-reality.html



Best regards  :)

Offline Manuel Cortes

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 276
Re: VideoF2B
« Reply #1 on: November 09, 2018, 07:48:04 AM »
Impressive work Alberto.

Congratulations!!!!!!!!!!!!


Abrazos.

Manolo.

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #2 on: November 09, 2018, 06:22:17 PM »
Very nice.  As soon as I go to train mode this is the first thing I am implementing. Would it be possible to get distortion files ( .npz file) for the popular phone cameras?
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #3 on: November 10, 2018, 03:22:33 AM »

Thank you!  H^^

The idea is that the user can generate the .npz file. You only have to print the pattern and make a small video of it, the program to generate the distortion file is included in the file (CamCalibration).
Might be useful to share calibration files, but is not really needed.

Best regards  :)

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #4 on: August 24, 2020, 07:45:51 PM »
Hi Alberto,

What do you think of the idea of fitting the traced flight path against nominal figures as they are defined by contest rules, and then automatically calculating a score per maneuver / per flight based on how closely the actual path matches the nominal definition?

Cheers
Spice is the variety of life.

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #5 on: August 26, 2020, 10:34:33 AM »
Hi,

It is possible, but would require two cameras to get the 3d trajectory and quite a lot of maths and work.
I have an unpublished version where you can draw the perfect loops and eight figures along with the reference lines of the v0.4 version. I stopped its development because I could not draw a 4 leaf clover compliant with the rules due to bad definition (This might have changed since). This would be the first problem.

Currently I don't have the time needed to continue developing the program. The source code is included in the downloads and I would be willing to help anyone interested in expanding it.

Regards  :)

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #6 on: August 26, 2020, 11:27:25 AM »
Hi,

Two cameras would help, but they're not strictly necessary to obtain the 3D trajectory.  The math is not too difficult, either.

Cheers
Spice is the variety of life.

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #7 on: August 26, 2020, 11:54:42 AM »
Well, it's not hard in an ideal world.
When the plane is near to the tangent between the sphere and the camera cone, a small angular error in the detection means meters of error in the 3d position, making it useless.

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #8 on: August 26, 2020, 06:28:33 PM »
What kind of detection errors (in terms of pixels) have you seen?
Most maneuvers of interest are performed on the far side of the sphere against a clear sky where the detector works best, especially with a high-contrast aircraft. With a properly positioned camera and the right lens, the aircraft's time in that high-error region you mention can be minimized.

I studied a calibrated 1920x1080 video with a 21 m sphere closely and came up with some numbers that give me hope.  I sampled a horizontal line of pixels in the video, projected each one to XYZ points on the sphere, and studied the distances between neighbor points (i.e., the error related to neighbor pixels).  As expected, the plot looks like a sharp "U" shape.  Most of the points on the interior of the sphere are almost the same value.  The magnitude of the results surprised me (in a good way).  The points at any given pixel inside the sphere are about 2-3cm apart on the "near" side, and 3-5cm apart on the "far" side.  The interior error is mostly flat like that.  As we get closer to that intersection of sphere and camera cone, the error rises sharply...but only up to about 40-50cm.

Of course, that's the error distribution between adjacent pixels anywhere in the sphere.  It's more interesting when I start looking at the cumulative error with larger pixel distances.  For example, tith an error of 5 pixels the worst cumulative error near the tangent locations rises to 1 m.  When looking at 10 pixels, the worst cumulative error is 2 m.  At 20 pixels, it's 3.5 m.  This is basically saying, for example, "if the airplane is actually near the tangent AND the detector falsely detects it 20 pixels away from its real position toward the interior, the real-world error is about 3.5 meters."  Otherwise, the same 20-pixel error away from the tangent is on the order of only 75cm.

Have you seen detection errors of that many pixels in your recordings?  I have not, so I'm really hopeful  ;D but then again, I only started looking deeply into this application's code about a week ago.

Even with errors on the order of 2-3 meters I believe it's possible to correct for outliers programmatically based on track history and physics.  There are also reliable techniques for filling in missing points based on nearest neighbors, etc.

I don't believe this is useless at all.  I believe this application has a future as a judging and training tool, and I want to help.
Spice is the variety of life.

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #9 on: August 28, 2020, 10:57:52 AM »
Hello,
Your results are interesting. I'm not sure if the naive method the program uses would work in full HD. The detection is made in 640x480 if I remember well, it simply follows the biggest blob of moving image. I made some quick tests of simple filters but the results did not improve.

When the plane is close to the camera it is usual to see the detection jump from one wing to the other, for example, and this happens close to the worst precision region. This might be solved refining the detection. Maybe using a kalman or particle filter could help.

I recently started my PhD and it will keep me busy for some years. If you want to experiment I can offer some help.

Regards

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #10 on: August 28, 2020, 12:04:54 PM »
Hi,

I looked even deeper into this and I see the real problem.  It is not the error between pixels, it's the effective "dead zone" at the tangent.  With one setup, I discovered that the deadband is up to about 5.6 meters. ~^

My solution is to retry my setup by placing and orienting the camera more carefully such that the top of the tangent is below the 45° reference elevation and the camera's FOV vertically fits the far side of the sphere's equator and the nearest points of an overhead eight maneuver in the frame.  I believe it's possible with my current camera and lens.  With that orientation, all stunt maneuvers on the far side will never cross the tangent (I'm deliberately ignoring wingovers and level flight).  For more freedom I would need a slightly wider lens, but one thing at a time eh?

The cool thing about placing the camera like I plan is that all of the interesting stunt maneuvers will occur on the far side of the tangent locus, eliminating the need to guess where the airplane is. Bonus!

BTW, I perform the detection at [960 x whatever] (about quarter frame size).  It's a little slower, but I'm experimenting. "Make it run, make it right, make it fast". For now I'm bouncing between the first two 8) ~> ???

And yes, the detector jumps quite wildly when the airplane orients itself in certain ways to the camera. Most notably that happens at left and right in level flight, and sometimes during actual maneuvers.  We don't care much about level flight, do we?

So you've made a "Pretty horrible Decision" to become a "Professional hair Designer", huh? Good luck!  ;D

I should have some nice results soon.  Do you mind if I post v0.4 code in my public GitHub repo and develop it from there? Then the world can ridicule my work to their hearts' content  LL~

Cheers
Spice is the variety of life.

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #11 on: August 29, 2020, 03:17:49 AM »
Hi,
The deadband problem is what I failed to explain :)

My best guess for single camera would be something like this placed inside the circle. The only obstacle would be the pilot. I am not sure if the camera calibration process is valid for such a wide angle lens.

https://hobbyking.com/es_es/cam-360.html

I made a repo with my last version, so you can make a fork from there. I suggest you start from there because it was migrated to python 3:

https://github.com/alsolera/VideoF2B

BTW, it was my first python and opencv code, be patient! You can "pip3 install opencv" but to save videos in h264 codec you need to need to build opencv from source.

Keep us up to date  H^^

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #12 on: August 29, 2020, 08:52:13 AM »
Hi,

Excellent! Now we're on the same page regarding the issue 8)

I found it difficult to calibrate the camera at such wide angles even with my AS100V.  It was hard to achieve an error-free execution of the cam calibration exe, and even when I did the result looked strange to say the least.  However, to be fair I did those calibrations "blind" since the camera has no display feedback.  I will try it again with an attached monitor sometime.

However, I don't believe that such a drastic wide angle is even necessary.  I'm using my old Sony A57 with the beautiful 16-50 f/2.8 SAL1650 lens set to 16mm.  With the APS-C sensor crop, that's equivalent to 24mm in full-frame 35mm sensors/film.  By my calculations, I have a narrow window between 25m and 29m from sphere center, and I can place the cam up to 1m below the equator to achieve the result I'm looking for.  Calculations on paper are cool, but I'm excited to experiment.  Today's weather is not rainy like it was predicted, so I'm going to try the setup outdoors and report results.

Thank you for creating the repo yourself; it's more proper that I fork it and submit PR's instead of starting it myself, since you're the original author.  I will check it out and bring us up to modern Python software production practices, no worries  8)

Cheers  H^^
Spice is the variety of life.

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #13 on: August 29, 2020, 09:36:47 PM »
Well, I feel dumb today.

When I calculated the vertical angle of the FOV, I totally forgot the fact that even though my cam's photographic aspect ratio is 3:2, the video aspect is 16:9, which cuts off a big chunk of top and bottom, significantly reducing the vertical angle.  Unaware of this goof, I went to the field and recorded the setup from a few different vantage points, and then I was scratching my head for a while  HB~>

When I recalculated things later, it turns out that my focal length provides a vertical angle that is juuuuust shy of the minimum angle that I need  :-\  the minimum angle is even a little smaller because the cam calibration removes a border of about 9-10 pixels all around the original frame.

I want to see the visible edge (tangent) of the sphere slightly below the 45° elevation at the top of the frame, and the center of the flying circle at the bottom of the frame.  That's not possible with a 24mm equivalent lens, so I need something a little shorter.

Time to recharge my brains  Z@@ZZZ

On the bright side, I figured out how to draw the visible outline of the located sphere from the camera's perspective in the video  8)  It's a really nice guide in addition to the meridians and parallels already being drawn.
Spice is the variety of life.

Online Trostle

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 3338
Re: VideoF2B
« Reply #14 on: August 29, 2020, 11:20:33 PM »
Hi,

I have an unpublished version where you can draw the perfect loops and eight figures along with the reference lines of the v0.4 version. I stopped its development because I could not draw a 4 leaf clover compliant with the rules due to bad definition (This might have changed since). This would be the first problem.


Regards  :)

Keith Renecle did an outstanding job of developing a 3D presentation of the F2B pattern.  He found, as you did, the four leaf clover cannot be flown as described in the AMA or in the FAI rule book.  The problem with the rule book description is the constant 42o elevation for the "horizontal" intersections.  If the four loops are to be the same size, with the bottoms at the "normal flight altitude" (4-6 feet), the tops to be tangent to the vertical plane through the center of the circle, and all the loops are to be tangent to each other, the "horizontal intersections" cannot be a constant 42o elevation.  If you have the time to work on your simulation, try letting those "horizontal intersections" be tangent to a 45o plane through the center of the circle.  The four loops then all fit as described on the hemisphere and will all be tangent to each other.

It is easier to fly and it is easier to judge and it looks better.

If you do a search on Renecle's work, you will see this.

Keith

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #15 on: August 30, 2020, 07:55:34 AM »
Hi Keith,

I can define it as the "Great Circle Clover", no problem. Tangents are beautiful :)
For reference, here is the discussion I found so far: https://stunthanger.com/smf/rules-discussions/fai-4-leaf-clover-manoeuvre-description/
That thread is almost 2 years old; has any agreement been made on that front?

Cheers
Spice is the variety of life.

Online Trostle

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 3338
Re: VideoF2B
« Reply #16 on: August 30, 2020, 09:03:58 AM »
Hi Keith,

I can define it as the "Great Circle Clover", no problem. Tangents are beautiful :)
For reference, here is the discussion I found so far: https://stunthanger.com/smf/rules-discussions/fai-4-leaf-clover-manoeuvre-description/
That thread is almost 2 years old; has any agreement been made on that front?

Cheers

Of all the threads on the problems with the four-leaf clover, that thread you found fairly well explains/shows the problems with the rule book descriptions.  Peter Germann started that thread and suggested that a change to the FAI rule book was being considered.  I do not know the status of that effort.

There is a change proposal for the AMA four leaf clover rule currently pending.  There would have been an initial vote and earlier this year and a final vote (assuming the proposal passed the initial vote) later this year, but the formal change process for all proposals for all events has been delayed a year for Covid virus thing.  (I will not go into the reasoning for this action here.)  The AMA proposal is not as complicated as that explained by Germann early in the thread you found.  I have no idea on how the AMA CL Aerobatics Contest Board is going to act on this proposal.

The AMA change proposal for the four leaf clover is at

https://www.modelaircraft.org/sites/default/files/events/ruleproposals/CLA21-08.pdf

Keith

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #17 on: August 30, 2020, 08:59:40 PM »
Thanks for the AMA info Keith.  I guess it's a waiting game for now until the proposals on both sides of the world sphere are reviewed/approved/rejected etc.

As far as this software application is concerned, the clover leaf's definition can be defined/redefined at any time to match whichever rules are in effect upon the app's release.  The nominal geometry itself is simple enough.
Spice is the variety of life.

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #18 on: August 31, 2020, 05:59:37 AM »
Great work gentleman. I sure would love to see the auto scoring implemented. I will download and start using the software shortly. Since I fly with no coach 98% of the time I can definitely use this. Thanks a lot for your efforts.
Placing two cameras outside the circle is a good idea and it shouldn't be a big deal. What if you have a specific placement for the camera say 100ft from center of circle 90 degrees apart 5ft above ground. To set cameras place a target say 5 ft above the ground directly in the center of the circle. then with the augmentation turned on or just s dot presented to the screen you align the dot with the target  and you are done.
« Last Edit: August 31, 2020, 06:18:46 AM by TDM »
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #19 on: August 31, 2020, 10:28:37 AM »
Picture your eye at the camera's position.  Look up at the flight hemisphere.  You will see the visible outline (the "edge", if you will) of the hemisphere.  If you draw a bunch of lines from the camera position (a point) out to all the points on that outline, the lines will trace a cone.  The intersection of that cone with the hemisphere is a circle (don't take my word for it, prove it for yourself).  That circle is a minor circle whose center lies on the line joining the camera's center with the sphere's center.  We see the half of that circle that lies above the sphere's equator.  Finally, imagine that circle slicing the hemisphere into two chunks, as if with a knife: the part that's closer to the cam, and the far part.  The "near" chunk is smaller in volume than the "far" chunk.  With me so far?

Now... The issue at hand is that there is a deadband in that circle's region.  Alberto referred to this as the "tangent between the sphere and the camera cone" in an earlier post here.  An image is composed of pixels, which are whole chunks (I'm not aware of subpixeling detection in the image processing library yet). That means we can only detect the airplane's position in video at whole numbers: pixel 920, then pixel 930, for example. There is no in-between.  When the airplane passes through that outline we defined above, its 3D location "skips" between the near and the far side of the outline plane.  I found that the skip is quite large -- in one case (21m lines) it's 5.6m (over 18ft!).  That's unacceptable for 3D tracking.

To alleviate the issue, I am experimenting with the following idea.
Most stunt figures are performed on the far side of that tangent slice, and at 45° elevation or higher (let's ignore level/inverted flight and wingovers).  If we place the camera such that it sees at least the 45° elevation or lower, AND sees the pilot's feet within the frame, then one camera will suffice.  We need to see the 45° elevation because the overhead eight is executed in that region near the tangent.  All other figures are performed well away from it on the far side.

The constraint then becomes the focal length: we need a wide-angle lens that will fit the picture described above in the frame.

Some numbers for instance:
Sphere radius: 63 ft (60ft lines plus a few feet of arm)
Camera location at 89 ft from sphere center, and 3 ft below equator
This results in elevation angle at tangent equal to about 43°
The minimum view angle is about 48° vertically in the camera's field of view.
For reference, a 10mm lens on an APS-C sensor with 16:9 video crop has a 68° vertical angle of view, so it *should* fit.

I'm renting such a lens this week to try out the idea.
I'm planning to go to the NYCLST 2020 Stunt Championship this weekend.  If I can make an arrangement with the CD, I'd love to try it out.  As a bonus, I'd get recordings of stunt performances by Expert pilots whose pattern figures are close to perfection :)
« Last Edit: August 31, 2020, 11:42:14 AM by AMV »
Spice is the variety of life.

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #20 on: August 31, 2020, 11:18:02 AM »
Hi,
Very good explanation, I hope you prove me too pessimistic!  :)

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #21 on: August 31, 2020, 11:27:03 AM »
Thanks Alberto, I hope I'm not wrong! Saturday's mishap discouraged me for a while  ::)
Spice is the variety of life.

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #22 on: September 01, 2020, 06:38:11 PM »
Hi Alberto,

While waiting for the rental lens (delivery expected tomorrow) I have been experimenting with fisheye calibration.  It has been a two-day journey to get the API calls right, but the result looks very promising so far!  The cv2.fisheye module is available in opencv since release 3.x.  I'm using opencv 4.4.0.  The fisheye model is more appropriate for ultra-wide angle lenses.  I'll have more details once I work it out more completely.

Cheers  H^^
Spice is the variety of life.

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #23 on: September 01, 2020, 09:57:37 PM »
Hello everyone,

All was not lost during my field tests on Saturday.
Attached is a teaser to demonstrate my long-winded explanation from before, because a picture is worth all those words and more.
The virtual sphere you see has a radius of 20m (65.6ft for us 'Muricans).
The green ellipse is actually a perfect circle at the 45° latitude.  I drew it in green instead of the usual white because the sky was angry that day.
The huge white ellipse at the bottom is the circle at level flight.  As you can tell, the camera was located below the equator by about 1m (3ft).
However, the really interesting part is the magenta-colored curve.  That is the outline of the visible edge of the sphere from the camera's point of view.  Note how two of the meridians (black and gray) touch it at perfect tangents.

The important idea with this camera arrangement is that we want the green circle to be completely under the magenta curve, never crossing it.
Does anyone still doubt that 3D tracking cannot be done with one camera?  ;D

For the astute observer, you will notice how heavily distorted the edges appear.  The meridians should be vertical at the equator but they are not.  However, notice the frame of the soccer goal post on the right.  It shows that the camera's "fisheye" view has been correctly undistorted because straight lines remain straight.  Y'all should see that goal post before the correction...

Cheers  H^^
Spice is the variety of life.

Offline Keith Renecle

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 887
Re: VideoF2B
« Reply #24 on: September 02, 2020, 12:57:33 AM »
Thanks again to Alberto for the updated program. It's a lot of work and it is greatly appreciated. The Clover has been redefined in the F2B rules now and is technically correct. I've added some of my 3-D drawings that I drew using Blender 3-D, below. This subject has been discussed many times before and you'll find some strong feelings about the pro's and con's of a computerized stunt judging system. I'm one that is all for the concept, and Alberto's system in my mind is a giant leap forward for serious practice and also for judge training. I would for sure add that it can be used to access the good judges from the bad ones, but in this politically correct world with it's cancel culture, I guess that one would have to tread lightly here!

I've also had many, many discussions with really good design engineers and computer guys and it is for sure not an easy project to get right. It is technically feasible, but would require someone, or a team of people to spend a huge amount of their valuable time with very little remuneration. The camera option for actual competition judging is not easy to implement, not only for lens distortion, but pilots do not always fly the pattern where you want them to i.e. directly opposite your camera. Ball trackers use around 6 camera's to calculate accurate trajectories. The other option that uses an onboard tracking system with 3-D co-ordinates would give better, and more reliable data.

Just btw Mr AMV, would you mind giving us your name please? Thanks!

Keith R
Keith R

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #25 on: September 02, 2020, 01:11:07 AM »
Hello,

I knew of the existence of the cv2.fisheye module but had not found it neccesary for my idea of application. The calibration looks very good indeed!
I am looking forward to see your first results.

I totally agree with you Keith. I find it technically feasible but hard to implement. At least now we doubled the number of potential developers  :D
Thank you for your work with the clover definition.

 H^^

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #26 on: September 02, 2020, 07:21:03 AM »
Ah yes, where are my manners...

Hello, my name is Andrey.
I'm an engineer who loves math, physics, science, and all their practical applications that improve our daily lives.

My day job is with a company here in Rochester, NY that manufactures precision measuring equipment.  I am one of the API testers of our in-house software.  It involves .NET C#, and Python software.

In my free time I love to develop my Python software skills. I also tinker with electronics and embedded systems, mostly with MSP430 chips.

I am extremely interested in helping to develop the VideoF2B application toward automated scoring functionality.  I'm primarily here for the thrill of discovering new things and developing my software skills.  Working out the details of 3D geometry, physics, optimization, etc., is what's exciting to me here.  Even if it doesn't work out for official contest use, I'll work with it for private use as a training tool.  Renumeration is not the goal; this is a hobby, after all.

I've also recently taken up control line flying again with my brother for fun.  I'm a total beginner in stunt, and we practice some combat skills as well.

Cheers,
-Andrey
Spice is the variety of life.

Online Trostle

  • 22 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 3338
Re: VideoF2B
« Reply #27 on: September 02, 2020, 01:52:28 PM »
The idea of having computerized judging has been conceptualized for some time.  With the work that is being shown here, it appears that we might be one giant leap forward to have that possibility become reality.  So, it seems the computer will be able to detect errors from the rule book description for a given maneuver.   If and when this can be done, then fine.  I then think the next hurdle will be to have the computer access a score for the maneuver.  If a deviation is detected, and there will be for almost any maneuver by any pilot for any given maneuver, what will be the point deduction for that error and to make it more complicated, what will be the deduction based on degree of error?  And the problem is made even more complicated when the computer detects multitude of errors in any one maneuver.

These problems are what a human judge has to consider and that is why they are paid so much.

Keith (the other Keith) T

Offline Ken Culbertson

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 6035
Re: VideoF2B
« Reply #28 on: September 02, 2020, 03:17:23 PM »
The idea of having computerized judging has been conceptualized for some time.  With the work that is being shown here, it appears that we might be one giant leap forward to have that possibility become reality.
My daughter was a college swimmer.  A couple of times a year they would put a bunch of sensors on her and film her with some special camera from under water.  I understand Hollywierd uses the same technology to get life like animation.  I can see putting a sensor on the plane, filming so to speak then evaluating and assigning a score.  You would need to input line length and mark the starting point for each maneuver.  From there the computer could draw the perfect maneuver adjusted for the starting point and camera angle and track the deviation every foot if you so desired.  Then you start down the list of things that mitigate like wind shift or a bad entry corrected and a perfect maneuver that follows but it is shifted away from the computer track and so on, and so on.  Great pipe dream but until some computer animation PhD candidate gets a million dollar grant to develop it then it will remain just that, a pipe dream.

Anyway, what fun would a contest be if we couldn't bitch about the judges.

A better use for this technology is for a training tool for judges.  We all try and stand directly up wind but do we always move in time when the wind shifts?  Or you shift but the pilot doesn't because he didn't think you did?  Learning what properly flown maneuvers look like from different angles is a great training tool.

Ken
AMA 15382
If it is not broke you are not trying hard enough.
USAF 1968-1974 TAC

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #29 on: September 03, 2020, 06:52:23 AM »
Alberto and Andrey I just want to say thank you so much for your efforts I really appreciate it you two gentleman rock.  #^ #^
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #30 on: September 08, 2020, 08:17:57 PM »
Thank you Mr. TDM! I hope something great will result from our joint efforts.

-Andrey
Spice is the variety of life.

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #31 on: September 11, 2020, 07:50:11 PM »
Hi everyone,

Here is the YouTube channel where I will post traced videos: https://www.youtube.com/channel/UC_qSycHQlXgv9PKjmaHFKpw
So far, I've processed about half of the videos from the NYCLST 2020 Championship meet that took place last Saturday, and I hope to finish the rest this weekend.

I plan to post videos of more detailed 3D-tracked analysis, so stay tuned.

P.S. Alberto -- I found an easy way to speed up the processing! I get 2X improvement in overall FPS! You may find it useful for live video. I'll send you my PR in GitHub later.

-Andrey
Spice is the variety of life.

Offline Air Ministry .

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 4978
Re: VideoF2B
« Reply #32 on: September 11, 2020, 11:16:31 PM »
" The idea of having computerized judging has been conceptualized for some time.  With the work that is being shown here, it appears that we might be one giant leap forward to have that possibility become reality.  So, it seems the computer will be able to detect errors from the rule book description for a given maneuver.   If and when this can be done, then fine.  I then think the next hurdle will be to have the computer access a score for the maneuver.  If a deviation is detected, and there will be for almost any maneuver by any pilot for any given maneuver, what will be the point deduction for that error and to make it more complicated, what will be the deduction based on degree of error?  And the problem is made even more complicated when the computer detects multitude of errors in any one maneuver.

These problems are what a human judge has to consider and that is why they are paid so much.

THIS is why they should loose TEN POINTS for every inch deviation from the flight path .  ;D LL~ S?P

Actually think six foot out ( or more ) would be ZERO , so three foot , 50 % . over six inches , five or ten percent .

As ive prattled on before , miniturised position transmitters are feasable . Matchbox sized. sub One Ounce .

Then a data aquireer at the circle center , would get it all , at least on a radian / tangentally .
Likely Line Cr to Cr , plus arm , plus say a foot or two , to get the sphere diameter ?

Though , as shown here , the wingover should catch the top limit, division from there would define the 45 line etc , that horizontally gets the virtical legs for several manoures .

GOOD EFFORT though . Good to see the tracking overlay of repeat manouvres . H^^

Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #33 on: September 12, 2020, 02:15:41 AM »
P.S. Alberto -- I found an easy way to speed up the processing! I get 2X improvement in overall FPS! You may find it useful for live video. I'll send you my PR in GitHub later.

-Andrey

Nice finding Andrey. I am sure that there are other optimizations possible based on the principle of static camera. Drawings for maneuvres are calculated for each frame for example.
I will approve your PR later from the computer.

Best regards  H^^

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #34 on: September 12, 2020, 07:04:41 AM »
The rule book allows a 2ft tolerance on consecutive overlapping maneuvers.
Here is an idea for a judging logic.
 
1 make an average path for the maneuver and a min/max
2 scale the average to ideal shape compare the scaled average to ideal shape and give ti a score based on how well do they fit. 100 percent fit is a 10 and a so on. This will create the overall shape score.
3 also use the average to judge the level 45 90. the closer it fits the targets the better the score. If on target you get 100 if 30 degrees off you get 0. This will make the basis for size score.
4 use the min/max to score the overlaying of flight path etc. this will judge how well the plane follows the original path. The closer they match the more on top of each other they are.
5 give each of the three categories a score, one for shape one for size and one for overlaying, with a value 0-10.
6 average the three scores and that is what you get in FAI.
7 AMA you can use the FAI score times 3 and add 10 to get an AMA score.

I hope this helps.

Based on he stuff that can be collected this would account for all the scoring factors. Sorry there is no halo criteria.
 
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #35 on: September 12, 2020, 07:13:50 PM »
The rule book allows a 2ft tolerance on consecutive overlapping maneuvers.
Here is an idea for a judging logic.
 
1 make an average path for the maneuver and a min/max
2 scale the average to ideal shape compare the scaled average to ideal shape and give ti a score based on how well do they fit. 100 percent fit is a 10 and a so on. This will create the overall shape score.
3 also use the average to judge the level 45 90. the closer it fits the targets the better the score. If on target you get 100 if 30 degrees off you get 0. This will make the basis for size score.
4 use the min/max to score the overlaying of flight path etc. this will judge how well the plane follows the original path. The closer they match the more on top of each other they are.
5 give each of the three categories a score, one for shape one for size and one for overlaying, with a value 0-10.
6 average the three scores and that is what you get in FAI.
7 AMA you can use the FAI score times 3 and add 10 to get an AMA score.

I hope this helps.

Based on he stuff that can be collected this would account for all the scoring factors. Sorry there is no halo criteria.

You, sir, are a gentleman and a scholar. This is the first analytical breakdown I've heard that I can implement directly to all the math I'm about to do. Thank you! H^^
Spice is the variety of life.

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #36 on: September 12, 2020, 07:26:56 PM »
As ive prattled on before , miniturised position transmitters are feasable . Matchbox sized. sub One Ounce .
Yes, this is also possible and I'm open to development along this line as well.
Are we talking about IPS (indoor positioning system)? Do you mind linking me to the transmitters you mention?

Then a data aquireer at the circle center , would get it all , at least on a radian / tangentally .
If it's IPS, I believe they use at least three "beacons" to scan the space, detect the transmitter, and triangulate its position. I've seen a laser-based system that is accurate to just a few centimeters (1-2 inches for us 'Muricans).

Likely Line Cr to Cr , plus arm , plus say a foot or two , to get the sphere diameter ?
Though , as shown here , the wingover should catch the top limit, division from there would define the 45 line etc , that horizontally gets the virtical legs for several manoures .
The system inherently collects data points in 3D.  There would be no need for any line measurements in the field, nor any guessing. For example, a cloud of points from one lap of level flight and the wingover would be enough to construct the initial flight sphere, and any flight points collected after that could serve to refine the sphere and even inform us of the pilot's stepping pattern throughout the flight.

I'd be interested to know how affordable such a system is.  I've only given IPS a cursory look so far.
Spice is the variety of life.

Offline Air Ministry .

  • Trade Count: (0)
  • Admiral
  • ******
  • Posts: 4978
Re: VideoF2B
« Reply #37 on: September 12, 2020, 08:43:47 PM »
Yes, this is also possible and I'm open to development along this line as well.
Are we talking about IPS (indoor positioning system)? Do you mind linking me to the transmitters you mention?

Guess its these guys , http://www.yacht-bot.com/pages/company .

Igtimi Limited is a privately owned New Zealand Company, established in 2008. Igtimi is an electronics and communications technology development house. We develop, manage and support innovative ideas and solutions. Igtimi’s forte is specialised GPS telemetry, control and communication solutions.

Was sitting around at Lake Pupuke , a chap said theyed done the telemetry for the Americas Cup , signal was Ea 10 seconds / , but 10ths or 100s ' pulse ' was feasable . Could be half the size of a matchbox & sub one ounce .

Now , if I had his name & he did it as a ' Homer ' it'd be 50 bucks . But as a corporation theyed likely want 50.000.000 .  >:(

Three Position ( outside the circle ) would be ideal . Think he said G P S on the Am. Cup .


Offline Alberto Solera

  • Trade Count: (0)
  • Lieutenant
  • ***
  • Posts: 51
Re: VideoF2B
« Reply #38 on: September 13, 2020, 09:32:48 AM »
Maybe be should open another thread for that, but about IPS:

https://www.pozyx.io/

I have some good words about this system, but it is a bit bulky as is.

Offline Ken Culbertson

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 6035
Re: VideoF2B
« Reply #39 on: September 13, 2020, 12:28:59 PM »
It's funny all you gear heads are trying to come up with computer scoring. It mostly comes down to forming tribes in a popularity contest just like every other human endeavor. How do you calculate in the nice guy icon factor or the out of town unknown guy factor?


Motorman 8)
I have it from reliable sources that this sort of thing never happens. LL~ LL~ LL~

Ken
AMA 15382
If it is not broke you are not trying hard enough.
USAF 1968-1974 TAC

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #40 on: September 14, 2020, 06:56:04 AM »
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #41 on: September 14, 2020, 07:57:25 AM »
This might be of interest.
https://www.pyimagesearch.com/2015/09/14/ball-tracking-with-opencv/

Yes sir.  I've been following Adrian's work for some time.  This is exactly the approach of the current software, there's just also background subtraction involved before finding the contours: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_video/py_bg_subtraction/py_bg_subtraction.html

From there, I take the detected image point and project it from the camera out to the virtual sphere, looking for intersections.  The result is typically two XYZ points. Next step is to determine which of the two points is actually the model.  For now I'm cheating because I've been analyzing points on the far side, and the results have allowed me to tinker with fitting them to a predefined path.  More generally, I believe applying basic physics of motion (for example, the model's acceleration is only within a certain range) would greatly help with not just determining which point is the correct one, but also with eliminating outliers (pesky field flies, people/vehicles moving outside the circle, etc.)
Spice is the variety of life.

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #42 on: September 14, 2020, 08:57:45 AM »
I was thinking to ask how do you filter out those unwanted pesky targets. You are already on that. I saw somewhere a taking software that was using some flat round reflective stickers.
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #43 on: September 14, 2020, 09:37:39 AM »
That's the idea in my head for now, anyway. It remains to be proven in code as I haven't implemented that part yet.  I expect it won't be perfect; for example, if some stray item is detected near the model in a frame, it may pass the physics test and be included.  I'm also not sure yet how I will deal with the model going in and out of frame between laps.  These are all challenges I'm prepared to resolve.

The videos from Round 2 and finals should be a good test of this strategy. The wind changed in the afternoon and I had to point the camera toward the road, where passerby cars and big trucks kept stealing the tracing show :(
Spice is the variety of life.

Offline Ken Culbertson

  • 24 supporter
  • Trade Count: (0)
  • Admiral
  • *
  • Posts: 6035
Re: VideoF2B
« Reply #44 on: September 14, 2020, 03:34:40 PM »
That's the idea in my head for now, anyway. It remains to be proven in code as I haven't implemented that part yet.  I expect it won't be perfect; for example, if some stray item is detected near the model in a frame, it may pass the physics test and be included.  I'm also not sure yet how I will deal with the model going in and out of frame between laps.  These are all challenges I'm prepared to resolve.

The videos from Round 2 and finals should be a good test of this strategy. The wind changed in the afternoon and I had to point the camera toward the road, where passerby cars and big trucks kept stealing the tracing show :(
I have several of my old tracings (older version, much older) where it takes off after a bird.

Ken
AMA 15382
If it is not broke you are not trying hard enough.
USAF 1968-1974 TAC

Offline jerry v

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 199
Re: VideoF2B
« Reply #45 on: September 14, 2020, 07:06:23 PM »
It's funny all you gear heads are trying to come up with computer scoring. It mostly comes down to forming tribes in a popularity contest just like every other human endeavor. How do you calculate in the nice guy icon factor or the out of town unknown guy factor?


Motorman 8)
The subjective opinions of the judges is the main reason I never liked CL stunt. When I was flying F2A speed there was just a chronometer involved and anybody interested can double check the official time.
This work of the “gear heads “ is making all pilots equal before the rules. Of course, we are humans and should be judged by humans, not machines. This project, in the case of its success, will be a great tool for practice and, most importantly, the ultimate tool to settle disputes.

Jerry
Variety is the spice of life.

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #46 on: September 14, 2020, 07:30:50 PM »
I have several of my old tracings (older version, much older) where it takes off after a bird.

Ken

Yes sir. The logic in the current version 0.4 is still the same, so it will still trace the largest thing that moves.  Without the 3D information, there is no way (aside from a trained AI or similar) for it to intelligently discern between a model and a fly/bird/truck.  Ignoring those stray "flyers" will hopefully be a nice side effect of this upcoming work on the software.
Spice is the variety of life.

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #47 on: October 20, 2020, 07:33:40 AM »
We need this system like air.
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline TDM

  • Trade Count: (0)
  • Captain
  • *****
  • Posts: 844
Re: VideoF2B
« Reply #48 on: January 22, 2021, 07:12:28 AM »
What is important to have in a camera dedicated for this purpose.

Resolution
Frames per second
Lens

Thanks for information.
Each goal you meet is a moment of happiness
Happiness is the harmony between what you think and what you do. Mahatma Gandhi

Offline AMV

  • Trade Count: (0)
  • Commander
  • ****
  • Posts: 152
Re: VideoF2B
« Reply #49 on: January 22, 2021, 08:43:42 AM »
Camera resolution and frame rate can be whatever your computer processor can handle.  These days, that's 1920x1080 at either 30 or 60 fps.

The lens choice needs to be such that the vertical view of a 16:9 frame fits the center of the flying circle (pilot's feet) near the bottom of the frame and the 45° hemisphere parallel near the top of the frame, with some space to spare at top and bottom of the frame.  The extra space is necessary because a few of those pixels will be lost after camera calibration.
In practical terms, for a 68-ft flying circle you need a lens with a 15mm focal length equivalent in 35mm format (i.e., for APS-C that's a 10mm lens).  Place the camera near the ground, about 1m below the pilot's shoulders, and 25-28m away from circle center.  Adjust the distance and height in the field as needed.

Cheers H^^
-Andrey
Spice is the variety of life.


Advertise Here
Tags: camera video 
 


Advertise Here