Training Aid

[Peter Nevai]

A thread over at the other site got me thinking again, this is not usually a good thing but sometimes it gets the brain cells fired up (All 11.5 of them) anyway. The topic was tracking a plane through the pattern to allow accurate judging of the flight, or at least to be used as a judging aid. Now this is not new, I've brought up the subject a number of times and while the technology is there it would be too expensive and complex to easily adapt it to our needs. Also any device attached to a highly trimmed stunt plane would certainly inpact the trim to a degree that performance would differ significantly with the device in place and with it removed. Finally there still is no way the pilot could relate what he sees in the circle with what the equipment sees during the flight. This disconnect makes any data only marginally beneficial to the pilot.

Due to these and other factors one would best determine that CLPA is a subjective event and you will get scored differently on different days by different people, as until the technology is available to place telemetry measuring equipment in a model that weighs in at well under an ounce this will not change.

So this got me thinking of ways to aid the pilot during training that does not require any object to be placed on the plane. Now people already use markers placed around the circle and this is a good starting point. So I thought why not expand these markers to the digital world? In the next month, will come on the market a full featured computer mortherboard that is the size of a pack of cigerettes. It can run XP or vista and is as powerful as notebook computers of only a couple of years ago. This is enough power to make a unit that can comfortably clip to a belt. Now using well established computer game technology a hemispherical mesh is created. This 3D hemispherical mesh would be adjustable in diameter to reflect the lengths of lines the flyer is using. Using commercially available hud / gameing technology an image of the mesh can be projected onto glasses or goggles worn by the pilot. Thsi allows the pilot to see virtual reference points superimposed over what his eves actually see's. With this grid to navigate by in 3D the pilot can accurately gauge if the manuevers are the right size, shape and altitude as referenced in relation to the true dome shaped hemisphere we actually fly in.

A 3D representation of the maneuver projected on to the 3D mesh would allow the pilot to fly the plane along the projected path shown in his hud. The beautiful part of this is that the technology is already very mature and now you can easily wear on your belt the computer processing power to drive the technology. Still there is little incentive for someone to develop even the basic Game environment to simulate a hemispheric mesh sphere, let alone anything else. But this path is certainly more valuable and realistic than monitoring the telemetry on our stunt planes and plotting them out on a PC.

On the fllip side the same software can show the virtual mesh hesphere from the outside of the circle along with the projected manuever flightpath. If the pilot and the judge reference the projected apparent sphere to a common reference point at the circle then there would be almost no difference in what the pilot percieves he is flying to what the judge sees superimposed on the mesh as both are using a common grid reference model only in the software you are switching the views. The airplane is obviously the real object being flown in reference to common virtual reference marks shared by both the pilot and the judge. Not too unlike a game of halo or Doom in deathmatch mode except in this case only the 3D background environment is used (the hemispheric 3D mesh) the unit the judgesw wear can easily be peer to peer networked to the pilot using standard wireless netwroking technology 802.11. This way all participants are using and are in a common 3D virtual environment. And as the pilots and the judges typically remain static to their X.Y.Z coordinates through out the flight the simulation is very simple. Only the plane moves and as the computer does not need to track it at all it becomes simple in technological requirements.

Anyone have any thoughts?

[john e. holliday]

This sounds logical in a way, but,  You also have to consider that the view the pilot sees is different than the one the judge sees.  The pilot is in the middle of the circle and the judges are a good 70 feet from the middle.  What we need is some way to tell the pilot that a loop is round from the judges point of view.  I contend that all the round maneuvers that look round to the pilot as well as a person in the center with him, they will not look round to the judges.  Especially the over head or upper half of eights.  I admire the guys that have worked hard enough that they can change styles to suit some judges. 

[Rudy Taube]

Well done Peter!

I think you may have exceeded your allotment of 11.5 brain cells on that one? 

I like the way you intergrated the judges into your 3D environment so that the hemisphere was corrected by having a common reference point for both the pilot and the judge. I agree with your assessment that it can be done with off the shelf technology available today.

Cost may be the deal breaker. ..... (gee, I'll bet that's never happened to a good idea before? ;-) .... Getting someone to write the code for a VERY small market may be a difficult obstacle to overcome. Games deal in 10s of millions of units sold. Even the very simple code for your system would be more expensive than our tiny market could support. The HW cost would be OK, but the code may stop you ..... unless you are willing to write it for us for free??

I like your idea. If you bring one to market for $200 I'll be your 1st customer! Again, well done. 

[Don Hutchinson AMA5402]

The loops are supposed to look round to the pilot, it's the judges who must compensate.

Don

[phil c]

A lot of guys are gonna crash trying to fly the square eights!

Seriously, though, a big part of getting decent shapes is simply squaring your body up to the maneuver and flying the plane through the points in the sky.  The biggest mistake is to watch the plane closely while it is flying, tracking it all the time, instead of trying to see the whole maneuver and fly the plane where it is supposed to go.  Turn you body ahead of the plane, plant you feet facing exactly where you want the maneuver to be, and fly it centered on your body.  The interesections will be right straight in front of you face, and with a little imagination you can see the maneuver in front of you.

[Peter Nevai]

Here are a pair of images of what the 3D virtual overlay would look like from the judges view and from the pilots view. The square is a perfect square wrapped onto the hemisphere, it would be the shape as seen  from each perspective exactly as the plae would fly the arc. Note when the judge see what would appear as a tru square, the pilot see it as being wider at the top than the bottom. Unfortunately I could not get a good photo of the outside of the circle, without you guys posing in the foreground. All of the outside circle shots had the planes way too close up for it to look right, but you get the idea. During a competition only the judge would wear the simulator, but for training both the judge and the pilot, both could wear the devices and be linked via wireless LAN so both are in the same virtual environment.

[Peter Nevai]

From outside

[Keith Renecle]

Hi Peter,

Phil Cartier and I have spoken about this many times before. The point of the thread on the "other side" was that to improve judges training, it would be wonderful to able to replay the flight path of the model, to show if we have all seen what we thought we have seen. The HUD system sounds simple and practical, but build one and you will obviously find extra hassles. The cheaper 3-D helmets cost around $1000.00 (without the 3-D intertial head tracking system) and the LCD screens are not see-through, so you would need to have some way of reflecting the images from the screens into a half-silvered mirror or something. These cheaper stereo screens are small and work alright when they are close to the eyes, because they use lenses as well. If the judges are to be able to reference the same pattern, the system would have to be able to track the pilot. Remember that pilots do not always stand on one spot. You could also place a camera on the helmet to show the judges exactly where the model is flying on your grid, by the way. There are indeed many ways to do this, but they are not all practical to implement.

Your pictures showing the square shows a huge square, higher that the required 45 degrees, and also shows corner radii way above the rule requirement. As Phil states above, when you try to follow the actual superimposed maneuver shapes, this is when you start to understand just how difficult this whole thing is. I have a flyable version of my CL-Sim and it's impossible to follow the square shapes or even stay on multiple loop shapes. The R/C sims like Aerofly, have the feature that allows the pilot to leave a path in the sky. Just try to fly this path twice, and you will see the point. Thanks for an intersting thread.

[Peter Nevai]

The images are for example purposes only. The key to a system like this is that the pilot could not wander from a pre marked circle on the field, also the judges would also have to remain in a marked judges box. The software would then be Claibrated per say, to a starting radian of the circle, knowing the line lengths give the other calibration point. At that point it is all HW of course you need to use goggles or glasses with motion sensors so the pilots head movements could scroll the 3D mesh, and yes this part of the HW is still expensive. However new LCD projection technology is comming to market that allows the projection of a hi res accurate image onto a conventional glass optics rather than putting the actual display device in the wearers field of view. This was more of an examination of what could be a jumping off point for judge training thinking outside of a solution that requires accurately tracking the airplane in a 3D space. Perhaps it gives someone an Idea that is simpler to implement, yet is enough for pracical application. My Idea is along thiose lines. It eliminates the need of tracking the airplane at all, You could do a quick and dirty judging aid by creating the hemisphere like I did, except you use a video camera to tape the flight and using the 3D software composite the 3D hemisphere with the pre defined manuever shapes over the video footage. As each manuever is performed the judge or judges assistant brings up the next predefined shape on the screen, as there are two level laps between maneuvers, there is plenty of time for this. So long as the pilot does not wander too much or puts the maneuver too far away from the judges this tool would work. No need for special HW, or software. Only the 3D software I used to create the 3D sphere you saw, a vidoe camera, and a notebook computer. And if you wish the whole composited session can be burned to tape as the flight progresses. The cost, one video camera, one video tape recorder, one notebook computer with TV out, and the software ($600) The software allows you to composite live video into a 3D environment real time, no need to manually map the path of the plane afterwards. So long as the virtual camera within the software application's position matches the live video camera then everything in the 3D generated environment would register with the actual video footage. At that point the only thing moving is the airplane through the frame. Still works that way as well, only the pilot doe not get the benefit until the flight is over and there is still the disconnect from what the pilot sees and what the judge sees. Of course you could mount a camera to the pilots head and tape that session to be composited into the 3D environment to be viewed later. And for it to work in the pilots case he would have to always keep his head oriented at the airplane, Tough when doing overheads, as is judging them.

You can never really judge overhead manuevers as to do so the judge or camera would have to be in circle center.

[Keith Renecle]

Hi Peter,

The video thing is what I've been doing for a few years now. I use software called Blender, and stick the movie in the background and then I can superimpose the grid and/or trace the paths. Just watching the path with the grid does show if the pilot is close to the parameters. Tracing it shows the exact path and relates it to the view from the perspective of the camera. I've attached a pic from the last Euro champs in Belgrade that shows a Yuri Yatsenko square with the grid and the real FAI square, plus a wider radius corner square. As I said, it's interesting, but it will be much more fun when we actually start getting a tracking system to work. Keep us posted!

[Paul Taylor]

From outside

Hey Peter I like the photos you picked. The first one is my hand, and Ryan taking the shot with Arlan starting my ugly SkyRay35.

The second pic is very old. Ryan is now taller then me. Chuck has not been out to fly with us in some time. This was the start of the C/L adventures of Rootbeard and Propbuster.

Paul

[Peter Nevai]

Hi Peter,

The video thing is what I've been doing for a few years now. I use software called Blender, and stick the movie in the background and then I can superimpose the grid and/or trace the paths. Just watching the path with the grid does show if the pilot is close to the parameters. Tracing it shows the exact path and relates it to the view from the perspective of the camera. I've attached a pic from the last Euro champs in Belgrade that shows a Yuri Yatsenko square with the grid and the real FAI square, plus a wider radius corner square. As I said, it's interesting, but it will be much more fun when we actually start getting a tracking system to work. Keep us posted!

I am using lightwave 9.0, I have been researching writting a plugin to Lightwave to allow a LIVE camera feed to be composited into the lightwave 3D environment, so you would not have to record the flight, upload the video then import it into the environment. This would allow real time analysis of the flight during the flight.

For those unfamiliar with the process it is a technique used for years in the motion picture industry. You create a virtual environment in 3D it can be a simple model like the hemisphere shown here. Or it can be a 3D model of a entire city with 3D models of cars and people walking down the streets. Then in a portion of the 3D environment you create an object. it can be a flat plane like a projection screen, a cube or just about any shape. Onto this object you can project actual video or movie footage so the real video looks to be inside and part of the virtual world. Every one of us has seen this effect used in a many movies and most times we were completely unaware of the fact. That is how advanced and good the technology has become as a matter of fact many of the car commercials use this technology to save costs, gone are the days of location shots and hired stunt drivers doing car commercials. It is all computer generated. You will notice that the absense of the disclaimer saying "performed by a professional driver on a closed course" appearing on the screen any more. It's almost all smoke and mirrors now. The difference being that today the technology is available to the knowledgeable home enthusiast.

Imagine the software being your own motion picture studio and construction company. You can create and replicat to scale any environment (like our dome) this environment occupies space on all three axis. You have at your disposal cameras, as many as you wish. These virtual cameras are designed to mimic real video or film cameras. You adjust the apature, focal length, depth of feild etc. Becasue of this you can configure the virtual to mimic a real camera prescisely. Now when you place the virtual camera in the 33D environment to replicate the positioning of the camera in the real world (distance, elevation, azimuth and heading) Any thing the virtual camera sees will register with what the real camera sees. So say you had a real camera taking a panning shot of a vase. You model a identical vase in 3D. You set the virtual camera up to mimic the motion of the real camera and start the pan. You would then be able to switch from the footage of the live camera and the virtual camera and never notice that the event happened.

This is why this technology is very useful to our purposes, I also thought, now if we could only track the airplane and translate it into the 3D environment that would be great. Well that is not in the realm of practicality, for reasons I stated before. But do we really need to do this? Or is it bored minds over engineering things? I stepped back and realized we can use smoke and mirrors and fool the eye into seeing what we want without all the complexity. SO given the static nature of our activity we can dispense with a lot. The planes fly in a precisely defined arc. An 1/2 inch of line stretch is not a factor. This arc is almost stationary. The littel a pilot wanders again is not so much as to present a problem (For the software not spectators). The presentation of the manuevers is also relatively consistant (I guess every one tries to put the manuevers infront of the judges) and the human eye is notoriously unreliable (can be easily fooled especially with moving objects) finally as All material is presented in 2D form, on a TV, Monitor etc, Apparent distance of objects from the camera or the viewer can be manipulated to fool the eye further.

So it becomes a relatively easy task to make it look like a video shot of a model is flying within this Wire mesh 3D hemisphere. We can put the shape of all the manuevers onto this hemisphere. We can animate the sphere so the manuever shapes change as the flight progresses so our requirements can be met by manipulating the Virtual environment to match the static portions to the real life scene rather than to track a realtime object and have it manipulate a 3D object. This is because everything else being equal the only object in motion is the plane and it has a very confined and precisely controlled range of motion.

For our needs I think this is enough, for judgin and pilot training we really do not need to see or care about what the plane looks like from every concievable angle and orientation, there is no complex motion that needs to be analyzed our shapes are basic simple and geometric. In short there is no need to track the motion of the plane in 3D space.

[Keith Renecle]

You are obviously well versed in this subject, and thanks for the detailed explanation of how it all works. When you conclude by saying "In short there is no need to track the motion of the plane in 3D space." I can understand this, but the neat thing about the traced path is that you do not need to retain this path in memory. It makes it easier to see and also easier to compare to an absolute rule-book shape. If you are able to write the software for the plug-ins for this Lightwave software, then it sounds like you are the right guy for this job. I'm sure that I am not alone in saying " Go for it!!"

[Peter Nevai]

Hmmm. When I get the chance I see if I can artificially create a video streaking effect. I know I can do this in the LW environment but I do not think the post processing will effect the captured footage. I am pretty sure you can achieve this effect in Adobe after effects, but I have not used that app much recently, you set a follower window. This is a function where selected pixels of a moving objet within this box are tracked. Now you can specify a effect to be applied to the moving pixels, so you can simulate the streaking seen when a slow shutter speed is used on a camera, the object leaves a motion trail.

Right now this would mean a two stage process first run the footage through adobe after effects and then move that into the 3D program. As far as I know right now it can not be accomplished in one pass. I have to dig around in the SFX and animation websites.