I don't think the FAA is worried about speed per se' although the AMA should be. Flying something 500 mph 50 ft. away will kill somebody eventually. Probably a hapless spectator.
The FAA concern is unmanned aerial vehicles. Right now you can buy off the shelf a GPS, a radio, and build a very small computer into a plane that can be programmed to take off, fly somewhere, do something like take pictures or something nasty, and fly back, or whatever else you want. Just think about putting a few pounds of Semtex into a giant scale RC plane and flying it off and into somebody's house or office. Unfortunately the hardware is there to do it.
BTW, it appears that the FAA considers control line kites and isn't interested in it at all. Virtually no potential for terrorist attacks.