Hi Richard,
I don't think it's size that really matters as much as weight. Of course the two are definitely linked a the larger airplanes tend to weigh more.
For Stunt I think something in the 4-5 lb weight is about the limit of what can be flown with any precision, unless you're built like Arnold S.
I've had a couple of stunters in the 74 oz range and they become a handful to fly when the wind blows hard because they pull so hard down wind.
For scale or sport a lot more can probably be tolerated by most folks. I personally flew a scale B24 that weighed 16 lbs a number of years ago and it wasn't much fun even with a throttle. It took a lot of effort and concentration to fly it with all the pull it generated. I'm not really a big guy, 180 lbs, but not a little fellow either and fairly strong.
There are official weight limits for competition, but personally I believe the pratical limits are about what I mentioned above.
Best weight for a stunter for me seems to be under 60 oz or so on about 65 ft lines. Even then the pull can get pretty hefty in strong wind like is often found in competition.
Speed of course is another important factor and AMA Fast Rat became almost a thing of the past when speeds reached the 160+ class...most folks simply found it too physically demanding to fly the things, especially in traffic. Most of them weighed agound 32 oz or less but the speed made the pull astronomical...can you spell Big Arms and a lot of body lean.
I think the best discription I heard of the event was "It's like picking up a 60 lb tool box and running in place with it for about 5 minutes...Some folks might think that's fun...
I'm much too old now for that kind of FUN.
Hope this helps some!
Randy Cuberly