You can get a pretty decent borescope attachment for a smartphone. Gives you a pretty good look prior to wholesale surgery. Best to have a strategy of isolation; keep dividing the problem in half if you can. Sounds like you did that by ruling out the entire elevator setup. You might learn something if you changed orientation. One thought there is to attach one of your old, heavy handles directly to the leadouts and put that wingtip straight down. Now try wiggling the elevator. Then try wiggling the flaps. I have rebuilt and/or repaired quite a few OPPs (other people's planes) and find that control installation does not always get the attention it deserves. And, simply put, a lot of commercially available bellcranks have waaaay too much wobble. You can fix that, but most people don't. And it can lead to slop and glitches. One test that is just as important as feeling how smooth the controls are is an end-to-end slop test. Grab the leadouts with one hand right at the wingtip and put tension on them and don't let them move. Now wiggle the elevator. Then wiggle the flaps. Is there a lot? Does it go clunk-clunk? A lot at the elevator but not much at the flaps? A lot at both? With even a small access hole to see the flap horn you can reach in with large forceps and grab the flap pushrod. Wiggle it with the leadouts locked. Springy? Clucnk-clunk? All diagnostics to help isolate. Oddly, I have had to cut into some really nicely finished planes that on the outside looked great. But the controls were, um, slightly worse than poor. You just can't tell without testing, and even then, they can feel fine right up until some poorly executed construction locks up or saws thru something.
Don't trash it! If you get frustrated come back to it later. Good luck!
PS--I almost chuckled when you talked about the file using inside the plane. I made a special file quite a few years ago that is really useful. It is a coarse rat tail file that I heated and put a 30 degreeish bend about in the middle. Amazing for accomplishing stuff that seemed too hard to get to....
Dave