I am simply not skilled enough to sand a clear filler coat without sanding down through the base primer underneath. The only way that I can reliably avoid sanding through is to have a base of at least a slightly contrasting color, and pigmented primer. As soon as I see the last of the primer feathering away, I stop (well, at least I try -- then curse myself if I give in and sand "just a bit more"). Then I put on another coat of filler, and start sanding again.
The idea of fill/sand is that you build up everything, then sand down the high spots to the base material -- if you're sanding into the base material then you've gone too far, and (as you're finding) you may well have to do a bunch of repair just to get yourself back to where you were before you sanded that spot "just a bit more".
I haven't been painting with dope for ages, but I'd seal the polyspan with clear, put on a coat of whatever dark I had lying around (maybe mixed with clear -- it doesn't need to be pretty, just dark), then use Brodak's gray primer for the filler. Then, any time I started seeing dark, I'd stop sanding that spot.