Rust is not as good an abrasive powder as carborundum, but it can be pretty effective. If you don't want to fill your engine with rubbing compound and run it, then don't run it with rust in it.
Before I read this thread I'd have taken it down completely (i.e. every screw unscrewed, liner out, piston pin out, etc. -- basically anything that can be done with hand tools), crock-potted it, then cleaned it with a toothbrush, soap & water, and maybe a toothbrush-sized wire brush before putting it back into service. Now that I've read the thread, I'd hit it with Naval Jelly, too.
If there's rust on any of the crank bearing surfaces I'd do two more things: one, I'd think about getting a new crank, depending on how much good material is left, and two, I'd worry about rust having embedded itself into the bearings in the case and the con rod. My one and only attempt at engine building wore out in the same run where it became broken in, because I didn't wash the lapping compound off of the piston well enough. So I'm kinda freaked out about stray abrasives left in an engine.