Jim,
I will try!
#1 the A123 aren't the lightest things to begin with, so that's a small issue I think, but not probably the biggest one.
I fly a Brodak Electric Clown Arf on a 3s2100mAHr Lipo pack (5.5 oz) and it has plenty of power. I have also used a 3014-16 Scorpion motor which in my case (since I wound it myself) had a kV of ~1250 (stock is 1187). I used an APC 10-7 Electric prop turning at about 8500 rpm. This gave me ~4.9 to 5.0 s laps. The flying weight is 28oz charged up and ready for takeoff!
I am guessing in your case that you are using a lower pitch prop and therefore need more rpm to make the needed thrust to fly the pattern. However the A123 batteries have a lower output voltage than the lipo, and as a result, you probably don't have a high enough kV to get the rpm you need (or conversely, with that motor, you don't have enough voltage in the A123 pack). I believe you have enough stored energy, but it is coming out too slowly! At least you have enough for longer than 2 minutes I think. You didn't say how much your charger is putting back into the pack after a flight.
So what to do---the easiest is to change props--go either to a higher pitch prop--like I did---an APC 10-5 might be marginal for me I think--I can't turn it at a high enough rpm with a 3s pack after 4 minutes of flying---or go to a larger diameter prop-- like the APC 11-5.5 prop. Either one will give you the thrust at the lower rpm that I "think" you can't get with your current prop. The idea is to have an rpm that the battery/motor combo can provide for the entire duration of the flight.
About your flight time, going up a cell will give you a longer flight, because you will draw fewer amps (Power=volts *amps and for a given airplane setup, the power needed will have a fixed maximum and minimum). The 'conversion' occurs in the ESC (high voltage-low amps from the battery to lower voltage-higher amps in the motor.
Most of us fly with an governor which allows the ESC to raise the throttle setting during the flight to compensate for the dropping battery voltage. That means of course that we are not flying at the start with wide open throttle, probably more like ~70% throttle. Even without the governor, most timers allow you to ramp up the throttle as the battery winds down. You have to play a bit with the initial and final throttle settings to get a reasonable ramp-up rate.