Jim,
The Speed control (ESC) does a pretty good job of matching the battery to the motor without losing too much efficiency -more or less borne out by my experience. However you will gain some efficiency if you match the battery voltage to the motor --maybe in the 10% range of so.

I have done what you are asking right now with my Electric Nobler--bigger than 1/2a of course. I have run the entire pattern using ~2200mAHr from a 3s2p 4200mAHr pack (two 3s2100mAHr packs connected in parallel), and now I fly the same motor using ~1700mAHr from a 4s1p 2100mAHr pack.  So I am using a lower throttle setting with the 4s than the 3s, but the motor itself is seeing the same nominal voltage to make the 8100 rpm that I have set the governor to provide.

Of course the higher cell battery gives one advantage, the ESC will change high voltage-low current to lower voltage-higher current, but not the other way! So it is a more flexible in that regard.