Electrochemical cells, in general, tend to exhibit a characteristic of a fixed voltage (temperature dependent, as Dean mentioned), in series with a resistance that varies with temperature, cell construction, and state of charge. Again in general, that effective resistance goes up as the charge diminishes -- I believe that the primary mechanism for this is either that the electrolyte gets depleted (as in lead-acid cells) or that the outsides of the electrodes get discharged first, so all of the various ions involved in the battery chemistry have to migrate through the ever-thicker discharged layer before they get to something active.
If you push that model much at all you find out that there's a time dependence -- start sucking current from the cell and the voltage doesn't immediately drop: it quickly goes down to a plateau, then sinks slowly. Nothing is a nice exponential, and in particular the recovery when you take the load off can take minutes or hours (at least with NiCd cells -- I'm not as up on this new-fangled LiPo stuff as I should be). So you end up with a model something like this:
___ ___
.-------|___|------- // ------|___|--o------o
| Rn | R1 |
/'\ | |
| | --- * * * --- Vterm
\./ Vopen --- Cn --- C1
| | |
| | |
'---------------o--------------------o------o
(created by AACircuit v1.28.6 beta 04/19/05 www.tech-chat.de)
Go much beyond this in detail and things get wacky: For the most part the ultimate open-circuit voltage is a constant that depends on the cell chemistry, but severe over- or under-charge changes that chemistry. Again with NiCd cells I know that the actual oxidization state of the stuff getting plated onto the electrodes changes with significant over-charge, while if you manage to completely deplete a cell of either the plating on one or the other electrode, or the electrolyte, or whatever, then the open-circuit voltage will change (probably permanently).
LiPo's are different: they don't work by plating material onto the electrodes. Instead, they work by stashing ions into the nooks and crannies of the electrode material -- in fact, one of the (or perhaps the) failure modes of overcharged LiPo cells is that actual lithium gets plated onto the pertinent electrode, and starts reacting with the electrolyte. There's also some failure mechanism that happens when the cell voltage gets too low (I think that one isn't dangerous by itself, but if you try to charge a cell that's discharged too far then you just plate lithium out onto an electrode, and hilarity ensues).
If you really want to know this stuff, find "battery university" on the web and read what they have to say. You may also want to find a copy of "Rechargeable Batteries Applications Handbook", published by Newnes. If I recall correctly, Red Shoenfeld (of "battery clinic" fame) actually had a hand in the writing of this book (it's authorship is attributed to "staff"). It says nothing about LiPo batteries, but after you read it the things you do find on the web about LiPo's will make tons more sense.