Nothing adds up here. With that motor & battery you should be able to get 13k RPM unloaded (unloaded = no prop). A 12-4 shouldn't load the motor that much, and besides, if you managed a 6 minute flight then the amount left on the batteries indicates you were only pulling 25A on average, which is well within the capabilities of the speed controller.
This probably means that there's one problem that we're not getting from your description, or you have more than one problem that are causing the symptoms separately.
You did not say what plane you're working with, or how much it weighs. Nor did you tell us the model of the motor. Knowing that should help.
If you can, take a picture of your plane and post it. People will want to see your general wiring arrangement, and they'll want to get an idea of the airflow over your motor, ESC, and battery.
The KR timer regulates speed: it's possible that you don't have it set up correctly, or that it isn't set up to regulate at faster speeds than what you're getting. Hopefully someone who's familiar with the timer will weigh in.
If you have any RC gear or a servo tester you can do a speed test. Set the thing up with a receiver (or servo tester), put a small prop on the motor or a dummy prop made from balsa, turn it on, go to full throttle, and see what the motor tachs at. If it can't get above 12000, it's not the timer.
If you have the equipment, checking the voltage and current on the ground, both running and not, is a good idea.
Check that you have cooling, like Bob suggested.
Also, unless your plane weighs over 65 ounces, check that the motor isn't rubbing anywhere and that it has good bearings -- friction at the motor would account for both the high power drain and the motor heating, although it wouldn't account for the ESC getting warm.
I can't think of anything else to suggest at the moment.