There are window heat pumps that are reasonable in price and can be installed diy easily. Make sure it has a reverse cycle and not just heater coils.
The limited experience I have is to check and double check. There are typically 3 ratings, cool, heat, and backup heat. A/C with heater coils list only cool and heat. I was looking at this and a split-system for the loft of my house, and ended up pushing a duct through an interior wall from the basement. When i get a large shed or garage built i will be investigating this again.
Phil