I went through this a few years ago. Everything with more than 2 blades leaves me irritated and razor burnt. The $20+ for 3 cartridges was killing me too. I started here
http://www.badgerandblade.com/be careful, they encourage you to try everything. I just about did. Straights, DE, I have a couple SE too.
My favorite vendors
http://shoeboxshaveshop.com/unless you want straights, or natural handles, or coticule hones this place photograph everything
http://thesuperiorshave.com/If you are starting in straight razors check out the sight unseen deals here
http://www.whippeddog.com/I can get away with shaving every few days. My wife prefers me NOT shaved, but I am not allowed to grow a big beard... Go figure.
My typical shave is
http://shoeboxshaveshop.com/omega-50068-quotbambin50068.html (this is my travel brush, fits in a small medicine bottle, I have been frequently shaving at the Y lately)
or a large badger in a handle I turned on my lathe. The knot is a large "finest" in "fan" shape from
http://www.thegoldennib.com/(I also like their Super badger. I made brush handles as a Christmas gift for all the men in my family and used these knots in finest and super)
I typically use Arko shave stick, which is pretty intense when fresh so don't be afraid to air a fresh stick in the garage for a couple months.
I also really like CADE soap from L'Occitane.
The Merkur 1904 in Open comb is my preferred razor.
http://shoeboxshaveshop.com/merkur-1904.htmlwith Personna "medical" blades (From ebay for about $20/100 blades) or Personna "red", or generic "made in USA" blades from the grocery store (about $5/5 blades) or Wilkenson Sword (Walmart, $5/5 blades)
I do recommend using a brush. It is worth the effort, even if you are just using canned creams. The brush lifts the beard and makes shaving easier.
Phil