That's pretty much how I had my two-handed character set up--spells and stealth are for cowards! (At least in combat, he wasn't above using stealth to silently pick a lock so long as there wasn't any fighting going on at the time). It does slow your levelling somewhat, though, because smithing and alchemy are really slow skills to raise.