There's an option for that when you first start a new character I believe.

A female character with the lady in waiting background I believe gives you a good horse to start with.