The idea of the exiled House acting like little royalty of Rukon is perfectly fine. Just edit your backstory to accommodate for the detail and it's good.