You would note that with the way the village is situated and with the surrounding mountainous terrain, you would have to pass through the village to reach the ruins in any amount of time.