Basically, nobody is getting in unless you all become invisible. The golems still haven't noticed you yet.