Or it could be that the inferred "only casters can use the portals" rule is a "not allowed" thing rather than a "not physically possible" thing (i.e. the portals will work for a non-caster, but the powers that be in the Magic Kingdom will take strong exception to the presence of a non-caster). If so, going through the portal as a desperation measure and worrying about that problem later makes sense.