The issue is that creatures rarely fill any of their component spaces (except, like, gelatinous cubes). The rules allow corporeal creatures of any two sizes to share space in a variety of circumstances, which already implies that sharing space isn't physical overlap.
You can't reliably assume that a medium-sized ghost in the central space of a huge-sized giant is automatically touching, because there's enough space for the two of them not to touch. A minimum of effort (that is, an attack roll) remains necessary.