Saw it today, very much enjoyed it and looking forward to seeing what effect it has on Agents of S.H.I.E.L.D.
Saw the trailer for X-Men too, and that looks really good as well.
Funnily enough, I think the exact opposite - I really like things that interconnect like that. It makes the universe have a better sense of richness and scale. The DCAU and the Marvel animate series of the time did it really well, I though and the Marvel Cinematic Universe is practically a dream come true. I am greatly saddened we won't see Wolverine or Spider-Man showing up as a cameo or guest-stars in an Avengers movie, personally.
Granted, the Marvel universe in particular could do with not writing its entire populace like barely-functional bigots and reactionary nut-jobs 99% of the time and act something like resembling real people and not caricatures, but that's bad writing not a fundemental problem with the X-Men and Avengers being in the same world.