Meh...this is where I'd boost it up to the 1d8 + Strength and the knocking prone, because it is harder to throw someone into a wall, and PCs should be rewarded for RP rather than penalized because they should have logically swung a sword. Basically, keep the damage approximately equal to their weapon damage (allowing Power Attack for a throw...things like that), and tack on a small bonus (knocking prone, in this case) as a reward for the extra effort required to grab, lift, and throw a struggling enemy.
Yes, as the poster above me says, it's not the most plausible way of attacking. But, in the end, we're playing a game, and players want to do things they thing are awesome. If that includes flinging people into walls, well...seems to me it's your job as a DM to make it as awesome as they think it should be, within reason. This probably means a bit less realism in return for more player enjoyment...which should always be the highest priority.