I mostly use them for starting wealth determination; that said, it is true that the game balance is based on those wealth levels, and being far below/above it will make the party weaker/stronger; especially being short on it can make martials weaker by making their attack/ac/saves too low. (casters of course can just be poewrful anyways, and are a bit less sensitive to wealth).
A good dm, or a powerful party, can work around alternate wealth levels fine if they're aware of it; but if they're not it could cause a problem. I'd say check with your dm about it to amke sure he's aware of it.