Skill point are said to represent real, full training in a skill. In a modern world, unless you can professionally train dogs, fix cars or play cards, you don't have skill points in those.
The other problem is that if you want to build a character who can do what they do well, for example a frontline melee-type who can guard(spot/listen), be stealthy(move silently, hide), scare enemies(intimidate) and ride a horse(ride). The end result is having no skill points to spend on, say, Craft(flutes) or the like, something that you would have due to hobbies, but can't get without giving up something more important to the concept. It's like having to take parts off of a car. Will you take off the things on the outside that you like, but don't need? Or will you get rid of the parts that are fairly important, but not really necessary?