It's a lousy starting point. Depending on the age, a child will not be anywhere near as dextrous as an adult, never mind more so. All the ability scores of a child will be substantially lower than an adult's. Strength and Consitution will probably reach parity long before everything else.
-4 to physical scores, -2 to mental scores, Small size?