Well, suppose you have two people (Jeb and Bob) who want to write some additional stuff as modifications to D20L core. One (Jeb) wants to write a book on dance fighting, and another (Bob) wants to go into Star Wars space fantasy stuff. If you don't think about possible mod support while writing the core rulebook, you may end up in a situation where both of these writers find it very hard to make any changes/additions without seriously breaking things. For example, suppose that Jeb wanted to write a dance-fighting spellcasting class. Jeb wants to give this class a special spell list, tailored towards dancing. So he comes up with some new spells, new mechanics(how to start dance-offs, how they work, how they function with core spells), and gives that class some core spells at weird spell levels, like Freedom of M. at lv 3, balancing other class features/spells around that to insure it's not gamebreaking at those levels. Well, in Pathfinder, this would cause serious problems. For starters, due to ripple effects caused by magic creation rules this change should drop the price of Rings of Freedom of Movement significantly, affecting medium- and high-level combat all over the system. So if you want to allow people to create classes with different spell lists, you may need to think about fixing this issue(among others).

In itself, that's not terribly hard to fix-just insure that your core system doesn't cause these ripple effects anywhere and you'll be fine.

Now suppose Bob releases his modbook(which has lightsabers and Force), and a table somewhere decides to try employing both at once. If that possibility is something you want in your system, then you may want to write some rules on the order in which modifications should apply, such as "First books that modify major mechanics, then books that modify minor mechanics, then books that modify specific classes, then books that modify specific class features", so as to insure that contradictions don't arise. Likewise, if you yourself plan to release supplemental books that modify the core system in some way without being patches, such a "load-order" might be very important. You may be familiar with the concept from Skyrim mods.

My fence of text may seem a bit all over the place, so here is my main point: if you don't design the system to handle user and/or other author modifications from the start, it's going to be a terrible mess when someone does try to modify it.