Rule of thumb: People get most of their food from sources that get the most energy and nutrition from the least amount of time and work and is sufficiently reliable to be available every season.
Big animals are great if you can kill them without great danger to yourself and they are easy to find. That's a lot of meat for relatively low work.
Fish are also a good idea, but those would probably be available in large enough numbers to surive feeding villages in very large underground lakes submerged caves. If they go extinct after a year or two, they don't make a good long term food source. While there is not a lot of meat on a small fish, they often live in great swarms, at least in rivers and oceans, that can be caught with nets, bringing in a lot of food with a relatively low amount of work.
While wood and grass is inedible for humans, mushrooms make good food for people and they can grow without light, which makes them good primary food sources for underground worlds. Not sure if humans would get sick from eating them every day in large quantities, but in an entirely underground world, their digestion would have adapted to it and they would be perfectly fine.