As stated, this says more about your personal experiences than it does about the RPG industry. Further, no one is claiming that the issues that exist with D&D aren't issues, but they are saying that the fundamental change in the industry has or is in the process of happening, and that the primary list of complains in the OP are really complaints about D&D, and even further, very specific forms of D&D. A more apt analogy is that the OP is complaining that the movie industry needs to cut its action movie baseline in favor of other things because action movies make up the summer block busters, and this thread is pointing out that if action movies aren't your bag, there's a lot of movies out there that do other things, even if they're not summer block busters.
Some of this is because combat is also one of the harder things to adjudicate without rules, in part because acting things out for combat is difficult, but also because most people don't have experience with combat, where as things like suspicion, argument, debate, trading, haggling, begging, and leading are things that many or most people have at least passing familiarity with and its general effects. Now we can argue over whether more rule space should be dedicated to these things, but most game makers are approaching this from the "I don't need to give you rules on how to adjudicate an argument, because most people already know how to do that". Still there are plenty of games that dole out rules on plenty of other things. Heck going all the way back to the beginning, Traveller delegates 15 pages of the 129 page rule book (The Traveller Book) to personal combat, and another 7 pages to starship combat. That's 17% of the rules, and it's actually less than that because those sections include things like general information about being wounded and how the gravity wells of planets affect ship movement. An equal 22 pages are dedicated just to the building and running of starships outside of combat.Basically every TRPG on the market gives more focus to its combat rules than other types of conflict. And why wouldn't they? That's what the market clearly wants, based not only on market data but on playtests and observing player stories and basically everything else.
TRPGs have pigeonholed themselves, and it's going to take some serious work to dig them out. But hardly anyone cares enough to even start digging. If anything is going to kill this medium, it's apathy towards change.
Some of this is because simple combat is the most basic and easiest to develop conflict. Of the 3 main conflicts, "man vs self", "man vs nature" and "man vs man", MvM is the most common one to start stories with because it's simple to develop. It requires very little setup, it requires very little plotting and it's easy to understand. Even basic fairy tales tend to focus on this, because it's simple for children to grasp the concept of one person harming another. See also comic book super heroes. The deeper concepts of self destruction and societal pressures are harder to grasp and take more legwork to produce a workable story from. As a result, those themes are explored in more "advanced" versions of media, and in RPGs that publish more than their core books, a lot of that additional material is covered in later books.
But from a new player (and new GM if you have a game that requires a GM) it's much easier to start with a "these people over here are bad, stop them with force" scenario than a "this dystopian society has created all these subtle insidious pressures on you and your loved ones, vive la revolution!" MvM scenarios are (or can be) small, compact and isolated. If a scenario doesn't work, get rid of those people, bring in new ones. MvN scenarios are sprawling things, and if they don't work, you have to change the whole setting. And MvS scenarios are usually introspective, they can be done in a group TTRPG (something like Monster Hearts), but they can also wind up being dangerously personal. Think about how uncomfortable it is when that one guy won't stop acting out his sexual or power fantasies in D&D. Now imagine new players trying to role play out an introspective Man vs Self conflict, that could get uncomfortable or offensive really quick.
Nothing to add here, just really quoting for emphasis because this is the truth.
But what is the RPG medium as a whole if not the combination of all the pieces of that industry. D&D does not an industry make. It's one part of the industry, and yes it's a big part, but to go back to my previous analogy, summer blockbuster action movies are a big (if not the biggest) part of the movie industry. But I don't think anyone would take you seriously if you came into a forum and said "the movie industry needs to drop its action oriented roots, and the small smatterings of films that aren't action oriented don't change the fact that the medium is all about action movies"
Frankly this isn't D&D's responsibility and it's not really something you see other mediums do. Yes, movie theaters run trailers, and books will often have ads for other books by the publisher, but those are often done by the publishing houses or the sellers, not by the individual films and studios. Jason Bourne movies don't have end credit sequences that say "hey, if you didn't like this film, check out Life of Pi or The King's Speech"
But you have hit on something here, which is that the RPG industry, unlike many other industries doesn't seem to have a central "industry hub". TTRPGs are all over the web, blogs and forums and all sorts of things, but they're also very niche. Blogs are dedicated to one game, or one or two similar games, forums are dedicated to either one genre, or often one game. Even general forums tend to consolidate. Look at GitP here. We have an RPG forum, a D&D5 sub, a D&D4 sub, a D&D3 sub and then an "everything else" sub, and the top level RPG forum tends to be about 80% general D&D.
Where it's the TTRPG's Gamespot.com? Its Gamestop? Where is its polygon? Where is the TTRPG's movie reviewer column in the NYT? The closest things we might have are rpgnow.com and maybe EnWorld. And let's face it, even as generic as the EnWorld front page might be, look at the forum, in the general TTRPG section, we have the following sections: D&D5, RPG General, D&D AL, Pathfinder/Starfinder (more or less D&D3), Older D&D, Star Wars, Character Builds and Opt. (almost entirely D&D), EnWorld Publishing, and RPG Ratings. So 5/9 sections are pretty much D&D.
I think a lot of this has to do with the fact that unlike almost any other medium or hobby, TTRPGs requires heavy personal investment in a single system. Let's face it, ours is a nerd hobby, which means to get the most out of it, you have to learn the little details, and people only have so much time to devote. Switching movies, video games and books is easy by comparison. But switching systems means learning completely new rules, new assumptions and often finding completely new players. So rather than go wide, TTRPG fans like to go deep. That's why character build forums exist, because it lets people invested in a particular game dig deeper.
But for the new person getting into TTRPGs, it means they're often faced with deciding on one thing they want to get deep with. D&D is the "lingua franca" for TTRPGs, so that's what most new people get or are brought to first. Not because it's the best, but because if you're only going to be able to invest in one, it's the most likely to pay off for you as a player.
I'd say our FLGS's can go a long way towards that. They've done really well bringing board games into the public eye with open gaming areas with a big library of games players can just come and sit and play. We need something sort of equivalent for TTRPGs. The problem is, unlike a board game, TTRPGs take a lot of specialized knowledge to run, and a lot of upfront planning. Which means your FLGS really needs essentially "pro" DMs to have enough breadth, and to do that, means they need to charge money, which in turn raises the barrier of entry again. The other strike is that a lot of the "non-D&D" rpgs out there aren't always set up for simple drop in and drop out play. Some are (Fiasco comes to mind), but others work best over long time periods (even D&D, at least the newer versions to a large extent) which makes coming up with "samples" for players much harder. It's easy to hook a player on some basic "boff-an-orc" playing. It's a bit harder to rope them into the complexities of say Vampire in a single shot.
Frankly, rather than the wargame roots, the hobby really needs to shed it's "nerd" roots. Don't get me wrong, I'm not disparaging anyone's love for the details, or for the character building mini-games or anything like that. But it's truly ridiculous that most TTRPG core books are in the near 200 page plus range. Compare the average board game rule set. And yes, TTRPGs do more, but even something like Fate Core is 20 PAGES of stuff just to build your character, not even to understand the rules. (for reference, Traveller (84) is 13 pages, by Mongoose 1e, you're up to 43(!) pages, not counting skills. FAE is a mercifully short 5 pages, which compares well to 4e D&D's ridiculous 163 pages)
My point here is that whatever game we come up with to supplant D&D as the gateway drug has to acknowledge, and be ok with the fact that 90% of the people who ever touch it will never sweat the details and that's OK. It's ok for the folks that want to dig deep to be able to nerd out (I love me some GURPS) but that has to come AFTER we've hooked them, not before. Personally, I think maybe what we could use is something similar to the D&D AL / Pathfinder Adventure Paths sponsored gaming thing, but for PbtA games. There's enough PbtA hacks to cover almost every genre, the playbooks tend to be almost as good as a pre-gen while still giving the players some choice, and the whole action/reaction flow of the game, with the common language moves (as opposed to specialized terms) tends to (with a good GM) keep it flowing without needing to get bogged into the details. I envision a sort of week schedule at FLGS locations where each night is a different genre in a PbtA game, rather than a different "game" as it were, with an eye towards pushing the converts either to that nights' PbtA game, or the other games that seek to specifically (and nerdily) emulate that genre better.
You're experience is very different from mine. I've found (even in D&D combat heavy games) that camaraderie and teamwork is in plenty of abundance. Then again, it probably depends largely on whether one is gaming to play RPGs, or gaming to socialize. I tend to game to socialize.
I think another thing we have to break away from as an industry (and again, this is largely an issue because we're a nerd hobby) is this idea that we can't make mistakes. Don't get me wrong, if you find you're making a mistake, and it could make things better to correct it, then by all means correct it. But if you're having fun, and you're playing your games and everything is going swimmingly, mistakes or no, it doesn't matter. What matters is you're gaming. Mistakes are house rules, nothing more, nothing less.
It seems rather unfair to state on the one hand that one can't use the indie RPG scene to describe the state of RPGs and the RPG industry, but then on the other hand use the indie video game scene to describe the state of video games and the VG industry. Either indie's count or they don't. I'm of the opinion they do. The question is, what can we come up with for the TTRPG industry that makes finding the indie stuff easier too? We already sort of have a "steam", it's rpgnow.com.
I think rather than a "steam" we need the TTRPG equivalent of mobile apps, and of Nintendo. First thing, we need the "mobile apps" equivalent to pick up the casuals. Something small, simple, that you can get people playing without them having to commit to being a TTRPG player (how many mobile game players call themselves "gamers"?). We need the Nintendo of the TTRPG world to. The company that's not concerned with what the industry as a whole is doing, they're concerned with making their own fun, first party off the wall stuff, but with the clout and exposure to get shelf space next to the dinosaurs (yes I know technically Nintendo is older than the Sony and Microsoft and (modern) PC gaming scene, but in this context dinosaur is referring to the size and pondering slowness)
A last thought, I think we need to put some serious effort into stamping out gate-keeping in the industry. Edition flamewars are not just internally destructive, when they leak into the public they're off-putting to those looking to get in. Fate, or PbtA, or Mouseguard, or GURPS or D&D, or Dread or CoC, it shouldn't matter. By the book, or a comedy of errors and mistakes, it shouldn't matter. Causal and silly or deep and introspective, it shouldn't matter. And yes, heavy combat focus, or investigation or politics, it shouldn't matter. What should matter is that you're playing, that they're playing, and that we're doing everything we can to make it easier for them to play. Even if that means sometimes we go out of our way to learn something new and play it. We're the old guard, and we're the computer, we have a responsibility to help give new players the steps to get into the community. I'm not saying if you love D&D, or you love FATE and someone comes along and wants to do Hero that you have to give up FATE and D&D. But I'm saying maybe invest in some HERO, learn it, give the player a chance to play (and be honest that you're learning too) and then help give them the resources the community and the industry has to let them find more and let them fly.