I am a long time reddit user and am of the opinion that the "mainstream" subreddits serve as honeypots to attract disrespectful juvenile users and should be kept for that purpose. Making it easier to find the most interesting subreddits will only attract the wrong kind of people.
I'm a long time redditor as well (4 years or so) and I've found that the subreddits are really not nearly as good as they used to be. /r/programming used to feel a lot more like HN, where you'd have a lot more researchers and well known experts in various areas chiming in. Things like /r/compsci seem to be mostly cs undergrads with a fairly underdeveloped sense of what the field entails. All of more niche subreddits I subscribe to tend to be so small there's almost no conversation going on.
What reddit has become is my guilty pleasure/trashy magazine. I find I still enjoy it but not for any of the reasons I used to.
Basically Reddit is like hanging out in a bar. There may be some good, serious conversations, but most of the time it's just having fun and talking bullshit. And I'm fine with that.
To add to this, it also feels as that in most cases, for every topic that a niche subreddit (or multiple similar subreddits) covers, there usually exist better online communities dealing with the same topic elsewhere on the Web.
For example, let's say you're interested in airplanes and aviation. Relevant subreddits such as r/aviation just doesn't compare to communities like airliners.net with its sheer amount and high quality of content.
Quality of content aside, other communities in such instances also seem to have better organization, more strict moderation (although some subreddits do have very good, dedicated moderators), and certain culture that's not prevalent on Reddit (e.g. compared to some other communities that I visit, there doesn't seem to be a culture of searching before posting on Reddit, ultimately resulting in a lot of reposts), leading me to go to those places rather than the subreddits that cover the same topic.
You mean like /b/ was, by the end, supposed to act as a buffer on 4chan to keep out the new posters from the other boards while they were still adjusting to the local "etiquette"?
That worked for a while, and then when that board became so bad that even new users wouldn't find any enjoyment in posting in it, they simply stopped lurking and carried over their bad habits to other boards. It spread through the whole site at an alarming rate. You can't try to separate the good content from the bad, or at least not for a long time because the bad always finds ways to spread, and I doubt the separation into "sub-reddits" instead of boards has any meaning in this case.
