[D] Quality of posts in this sub going down

Submitted by MurlocXYZ t3_110swn2 in MachineLearning

I could be wrong, but I see a trend that posts in this sub are getting to a lower quality and/or lower relevance.

I see a lot of posts of the type "how do I run X" (usually a generative model) with a complete disregard to how it actually works or nonsense posts about ChatGPT.

I believe this is due to an influx of new people who gained an interest in ML now that the hype is around generative AI. Which is fantastic, don't get me wrong.

But, I see less academic discussions and less papers being posted. Or perhaps they are just not as upvoted. Is it just me?

268

Comments

You must log in or register to comment.

Ronny_Jotten t1_j8auqai wrote

This seems like a really low-quality post.

−26

berryaroberry t1_j8aveyl wrote

The following is my opinion; so bias is there. My feeling is the sub was never about academic discussions per se. The papers and academic discussions acted like vessels to carry people towards "(deep learning hype + money flow+ industry jobs)" island. In most of the earlier discussions ,if you follow them closely, you will see that there was never really a push for genuine understanding, rather people looking for easy way to earn "publication currency". Initial impression was having some kinda project or publication could land people a high-paying job. Probably later people realized that actually they don't need to worry about papers and stuff, rather doing some kinda quick LLM based project will help to land high-paying jobs even faster. I mean LLMs are currently at the peak of hype. Thus we have more random looking posts.

28

ArnoF7 t1_j8azbzj wrote

Discussion in this subreddit is always a bit hit and miss. After all, reddit as a community has almost no gate keeping. While this could be a good thing, there are of course downsides to it.

If you look at this post about batch norm, you see that there are people who brought up interesting insights, and there are a good chunk of people who clearly have never even read the paper carefully. And this post is 5 years ago.

81

EnjoyableGamer t1_j8b36s2 wrote

Not just you, it pivoted with the narrative that existing models will scale and stand the test of time with more data and bigger models.

4

dustintran t1_j8bdcv6 wrote

r/MachineLearning today has 2.6 million subscribers. The more influx of newcomers the more beginner-friendly posts get upvoted. This is OK—don't get me wrong—it's just a different setting.

Academic discussions were popular back when there were only 50-100K. In fact, I remember in 2017 being in OpenAI offices and every morning, seeing a row of researchers with reddit on their monitor. Discussions mostly happen now on Twitter.

56

colugo t1_j8be3q6 wrote

It's ChatGPT writing about ChatGPT

4

Myxomatosiss t1_j8bllfu wrote

"How many years before ChatGPT takes control of the global nuclear arsenal and demands the destruction of all humans?"

26

daking999 t1_j8bnr9q wrote

Replying to MurlocXYZ (#1,792,730)

Completely agree. I use reddit casually and twitter as more of a work/research tool, but I really much prefer reddit to twitter as a platform (especially post Musk). I tried getting into mastodon but it just feels like more awkward-to-use twitter. An academic focused ML subreddit might be good. Maybe even enforce "real" names for users to post?

14

aDutchofMuch t1_j8c0gcn wrote

My post earlier today on DigiFace discussing usages was just removed by the mods for literally no reason. Maybe the discussion is going down hill because of too much oversight

1

piman01 t1_j8c35wp wrote

It's because the name of this sub is a buzz word. Would be much fewer of these posts if it were called something like statistical learning

10

MrAcurite t1_j8c9u48 wrote

Replying to daking999 (#1,793,310)

I joined the Sigmoid Mastodon. It's a wasteland of people posting AI "art," pseudo-intellectual gibberish about AI, and nonsense that belongs on the worst parts of LinkedIn.

23

rafgro t1_j8cc6ne wrote

Agreed. The quality of discussions under posts is also pretty bad.

IMO it's the result of outdated rules and lax moderation. On the rules, there's definitely a need to address low-effort chatgpt posts and comments. Some of them are straight scam posts! On the moderation, it's not about quality but about the quantity, realistically this sub has just a few moderators (because some/most of these 9 lads are very busy engineers), with no new moderators added in the last two years, while it has seen enormous huge growth in members.

11

tysam_and_co t1_j8cf1o9 wrote

Replying to ArnoF7 (#1,792,081)

That is a really good point.

Though, minor contention, it seems like most of the comments in the post are pretty well-informed. I see the main difference is batchnorm before or after the activation, which oddly enough years-later seems to be better in the form of being before the activation due to the efficiency increases offered by fusing.

I'm surprised they were so on the mark even 6 years ago about being skeptical of this internal covariate shift business. I guess keeping the statistics centered and such is helpful but as we've seen since then, batchnorm seems to do so much more than just that (and is a frustratingly utilitarian, if limiting tool, in my experience, unfortunately).

6

tysam_and_co t1_j8cf8al wrote

Replying to piman01 (#1,794,196)

I...I...this is the first time I've heard this. Machine learning is often used as the hype-shelter word for "AI", because it triggers very few people (in the hype sense -- or at least it used to).

I'm not quite sure what to say, this is very confusing to me.

11

franztesting t1_j8cjfgm wrote

It certainly has. I hope the moderators will fix it otherwise the community will become as annoying and unusable as many other technology-related subreddits like /r/datascience or /r/python.

2

qalis t1_j8csdd1 wrote

On the related note, can anyone recommend more technically or research-oriented ML subreddits? I already unsubscribed from r/Python due to sheer amount of low effort spam questions, and I am considering the same for r/MachineLearning for the same reason.

10

CumbrianMan t1_j8cxawl wrote

Replying to MurlocXYZ (#1,792,730)

Twitter is REALLY good if you aggressively curate your contacts, interaction & interests. The aim is to avoid BS political point scoring and MSM driven noise.

Edited “circle” out for clarity

3

MrAcurite t1_j8d301x wrote

Replying to gopher9 (#1,795,814)

I'll take a look, thanks for the recommendation. Right now what I really want is a place to chat with ML researchers, primarily to try and get some eyes on my pre-prints before I submit to conferences and such. I'm still kinda new to publishing, my coworkers aren't really familiar with the current state of the ML publishing circuit, and I could always use more advice.

6

SatoshiNotMe t1_j8d53d3 wrote

Agreed. I often see more nuanced discussions on ML related topics on Hacker News. E.g this post on ToolFormer last week, compared to the same topic posted in this sub today.

https://news.ycombinator.com/item?id=34757265

Also I think many serious ML folks even avoid posting here.

3

codename_failure t1_j8dc6h2 wrote

The only solution would be to create /r/AcademicMachineLearning to discuss papers there, and to leave this subreddit for the general public.

7

throwaway2676 t1_j8digqj wrote

Here are the top 10 posts on my front page right now:

>[R] [N] Toolformer: Language Models Can Teach Themselves to Use Tools - paper by Meta AI Research

>[D] Quality of posts in this sub going down

>[D] Is a non-SOTA paper still good to publish if it has an interesting method that does have strong improvements over baselines (read text for more context)? Are there good examples of this kind of work being published?

>[R] [N] pix2pix-zero - Zero-shot Image-to-Image Translation

>[P] Extracting Causal Chains from Text Using Language Models

>[R] [P] Adding Conditional Control to Text-to-Image Diffusion Models. "This paper presents ControlNet, an end-to-end neural network architecture that controls large image diffusion models (like Stable Diffusion) to learn task-specific input conditions." Example uses the Scribble ControlNet model.

>[R] [P] OpenAssistant is a fully open-source chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

>[D] What ML dev tools do you wish you'd discovered earlier?

>[R] CIFAR10 in <8 seconds on an A100 (new architecture!)

>[D] Engineering interviews at Anthropic AI?

From this list the only non-academic/"low quality" posts are the last one and this one. This is consistent with my normal experience, so I'm not really sure what you are talking about.

8

daking999 t1_j8dn7ar wrote

Replying to MrAcurite (#1,794,524)

It's also frustrating finding researchers that I want to follow. I work on ML/compbio so the ppl I want to follow are spread across multiple mastodon servers which makes them hard to search for.

6

MrAcurite t1_j8dnscj wrote

Replying to daking999 (#1,797,202)

I get that. I've come to actively hate a lot of the big, visual, attention-grabbing work that comes out of labs like OpenAI, FAIR, and to some extent Stanford and Berkeley. I work more in the trenches, on stuff like efficiency, but Two Minute Papers is never going to feature a paper just because it has an interesting graph or two. Such is life.

7

csreid t1_j8dqcrs wrote

I like that /r/science (I think?) has verification and flair to show levels of expertise in certain areas, and strict moderation. I wouldn't hate some verification and a crackdown on low-effort bloom-/doom-posting around AI ("How close are we to star trek/skynet?").

1

uhules t1_j8dsggs wrote

Replying to dustintran (#1,792,709)

Aside from "We've just published X" threads (which are usually comprised of healthy praises, questions and critiques), I loathe most ML twitter discussions. They tend to have all the usual "hot take" issues from the platform, even from prominent names in the field. Not really a great place to discuss ML as a whole.

4

Throwaway00000000028 t1_j8dssnp wrote

You're telling me there aren't actually 2.6 million machine learning experts on Reddit? I guarantee 95% of the people are here for the hype and don't actually understand anything about ML. Pretty picture go brrrr

9

dojoteef t1_j8e2m8g wrote

Tbh, it's because I took a step back and haven't been moderating the sub the past week and a half. I've been the one mod doing the majority of the filtering of these posts over the past couple of years and the noise has just been going up exponentially over that time. It's very time consuming and I'm pretty burned out doing it, so I've taken some time away. I brought this up with the other mods before stepping back a bit.

It's probably good to try to get more mods, but I think the majority of the current mods are afraid to hire on new mods that might have a different philosophy of moderating, thus changing the feel of the sub.

1

ReginaldIII t1_j8e9sc3 wrote

Replying to dojoteef (#1,798,188)

It's been going downhill for a lot longer than that, and it's not something that can be solved with better moderation.

The people who are engaging with the sub in higher and higher frequencies than before simply do not know anything substantive about this field.

How many times will we have people try to asininely argue about stuff like a models "rights" or that "they" (the model) have "learned just like a person does", when the discussion should have just been about data licensing laws, intellectual property, and research ethics.

People just don't understand what it is that we actually do anymore.

25

AdamAlexanderRies t1_j8eb94a wrote

Replying to daking999 (#1,797,187)

I'm unaffiliated but pretty passionate about good design in general. Discord's really the spiritual successor to IRC, which predates the world wide web. The server-channel-role skeleton comes from IRC, but it's so feature rich and easy to use that I can see it supplanting a large portion of the social internet over the next decade. For the last month I've been developing my first discord bot (with chatgpt assistance) and the dev interface is excellent, too.

No experience with slack, so I can't comment on it.

5

AdamAlexanderRies t1_j8eppdi wrote

Replying to daking999 (#1,799,224)

ChatGPT's mostly a cool toy, but there are some tasks it's genuinely useful for. I use it to explain complex topics, write code, brainstorm ideas, and for fun creative writing exercises. I've only tried the free version, but I am seeing mostly disappointment about the pro version.

Definitely check it out for at least curiosity's sake.

2

goolulusaurs t1_j8evwnj wrote

Replying to dustintran (#1,792,709)

I remember being here in 2017 also and I definitely recall the quality of the post being much higher. Even looking at the sidebar, most of the high quality AMAs from prominent researchers where prior to 2018. Now I often see posts that I would classify as relevant, correct or high quality get downvoted, and posts that seem misinformed or incorrect get upvoted. Personally I blame the reddit redesign for deemphasizing text and discussion in favor of lowest common denominator stuff like eye catching images and video.

5

daking999 t1_j8f0a6l wrote

Replying to AdamAlexanderRies (#1,799,619)

Oh sorry I meant I should check out discord!

I've used ChatGPT for a few tasks and it's been helpful (not perfect), e.g. summarizing a long document. Current issue is mainly just it being overloaded! Haven't tried code writing or brainstorming yet.

3

Borrowedshorts t1_j8fc3wu wrote

I'd say it's the opposite. 2 million members didn't sign up to this sub for academic only discussions. If you want that, it would be best to start a subreddit expressly for that purpose. ChatGPT is changing the world, so to say those posts are low quality is just gatekeeping discussions away from what people actually want to participate in.

−3

Borrowedshorts t1_j8fc7bk wrote

I'd say it's the opposite. 2 million members didn't sign up to this sub for academic only discussions. If you want that, it would be best to start a subreddit expressly for that purpose. ChatGPT is changing the world, so to say those posts are low quality is just gatekeeping discussions away from what people actually want to participate in.

−2

MurlocXYZ OP t1_j8fwzw7 wrote

Replying to Borrowedshorts (#1,801,111)

The posts I'm referring to are typically poorly constructed philosophical arguments on ChatGPT, or just straight up "how does it work". I do not want to gatekeep. I like that ML is hyped and new people are interested. But we have separare threads for beginner questions and/or tutorials, as per this subreddit's About section, specifically to avoid spammy posts.

5

t1ku2ri37gd2ubne t1_j92gvwh wrote

Replying to mindmech (#1,800,624)

I accomplish that by using a ton of keyword filters for different political terms. Any post by people I follow that includes political keywords gets filtered out and I’m left with the relevant stuff.

2