I cannot say how valid it is but it is interesting because everyone else is saying the contrary (i.e., according to his data, the volume has gone quite rapidly down post-2021).
Oh no, people are mean to me :-O. I mean, sure, we can talk about sociopaths and whatnot, who may or may not have their means, but do we need such role models? Ends, not means. Kant?
Arguably, all venues, whether journals or conferences, should nowadays have sections for these instead of outsourcing them to a specific venue (which is still better than nothing, of course).
That's not right; retractions should only be for research misconduct cases. It is a problem with the article's recommendations too. Even if a correction is published that the results may not hold, the article should stay where it is.
But I agree with the point about replications, which are much needed. That was also the best part in the article, i.e. "stop citing single studies as definitive".
I will add it's a little more complicated than I wanted to let on here as I don't identify it in the process. But it definitely was misconduct on this one.
I read the paper as well. My background is mathematics and statistics and the data was quite frankly synthesised.
Okay, but to return to replications, publishers could incentivize replications by linking replication studies directly on a paper's website location. In fact, you could even have a collection of DOIs for these purposes, including for datasets. With this point in mind, what I find depressing is that the journal declined a follow-up comment.
But the article is generally weird or even harmful too. Going to social media with these things and all; we have enough of that "pretty" stuff already.
However there are two problems with it. Firstly it's a step towards gamification and having tried that model in a fintech on reputation scoring, it was a bit of a disaster. Secondarily, very few studies are replicated in the first place unless there is a demand for linked research to replicate it before building on it.
There are also entire fields which are mostly populated by bullshit generators. And they actively avoid replication studies. Certain branches of psychology are rather interesting in that space.
> Certain branches of psychology are rather interesting in that space.
Maybe, I cannot say, but what I can say is that CS is in the midst of a huge replication crisis because LLM research cannot be replicated by definition. So I'd perhaps tone down the claims about other fields.
Tell me about it. But the grand question is: what to do about it? I'd suppose we'll see something similar to other domains (such as open source), including more rigorous PGP-style trust chains. ORCID is already getting there.
For starters, researchers who publish AI-generated work without disclosure should be barred from federal grants for N years. The institution they’re at should face penalties for the number of these researchers in their employ. This lets us at least label American universities as a safe source of research.
reply