Tom Dietterich (Editor in Chief at arXiv) posted on LinkedIn the other day that they’re having trouble keeping up with the onslaught of AI-generated papers. Lots of suggestions in the comments but no magic bullets.
It’s ironic that the LLMs which benefit so much from reading arXiv papers of yore are now being used to pollute it. Personally if I see a single author post 2023, I assume it’s junk, especially if they’re not from an actual research institution. Not all solo independent researchers are phonies but … many phonies are solo independent researchers.
OpenReview is okay but even some of the reviewers are apparently using LLMs or just hardly reading.
A recent review of mine had a LLM-ism at the very end, "would you like me to format this into a formal peer review report?" So they very likely copy-pasted their whole review :). I'm pretty down on academia atm ;-;
I think this is more of a systematic issue. I review now since 2-3 years, I do not get paid, which is fine. However, it takes always a huge amount of time without really having anything from it, but I do it because it is important work.
There now so many researcher that need to publish which explains the flooding, LLM only speed it up, so reciprocal reviews take place. So now you are forced to review and you are having less and less time. So it’s a natural choice for you if you already took an LLM to write a paper to use it to review.
Perhaps one solution would be much harder entry barriers, and enforcing some guidelines. For example that a supervisor can not have more than 5 papers and PhD students only need one real paper on a major conference/journal.
Asking earnestly: is ArXiv a valuable resource? I used it few years back when finishing my (late) Master's, but also saw a lot of crap published there (primarily for promotion/visa purposes) so it kind of took the shine out of the service for me.
The preprint papers on arXiv are like 99% the same as the published versions, except they're free instead of locked behind a multi-thousand dollar/year paywall.
It’s good for getting free access to preprints which are often close enough to the paywalled real papers in journals. If I find a paper I want (or more realistically, if ChatGPT finds a paper it wants for me), but it’s behind a paywall, odds are the authors put a preprint on arXiv.
Either you setup feeds for the specific topics/subjects you care about, scan what you come across once a week, or you use it to get access to papers that are usually behind some paywall. I don't think the intention nor the value comes from just haphazardously reading through everything in some section.
It's not peer-reviewed and supposed to free and accessible from both sides so the results kind of makes sense.
It’s ironic that the LLMs which benefit so much from reading arXiv papers of yore are now being used to pollute it. Personally if I see a single author post 2023, I assume it’s junk, especially if they’re not from an actual research institution. Not all solo independent researchers are phonies but … many phonies are solo independent researchers.
OpenReview is okay but even some of the reviewers are apparently using LLMs or just hardly reading.
There now so many researcher that need to publish which explains the flooding, LLM only speed it up, so reciprocal reviews take place. So now you are forced to review and you are having less and less time. So it’s a natural choice for you if you already took an LLM to write a paper to use it to review.
Perhaps one solution would be much harder entry barriers, and enforcing some guidelines. For example that a supervisor can not have more than 5 papers and PhD students only need one real paper on a major conference/journal.
It's not peer-reviewed and supposed to free and accessible from both sides so the results kind of makes sense.