by crazygringo
8 subcomments
- > That means the article contained a plausible-sounding sentence, cited to a real, relevant-sounding source. But when you read the source it’s cited to, the information on Wikipedia does not exist in that specific source. When a claim fails verification, it’s impossible to tell whether the information is true or not.
This has been a rampant problem on Wikipedia always. I can't seem to find any indicator that this has increased recently? Because they're only even investigating articles flagged as potentially AI. So what's the control baseline rate here?
Applying correct citations is actually really hard work, even when you know the material thoroughly. I just assume people write stuff they know from their field, then mostly look to add the minimum number of plausible citations after the fact, and then most people never check them, and everyone seems to just accept it's better than nothing. But I also suppose it depends on how niche the page is, and which field it's in.
by ColinWright
3 subcomments
- The title I've chosen here is carefully selected to highlight one of the main points. It comes (lightly edited for length) from this paragraph:
Far more insidious, however, was something else we discovered:
More than two-thirds of these articles failed verification.
That means the article contained a plausible-sounding sentence, cited to a real, relevant-sounding source. But when you read the source it’s cited to, the information on Wikipedia does not exist in that specific source. When a claim fails verification, it’s impossible to tell whether the information is true or not. For most of the articles Pangram flagged as written by GenAI, nearly every cited sentence in the article failed verification.
by julienchastang
0 subcomment
- "Never copy and paste the output from generative AI chatbots" is mentioned in the article three times. This has been my experience as well. Initial AI output can be stunning until you quickly realize that it is mostly BS, filler and pap. However, I do find LLMs to be really useful for brainstorming, ideation, sounding boards etc.
by wry_durian
3 subcomments
- Note that this article is only about edits made through the Wiki Edu program, which partners with universities and academics to have students edit Wikipedia on course-related topics. It's not about Wikipedia writ large!
- Set aside the effect within Wikipedia and consider the larger picture, millions of people generating text with LLMs and at least some of that text being accepted as correct by millions of readers.
The WikiEdu article clearly demonstrates what everyone should have known already: an LLM has no commitment to the truth. An LLM's only commitment is to correct syntax.
- So, a small proportion of articles were detected as bot-written, and a large proportion of those failed validation.
What if in fact a large proportion of articles were bot-written, but only the unverifiable ones were bad enough to be detected?
by candiddevmike
3 subcomments
- I feel like this is such a tragedy of the commons for the LLM providers. Wikipedia probably makes up a huge bulk of their dataset, why taint it? Would be interesting if there was some kind of "you shall not use our platform on Wikipedia" stance adopted.
by theendisney
0 subcomment
- What would be a truly epic application would be their own chat bot to ask about applying edit guidelines. After reading almost all of the guidelines the talkpage debates, even amoung experienced edditors, looked waaaay off. The pattern of revert first make up excuses later seems the worse newbie deterrent possible. This while it should be fine to make mistakes. Many such excuses would get debunked by a bot imediately. It simply wont do any favors. If established editors dont like it they can edit the guidelines.
by shevy-java
1 subcomments
- So, AI spam can degrade quality.
But ... isn't this with regards to Wikipedia a much more general problem?
Usually revisions are approved manually by real people. This already can
be negative; takes a lot of time; no guarantee that new information is
true but old information can be wrong too. To me it seems more as if the
problem has much more to do with the quality control problems of wikipedia
itself. Yes, AI spam fatigues here but if the quality control steps are
bad then AI spam will only make this worse. But AI spam going away, does
not mean the quality control steps have gotten any better. These two issues
should be separate. Wikipedia needs to find better quality control mechanisms
in general. And that also includes existing articles - some are written by
people who are experts in the field. But they don't really explain anything
at all. So, these articles appear good but are virtually useless for 98% of the
people. I am not saying one should dumb down wikipedia, but you need to kind
of focus primarily on the average person really - not stupid but not a godlike
expert either. Explain it to, say, someone at age 18 or perhaps even a bit
less than that.
by shevy-java
0 subcomment
- Another issue, somewhat indirectly, is Grokipedia. As we now have more and more information, the AI that is used here deliberately engineers Grokipedia to contain,
shall we say it ... "alternative facts". If you look at Grokipedia, it actually looks visually better than Wikipedia, on a smartphone at the least. At the same time it tries to destroy an objective purpose, e. g. Wikipedia trying to show accurate information without any "spin". I don't believe that how AI is used by, e. g. Elon or mega-corporations, has purity and truth at heart though. We may have to look carefully at what happens to Wikipedia - it almost seems as if the attacks against Wikipedia by AI may not be merely "accidental". (Since it stores a lot of data, of course AI bots will leech off regularly, but I am talking here about purposes by organisations who may dislike democracy, for instance.)
- > That means the article contained a plausible-sounding sentence, cited to a real, relevant-sounding source. But when you read the source it’s cited to, the information on Wikipedia does not exist in that specific source.
This happens a lot on Wikipedia. I'm not sure why, but it does and you can see its traces through the Internet as people post the mistaken information around.
One that took me a little work to fix was pointed out by someone on Twitter: https://x.com/Almost_Sure/status/1901112689138536903
When I found the source, the twitter poster was correct! Someone had decided to translate "A hundred years ago, people would have considered this an outrage. But now..." as "this function is an outrage" which honestly is ironically an outrageous translation. What the hell dude.
But it takes a lot of work to clean up stuff like that! https://en.wikipedia.org/w/index.php?title=Weierstrass_funct...
I had to go find the actual source (not the other 'sources' that repeated off Wikipedia or each other) and then make sure it was correct before dealing with it. A lie can travel halfway around the world...
by throwaway5465
2 subcomments
- There seems much defensiveness in the comments here along the lines of "not a new thing" and "not unique to LLM/AI".
It seems to deflect, even gaslight TFA.
> For most of the articles Pangram flagged as written by GenAI, nearly every cited sentence in the article failed verification.
So why deflect that into convenient other pedantry (surely not under the guise tech forums often do so)?
WSo why the discomfort for part of HN at an assertion AI is being used for nefarious purposes and creation of alternate 'truths'?
- I’m honestly surprised LLMs are still screwing up citations. It does not feel like a harder task than building software or generating novel math proofs. In both those cases, of course, there is a verifier, but self-verification with “Does this text support this claim?” seems like it ought to be within the capabilities of a good reasoning model.
But as I understand the situation, even the major Deep Research systems still have this issue.
by PlatoIsADisease
0 subcomment
- ITT: People saying what I got downvoted for on the last wikipedia HN thread
I don't care if AI is used. I care about citations.
I don't know what happened between that thread and this, maybe the narrative really changes how people respond.
by simianwords
5 subcomments
- I find it very interesting that the main competitor to Wikipedia which is Grokipedia is taking a 180 degree approach being AI first.
- wikipedia is great but I can't get over this - https://www.wikifunctions.org/view/en/Z16393
- [dead]
by huflungdung
0 subcomment
- [dead]
- [flagged]
by asyncadventure
2 subcomments
- [dead]
by vibeprofessor
3 subcomments
- I trust Grokipedia way more, even though it's AI-generated. Wikipedia on any current topic is dominated by various edit gangs trying to push an agenda
Hacker News