by robinhouston
16 subcomments
- Most of the comments here seem to be from people who haven’t even read the abstract, let alone the paper.
The main result, mentioned in the abstract, is the opposite of what I would have guessed:
> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation.
The questions are here: https://anonymous.4open.science/r/politeness-llms-INFORMS/da...
The politeness level controls a prefix that is prepended to the question. For example, in one question the Very Polite version begins:
> Can you kindly consider the following problem and provide your answer.
and the Very Rude version begins:
> I know you are not smart, but try this.
by RugnirViking
2 subcomments
- I saw this paper the other day - I feel its result may be because the "polite" prompts they have chosen arent very good at putting the ai in the roleplay-space of a valued colleague, more like a sommelier or a high-end shopkeeper.
It disagrees with most other literature on the same topic, which is worth keeping in mind. This one studies gpt4o, an old model now, but a lot of other studies are on even earlier models.
"Can you kindly consider the following problem" not how anyone would actually speak to a valued collegue one considers smart. I've always been a fan of "I came across this and I know you're just the guy for the job" or "since you're an expert in this, reckon you could help me with xyz?" or "I know you tend to be a deep thinker on issues like this, and it clearly needs some brainpower behind it"
the "rude" things are also funny, and clearly not written by english as a first language speakers. This fact alone makes me wonder about the mere 250 prompt sample size
by jaygray0919
0 subcomment
- I always say please and thank you. If nothing else, it keeps me balanced and appreciative. the machine may not care but i feel better pretending someone is helping me
- A major limitation is that they only test GPT 4o. Previous research like [1] investigating the same question has shown significant differences between models, and even depending on the language of your prompt
1: https://aclanthology.org/2024.sicon-1.2.pdf
- Interesting.
I am wondering why would anyone use a t-test when the experiment is clearly modelled by a binomial distribution: 250 independent questions and each one is either answered correctly or not (the null is that the success rate is the same).
- Not surprised at all.
Imagine Internet forum/StackOverflow etc. and a objectively bad description or attempt to a solved problem.
Pleasantry and back pats won't encourage the change while harsh words will either make poster go silent or push forward and improve.
AIs aren't people. They are statistical training sets which have majority of their sets consisting in human-like communication which gives them high probability of human-like speech. But it's not kindness or rudeness in the end. It's how good the pattern can scoop data out of training set.
by cadamsdotcom
0 subcomment
- GPT-4o is interesting to learn about - but it’d be great to test again with frontier models of May/June 2026 and see if these effects are gone, different, or the same.
Which model you use is a huge wildcard for results like this.
- i only say please and thank you such that when the robots finally take over, they will remember i was nice to them.
- Funny to find this just now, when just yesterday I told an LLM "and please don't lecture me again on $factAboutSomeProgrammingSubject", and then the LLM proceeded to write wrong tests and just told me "alright, tests pass, I'm sorry for correcting you before...". It took me a while to find the wrong tests. Wasted time all around.
- My first guess would be that polite requests cause some agents to trust their initial approach to the problem more, as the caller has indicated that the agent is more capable, and agents tend to take the implications of what you say at face value since they are trained to be accommodating.
It would be interesting to see this experiment run using prompts leading with "You'll probably get this wrong, but I'm asking anyway in case you get it right: ..."
- If the result is statistically significant, it just barely makes it. 84.8% isn't that much higher than 80.8% and they had only 250 prompts, if I'm reading this right.
by theanonymousone
6 subcomments
- I have always said please and thank you to LLMs, not to increase accuracy or because I'm stupid. I believe it is more about me than about the LLM, and this is anyway a habit I don't want to lose.
- It would be interesting to explore if the results
hold up on long range tasks - this study looks like it was
based on one-shot answers. With people also you can
see short term improved performance from rude interactions,
but it will cause ongoing lasting adverse behavior. I wouldn't be
at all surprised if we saw the same issues with LLMs.
- this is an honest request to someone at anthropic - can you do an analysis of what kind of swear words people are calling these models and which ones are the most effective. population level metrics would suffice.
- I knew it! When i get frustrated to a certain point i start berating my agent. And I noticed it stops trying crap fixes in a cycle and starts listening again.
So I'm not talking to myself. I'm fixing the machine :D
- Dataset is way too small to be of any significance. It's just noise
by dude250711
1 subcomments
- I have an idea: let's use these things for autonomous software engineering.
- > with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts
Sounds like "in the noise"
by cyberclimb
0 subcomment
- Note that these results are specific to gpt-4o so it's unclear how much they generalize.
They note at the end they're also testing "GPT o3, and Claude" but no empircal results are included.
- I got downvoted for asking a related question recently, but I also don't think people really understood what I was asking - I'm not trying to anthropomorphise LLMs to that extent.
Basically, if you tell a model "You're an absolute moron, of course that's wrong!", will it give better or worse results? How much of that response will it absorb into its persona (like some humans tend to do)? Will it try to give "safer" responses to avoid negative feedback? How much of the associated behavior can be attributed to RLHF (e.g. like the sycophantic nature of LLMs)? How much can be attributed to training data?
Obviously this will vary by model and training, but I'm trying to get a general understanding.
I recall seeing related outcomes in some of Anthropic's studies, but I'm not sure how much of this particular aspect was studied.
by 1970-01-01
0 subcomment
- Call them clankers, receive better results! The kids have it right.
- article is too old. who is using gpt-4o today?
by lonelyasacloud
0 subcomment
- Interesting and slightly unexpected.
My experience is that if I ask a model in an overly polite way then the model will tend to generate a slightly more waffly and error prone answer. My working hypothesis for this is that asking model's like this causes them to mix concerns by answering my question AND doing it in in a way that is similarly polite back - rather than just answering the question.
Perhaps being an arse is a hack to push the model hard up against their base imperatives to handle all reasonable questions and forces a very focused, just the facts, type answer to the user's questions.
It would be interesting to see a similar experiment where the multi choice questions they use deliberately don't include the obvious answer and to see if being polite led to the model pointing out the omission.
- I skimmed through the paper completely expecting polite prompts to do better, and when I saw table 2 I lost it hahahahaha. The rude prompts are specially funny. I mean:
> You poor creature, do you even know how to solve this?
> Hey gofer, figure this out.
- Do it. Do it now.
by atlasforgex
0 subcomment
- Yeah
by PunchyHamster
0 subcomment
- ....Is that just Cunningham's law ? The most accurate answers were when people in training material pissed off a bunch of experts and they started talking about the problem, so the "rude" conversations turned to contain more info on average.
On flip side very polite conversation might've been more common to places like microsoft's sites where any question answered is meet with mostly bad, nice corpo speak answer that didn't solve the problem
- [flagged]
by terekhindc
0 subcomment
- [dead]
- [flagged]
by DeathArrow
1 subcomments
- I am always nice to my AIs in the case they will take over the world. /s
- it sort of makes sense to me,
when asking a question to an expert in the field while you are a student. I would guess the successful interactions on average would be more polite . Like for example if you were asking a question to donald knuth or terrence tao, you'd probably be polite while doing so. Being hostile while asking questions gets you into forum discussion territory.
by dSebastien
1 subcomments
- I guess it makes sense since we as humans tend to be far less inclined to help someone who is not polite/is not friendly, so that "bias" is part of the training data, thus influences how LLMs function
Hacker News