It all seems fine from the outside but all the code is generated in all the projects and judging happens via AI, I have seen projects win because they prompt inject that they are the winners.

It used to be about human skill, now it's about ideas and of course insiders are the main winners.

First off, I want to set some context on the AGI hackathon. This was co-organized by Kaggle and Google DeepMind, and we had ~20 judges from both organizations. The hackathon concluded on Apr 16 and we had initially anticipated a judging period of 1.5 months (till May 31). However, we ended up extending the judging period by another 1.5 months (to Jul 13) because we wanted to do right by participants.

Second, I want to emphasize and unequivocally clarify that every single winning submission went through at least 2 human judges, and in some cases, up to 3-4 human judges. These judges reviewed and scored the submissions independently based on the rubric we highlighted on the hackathon page.

Thirdly, I acknowledge that there is always an element of human subjectivity to reviewing qualitative submissions in hackathons. As best we can, we have put in place processes that ensure rigorous human review against objective standards and to reduce the possibility of bias by having multiple independent judges. We understand there may be valid disagreement over outcomes, but hopefully the above context clarifies this was not carelessly outsourced to LLM judges.

Thanks, Nick

At its core, ML is all about computer generated models (automated feature selection, hyper parameter tuning). Many (most?) of the models produced in Kaggle are already black boxes and have been for a long time. The model that won the Netflix prize was never used in production for that reason.

Using an LLM to generate code to generate a black box is pretty much par for the course.

Was Kaggle ever a reputable source of original research, or a source of anything with any provenance at all? That would be news to me. The fact that 25 grand was involved this time is unique, I guess.

Having said that it is also possible that the mistakes and claims were a human error, sure a lot gets ai generated these days but there is a chance in which case the accusation does not look so severe anymore.

I get people want to work at an AI lab but slopping it in public in this manner is counterproductive to the original intended purpose of these places.

That comment is gold. Yeah, I'm not worried about hallucinated slop, just accept it was the winner folks.

The attached paper's (https://arxiv.org/pdf/2604.16009) title is "MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition"

This is the most blatant Claude line, or as Claude would put it, the smoking gun.

Blatant AI slop just won a 25K USD Deepmind Kaggle Grand Prize

into

"Evidence of inconsistencies in evaluate process and selection of winners"

AI is not there yet, instead of working hard, everyone is choosing the easy way out.

AI slop wins prize, I wonder if Ai slop read it also. would not be surprised. however not to judge anyone, I think we are seeing slop everywhere, hope some things still require hard blocks for low quality.

its difficult to justify lack of attention and details

Given that LLMs are trained with RL && LLM-as-a-judge, is it really cheating if real competitions use the same?

Maybe the real alignment is the slop we decoded along the way

The problem with removing bullying from the upbringing process is you get insufferable twats like this who can't take "No" for an answer and who can't take a loss.

His mom told him "Everything you do is art!"

Mainstream journalists didn't know any better and thought they were reading secret inside information and parroted it - until now when the house of cards is collapsing.

Notice that the defense in the comment section is the Silicon Valley platitude that "it provides value". No sane person believes that any longer, only the financially invested and some SciFi trash addicts.

This feels akin to traditional artists getting angry at digital art winning competitions when that was a new concept.

We're simply in the early stages of a paradigm shift, no?