10 years ago, we wrote exams by hand with whatever we understood (in our heads.)

No colleagues, no laptops, no internet, no LLMs.

This approach still works, why do something else? Unless you're specifically testing a student's ability to Google, they don't need access to it.

For most of us--myself included--once you graduate from college, the answer is: "enough to not get fired". This is far less than most curriculums ask you to know, and every year, "enough to not get fired" is a lower and lower bar. With LLMs, it's practically on the floor for 90% of full-time jobs.

That is why I propose exactly the opposite regimen from this course, although I admire the writer's free thinking. Return to tradition, with a twist. Closed-book exams, no note sheets, all handwritten. Add a verbal examination, even though it massively increases examination time. No homework assignments, which encourage "completionist mindset", where the turning-in of the assignment feels more real than understanding the assignment. Publish problem sets thousands of problems large with worked-out-solutions to remove the incentive to cheat.

"Memorization is a prerequisite for creativity" -- paraphrase of an HN comment about a fondly remembered physics professor who made the students memorize every equation in the class. In the age of the LLM, I suspect this is triply true.

I think this is changing rapidly.

I'm a university professor, and the amount of students who seem to be in need of LLM as a crutch is growing really exponentially.

We are still in a place where the oldest students did their first year completely without LLMs. But younger students have used LLMs throughout their studies, and I fear that in the future, we will see full generations of students completely incapable of working without LLM assistance.

You are an outlier. When I was in school any outside assistance was tantamount to cheating and, unlike an actual crime, it was on the student to prove they were not cheating. Just the suspicion was enough to get you put in front of an honor board.

It was also pervasive. I would say 40% of international students were cheaters. When some were caught they fell back on cultural norms as their defense. The university never balked because those students, or their institutions, paid tuition in cash.

Makes me wonder if they should also get a diploma together then, saying "may not have the tested knowledge if not accompanied by $other_student"

I know of some companies that support hiring people as a team (either all or none get hired and they're meant to then work together well), so it wouldn't necessarily be a problem if they wish to be a team like that

I wished other universities adapt so quickly too (and have such a mindful attitude to students e.g. try to understand them, be upfront with expectations, learning from students etc).

Majority of professors are stressed and treat students as idiots... at least that was the case decade a go!

This is radically different from the world that's been described to me. Even 20 years ago cheating was endemic and I've only heard of it getting worse.

> It took me 20 years after university to learn what I know today about computers. And I’ve only one reason to be there in front of you: be sure you are faster than me. Be sure that you do it better and deeper than I did. If you don’t manage to outsmart me, I will have failed.

Regarding the collaboration before the exam, it's really strange. In our generation, asking or exchanging questions was perfectly normal. I got an almost perfect score in physics thanks to that. I guess the elegant solution was still in me, but I might not have been able to come up with it in such a stressful situation. 'Almost' because the professor deducted one point from my score for being absent too often :)

However, oral exams in Europe are quite different from those at US universities. In an oral exam, the professor can interact with the student to see if they truly understand the subject, regardless of the written text. Allowing a chatbot during a written exam today would be defying the very purpose of the exam.

I'm sympathetic to both sides here.

As a professor who had to run Subversion for students (a bit before Git, et al), it's a nightmare to put the infrastructure together, keep it reliable under spiky loads (there is always a crush at the deadline), be customer support for students who manage to do something weird or lose their password, etc. You wind up spending a non-trivial amount of time being sysadmin for the class on top of your teaching duties. Being able to say "Put it on GitHub" short circuits all of that. It sucks, but it makes life a huge amount easier for the professor.

From the students point of view, sure, it sucks that nobody mentioned that Git could be used independently (or jj or Mercurial or ...) However, Github is going to be better than what 99.9% of all professors will put together or be able to use. Sure, you can use Git by itself, but then it needs to go somewhere that the professor can look at it, get submitted to automated testing, etc. That's not a trivial step. My students were happy that I had the "Best Homework Submission System" (said about Subversion of all things ...) because everybody else used the dumbass university enterprise thing that was completely useless (not going to mention its name because it deserves to die in the blazing fires of the lowest circle of Hell). However, it wasn't straightforward for me to put that together. And the probability of getting a professor with my motivation and skill is pretty low.

In my experience LLMs can significantly speed up the process of solving exam questions. They can surface relevant material I don't know about, they can remember how other similar problems are solved a lot better than I can and they can check for any mistakes in my answer. Yes when you get into very niche areas they start to fail (and often in a misleading way) but if you run through practise papers at all you can tell this and either avoid using the LLM or do some fine tuning on past papers.

I have corrected exams and graded assignments as an external party before (legal requirement). The biggest problem with LLMs I see is that the weak students copy-paste commands with unnecessary command line switches. But they would have done the same with stack overflow.

Some also say they use LLM to help improve their writing but that's where the learning is so why????? I think it's the anxiety for failing, they don't seem to understand I'll not fail them as long as their incoherent text proves they understood what they were doing.

Having graduated and knowing how things are ought to look, taking exams are so much less scary now because I'm confident I will be failed for being incompetent, not because I didn't write properly. Not all students have the same privilege, they gain it over time.

It does help that computer science assignments and papers are pretty damn standard in form.

I think not all exams can occur like that. In some cases you just have to test one's knowledge about a specific topic, and knowing facts is a very, very easy way to test this. I would agree that just focusing on facts these days is overrated, but I would still reason that it is not a useless metric still. So, when the author describes "bring your own exam questions", it more means that the exam itself is not so relevant, which is fine - but saying that university exams are now useless in the age of autosolving chatbots, is simply wrong. It just means that the exam itself is not important; that in itself does not automatically mean that ALL exams or exam styles are useless. Also, it depends on what you test. For instance, testing solving math questions - yes, chatbots can solve this, but can a student solve the same without needing a chatbot? How about practical skills? Ok, 3D printing will dominate, but the ability to craft something with your own hands, that is still a skill that may be useful, at the least to some extent.

I feel that the whole discussion about chatbots dumbs down a lot. Skills have not become irrelevant just because chatbots exist.

When I was a student, professors maintained a public archive of past exams. The reason was obvious: next time the questions would be different, and memorizing past answers wouldn't help you if you don't understand the core ideas being taught. Then I took part in an exchange program and went to some shit-tier uni and I realized that collaboration was explicitly forbidden because professors would usually ask questions along "what was on slide 54". My favorite part was when professor said "I can't publish the slides online because they're stolen from another professor but you can buy them in the faculcy's shop".

My uni maintained a giant presence on Facebook - we'd share a lot of information, and the most popular group was "easy courses" for students who wanted to graduate but couldn't afford a difficult elective course.

The exchange uni had none of that. Literally no community, no collaboration, nothing. It's astonishing.

BTW regarding the stream of consciousness - I distinctly remember taking an exam and doing my best to force my brain to think about the exam questions, rather than porn I had been watching the previous day.

> I was completely flabbergasted because, to me, discussing "What questions did you have?" was always part of the collaboration between students.

I suspect that lots of intelligent and diligent students hate our new world of AI because they probably find it more likely now that they could be accused of and disciplined for something they didn't do.

They should be encouraged to read and review the LLM output so they can critically understand it and take ownership of it.

And if you want to test something else than whether the student can provide answer to your question, then why do you ask this question? Give a task that shows what you care about.

Our written assignments were a lot of "have an LLM generate a business proposal, then annotate it yourself"

The final exam was a 30 minute meeting where we just talked as peers, kinda like a cultural job interview. Sure there's lots of potential for bias there, but I think it's better than just blindly passing students using LLM's for the final exam.

Bravo.

Also, the take on AI is a stark contrast to much of what we've seen by other educators.

https://www.youtube.com/watch?v=JcQPAZP7-sE

LLM reasoning models are very good at searching well documented problems. =3

I still think pen and paper is king among students.

>I thought this was fair. You can use chatbots, but you will be held accountable for it.

So you're held more accountable for the output actually? I'd be interested in how many students would choose to use LLMs if faults weren't penalized more.

Is it possible, and this is an interesting one to me, that this is the smartest kid in the class? I think maybe.

That guy who is playing with the latest tech, and forcing it to do the job (badly), and could care less about university or the course he's on. There's a time and a place where that guy is the one you want working for you. Maybe he's not the number 1 student, but I think there should be some room for this to be the Chaotic Neutral pick.