I have written an article on how I have adjusted my classes to the situation:
https://htmx.org/essays/universities-and-ai/
Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.
Now, I require of students to submit written assignments done by hand. That way, I can at least be certain that there's some learning involved, even if they resort to AI to produce the relevant written part, because evidence points out that writing by hand reinforces learning.
I read that other professors resorted to requiring manual typewriters, which I also hate with a passion.
That is, AI is negating decades of enablement achieved by technology.
Honestly, between these circumstances and the fact that we are "enjoying" (?) brave new prices for RAM and SSDs, I'm finding AI increasingly harder to like.
We're currently designing a new intro systems curriculum, and we're thinking of it as an adversarial problem. That is, we're designing the course to ensure that a student optimizing for the best grade per unit work still meets our learning objectives. That means, as everyone else is saying, paper exams, but also 1-on-1 interviews to check that students understand each assignment they turn in. These interviews feature both factual questions ("You're using this macro from that library. What does it do?", "Please describe what this function does and how it works.") and conceptual questions ("Why is this code structured this way instead of $whatever?", "How else did you try solving this?", etc.) This doesn't stop students from generating code, but at least they have to understand that code in detail.
This is not as good as writing the code yourself, but how much worse is it? For math classes, this gap is gigantic. Obviously, understanding someone else's proof is much easier than writing your own. For programming classes, I think (without evidence) that the gap is somewhat smaller.
My experience from the past is that when this kind of evaluation is made clear up front, the students know what to expect and either do fine or drop the class in the first week. If you start with take-home exams and then spring paper exams on them halfway through the course, then half the class is cheating and won't be able to recover, as we read in the article.
In general, our students are somewhat motivated by an abstract desire to learn, but are much more motivated by grades. If there exists a straightforward path through your course that leads to a good grade without doing much work, most students will take it. (Our undergrads' course review website is literally called "Layup List." They are actually this shameless.) It's our job as instructors to ensure that all paths leading to a good grade either require learning the material or are more difficult to pull off than just learning the material.
It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place. We just need to better align the evaluation metrics with the outcomes that we're looking for.
Is anyone surprised that the academics are essentially being left on their own about is? University management has become increasingly disconnected from the academics doing the work on the ground at least during the last 20 years that I've been in academia. Covid was really an eye-opener in this respect, at our university academics were told with very short notice to make all lectures remote. There was essentially zero support from central management, there was not even budget to buy headsets or cameras and academics were expected to use their research funds.
At the end of the pandemic academics were thanked with two years of 0 and 1% salary increase, due to a tough economic situation. During the same time the president received a 8% raise. The tough economic situation was entirely due to mismanagement, i.e. the university had invested pension funds into fixed interest investments which did not keep up with pension growth (during a time when stock markets were going from strength to strength).
Perhaps this class isn't too hard for Stanford students, but I have to wonder about cheating when the averages on nearly every single assignment is that high. One clue was office-hours: Whenever I dropped by office hours, there was always a line (didn't matter the class). I quickly learned that TAs would often drop a hint that would make particularly hard problems easy to complete. It provided an unfair advantage to students who could attend office hours.
It also reminds me of the huge scam of cheating on the USMLE amongst Nepali medical students: https://www.medpagetoday.com/special-reports/features/113627
I've met many wonderful international medical graduates. Many have shockingly high USMLE scores. It is true that there's no time limit on their preparation (U.S. students have under 2 years to prepare during medical school, international students may wait years after graduation before taking the exam). Before that scandal, I never would've thought someone could cheat on the USMLE. Prometric test centers are crazy locked-down. But that's not how they did it. They did the long-game. Prior test takers would remember a handful of questions and just add them to a secret database. After many years, that database contained nearly every question on the exam. Test takers would work extremely hard - memorizing every single question. The reward of a U.S. residency is life-changing, I get why cheating was(is?) so rampant.
What an oxymoron. I agree with the others here that AI isn't the problem.
If you're the type that applies (and is accepted) to those schools, you are likely very informed on careers out there. So you also know what is at stake.
Good grades, or at least above a certain cut-off point, will open doors to prestigious jobs, as well as further studies. Finance, Law, Tech, you name it.
For these students, the stakes are high - and more akin to professional sports and draft season. I'm not saying this as an excuse to their cheating, but rather what the reality is for them. Again, not only is your competition smart and hardworking to begin with, but this sort of cheating is basically equivalent to academic and intellectual PEDs.
You could be studying English at Brown, with the intent to land a job at some management consulting firm or bulge bracket investment bank (1 out of 4 students at Brown end up in finance and consulting). Work that is miles away from your major, and where you're being provided the necessary training when you join the firms.
It is stupid, but understandable. And if you know others are doing it, it really only impacts you negatively if you don't. When so many of your opportunities come down to a two decimal number, people start taking risks.
It was certainly not difficult to cheat at a "closed book" take home exam before.
Based on my own experiences and those of people I’ve watched, I have three points. Sample size of 1 and all that; but it’s a lowkey passion topic for me.
- The argument that AI is “the next calculator” and education testing and overall methods need to adapt, is true.
- The distance between learning concepts to passing memorization tests(or worse, non-reality test specific logic) has grown significantly in my lifetime, and the AI education problems are really a mix of the cascade impacts of this issue + the AI impacts on conventional education measures.
- The problem with students who use AI to get past (often arbitrary) difficult courses or testing scenarios may look like one problem compensating aka solving for another (and I love AI for it!) but the parallel problem of AI (and our education system baseline) enabling students to normalize non-cognitive drone like approaches to any problem is super problematic.
The last point is an admittedly recent eye opening moment for me. Working with younger students recently, they are shocked at anything less than a fully clearly defined problem (education system training) and anytime thinking is required, they go to AI (which is fine!) and they can’t think beyond the AI output (which is super not fine!) The latter has been astonishing for myself and coworkers and really has us reconsidering our young talent programs.
Back in my day, you could also just Google the problems and find the solutions. What mitigated cheating at UVA was the honor code and each professor's faith and trust in our integrity. That culture was enough to not cheat.
Imo, the fix should be to work on culture. Cheating should always be a tempting choice, so that the student may challenge their integrity, which is a muscle that can atrophy.
IMHO to solve many problems we should go with Ivan Illich's ideas: https://en.wikipedia.org/wiki/Deschooling_Society and make education about education, not testing and certification.
I never understood this behavior from undergrads though, you're paying so much for an education and then you just skip the education part? Why bother?
Serrano did not void the midterm exam, but warned students that the final one, which counted for 50% of the final grade, would be held in-person. He also said that if the grade distribution was not similar to the midterm, only the final exam would be taken into account. The average score dropped to 48 out of 100. Of the 89 students who did the midterm exam, only 59 showed up for the final one. And of the 27 who did not show up, 22 had scored a perfect 100 in the midterm exam."
I’ve talked to a bunch of teachers and school leaders, and see three main ways schools are handling AI use in assessments:
1. Punish it: Detect AI use on homework and take home exams; treat it as cheating.
2. Prevent it: Move to live assessments – oral or offline – that are hard to cheat on.
3. Embrace it: Assess the process, not the output.
The second one seems to be the only real answer for foundational subjects. And the third one can also work for more creative or project-based work.
if ai doesn't help, then it won't help. if it does help, then you should use it. the metric is your output of whatever is being tested. writing an essay well and clearly understanding the material. solving a pset. whatever.
if you give access all the time for that, and then you test on a hard problem that could be done with or without ai, then it's fair. e.g. "clearly explain these four sentences of Y." obviously ai researching loosely and blathering isn't useful. won't be high signal / dense and correct and worthy of an 'a'. but someone who can harness ai and someone who knows the material well in the end will be rewarded the same by society. what you are testing is correctness and information density in a response. so you have to start now in accepting the reality that those who use ai to get there should be rewarded just the same as those who don't.
the burden is on educators to be as good as they can with ai if it is relevant or not if it is not relevant (and schools to fund them and ai companies to fund them if they have excess capital and are humanitarian).
and note the hard part even for us engineers at tech companies is in the correctness. it is very hard. but the sooner we start teaching how to do things correctly with ai, the more prepared the next generations will be.
So just mentioning my own anecdata here; I've had reasonable success converting a "code + written report" assignment into a "code + structured video report", where the code is assessed "exclusively" through the video report, and where the student needs to demonstrate ownership and understanding of the code, explain and defend design decisions, etc.
Yes, in principle it is possible for the student to generate a "script" and narrate it during the video presentation, but this is fairly easy to detect, and in general, given enough flexibility / subjectivity in the marking scheme, you can always downmark such students on the basis that their presentation demonstrated lack of understanding and ownership of the codebase, without having to go into too much specifics.
Obviously one implication of the above is that you no longer care if their code was "manually" written vs AI "assisted"; only if they have a good grasp / ownership of it, and can defend the broader architecture and design decisions. But I feel that this is realistic in the age of AI, since this is largely what will be expected from their future employers too.
I think the tricky part here is essays, but maybe they should be replaced with an oral exam + essays? A lot of the humanities and classics courses I took were pretty small.
Overall I don’t think these problems are that hard, I just don’t know if the incentives of universities align with fixing them. If their reputation is all that it’s for, grades don’t really matter as long as the $$$ is coming in.
Exams are definitely in person. Homework is not graded, except that once a week, one problem would be randomly selected from the homework for an in class quiz. The numbers would be changed up.
From my perspective, you didn't really have a choice but to be able to solve all of the problems on the homework unless you wanted to gamble failing that week's quiz. The answer key and AI are always available, but the learning got to happen on my terms. It was a good balance. And grading for the professor was relatively light since it was only one problem per student.
I have no doubt that AI was used for this, but I am shocked that this is even a thing. Before AI, any rich or connected student (which I assume is essentially all ivy league students) could just get someone to take the test for them?
All I know is I wouldn't want one of these cheaters performing open heart surgery on me, designing the airplane I'm flying in, or managing my financial affairs. Or marrying my daughter.
I would have thought the defense of cheating to be a much more marginal position.
I seem to recall they submit the exam via usb drive.
now they may have a closed exam mode where they can finish the exam without network and the. Submit over the network.
Of course this type of exam must be conducted in person or else the examinee could simply use a phone and type the ai provided answer.
Also is the professor living under a rock? Everyone is going to use AI if they can. So if you want them to not, asking them nicely isn’t going to cut it.
"Of the 89 students who did the midterm exam, only 59 showed up for the final one. And of the 27 who did not show up"
59+27=86
Maybe the article was written with AI that doesn't check its math.
But being upset that people use available technology to solve problems is quite an exaggeration and makes the guy close to being a luddite. He can just say "Hey, we do exams on paper and in class next time" and be done with it if he does not like technology.
...
"But it also hurts him that the one time in 34 years that he decided to offer a take-home exam, for highly justified reasons, the response was wide-scale fraud."
-----------------
Not to in any way defend or condone academic misconduct, the fact that this was his teaching-career-first take-home exam is probably relevant. Take home exams can be fiendish. I remember having one in grad school where we were given a very insufficient 36 hours to complete it, and many people just didn't sleep. That was from a prof who knew what he was doing. This guy may have accidentally made his exam absolutely sadistic.
Couple this with the fact that students often have other exams they need to be studying for in the same time window. The pressure can be immense. The temptation to use AI to help is going to be hard for many to resist unless the penalties are severe and strictly enforced.
AI cheating is probably going to be a problem going forward in all situations, but open-book, take-home tests are going to bring it out more strongly than other test formats.
This is a trap. I understand they've done this in the past, but profs are paranoid now.
I don't believe he's 100% correct on each incident of fraud and he's going to ruin students [academic] lives because of it.
Maybe the students cheated, maybe they didn’t. But the article does not seem to provide actual proof that they used AI. It mostly presents circumstantial evidence: unusually high scores, similar answers, and a later drop in performance on an in-person exam.
I'd like a citation for this being the "biggest known scandal" in the "entire Ivy League". Frequently such situations are kept somewhat quiet, for a variety of reasons. But fifty students is not a large number in courses that can enroll hundreds or up to a thousand students.
I would be truly ashamed to assist a top notch course only to turn to a computer for answers.
I would prefer to fail forever and learn the hard way before doing some cheap copy/pasta.
He understands the stakes here. If a university degree becomes useless, then what?
https://web.hedc.shizuoka.ac.jp/msg-from-center/creating-mea...
Edit: Didn't realize the original URL was paywalled. Sorry. This newer URL is open. Apologies.
I was very fortunate to attend Brown University for grad school, and consider it a great place.
Why would many people who were also fortunate to attend there not honor that opportunity?
A single person can easily do that using glasses with a micro-camera & rice-sized earbuds, and almost impossible to be caught.
I went an ungrad school that was top-5 in engineering. But my experience - and in the experience of other people I've talked to - formal undergrad education was, and always has been, a farce. At best, you learn through working on projects that are meaningful to you and learn "how to be an adult" (and later, you learn how to manage the enormous financial debt you acquired). But more typically, it's pure credentialism - no one cares what your grades were, only what school you graduated from.
The amount of actual learning that goes on from classes is minimal, but somehow we can't shift the overton window away from this silly game of grades that don't measure anything meaningful.
After graduating, I've was asked about my grades exactly twice in my life -- once when I applied to a master's program, and at one job interview (the company had a policy of asking about GPA for anyone who graduated less than 10 years ago).
I'm pro-education but anti-school, and all this nonsense makes me this way even more.
Running all your students exams through an AI checker is one of those Temptations of AI I am sure.
Accurate and high-quality exams are a solved problem. The issue is that universities aren't necessarily judged on teaching quality and opt for examination methods that scale well.
The obvious problem with that is it is terribly expensive. You need Masters or Doctoral level people for long periods alone with students and you need to trust that these proctors won't be some form of -ist towards the students and also that their grades will be fair.
It is by design not something that scales.
But it seems that is where the path lies at this point.
Essentially, the aristocracy gets education again and the plebs get to fight/cheat it out amongst each other and paying to do nothing in the end.
Damn it
The problem is that kids, even (or especially?) in the elite schools, are treating college as a box to tick. At the Ivies, for many students, it's nothing more than the requisite means to get their ticket punched so they can become/remain part of the elite class. This has been the conventional wisdom for half a century.
Now, this paragraph doesn't apply to an economics class like his, which is actually a useful degree, but many students will never use what they learn in college in a professional capacity, so it barely even matters if some (maybe even most?) students cheat because outside of the useful degrees, it's mainly a sorting hat to determine who works at Starbucks to pay back student loans, while rolling their eyes because of their 'valuable degree,' and who works there unironically, to pay their bills.
What needs to happen though is that students at all levels need to either believe that they need to learn what they're choosing to take courses in, or even better, actually innately want to learn it to satisfy their own interest. Either will do. If you have neither intention toward the material, of course you'll cheat your way through it. No student who actually wants to learn would waste their time and money taking a class only to not learn, and cheat their way through it.
I’ve studied in classes of 40 to 50 students to one teacher. The challenge of grading students at that ratio, means you need automation. It also reduces the kind of teaching possible. You really can’t provide one on one attention when you have multiple classes of 40+ students to teach.
Even the idea that MIT is somehow better than some other universities is itself a fiction. People are conflating financial success with academic ability. The former is mostly the result of social connections which are formed within the academic institutions and have very little to do with actual capabilities.
Universities should just sell degrees for a high price without requiring the students to attend. If they're rich enough, their skills aren't going to matter anyway; they'll succeed in their careers regardless so the university will still look good. 'Academic integrity' will be intact. Especially true for business, economic degrees or other humanities.
I hope it does.
Since students are notorious for being cheap
Of course 70% or so usually crashed out in particular in Calculus and I suspect given that US education is paid for daycare that's exactly the thing that can't happen which is why they're never fixing it
These news articles are just tiresome at this point. Obviously folks cheated previously, obviously it's easier now, obviously the answer has been to not have take homes all along.
If it's a math for finance course then some kind of model building for the midterm and being marked on the quality of the model or something. If AI becomes so good that it always chooses the best fitting model and requires no numerical optimization then they will have to change the courses to be more like UChicago where it's primarily undergrad directed research but AI assisted.