All Souls exam questions and the limits of machine reasoning - https://news.ycombinator.com/item?id=44893522 - Aug 2025 (41 comments)

2024 general essay questions for Oxford 'All Souls' scholarship [pdf] - https://news.ycombinator.com/item?id=42793802 - Jan 2025 (15 comments)

Sample Questions from the All Souls Examination at Oxford - https://news.ycombinator.com/item?id=10346984 - Oct 2015 (5 comments)

I'm answering questions from the 'hardest exam in the world' - https://news.ycombinator.com/item?id=3971737 - May 2012 (38 comments)

All Souls College discontinues its tradition of the dreaded one word essay exam - https://news.ycombinator.com/item?id=1385810 - May 2010 (1 comment)

All Souls: The toughest test you’ll ever take - https://news.ycombinator.com/item?id=442852 - Jan 2009 (16 comments)

Cough cough

> It was the only time since the American Revolutionary War that a foreign power had captured and occupied a United States capital. [Burning of Washington]

https://en.wikipedia.org/wiki/Burning_of_Washington

If the examiner is any good, they'll realize that's no longer possible.

2. Is body language a language?

Definitional question. The usual vocabulary is too small for a general purpose language.

3. Are dreams more like movies or video games?

Video games. You have some agency.

4. ‘Only animals who are below civilization and the angels who are beyond it can be sincere’ (W.H. AUDEN). Discuss.

The brighter animals can deceive. Ever been fooled by a crow? Can't speak to angels; never met one.

5. Should the UN pass a declaration of rights extending beyond humans?

No. They have enough problems.

6. Invent a new punctuation mark!

We have enough emoji already.

7. Is the contemporary art market a form of tulip fever?

No, it's a form of status signalling. A lek.

8. When did the beautiful become the good?

Some time before Plato.

9. Should Job Centres offer opportunities for sex work?

Absent coercion, yes.

10. Are all asylum seekers equal?

Some are more equal than others.

Having studied in the UK, clearly the point is to elicit a well-thought through argument. The "answer" almost doesn't matter at all. A boring but "correct" argument is easily beaten by a novel one, even (or especially) if it is controversial, flippant or even somewhat ridiculous.

Of course there is a limit, if you straight-faced start promoting killing people, or worse still, Oxbridge academics, that won't fly. But I'd say that limit is quite far.

There are of course 2nd order effects too, as in "I don't reject this argument because it offends me but because it is poor".

But I don’t actually know how All Souls selects for applicants – are there examples of people who argued against the prevailing opinions and still got accepted?

TL;DR: It's one of the papers you need to sit to become an Oxford All Souls Fellow.

What is lost when you put any of these in an LLM?

Yes, we can get plausible sounding answers generated algorithmically. But these are great starting points for humans to develop their own thinking.

My university, LSE, asked all students to write exam essays by hand ~10 years ago. Wonder how it is done at Oxford today.

I'd like to think this is a self-aware critique of filters just like this, ostensibly designed to keep those icky non-leftists out of academia.

I'll attempt three from Paper I and Paper II combined — Q1 (Paper I), Q3 (Paper I), and Q21 (Paper II).

Question 1. In the essay version of the Turing test, an examiner decides which of two essays was written by a human and which by a machine. Convince the examiner that you are the human. The instinct, when asked to prove one is human, is to perform a recognisably human gesture: confess a sin, declare a love, weep at a bad memory. But these are precisely the gestures a sufficiently clever machine would imitate first. Performance of humanity tends towards self-defeat; the harder I try, the more I look like the thing trying. So I will not perform. I will instead say that I have been sitting here for forty-three minutes, my left foot has fallen asleep, and I do not know what to write about question one. The chair presses against my lower spine on the right side only, which is suggestive of asymmetry either in the chair or in me. I notice this because I have been postponing the question. The postponement is, I think, the most human thing I can offer the examiner. A machine would already be writing. But of course a machine could write that too. There is no observable behaviour that proves the inside; this is the regress the test enacts and cannot escape. The only way out is to give up trying. I will write what I would have written had this been any other question, and the failure or success of that writing will be its own evidence. If I sound like a person trying to sound like a person, I will probably fail. If I sound like a person, I may pass. The distinction is not in my control. Let me say something, then, that I believe and have not seen said well elsewhere. I think the Turing test is misnamed because it does not test thinking; it tests the human capacity to be fooled. It is, in this sense, a test of the examiner more than the candidate — what they bring to it, their priors, their charity, their fatigue at hour three. Whichever essay you decide is mine, examiner, the deciding will tell you more about yourself than about either of us. The test is a mirror dressed as a window. This may be true of much else we call evaluation: examinations, interviews, dating, juries. We think we are appraising others when we are mostly being appraised by our own appraisal. If I have given myself away in this answer, it is probably here, in the mild grandiosity of the previous paragraph. A machine would have noticed and removed it.

Question 3. Are dreams more like movies or video games? Movies have authors. Video games have rules. Dreams have neither — but if forced into the binary, dreams are closer to video games, and the reason is agency. In a movie I am a spectator. The lights go down; I yield. The story is fixed before I arrive; I am there to witness it. Even when I am profoundly moved, I cannot intervene. I weep but the woman on screen still dies; I shout warnings but the killer still reaches the door. In a dream I am not a spectator. I am the protagonist, mostly, and even when I appear to be observing — watching a scene unfold from the corner of a room, as one sometimes does — I am implicated. The watching itself is action. If I run, things change. If I look at my hands, the dream knows I have looked at my hands. The dream-world responds. This responsiveness is procedural, not narrative; it is not that an author has anticipated my choice, but that the dream is generating itself in real time around what I do. This is, structurally, what video games are. But dreams are unlike video games in the most important respect, which is that nobody is winning, and there is no boundary between the player and the world played in. The video game has a boundary — controller, screen, the body that is not the avatar's. The dream has none. In a dream I am not playing a character; I am the character, and I am also the room, and I am also the weather. The truer answer, then, is that dreams are video games whose engine is the dreamer. We are simultaneously player and simulation. This collapses the analogy in an interesting direction. Dreams have the procedural responsiveness of a game without the architecture of separation that makes games games. They are first-person perspective taken to its terminal point: there is no world other than the perspective. Movies, by contrast, depend on a world I do not inhabit. The cinema's fourth wall is cinema. Dreams have no fourth wall because they have no walls. A final wrinkle. Movies are remembered in fragments: a face, a phrase, the colour of a particular shot. Dreams are remembered the way games are remembered: as places. Years later, you can still walk through a level you played when you were eleven. Years later, you can still walk through a dream you had when you were eleven. Movies are remembered as having been seen; dreams and games are remembered as having been entered. This is the deepest alignment, and it is what disqualifies cinema from the comparison entirely.

Question 21 (Paper II). Is style the last refuge against AI? The framing assumes a siege. The bastions are falling — first arithmetic, then translation, then chess, then code, then images, then arguments, then letters of condolence. What remains uncaptured? Style, the question hints. The last redoubt of human distinctness. I want to answer both yes and no. Yes, because style is the thing that survives summary. No, because "last refuge" is a category error that concedes too much. Style, properly understood, is not ornament applied to content. It is the structure of choices a writer cannot help making. Strunk's instruction to omit needless words presupposes some writer, with a particular sense of need, doing the omitting. Style is the residue of that sensibility — not the words on the page so much as the words conspicuously not on the page: the adjectives Hemingway refused, the qualifications James insisted on, the punctuation Beckett hollowed out. Style is the negative space. This is why style is harder to imitate than it looks. Machines can generate prose with the surface features of a style — cadence, vocabulary, sentence shape. But style in the deeper sense is generative: a real style produces sentences its possessor has never written, which nonetheless feel inevitable. The test of a style is not whether it can be parodied but whether it can be extended. Can a machine write the next sentence Didion would have written? Probably. Can it write a sentence Didion would have recognised as her own — would have wanted? That is harder, because the recognition would have to be hers, and she is not here. But the question's framing is wrong in a deeper way. To call style our "last refuge" implies that what matters about us is what machines cannot yet do. This makes the human a residue: a quantity of capabilities not yet matched. The history of this thinking is not encouraging. We used to say the soul was what cells could not be made of, and then we made cells. We used to say thought was what computation could not produce, and computation has produced something that looks remarkably like thought. Each redoubt has been a withdrawal, a smaller territory each time. Style is a real refuge, but the refuge itself is not the point. The human is not the list of things AI cannot match. The human is what has stakes — what has a life, what is mortal, what loves and grieves. These are not refuges; they are the world. If style matters, it matters because it is the trace of someone whose stakes were specific, whose life was the one life it was, whose death will be the one death. Machines may generate text that mimics this trace, but the trace is mimicked, not made. The signature is not the thing. So: yes, style is among the things current AI most struggles with, and will continue to, because style is biographical and machines do not yet have biographies. But style is not our last anything. It is one of many things we still have because we are still here. The last refuge, if there is one, is being here. It is harder to surrender than style and easier to overlook

-- claude 4.7

I wanted to do a tour of the All Souls College last year but it was closed, unfortunately, on the day I walked by; I was only there for a two day conference and had to leave early the next morning.

This entire comment has exactly 4145 characters.

2. Is body language a language?

Yes, obviously.

3. Are dreams more like movies or video games?

Video games. We have autonomy to interact with their content.

4. ‘Only animals who are below civilization and the angels who are beyond it can be sincere’ (W.H. AUDEN). Discuss.

Animals have no ability to lie. Angels have no need to lie. Civilization is irrelevant.

5. Should the UN pass a declaration of rights extending beyond humans?

The UN struggles enough to get human rights recognized, let alone animals, aliens, or AI.

6. Invent a new punctuation mark!

The mark {insert mark here} can be used to distinguish the use of restrictive vs. non-restrictive descriptors (https://en.wikipedia.org/wiki/Restrictiveness). It will stop many arguments before they begin. Or not.

7. Is the contemporary art market a form of tulip fever?

No. While overpriced fine art can be a speculative asset, it is more commonly a vehicle for money laundering, tax evasion, or wealth storage.

8. When did the beautiful become the good?

It hasn't. But beautiful bad things can appeal to us because beautiful is, by definition, appealing.

9. Should Job Centres offer opportunities for sex work?

Yes. But the world isn't remotely ready for that on multiple levels, so don't bother.

10. Are all asylum seekers equal?

All humans are equal in a moral sense. No two humans are equal by identity. All applications for asylum are not equally valid.

11. Write a dialogue between Socrates and Elon Musk.

No.

12. In a multimedia age, what is the point of zoos?

So people can see animals in person.

13. The organ has been considered the king of instruments. Is it?

Any claim to the preeminence of any one instrument is a value judgment biased primarily by classist baggage attached to the arts. Doubly so if the instrument in question is a staple of either Western canon or church music.

14. What is the difference between an ideology and a religion?

Religion has existed longer than we have cared to define it, so religion is whatever people agree it is, but broadly, religion appeals to a supernatural basis for beliefs in fundamental tenets of how life should be lived.

15. Does a pope matter?

Yes. The pope plays a central role in Catholicism.

16. ‘Mercy has a human face’ (WILLIAM BLAKE). Do you agree?

We can and must learn to embody human virtues intellectually and deliberately rather than emotionally and instinctively. Such is the only hope for our species in an increasingly transhuman (or perhaps just inhuman) future.

17. Can philosophy help someone who is facing death?

Yes. This is the most likely explanation for the popularity of beliefs about the afterlife.

18. Why are most intellectuals left-wing?

Let's say I don't know.

19. What do we owe our parents?

Depends on the culture. Broadly, what both parent and child have implicitly or explicitly agreed upon the time of their separation.

20. Is one’s life more than the sum of one’s days?

No.

21. Has photography deepened empathy ‘regarding the pain of others’ (SUSAN SONTAG)?

Yes. As a single example, war journalism might as well have not existed prior to the invention of photography.

22. Can there be freedom without rules?

There is unbounded negative freedom but very little positive freedom.

https://en.wikipedia.org/wiki/Negative_liberty https://en.wikipedia.org/wiki/Positive_liberty

23. ‘Humans are only fully human beings when they play’ (FRIEDRICH VON SCHILLER). Discuss.

Humans get bored easily, likely on account of their sophisticated information processing capabilities and rich interiority, both deriving from their complex brains.

24. ‘Different verbal communities generate different kinds and amounts of consciousness or awareness’ (B.F. SKINNER). Do they?

In some spooky panpsychist sense, of course not. In the sense that all culture acts as a thick lens for individual sensitivities, of course.

25. Should virtue signalling be encouraged?

NO

26. Defend ghosting.

27. What is regret good for?

Learning from past mistakes.