by adamzwasserman
7 subcomments
- There's an interesting falsifiable prediction lurking here. If the language network is essentially a parser/decoder that exploits statistical regularities in language structure, then languages with richer morphological marking (more redundant grammatical signals) should be "easier" to parse — the structure is more explicitly marked in the signal itself.
French has obligatory subject-verb agreement, gender marking on articles/adjectives, and rich verbal morphology. English has largely shed these. If you trained identical neural networks on French vs English corpora, holding everything else constant, you might expect French models to hit certain capability thresholds earlier — not because of anything about the network, but because the language itself carries more redundant structural information per token.
This would support Fedorenko's view that the language network is revealing structure already present in language, rather than constructing it. The "LLM in your head" isn't doing the thinking — it's a lookup/decode system optimized for whatever linguistic code you learned.
(Disclosure: I'm running this exact experiment. Preregistration: https://osf.io/sj48b)
- Anecdotal data, based on a sample of 1 (aka me). I'm originally Polish, but I would say my mother tongue is English. I also learned Latin as a kid/teen. Then learning any other languages is much easier, I also learned German and some Swiss German dialects. I can also do Spanish, Italian, French, Dutch, Czech, some Serbo-Croation. I think being Polish makes learning languages easy - as we have a lot of creations in Polish that do not translate easily to other languages. I think in my case it's the same part of brain that processes both human language and computer language. My brain can do another fun party trick: I never learned cyrillic, but I can read it just fine, my brain does like pattern matching and statistical analysis when reading cyrillic.
I also learned to think in hmm "concepts", and then apply a language of my choice to express them. It's a fun skill to have :) Obviously works of Chomsky are great, especially exploring if language evolves mind or is the other way around, does mind evolve language? [let's skip his rather controversial political views lately].
by liampulles
6 subcomments
- What I'm curious about is what the language parts of the human brain look like for babies and toddlers. Humans obviously have a bunch of languages they can speak, and toddlers pick up the language that their guardians speak around their home, so there seems to be machinery there that is for the task of "online" learning.
by netfortius
1 subcomments
- Every time I read something like this reminds me of Maturana (of autopoiesis fame), who was among the first scientists from where I started gaining an interest in these areas. Relevant to his view, in the area of language, is the following:
"We human beings are living systems that exist in language. This means that although we exist as human beings in language and although our cognitive domains (domains of adequate actions) as such take place in the domain of languaging, our languaging takes place through our operation as living systems. Accordingly, in what follows I shall consider what takes place in language[,] as language arises as a biological phenomenon from the operation of living systems in recurrent interactions with conservation of organization and adaptation through their co-ontogenic structural drift, and thus show language as a consequence of the same mechanism that explains the phenomena of cognition:"
- > But what if our neurobiological reality includes a system that behaves something like an LLM?
It almost seems like we got inspiration from our brain to build neural networks!
- > But what if our neurobiological reality includes a system that behaves something like an LLM?
With every technological breakthrough we always posit that the brain has to work like the newly discovered thing. At various times brains were hydraulic, mechanical, electrical, like a computer, like a network. Now, of course, the brain has to be like an LLM.
- > It almost sounds like you’re saying there’s essentially an LLM inside everyone’s brain. Is that what you’re saying?
>Pretty much. I think the language network is very similar in many ways to early LLMs, which learn the regularities of language and how words relate to each other. It’s not so hard to imagine, right?
Yet, completely glosses over the role of rhythm in parsing language. LLMs aren’t rhythmic at all, are they? Maybe each token production is a cycle, though… hmm…
by moralIsYouLie
1 subcomments
- reads like a collection of HN comments by commenters who like to build "chapter 1" textbook agents using instant-noodle "training tools". "and what would be the time complexity?"
I can't do this anymore.
- Language and Reality: a Wittgensteinian Reading of Bhartrhari - https://wab.uib.no/agora/tools/alws/collection-9-issue-1-art...
New Vistas to study Bhartrhari: Cognitive NLP (Natural Language Processing) - https://arxiv.org/abs/1810.04440
by fallingfrog
0 subcomment
- I've had the experience of having migraines with aphasia- this is essentially a migraine aura that affects the part of the brain that processes language. I can confirm that while this was happening, i was aware of my surroundings and able to have thoughts, but I was unable to speak and unable to understand spoken or written language. It all just looked and sounded like gibberish. I thought about whether I should go to a hospital, what was going on, wondered whether my loved ones were concerned, and so on, but was unable to communicate any of those thoughts to other people. It was a bizarre experience.
- If the brain's language network is only for "packaging words" and not for actual logic or reasoning, why does writing or speaking our messy thoughts out loud suddenly make them feel more logical? Is language actually helping us think, or is it just a filter that forces our chaotic ideas into a structure we can finally understand?
by griffzhowl
1 subcomments
- One disanalogy between human language use and LLMs is that language evolved to fit the human brain, which was already structured by millions of years of primate social life. This is more or less the reverse situation to a neural network trained on a large text corpus.
- This feels too reductive to me. In particular, it makes a hard distinction between the thinking and the language. I fully accept that they are distinct, but how distinct? It is hard not to think that some thinking styles influence how something is heard?
Not just in full language, mind, but consider the last time you heard a song in a major key? Do you even know what that means? Because many of us do not.
Same goes for listening to people discuss things like sports. I'm inclined to think many people effectively run a simulation in their mind of a game as they listen to it broadcast. This almost certainly isn't inherent to the language, it is part of the learning of it, though. Think looking over lists of the moves in a chess game. Then go from that to laying out the pieces as they are after that list. Or calling what the next move can be.
Can this be a completely separate set of "circuitry" in our brains that first parses the language and then builds the simulation? I suppose. Seems more likely there is something that is active between the two that can effectively get merged in advanced practitioners.
- I wouldn't read too much into the LLM analogy. The interview is disappointingly short, filled with a bunch of unnecessarily tall photgraphs, and the interviewer, the one who brought up LLMs and ChatGPT and has a history of writing AI articles (https://www.quantamagazine.org/authors/john-pavlus/), almost seemed to have an agenda to contextualize the research in this way. In general, except in a hostile context such as politics, interviewees tend to be agreeable and cooperative with interviewers, which means that interviews can be steered in a predetermined way, probably for clickbait here.
In any case, there's a key disanalogy:
> Unlike a large language model, the human language network doesn’t string words into plausible-sounding patterns with nobody home; instead, it acts as a translator between external perceptions (such as speech, writing and sign language) and representations of meaning encoded in other parts of the brain (including episodic memory and social cognition, which LLMs don’t possess).
Hacker News