FRESH

Hacker News

Nested Learning: A new ML paradigm for continual learning

149 points by themgt

by abracos

1 subcomments

Someone's trying to reproduce it in open https://github.com/kmccleary3301/nested_learning

by aktuel

1 subcomments

There is also a related youtube video online: Ali Behrouz of Google Research explaining his poster paper entitled "Nested Learning: The Illusion of Deep Learning Architecture" at NeurIPS 2025. https://www.youtube.com/watch?v=uX12aCdni9Q

by heavymemory

0 subcomment

The idea is interesting, but I still don’t understand how this is supposed to solve continual learning in practice.
You’ve got a frozen transformer and a second module still trained with SGD, so how exactly does that solve forgetting instead of just relocating it?

by Bombthecat

1 subcomments

Damn, and before that, Titan from Google: https://research.google/blog/titans-miras-helping-ai-have-lo...
We are not at the end of AI :)
Also, someone claimed that NVIDA combined diffusion and autoregression, making it 6 times faster, but couldn't find a source. Big if true!

by panarchy

0 subcomment

I've been waiting for someone to make this since about 2019 it seemed pretty self-evident. It will be interesting when they get to mixed heterogeneous architecture networks with a meta network that optimizes for specific tasks.