FRESH

Hacker News

Transformers know more than they can tell: Learning the Collatz sequence

128 points by Xcelerate

by jebarker

1 subcomments

This is an interesting paper and I like this kind of mechanistic interpretability work - but I cannot figure out how the paper title "Transformers know more than they can tell" relates to the actual content. In this case what is it that they know and can't tell?

by rikimaru0345

3 subcomments

Ok, I've read the paper and now I wonder, why did they stop at the most interesting part?
They did all that work to figure out that learning "base conversion" is the difficult thing for transformers. Great! But then why not take that last remaining step to investigate why that specifically is hard for transformers? And how to modify the transformer architecture so that this becomes less hard / more natural / "intuitive" for the network to learn?

by Onavo

2 subcomments

Interesting, what about the old proof that neural networks can't model arbitrary length sine waves?

by niek_pas

3 subcomments

Can someone ELI5 this for a non-mathematician?