TurboQuant: Redefining AI efficiency with extreme compression
11 points by davidbarker
by Reubend
0 subcomment
This looks great, but I'm wondering how effective this would be for full model weights rather than just the KV cache. Their paper only gives results for the KV cache use case, which strikes me as strange since the algos are claimed to be near optimal.
Hacker News