FRESH

Hacker News

Rotary GPU: Exploring Local Execution for Large MoE Models Under Limited VRAM

40 points by dryarzeg

by martinald

2 subcomments

Why is this a paper? It's just using the n-cpu-moe option on llama.cpp? What am I missing here?

0 subcomment

by sandworm101

0 subcomment

Um, doesn't the 4060 laptop card have the ability to share system memory?
Wait... My mistake. Google AI says the 4060 mobile can access system memory but tech sheets say no.