FRESH

Hacker News

Home

Alternative(s) to run CUDA on non-Nvidia hardware

139 points by alok-g

by woctordho

5 subcomments

There's nothing wrong to run CUDA on non-Nvidia hardware. CUDA has an interface that is reasonably well-designed, well-documented/reverse-engineered, and battle-tested for decades. What we need is not to invent another interface just under the name of 'open standard', but to implement the same interface. ROCm is exactly doing this, and so are other hardware SDKs such as MooreThread and Alibaba T-Head.

by HarHarVeryFunny

2 subcomments

These efforts to support CUDA on non-Nvidia hardware seem to me misguided. If all you want is to be able to easily use non-NVidia hardware then high level tools like PyTorch already let you do that (and torch.compile uses Triton for target-specific optimization). OTOH if you want to be programming close to the metal to achieve top performance then you are probably not using CUDA in the first place, and using some CUDA translation layer on non-NVidia hardware would be an even worse idea.

by pjmlp

2 subcomments

Most of these "alternatives" focus on CUDA C++, and overlook what actually makes CUDA interesting.
Already in 2020,
https://developer.nvidia.com/blog/cuda-refresher-the-gpu-com...

by luciana1u

1 subcomments

every CUDA alternative follows the same arc: bold launch, works for 3 operations, then a Discord server where the last message is 'any updates?' from 2024

by puschkinfr

1 subcomments

In this context AdaptiveCpp should also be mentioned. Started as a SYCL implementation, but recently-ish added a compiler for compiling a CUDA dialect to GPUs and CPUs from basically all vendors

by u1hcw9nx

0 subcomment

Alternatives exist, but little demand outside hyperscalers and special uses.
Neocloud customers just want plug-and-play CUDA. It works, it's tested, it adapts faster, and has known performance. Alternatives give no significant benefits.
Things can change, but they are not changing now.

by maxloh

1 subcomments

There is also ZLUDA, which is open source and works on pre-compiled binaries.
https://github.com/vosen/ZLUDA

by alightsoul

0 subcomment

I fail to see how scale is not just another form of vendor lock in, given that their compiler is not open source. Every compiler used today except cuda's is open source. And Nvidia can get away with it because no one else cares about development experience

by lulzx

0 subcomment

I have been trying for cuda -> metal, to run it on mac, https://github.com/lulzx/cuda-metal

by woodrowbarlow

1 subcomments

i'm also interested in tenstorrent. they're building GPUs with cheap GDDR6 using a fast SRAM cache, and writing their own compiler stack (used instead of CUDA) that pipelines data to the SRAM ahead-of-time so you (in theory) never need to suffer the slow speed of GDDR6 for AI workloads. also they've got built-in SFP cages where the video ports would normally be.

by dachworker

1 subcomments

Why should I not just port my kernel to Triton? What's the appeal of Scale?

by leoh

1 subcomments

Why is London having such a comeback right now in AI?

by DiabloD3

5 subcomments

Its easier to just get rid of your legacy code entirely and use Vulkan for compute, or have your compiler emit SPIR-V directly.
No reason to tie yourself to Nvidia's moat.

by asdaqopqkq

0 subcomment

aren't llms smart enough to directly write custom kernels for custom hardware from cuda code?

by cactusplant7374

0 subcomment

Isn't the future of the industry specialized chips like those that Broadcom and Cerebras are making? I don't know how much longer I can tolerate 50 tokens per second. It feels like the dial-up era.

by tangsoupgallery

0 subcomment

[dead]

by mailship

0 subcomment

[dead]

by z0ltan

0 subcomment

[dead]

by mangoorangesoda

0 subcomment

[dead]

by villgax

0 subcomment

@claude add this to the graveyard of wannabees