Our container platform is in production. It has GPUs. Here's an early look
- So I just discovered that Cloudfare now owns the trademark for Sun's "The Network is the Computer".
"Cloudflare serves the entire world — region: earth. Rather than asking developers to provision resources in specific regions, data centers and availability zones, we think “The Network is the Computer”. "
https://blog.cloudflare.com/the-network-is-the-computer/
- > To add GPU support, the Google team introduced nvproxy which works using the same principles as described above for syscalls: it intercepts ioctls destined to the GPU and proxies a subset to the GPU kernel module.
This does still expose the host's kernel to a potentially malicious workload, right?
If so, could this be mitigated by (continuously) running a QEMU VM with GPUs passed through via VFIO, and running whatever Workers need within that VM?
The Debian ROCm Team faces similar challenge, we want to do CI [1] for our stack and all our dependent packages, but cannot rule out potentially hostile workloads. We spawn QEMU VMs per test (instead of the model described above) but that's because our tests must also be run against the relevant distribution's kernel and firmwares.
Incidentally, I've been monitoring the Firecracker VFIO GitHub issue linked in the article. Upstream does not have a use case for and thus no resources dedicated to implement this, but there's a community meeting [2] coming up in October to discuss the future of this feature request.
[1]: https://ci.rocm.debian.net
[2]: https://github.com/firecracker-microvm/firecracker/issues/11...
by LukeLambert
2 subcomments
- This is really cool and I can't wait to read all about it. Unfortunately, I've missed a month of blog posts because Cloudflare changed their blog's RSS URL without notice. If you change blogging platforms and can't implement a 301, please leave a post letting subscribers know where to find the new feed. RSS isn't dead!
by tarasglek
1 subcomments
- I love they built all this infra for running js to avoid building a container runtime and ended up building a container platform using all the hypervisors. On more serious note, I do not understand why they can't fix the 500mb upload limit. I hit that with r2 registry and ended switching away instead changing all the dockepush tooling. Not super excited re using more weird tools rather than fixing platform
by breatheoften
1 subcomments
- Why does it take 4 minutes (after being optimized from 8 minutes!) to move a 30 GB (compressed) docker image ...? The read slowness of docker registries continues to surprise me ...
- surprised they didn't go straight into cloud-hypervisor, though I haven't actually tested with gpu yet but it is on my todo list. OCI layers can use zstd compression. I wonder if they are defeating layer sharing by splitting in 500 mb chunks. Lambda splits your image into chunks and shares at the block layer (I believe even same chunk different (user's?) container on a single host). Esp for 15 GB images I'd think using lazy pulling with nydus/stargz or whatever would be beneficial. I'd like to test out snapshotting, though my testing already boots a guest and runs a container in ~170ms; and I'm not actually sure how you write the guest init to signal it is ready for snapshotting and then wait properly (maybe you just sleep 1000?) so it resumes from the snapshot in a good state. I know fly has written about their use of snapshotting but I don't think it went into that detail. Cool stuff overall though, not worrying about locations and the yucky networking to do so seems nice
- I like using Workers for smallish http services. The uptime, pricing, and latency are fantastic. I would never use them for anything complex as the vendor lock in is quite strong and the dev experience still needs to improve.
Containers on the edge with low cold starts, scalability, the same reliability as Workers, etc would be super cool. In part to avoid the lock in but also to be able to use other languages like Go (which Workers don't support natively).
by thefounder
0 subcomment
- So this will be similar to Google Appengine(now Google run) ? If that’s the case I would love to give it a try but then I need close SQL server nearby and other open source services as well
by dopylitty
3 subcomments
- I like the dig at "first generation" clouds.
There really is a wide gulf between the services provided by the older cloud providers (AWS, Azure) and the newer ones (fly.io, CloudFlare etc).
AWS/Azure provide very leaky abstractions (VMs, VPCs) on top of very old and badly designed protocols/systems (IP, Windows, Linux) . That's fine for people who want to spend all their time janitoring VMs, operating systems, and networks but for developers who just want to write code that provides a service it's much better to be able to say to the cloud provider "Here's my code, you make sure it's running somewhere" and let the cloud provider deal with the headaches. Even the older providers' PaaS services have too many knobs to deal with (I don't want to think about putting a load balancer in front of ECS or whatever)
- What I am always missing in these posts: How do they limit network bandwidth? Since these are all multi-tenant services, how do they make sure a container or isolated browser is not taking all the network bandwidth of a host?
- This seems like a pretty big deal.
I want to like CloudFlare over DO/AWS. I like their DevX focus too -- I could see issues if devs can't get into the abstractions though.
Any red flags folks would stake regarding CF? I know they are widely used but not sure where the gotchas are.
by CSMastermind
3 subcomments
- Does it say anywhere what GPUs they have available?
I really need NVIDIA RTX 4000, 5000, A4000, or A6000 GPUs for their ray tracing capabilities.
Sadly I've been very limited in the cloud providers I can find that support them.
- Lots of cool stuff in this blog post. Impressive work on many fronts!
If I understand correctly, you will be running actual third party compute workloads/containers in hundreds of network interexchange locations.
Is that in line with what the people running these locations have in mind? Can you scale this? Aren't these locations often very power/cooling-constrained?
by SantaCruz11
1 subcomments
- Edgegap has been doing this for 5 years.
- > Cloudflare serves the entire world — region: earth
Is that true for China though?
by lofaszvanitt
0 subcomment
- Why is Cloudflare trying to create a walled-garden internet within the internet?
by surfingdino
1 subcomments
- Looks like CloudFlare will soon be using "All other clouds are behind ours." slogan.
- > ”Remote Browser Isolation provides Chromium browsers that run on Cloudflare, in containers, rather than on the end user’s own computer. Only the rendered output is sent to the end user.”
It turns out we don’t need React Server Components after all. In the future we will just run the entire browser on the server.
- > We rely on it in Production
They really have a great engineering team
by Toritori12
0 subcomment
- [dead]
Hacker News