For example, the "light" test will do calls to "dynamodb". Maybe you benchmark python, or you benchmark the aws sdk implementation with different versions of python, or just you benchmark average dynamodb latencies at different times of the day, ...

And, at least for Python, his cpu and memory test code looks like to be especially specific. For example, if you use "sha256" or any other hashlib, you will not really benchmark python but the C implementation of the hashlib, that might depend on the crypto or gcc libraries or optimization that was used by AWS to build the python binaries.

- How is Rust only one order of magnitude faster than Python?

- How is Python that much faster than Node.js?

So I looked at the benchmark repo.

These benchmarks mean nothing folks!

Each of these benchmarks is just a SHA256 hash. This is NOT a valid way to compare CPUs, except if the only thing you will ever do with the CPU is to execute SHA256 hashes.

Hash functions are not representative of the performance of:

- Compression or decompression (of text, video, or anything really)

- Parsing

- Business logic (which is often dominated by pointer chasing)

So, you can safely ignore the claims of this post. They mean nothing.

In terms of perf, Node has a pretty snappy JIT, and seems to perform OK in the browser, but this seems like something's not right here.

~200ms requests even in the light case are on the very brink of barely acceptable.

On the other hand, Python performs very well, but it's alarming to see it gets slower in every release by a significant margin.

I found that Node 22 had ~50ms slower coldstarts than Node 20. This is because the AWS Javascript V3 SDK loads the http request library which became much heavier in Node 22. This happens on the newly released Node 24 as well.

I recommend that if you are trying to benchmark coldstarts on Lambda, you measure latency from a client as well. The Init Duration in the logs doesn't include things like decrypting environment variables which adds ~20ms and other overhead like pulling the function code from S3. The impact of this manifests when comparing runtimes like llrt to Node, the Init Duration is faster than Node, but the E2E time from the client is actually closer because the llrt bundle size is 4-5MB larger than Node.

Amazon was so ridiculously gimped and expensive that it was almost unfair, thus comparing ARM and X64 on Amazon thus runs into whatever arbitrary "savings" AWS does.

Tangent: when you have a lambda returning binary data, it’s pretty painful to have to encode it as base64 just so it can be serialized to json for the runtime. To add insult to injury, the base64 encoding is much more likely put you over the response size limits (6MB normally, 1MB via ALB). The c++ lambda runtime (and maybe rust?) lets you return non-JSON and do whatever you want, as it’s just POSTing to an endpoint within the lambda. So you can return a binary payload and just have your client handle the blob.

Then I deoloyed an ECS task with ALB and got something like <5ms.

Has anybody gotten sub-10ms latencies with Lambda Https functions?

If I want to use TypeScript for Functions, I should write to the v8 runtimes (Deno, Cloudflare, Supabase, etc) which are much faster due to being much more lightweight.

Not sure why this is phrased like that in the TL;DR, when ARM64 is just strictly faster when running the same nodejs workload and version.