FRESH

Hacker News

A single-file C allocator with explicit heaps and tuning knobs

54 points by enduku

by enduku

1 subcomments

I wrote this because I wanted more explicit control over heaps when building different subsystems in C. Standard options like jemalloc and mimalloc are incredibly fast, but they act as black boxes. You can't easily cap a parser's memory at 256MB or wipe it all out in one go without writing a custom pool allocator.
Spaces takes a different approach. It uses 64KB-aligned slabs, and the metadata lookup is just a pointer mask (ptr & ~0xFFFF).
The trade-off is that every free() incurs an L1 cache miss to read the slab header, and there is a 64KB virtual memory floor per slab. But in exchange, you get zero-external-metadata regions, instant teardown of massive structures like ASTs, and performance that surprisingly keeps up with jemalloc on cross-thread workloads (I included the mimalloc-bench scripts in the repo).
It's Linux x86-64 only right now. I'm curious if systems folks think this chunk API is a pragmatic middle ground for memory management, or if the cache-miss penalty on free() makes the pointer-masking approach a dead end for general use.

by ntoslinux

2 subcomments

What is the reason for the weird `{ code };` blocks everywhere and is the below code machine generated?
```c ((PageSize) (chunk->pageSize - ((PageSize) ((PageSize) ((PageSize) (sizeof(Page) + (sizeof(struct _Block))) + (PageSize) ((sizeof(double)) - 1u)) & ((PageSize) (~((PageSize) ((sizeof(double)) - 1u)))))) - ((PageSize) ((PageSize) ((PageSize) ((sizeof(FreeBlock) + sizeof(PageSize))) + (PageSize) (((((sizeof(double)) > (4)) ? (sizeof(double)) : (4))) - ```

by HexDecOctBin

1 subcomments

The classic Doug Lee's memory allocator[1] has explicit heaps by the name of mspaces. OP, were you aware of that; and if yes, what does your solution do better or different than dlmalloc's mspaces?
[1] https://gee.cs.oswego.edu/pub/misc/?C=N;O=D

by motters

0 subcomment

As a long time C coder I checked this out, because I have my own malloc replacement. The source code is nonsensical garbage.

by bitbasher

2 subcomments

by xantronix

0 subcomment

Brother. What is up with the Makefile rule 'test'? I don't mean to be harsh but is this performance art?
Edit: Homie. Why is bench.sh fetching external resources? Call me old fashioned, but it would be nice if when I cloned the repository (and checked out any submodules that may exist), I've got everything I need, right there.

by lstodd

1 subcomments

Zig got this right to such a degree that I'm sometimes tempted to export its allocators to C via FFI. Then I sober up a bit and just rewrite it all in zig, all its instability nonwithstanding.

by swetland

0 subcomment

by jeffbee

1 subcomments

"That costs ~5 ns when the line is cold"
I don't see how that could possibly be true. Sounds like a low-ball estimate.
Also i wish to point out that the "tcmalloc" being used as a baseline in these performance claims is Ye Olde tcmalloc, the abandoned and now community-maintained version of the project. The current version of tcmalloc is a completely different thing that the mimalloc-bench project doesn't support (correctly; I just checked).

0 subcomment