FRESH

Hacker News

Home

Show HN: Nimic – Pure Python as a systems language with AOT compilation

41 points by dima-quant

by dima-quant

4 subcomments

nimic is a lightweight pure Python package that emulates Nim types and constructions, making it straightforward to transpile to Nim and compile AOT. Key principle: nimic code is valid Python that runs unmodified in CPython and also transpiles to equivalent Nim code.
Because nimic code is just standard Python with type hints and ctypes shims, it is a fully valid CPython script, so you can use the Python REPL during development, drop a breakpoint in the middle of a heavy algorithmic loop and inspect the variables natively.
Zero Lock-In: You don't need a special runtime engine. If the Nim compiler is not available, your script still runs (albeit slower than standard Python due to some emulation overhead) on any machine with Python installed.
Seamless Distribution: You can use this to develop high-performance logic natively in Python, debug it with Python tooling, and then compile to a native executable or C-extension via Nim.
Why Nim? Its syntax maps well to Python, it is rather clear how to emulate its constructions in Python, and its performance is comparable to C (as it compiles to C). Port of the "trace-of-radiance" Nim project to nimic can be found in "ndsl_raytracer" in my GitHub repo (dima-quant). With the compiled executable the render time for a single 512x288 scene dropped from many hours in Python to just 10 minutes on a single M1 CPU core. The repo also includes nimic ppm to mp4 converter.
Similar projects: - Pyccel (https://github.com/pyccel/pyccel): Python extension language using accelerators - SPy (https://github.com/spylang/spy) is a variant of Python specifically designed to be statically compilable while retaining a lot of the "useful" dynamic parts of Python. - Codon (https://github.com/exaloop/codon) is a high-performance Python implementation that compiles to native machine code without any runtime overhead.
It is still work in progress, e.g. there is no JIT and multiprocessing support yet, but now I'm not sure what functionality would be best to implement next. Any suggestions?

by frumiousirc

1 subcomments

What is "AOT"?

by actionfromafar

1 subcomments

Similar idea as Shedskin which is a (large subset of) Python to C++ transpiler
https://github.com/shedskin/shedskin

by vshulcz

3 subcomments

The line I'd push on is "valid Python that runs unmodified in CPython." True at the syntax level, but the speedup isn't really coming from Nim as a backend, it comes from how much of Python's dynamism you can pin down at compile time. Refcounting semantics, __getattr__, values that change type at runtime, isinstance-based dispatch, monkeypatching in tests: the moment you statically commit to any of those, you've defined a subset, and the subset is the actual product, not the transpiler.
I've spent a fair bit of time generating specialized straight-line code for hot Python paths, killing the per-call attribute and dict lookups the interpreter does. The lesson was that dispatch-bound code claws back most of its overhead without ever leaving CPython. Where AOT-to-native actually pulls ahead is numeric and loop-bound work, where the interpreter loop and boxing dominate. Your 512x288 render is exactly that case, which is why it looks so strong.
So the benchmark I'd want isn't render time, it's what fraction of a real module transpiles with no rewrites. That number tells me whether this is a systems language or a fast path I have to hand-shape around. Codon and Shedskin both hit that wall. Curious where Nimic draws the line.