FRESH

Hacker News

Build a Deep Learning Library

134 points by butanyways

by megadragon9

2 subcomments

Thanks for sharing! It's inspiring to see more people "reinventing for insight" in the age of AI. This reminds me of my similar previous project a year ago when I built an entire PyTorch-style machine learning library [1] from scratch, using nothing but Python and NumPy. I started with a tiny autograd engine, then gradually created layer modules, optimizers, data loaders etc... I simply wanted to learn machine learning from first principles. Along the way I attempted to reproduce classical convnets [2] all the way to a toy GPT-2 [3] using the library I built. It definitely helped me understand how machine learning worked underneath the hood without all the fancy abstractions that PyTorch/TensorFlow provides. I eventually wrote a blog post [4] of this journey.
[1] https://github.com/workofart/ml-by-hand
[2] https://github.com/workofart/ml-by-hand/blob/main/examples/c...
[3] https://github.com/workofart/ml-by-hand/blob/main/examples/g...
[4] https://www.henrypan.com/blog/2025-02-06-ml-by-hand/

by amitav1

0 subcomment

This is cool! This summer I made something similar but in C++. The goal was to build an entire LLM, but I only got to neural networks. GitHub repo here: https://github.com/amitav-krishna/llm-from-scratch. I have a few blogs on this project on my website (https://amitav.net/building-lists.html, https://amitav.net/building-vectors.html, https://amitav.net/building-matrices.html (incomplete)). I hope to finish that series eventually, but some other projects have stolen the spotlight! It probably would have made more sense to write it in Python because I had no C++ experience.

by csantini

1 subcomments

Did something similar a while back [1], best way to learn neural nets and backprop. Just using Numpy also makes sure you get the math right without having to deal with higher level frameworks or c++ libraries.
[1] https://github.com/santinic/claudioflow

by silentsea90

1 subcomments

Isn't this what Karpathy does as well in the Zero to Hero lecture series on YT? I am sure this is great as well!

by yunnpp

1 subcomments

It's alright, but a C version would be even better to fully grasp the implementation details of tensors etc. Shelling out to numpy isn't particularly exciting.

by grandimam

1 subcomments

This is good. Its well positioned for software engineers to understand DL stuff beyond the frameworks.

by yazide

0 subcomment

by opan

0 subcomment

Perhaps obvious to some, but this does not seem to be about learning in the traditional sense, nor a library in the book sense, unfortunately.