FRESH

Hacker News

Home

Principles of Mechanical Sympathy

81 points by zdw

by bob1029

0 subcomment

> Most AI runtimes can only execute one inference call to a model at a time.
I don't know if the LMAX article makes much sense in this space. The latency of calling the models dominates any sort of local effects and the volume of the requests tends to be very low relative to any kind of meaningful system limits.
More broadly, parallel agent systems are of questionable utility compared to ones that operate over the problem serially. Parallelism bringing any uplift implies that the effects of different parts of the problem don't interact much. If things do depend on other things over time, then you have something that can only be solved with one serial narrative.
The case of high frequency trading works really well because the shared resource (global sequence) is extremely contentious and local. You actually get meaningful benefits out of these principles. OpenClaw is not the same kind of problem. Running an actual inference model on the CPU is maybe a different story.

by whizzter

0 subcomment

Historically I think the people that first ran into these issues and started talking about this was gamedevelopers even if it flew under the radar for many others since it was an era of less third party engines and code sharing.
The motivation was partially to solve processing on PS3's, the 7 usable Cell units have small memories of 256kb each so processing had to be moved to a compact/streamable memory format but developers were also concurrently fighting larger cache latencies caused by Entity Component hierarchies that had arisen as a solution to problems with inheritance using dislocated object hierarchies (ie separated linked objects).
Out of this grew the Data Oriented Design paradigm (that's intrinsically built on mechanical sympathy) that suited both Cell like streaming as well as minimizing cache effects for main-memory code where Entity Component Systems were reorganized to use cache efficient array processing instead of linked objects.
https://gamesfromwithin.com/data-oriented-design
Was the outside world oblivious? I'd say so to a large part, CPU's had gotten faster, more cores and memory latencies were masked to a certain degree for other kinds of code!
The Java people even pushed out their biggest mistake just as these effects started to become visible, the erased objects for generics that kind of locked in a design that requires generic list types to have disaggregated storage due to separate objects. In hindsight, this is IMHO the single biggest benefit that C# got over Java as their paths diverged as generic struct List<> object's have radically more efficent storage in C# compared to a Java ArrayList<>.
Afaik Project Valhalla still hasn't become mainstream even after almost _12_ years at this point, had they gone with proper generics at day one it would've been a trivial compiler upgrade.

by tommodev

0 subcomment

I abused this term a bit to help data scientists & data engineers understand why they should take interest in each others skillsets. I used to liken it to a formula 1 driver (scientist) and the car / pit crew (engineers).
Sure, you can maybe be a great driver without caring about the car or the crew, but it is definitely going to have its limits. Likewise, at the end of the day the crew is there to make the driver shine, and need to be invested in understanding how they operate.
Creates a much better sense of culture and collaboration, and overall better products, when everyone can see the part they play and how important the relationship is to their peers.

by kjellsbells

1 subcomments

It's high time that programming under resource constraints was rediscovered and valued. It's been so easy to throw hardware (or someone else's hardware, ie cloud) at problems that efficiency and cost per compute output has been forgotten. Maybe a few years of serious memory shortage will focus minds. Perhaps this is one of those things that needs to be recast in terms a CFO can understand before it gets attention: "you're paying XXX per compute node because your team writes flabby software. You could save $$$ if they fit into Y instead"

by aledevv

0 subcomment

> If a single writer handles batches of writes (or reads!), build each batch greedily: Start the batch as soon as data is available, and finish when the queue of data is empty or the batch is full.
> ..prioritize observability before optimization. You can't improve what you can't measure. Before applying any of these principles, define your SLIs, SLOs, and SLAs so you know where to focus and when to stop.
These principles apply not only to individual applications, but also to all systems as a whole. The single-writer principle improves performance both when writing/reading large databases and when reading/writing to RAM. Where input/output is intensive, performance improves even further.

by jandrewrogers

1 subcomments

A single fundamental principle underlies the implementation of “mechanical sympathy” in software: make the topology of the software match the topology of the hardware as closely as possible. All of the various tricks and heuristics are just narrow cases of this single principle. Hardware architecture is immutable but software architecture is not.
It was how I learned to design code for supercomputers and is remarkably effective at generating extreme efficiency and scalability in diverse contexts. Importantly, it generalizes to systems of every shape and size. While it is difficult to get good results on a supercomputer if you don’t understand this, it works just as well for a random web app.

by globular-toast

3 subcomments

I looked for a term like "mechanical sympathy" for years. In fact, I think I even came up with that independently when I was thinking about it. I remember asking on some internet forum if anyone else feels a sense of sympathy towards machines and most people made fun of me. A few admitted they didn't like crunching the gears in their car gearbox.
But I was talking about all machines. I don't like it when a washing machine is off balance and vibrating violently. I don't like to use my tools at, or anywhere near, there mechanical limits. When I observe normies they don't seem to care. They'll force and abuse things all the time. I did wonder if it was part of my predisposition towards engineering.