FRESH

Hacker News

Home

Against Query Based Compilers

80 points by surprisetalk

by recursivecaveat

4 subcomments

Mass unqualified imports is a feature that I really dislike in languages. At least please make it less syntactically convenient so people are not drawn to it. Trawling around to find out where symbols come from has wasted a lot more of my time than unqualified imports have ever saved. Using a very short alias gets you 90% of the way there anyways.

by armchairhacker

3 subcomments

> In Zig, every file can be parsed completely in isolation...every name needs to be explicitly declared (there’s no use *)...
> In contrast, you can’t really parse a file in Rust...Expanding macros requires name resolution, which, in Rust, is a crate-wide, rather than a file-wide operation...Similarly, the nature of the trait system is such that impl blocks relevant to a particular method call can be found almost anywhere...
matklad doesn't even mention dynamic languages, where perfect name resolution is undecidable. "Fast static analysis, easy static analysis, language designed for static analysis - pick two".
Rust's IDE integration is fast and deep, and I've heard TypeScript's is too, so "easy static analysis" may not be important today. I believe it will as coding evolves due to LLMs, albeit without evidence and I'm not quite sure how.

by smj-edison

0 subcomment

It seems like he's not arguing against query-based compilers per se, but rather he's arguing against the thinking that often comes with it. Namely, that you can amortize an expensive compilation process across incremental changes (which often doesn't work in practice because of the avalanche effect).

by Panzerschrek

2 subcomments

I have a language server implementation for my programming language. Since it has been developed much later than its compiler, no stuff like query-based redesign was viable to implement. So, I just adopted the compiler codebase for language server needs in a tricky way: on each document change I just redo parsing, but not other compilation steps and use parsing results to perform fetches from name tables built previously - for some older document state. This state is costly to update and it's done only once in a while and only if input document is syntactically-correct. This approach works surprisingly well (it's fast and completion is mostly correct) and doesn't require throwing the whole compiler code away.
Even more, I can't imagine how people write language servers with full reprocessing on each document change (even with query-based approach). How do they deal with broken syntax? isn't it generally impossible to recovery syntactic errors, so that the end result is as good as expected?

by noelwelsh

1 subcomments

The article doesn't mention the other side of the tradeoff, which is that features like Rust's traits or macros make the language more expressive. Given that Rust's LSP server is pretty snappy, these features don't seem to cause problem in practice for incremental compilation.

by snthpy

0 subcomment

> Similarly, the nature of the trait system is such that impl blocks relevant to a particular method call can be found almost anywhere. For every trait method call, you get a dependency on the impl block that supplies the implementation, *but you also get a dependency on non-existence of conflicting impls in every other file*!
The last clause is a great point and one I will try to remember.

by jitl

0 subcomment

i agree on the whole that query is a good second line of defense versus the primary strategy. however you can do query based strategy and still use a diff/patch approach for some queries, just make the previous state of a query one of its inputs.
query based compilers are equivalent to component UI frameworks / signals / self-adjusting computation / spreadsheets. signia signals library implements this idea of using the previous output + diffs to compute the next output (https://signia.tldraw.dev/docs/incremental#using-diffs-in-co...)

by zarzavat

4 subcomments

> In contrast, you can’t really parse a file in Rust. Rust macros generate new source code, so parsing can’t be finished until all the macros are expanded.
Rust needs macros because the language is extremely strict and verbose. Macros are the main escape hatch, allowing you to simplify your code while maintaining memory safety without a GC.
Zig doesn't have memory safety as a goal so it seems like an unfair comparison.

by jiggawatts

3 subcomments

> This is provably impossible to make usefully incremental. The encryption can be implemented as a graph of function calls, and you can apply the general incremental recipe to it. It just won’t be very fast.
The specific example is bogus.
Merkle trees and their many variants exist to solve precisely this problem.
For more compiler-specific scenarios there exist vaguely similar solutions to the issues introduced by incremental compilation such as splitting up monolithic executables into many small dynamically loaded libraries (only during development time), or taking that to the extreme, hot code reloading at the function level.
> ... only because Zig language is carefully designed to allow this.
Is the key point. Rust wasn't designed for incremental compilation, and most legacy languages like C only allow it in a useful way because they were designed in the era "kilobytes" of system memory and hence they're very frugal in their design.
Other than Zig, the only modern language I'm aware of that was "co-designed" for efficient compilation and incremental reload is Jai.
> Expanding macros requires name resolution, which, in Rust, is a crate-wide, rather than a file-wide operation.
Sure, but macros change infrequently, so that ought to be a highly cacheable pure function for most edits.
> The above scheme works only if the language has a property that changing the body of function foo (not touching its signature) can’t introduce type errors into an unrelated function bar.
AFAIK Rust and most other statically typed languages have this property. Optimisations such as inlining can mess with the cacheability of code-gen in all languages.