FRESH

Hacker News

Home

Node.js needs a virtual file system

151 points by voctor

by indutny

5 subcomments

Taking the question of whether this would be a useful addition to Node.js core or aside, it must be noted that this 19k LoC PR was mostly generated by Claude Code and manually reviewed by the submitter which in my opinion is against the spirit of the project and directly violates the terms of Developer's Certificate of Origin set in the project's CONTRIBUTING.md

by wccrawford

2 subcomments

I'm not convinced that allowing Node to import "code generated at runtime" is actually a good thing. I think it should have to go through the hoops to get loaded, for security reasons.
I like the idea of it mocking the file system for tests, but I feel like that should probably be part of the test suite, not Node.
The example towards the end that stores data in a sqlite provider and then saves it as a JSON file is mind-boggling to me. Especially for a system that's supposed to be about not saving to the disk. Perhaps it's just a bad example, but I'm really trying to figure out how this isn't just adding complexity.

by wei03288

0 subcomment

This resonates a lot. The number of times I've seen Node.js projects ship with fragile path-joining logic that breaks across OS boundaries is wild. A VFS layer would clean up so many edge cases.
One angle the article doesn't cover much: testing. Right now mocking the file system in Node requires either sinon stubs on every fs method or something like memfs. A built-in VFS would make it trivial to spin up an isolated file tree per test case. No temp directories, no cleanup, no flaky CI from parallel tests writing to the same path.
The performance concern is valid though. Any abstraction layer adds overhead, and for I/O-heavy workloads even a thin wrapper matters. I'd love to see benchmarks comparing direct fs calls vs a VFS proxy on something like a large Webpack build.

by lacoolj

0 subcomment

Using Claude for code you use yourself or at your own company internally is one thing, but when you start injecting it into widely-shared projects like this (or, the linux kernel, or Debian, etc) there will always be a lingering feeling of the project being tainted.
Just my opinion, probably not a popular one. But I will be avoiding an upgrade to Node.js after 24.14 for a while if this is becoming an acceptable precedent.

by giancarlostoro

0 subcomment

> I pointed the AI at the tedious parts, the stuff that makes a 14k-line PR possible but no human wants to hand-write: implementing every fs method variant (sync, callback, promises), wiring up test coverage, and generating docs.
This is the biggest takeaway for me for AI. It's not even that nobody wants to do these things, its that by the time you finish your tasks, you have no time to do these things, because your manage / scrum master / powers that be want you to work on the next task.

by PaulHoule

8 subcomments

Would be nice if node packages could be packed up in ZIP files so to avoid the security/metadata tax for small file access on Windows.

by gnarbarian

0 subcomment

one of the reasons I prefer deno is the availability of indexeddb (and all the other great stuff that comes with it out of the box)

by minraws

0 subcomment

Why is this not a library what is this insanity??

by butz

0 subcomment

How about trying to reduce dependencies? 11ty is going in correct direction, dropping significant chunk of various dependencies or replacing them with packages with no dependencies or using platform features, that becomes readily available.

by mg

2 subcomments

    You can’t import or require() a module
    that only exists in memory.

You can convert it into a data url and import that, can't you?

by austin-cheney

4 subcomments

Most of the 4 justifications mentioned sound like mitigations of otherwise bad design decisions. JavaScript in the browser went down this path for the longest time where new standards were introduced only to solve for stupid people instead of actually introducing new capabilities that were otherwise unachievable.
I do see some original benefits to a VFS though, bad application decisions aside, but they are exceedingly minor.
As an aside I think JavaScript would benefit from an in-memory database. This would be more of language enhancement than a Node.js enhancement. Imagine the extended application capabilities of an object/array store native to the language that takes queries using JS logic to return one or more objects/records. No SQL language and no third party databases for stuff that you don't want to keep in offline storage on a disk.

by syrusakbary

1 subcomments

Funnily enough, we just released Edge.js, which uses Wasmer under the hood for sandboxing Node.js apps.
With it, you have a virtual fs automatically, just by using the `node:fs` package (or any other filesystem calls!)
We wrote about this in depth here: https://wasmer.io/posts/edgejs-safe-nodejs-using-wasm-sandbo...

by mohsen1

1 subcomments

Yarn, pnpm, webpack all have solutions for this. Great to see this becoming a standard. I have a project that is severely handicapped due to FS. Running 13k tests takes 40 minutes where a virtual file system that Node would just work with it would cut the run time to 3 minutes. I experimented with some hacks and decided to stay with slow but native FS solution.
What I really want is a way of swapping FS with VFS in a Node.js program harness. Something like
```
     node --use-vfs --vfs-cache=BIG_JSON_FILE 
```
So basically Node never touches the disk and load everything from the memory

by Normal_gaussian

1 subcomments

yarn pnp is currently broken on Node v25.7+;
- https://github.com/yarnpkg/berry/issues/7065
- https://github.com/nodejs/node/issues/62012
This is because yarn patches fs in order to introduce virtual file path resolution of modules in the yarn cache (which are zips), which is quite brittle and was broken by a seemingly unrelated change in 25.7.
The discussion in issue 62012 is notable - it was suggested yarn just wait for vfs to land. This is interesting to me in two ways: firstly, the node team seems quite happy for non-trivial amounts of the ecosystem to just be broken, and suggests relying on what I'm assuming will be an experimental API when it does land; secondly, it implies a lot of confidence that this feature will land before LTS.

by sidewndr46

0 subcomment

Don't all projects eventually grow to encompass service discovery?

by notnullorvoid

0 subcomment

I could see something like this being useful if it could be passed to workers to replace any fs access inside the worker.

by ozlikethewizard

0 subcomment

I'm not convinced this needs to be in core Node, but being able to have serverless functions access a file system without providing storage would definitely have some use cases. Had some fun with video processing recently that this would be perfect for.

by gwbas1c

0 subcomment

Can you dynamically load code via eval?
(I know, I know, it's ugly and has its own set of problems)

0 subcomment

by adzm

0 subcomment

How does electron do this with its packaged files? I suppose it does not work with module resolution?

by verdverm

0 subcomment

Separate the valid critiques on other comments, Go's io.FS interface is really nice for making these sorts of things. Is there something like this in Node already? (with base implementations like host and in memory)

by themafia

0 subcomment

> You can’t import or require() a module that only exists in memory.
Sure you can. Function() exists and require.cache exists. This is _intentionally_ exploitable.

by bronlund

1 subcomments

Yeah. That’s what we need. More Node.

by westurner

0 subcomment

Is node::vfs the new solution for JupyterLite filesystems?
From https://github.com/jupyterlite/jupyterlite/issues/949#issuec... :
> Ideally, the virtual filesystem of JupyterLite would be shared with the one from the virtual terminal.
emscripten-core/emscripten > "New File System Implementation": https://github.com/emscripten-core/emscripten/issues/15041#i... :
> [ BrowserFS, isomorphic-git/lightningfs, ]
pyodide/pyodide: "Native file system API" #738: https://github.com/pyodide/pyodide/issues/738 re: [Chrome,] Filesystem API :
> jupyterlab-git [should work with the same VFS as Jupyter kernels and Terminals]
pyodide/pyodide: "ENH Add API for mounting native file system" #2987: https://github.com/pyodide/pyodide/pull/2987

by moralestapia

0 subcomment

>Let me be honest: a PR that size would normally take months of full-time work. This one happened because I built it with Claude Code.
The node.js codebase and standard library has a very high standard of quality, hope that doesn't get washed out by sloppy AI-generated code.
OTOH, Matteo is an excellent engineer and the community owes a lot to him. So I guess the code is solid :).

by leontloveless

0 subcomment

[dead]

by aplomb1026

0 subcomment

[dead]

by openinstaclaw

0 subcomment

[dead]

by rigorclaw

0 subcomment

[flagged]

by andrewmcwatters

0 subcomment

[dead]

by buttsack

0 subcomment

[flagged]

by petcat

5 subcomments

Are people still building new projects on Node.js? I would have thought the ecosystem was moving to deno or bun now

by pier25

2 subcomments

The Node team has lost the plot IMO.
By far the most critical issue is the over reliance on third party NPM packages for even fundamental needs like connecting to a database.

by torginus

2 subcomments

Why do people keep reinventing OS features?
There's Docker, OverlayFS, FUSE, ZFS or Btrfs snapshots?
Do you not trust your OS to do this correctly, or do you think you can do better?
A lot of this stuff existed 5, 10, 15 years ago...
Somehow there's been a trend for every effing program to grow and absorb the features and responsibilities of every other program.
Actually, I have a brilliant idea, what if we used nodejs, and added html display capabilities, and browser features? After all Cursor has already proven you can vibecode a browser, why not just do it?
I'm just tired at this point