by anthonypasq
3 subcomments
- Love to see these crazy kinds of experiments going on. Even if this doesn't 100% work or is prohibitively expensive for now, these are glimpses into the future in the same way people were talking about coding agents in 2023 when we just had tab complete.
- >The browser swarm from earlier this year peaked at roughly 1,000 commits per hour on Git. The new system peaks at around 1,000 commits per second.
>To facilitate this rate of activity, we built a new version control system (VCS) from scratch. Throughput was not the only reason to own this layer. Every change in the system passes through the VCS, so it is where collisions first become visible, and several of the coordination mechanisms in the next section are implemented directly inside of it.
Talk about inventing the universe to make a button.
by handfuloflight
2 subcomments
- > To test that progress, we returned to a task the old swarm had struggled with: building SQLite from scratch, in Rust, from nothing but its documentation.
Isn't SQLite's source code in its training data?
- I would have loved to see more of the harness engineering shared as code. Instead we are left with only the outcome.
I guess that makes sense since the harness is the product in the case of Cursor.
- How do we know if these models weren’t trained on Turso’s rewrite of SQLite in Rust?
It seems both likely that they were and impossible to remove that code from pretraining. Doesn’t that make this just about LLM memorization of the training set? What am I missing?
- Am I reading this right? Opus + Composer did a comparable job to Fable, at ~1/19th the price, and half the LoC?
by stillpointlab
0 subcomment
- Today it is SQLite, tomorrow Linux?
I'm not sure of the long-term viability of these brute force swarm approaches, but clearly the AI companies dig the marketing they get out of it. But the fact that they get so far at this stage suggests we have a very interesting 2 years coming up.
- This is almost a year behind Steve Yegge's first post on beads. Gas Town and Gas City provide orchestration for the swarm. So far I haven't seen a perfect implementation but this idea isn't new.
- > One reading is that it was more productive. Another is that most of those commits were busywork (thrash, contention, churn).
Kudos
- I initially thought that getting agents to work for longer and in large groups was the future, but I'm increasingly thinking that, at least for engineering, just one thread makes more sense. The agent pulls things into context as needed. One thing that I've been experimenting with is also letting the agent remove things from context, such as files. But just adding to the context and compacting when it's full seems like it might beat a lot of more advanced options. Because the model is good, it knows what to put in the summary; just enough for the model to be able to rebuild the context from that seed (e.g. pulling in relevant files/data into the context).
- Even at a much smaller scale of a few dozen parallel agents working I've found real benefit to a structured hierarchy of agents. Not just keeping the context clean and focused at the right level, but if you find that something has just gone off the rails, you can more easily rip out or even just fully delete a leg of work without damaging the original design. It's also nice when you find a design change is needed and you can communicate that to the top level and have it efficiently propagate down where it matters.
I like it so much I even recorded a video walking through the process!
https://www.youtube.com/watch?v=efUmcCiRoDU
(I do walk through how I do this in our app DevSwarm here)
- Seems an extreme system looking for a use case, but it's great to see research being done in this area
- This is super fascinating, and I loved seeing testing of where exactly you need frontier intelligence -- looks like coordination / planning, but not coding right now -- the article's a bit of a tease, as we can't play with such a harness, or their new version control system or get a workable artifact out of it.
That said, I love the work on figuring out these harness coordination jobs. While there are analogs to human management there is also this enticing feeling that, since the models are broadly deterministic, we might be able to get repeatable science-type lessons about managing them with enough testing.
- The issue is that the only models that could be trusted to work autonomously cost more than a human employee
- These tests are interesting but I can't help feeling that these are benchmaxxing results. "We made our harness make sqlite in rust" sounds like a much easier probelm statement than having to recreate an application like facebook. I think the concept of having to integrate is the real challenge. Re-writing code in new languages based on decent docs is a difficult but less useful measure of how AI can help replace engineers.
Imagine this: you need to build a new CRUD app but your AI agent spends 20M toks on just writing it's own custom login function, that's a waste of time and money. I would not like to have a new system developed for me in such cases. I think we need a test where the agents are more specifically tasked to find technologies worth integrating and hence write the fewest lines of code.
- This post describes ideas found in existing agent orchestrator systems since they've existed. You'll find the same "top level agent that breaks a problem down into spec / plan / implement / verify" pattern in get-shit-done or superpowers.
Is the original part the VCS that works at 1000 commits/second? Or spending a bunch of money comparing models?
by agustechbro
0 subcomment
- Wow, people have been working hard for a long time in Turso:
https://turso.tech/blog/introducing-limbo-a-complete-rewrite...
to do the same cursor did in justo no time?
- If we take coding off the table i don't see the case for large agent swarms. More is not always better in my experience experimenting with agents
- Anything architectural like this is super interesting. We keep getting more powerful models sure. We're getting agents ok. But this coordination of many agents. This is the part that's really going to scale very fast and across many many machines in parallel. Very cool.
- I find these blog posts (and the originals, with Anthropic's C compiler and Cursor's browser) somewhat funny, as if they have this enormous power to build ... but they can't build something unique or new. Like the software sucks, but look how powerful the process is (the models are indeed powerful).
And it's a bit of a shame: by virtue of their position (their embedding in the fabric of venture capitalism), it seems like they can only make a subset of things -- what they can make is dictated enormously by capital, as they are engines of capital.
Not sure the point I'm trying to make, I just find it amusing.
Perhaps the point is that it might be more worthwhile to give independent creators a billion dollars to play around with agent swarms if we want to keep diversity in the evolutionary algorithm that is the software industry high.
by paraschopra
0 subcomment
- I think experiments like these are exciting, but they only work when you have the full suite against which you calibrate.
What I'd love to see is a small group of humans steering a million agents.
- They have invented software time travel: Write software => then write manual => then feed manual to cursor to write the software.
by Schlagbohrer
1 subcomments
- Amazing test. I wish I could spend $2000 of compute on having my own swarm build some weird piece of software with 1000 commits per second.
by pianopatrick
0 subcomment
- I wonder what would happen if the agents did not have the spec. Like if you just said "build a simple SQL database that works as a single file", what would the agent swarm come up with.
- Can a swarm of agents write the 835 page manual?
by doctoboggan
1 subcomments
- I feel bad for the engineers at Cursor who have to use Grok in these sorts of experiments.
by alienbaby
1 subcomments
- Tinfoil hat: what if the prices are artificially inflated deliberately to price out casual users being able to field large swarms of frontier model agents because it could be just too dangerous.
by thisismyopinion
0 subcomment
- 1000 commits of slop per second.
- [flagged]
- [flagged]
- [flagged]
- [flagged]
by SnehRJoshi
0 subcomment
- [flagged]
by alexmercerdev
0 subcomment
- [dead]
- [dead]
by DekryptLabs
0 subcomment
- [dead]
Hacker News