The idea of having the elements anticipated and lowering the cognitive load of searching a giant drop down list scratches a good place in my brain. Instantly recognize it as such a better experience than what we have on the web.

I think something like this is the long term future for personal computing, maybe I'm way off, but this the type of computing I want to be doing, highly customized to my exact flow, highly malleable to improvement and feedback.

Interactive fiction / text-adventures written in the 20th century used a deterministic natural language interface with low load as an intentional flexible puzzle to solve, so the problem today is efficiency.

You could just as well argue to stop using modern bloated operating systems, websites, and apps. I understand that the processing required for LLMs can be much higher. But the side-effect of additional power needs will be a global push for more energy, which will result in more power stations being available for future industries once LLMs become more efficient.

If you want to reduce complexity overall and have simple, flexible interfaces and applications that use fewer of the worlds resources, I’m all for it. But don’t single out LLMs assuming they will always be less efficient. Cost will drive them to be more efficient over time.

And that’s perfectly fine.

Though the title in that sense is more of a click-bait.

Their very strength, of not being limited, is also a weakness - you only find the boundaries of what's possible by trial and error.

There is no latency, because the inference is done locally. On a server at the customer with a big GPU

Anyway, interesting tool and nice that it is implemented in Rust. Where is the prompt that tells the agent when to call the popup tool?

We no longer have StackOverflow. We no longer have Google, effectively.

I used to be able to copy pasta code with incredible speed - now all of that is gone.

Chatbots is all we have. And they are not that bad at search, with no sponsored results to weed through. For now.

LLMs offer a level of flexibility and non-determinism that allow them to adapt to different situations.

GUIs offer precision and predictability - they are the same every time. Which means people can learn them and navigate them quickly. If you've ever seen a bank teller or rental car agent navigate a GUI or TUI they tab through and type so quickly because they have expert familliarity.

But this - with a non-determinstic user interface generated by AI, every time a user engages with a UI its different. So they a more rigid UI but also a non-deterministic set of options every time. Which means instead of memorising what is in every drop down and tabbing through quickly, they need to re-learn the interface every time.

Natural language MUST be mixed with traditional UIs. Our world is filled with new software, new features, new concepts every day even for a regular person and certainly much more for developers than almost anyone else.

The thing I find most helpful with this sort of thing is "where the fuck is that settings" and "how do I get it to/I want to do x" navigating complex UX that is so feature filled that even the very best UX designers just can't hack it.

I feel like in many of these cases sure, let me use the regular UI. But also being able to ask "Hey, can I set my background to an image, where do I do that?" and being presented with the dedicated UI, or behind the scenes tool calls if no UI available.

Anecdotally: things I use ALL the time are, Help->Search on MacOS toolbar, cmd+shift+P menu in VSC, the search in Android settings, etc.

Author should take his own advice.

Interfaces are hard, abstraction is hard. Computer science has been working on making these concerns easier to reason about, and the industry has put a lot of time and effort into building heuristics (software / dev mgmt / etc frameworks) to make achieving an appropriate abstraction (qua ontology) feasible to implement without a philosophy degree. We, like biological systems, have settled on certain useful abstraction layers (OOP, microservice arch, TDD, etc.) that have broad appeal for balancing ease of use with productivity.

So it should be with any generative system, particularly any that are tasked with being productive toward tangible goals. Often the right interface with the problem domain is not natural language. Constraining the "information channels" (concepts/entities and the related semantics, in the language of ontology) to the best of your ability to align with the inherent degrees of freedom, disambiguated as best as possible into orthogonal dimensions (leaning too hard on the geometric analogy now). For generating code, that means interacting with tokens on ASTs, not 1D sequences of tokens. For comprehending 3D scenes, a crude text translation from an inherently 2D viewpoint will not have physics, even folk physics, much in mind except by what it can infer from the dataset. For storing, recalling, and reciting facts per se, the architecture shall not permit generating text from nonverifiable sources of information such as those vector clouds we find between the layers of any NN.

These considerations early in the project massively reduce the resource requirements for training at the expense of SME time and wages to build a system that constrains where there are constraints and learns where there are variables.