Hell man, I attended a session at an AWS event last year that was entirely the presenter opening Claud and writing random prompts to help with AWS stuff... Like thanks dude... That was a great use of an hour. I left 15 minutes in.

We have a team that's been working on an "Agent" for about 6 months now. Started as prompt engineering, then they were like "no we need to add more value" developed a ton of tools and integrations and "connectors" and evals etc. The last couple of weeks were a "repivot" going back full circle to "Lets simplify all that by prompt engineering and give it a sandbox environment to run publicly documented CLIs. You know, like Claude Code"

The funny thing is I know where it's going next...

"In tech, often an expert is someone that know one or two things more than everyone else. When things are new, sometimes that's all it takes."

It's no surprise it's just prompt engineering. Every new tech goes that way - mainly because innovation is often adding one or two things more the the existing stack.

This dismisses a lot of actual hard work. The scaffolding required to get SOTA performance is non-trivial!

Eg how do you build representative evals and measure forward progress?

Also, tool calling, caching, etc is beyond what folks normally call “prompt engineering”.

If you think it’s trivial though - go build a startup and raise a seed round, the money is easy to come by if you can show results.

I'm suspicious of their methodology:

> Open DevTools (F12), go to the Network tab, and interact with their AI feature. If you see: api.openai.com, api.anthropic.com, api.cohere.ai You’re looking at a wrapper. They might have middleware, but the AI isn’t theirs.

But... everyone knows that you shouldn't make requests directly to those hosts from your web frontend because doing so exposes your API key in a way that can be stolen by attackers.

If you have "middleware" that's likely to solve that particular problem - but then how can you investigate by intercepting traffic?

Something doesn't smell right about this investigation.

It does later say:

> I found 12 companies that left API keys in their frontend code.

So that's 12 companies, but what about the rest?

For example, in the 90's, a startup that offered a nice UI for a legacy console based system, would have been a great idea. What's wrong with that?

If an AI company has an AGI, what incentive do they actually have to sell it as a product, especially if it’s a 10x cost/productivity/reliability silicon engineer? Just undercut the competition by building their services from scratch.

The bigger question is, this is the same story with apps on mobile phones. Apple and google could easily replicate your app if they wanted to and they did too. That danger is much higher with these ai startups. The llms are already there in terms of functionality, all the creators figured out the value is in vertical integration and all of them are doing it. From that sense all these startups are just showing them what to build. Even perplexity and cursor are in danger.

There is absolutely no point of reinventing the wheel to create a generic LLM, spend fortune to run GPUs while there are providers giving this power cheaply

I don't believe any of this. Why aren't we questioning the source of how the author is apparently able to figure out some sites are using REDIS etc etc?

Either you have a smash-and-grab strategy or you are awful at risk analysis.

This is rapidly becoming a more extreme version of the classic "what if google does that?" as the foundation model vendors don't necessarily need to target your business or even think about it to eat it.

It doesn't have to be this way and it won't be this way forever, but this is the world we live in right now, and it's unclear how many years (or weeks) it'll be until we don't have to do this anymore

LLMs is the new AJAX. AJAX made pages dynamic, LLMs make pages interactive.

Honestly it sounds about right: at the end of the day, most companies will always be an interesting UI and workflow around some commodity tech, but, that's valuable. Not all of it may be defensible, but still valuable.

I have to wonder, are people voting this up after reading the article fully, and I'm just wrong and this sort of info dump with LLM dressing is desirable? Or are people skimming it and upvoting? Or is it more of an excuse to talk about the topic in the title? What level of cynicism should I be on here, if any?

Wait...nvm.

AI has created a new interface with a higher level abstraction that is easier to use. Of course everyone is going to use it (how many people still code assembler?).

The point is what people are doing with it is still clever (or at least has potential to be).

What differentiates a product is not the commodity layer it’s built on (databases, programming languages, open source libraries, OS apis, hosting, etc) but how it all gets glued together into something useful and accessible.

It would be a bad strategy for most startups to do anything other than prompt engineering in their AI implementations for the same reason it would be a bad idea for most startups to write low-level database code instead of SQL queries. You need to spend your innovation tokens wisely.

Any reasonably informed person realizes that most AI start-ups looking to solve this are not trying to create their own pre-trained models from scratch (they will almost always lose to the hyperscale models).

A pragmatic person realizes that they're not fine-tuning/RL'ing existing models (that path has many technical dead ends).

So, a reasonably informed and pragmatic VC looks at the landscape, realizes they can't just put all their money into the hyperscale models (LP's don t want that) and they look for start-ups that take existing hyperscale models and expose them to data that wasn't in their pre-Training set, hopefully in a way that's useful to some users somewhere.

To a certain extent, this study is like saying that Internet start-ups in the 90's relied on HTML and weren't building their own custom browsers.

I'm not saying that this current generation of start-ups will be successful as Amazon and Google, but I just don't know what the counterfactual scenario is.

Thus has it always been. Thus will it always be.

https://github.com/zou-group/textgrad

and bonus, my rant about this circa 2023 in the context of Stable Diffusion models: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

The author never explains how he is able to intercept these API calls to OpenAI, etc. I definitely believe tons of these companies are just wrappers, but they'd be doing the "wrapping" in their backend, with only a couple (dumb) companies doing the calls directly to OpenAI from the front end where they could be traced.

This article is BS. My guess is it was probably AI generated because it doesn't make any sense.

It's still early in the paradigm and most startups will fail but those that succeed will embed themselves in workflows.