FRESH

Hacker News

Home

The Bitter Lesson of LLM Extensions

135 points by sawyerjhood

by uriegas

2 subcomments

> "I expect us to go back to extending our agents with the most accessible programming language: natural language."
I don't agree with this. Natural language is so ambiguous. At least for software development the hard work is still coming up with clearly defined solutions. There is a reason for why math has its own domain specific language.

by mkagenius

6 subcomments

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work.
and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP.
But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills.
1. https://simonwillison.net/2025/Oct/16/claude-skills/

by j2kun

1 subcomments

I don't see how "they improved the models" is related to the bitter lesson. You are still injecting human-level expertise (whether it is by prompts or a structured API) to compensate for the model's failures. A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference.

by iamcreasy

0 subcomment

ChatGPT apps, announced this month, feels a lot like original ChatGPT Plugin announced 3 years back. The only difference is how plugin are invoked. For ChatGPT plugin, we have to choose one from a drop down, and for apps - we could just include a plugin name in prompt.
Is there any other difference in the end-user side?

by arjie

0 subcomment

Custom GPTs are pretty old, but I recently found a use for them. My wife wanted some meeting note-taking and task recording assistance and I found that making a Custom GPT with a trivial Notion API that was scoped to one page[0] with structure that was encoded in the API was a quick couple-hour thing that unlocked a lot of utility for her (the default Notion MCP is "too broad"). It helped that this Custom GPT sits in her ChatGPT UI and she doesn't have to have another app or whatever to make it work.
We liked it quite a bit, but it led to some funny things. We use Reminders to keep our home to-do lists, hers and mine in one list with two sections. I wanted to take this existing flow we had and make it work with a Custom GPT. It's practically impossible because Reminders:
* doesn't have a good API through EventKit
* requires a pop-up permission grant in the UI
So in the end, I did end up making somewhat of an MCP server for it, running it on an old Macbook Pro I had and then sticking Amphetamine on in closed-lid display-sleep mode hooked up to my Tailnet and exposed via a Cloudflare tunnel so that we could use ChatGPT to interact with the thing. Yes, you can see how insane that whole thing is. But there's quite a lot of value to have your AI agent just be the one thing.
0: https://wiki.roshangeorge.dev/w/Blog/2025-10-17/Custom_GPTs

by metalrain

0 subcomment

Skills.md will in time have same problem as MCP, they will bloat the context. I wonder if we could just have the scripts without the descriptions and LLM would have been trained to search the most useful things in specific folder.

by dsign

0 subcomment

I don't know, even ChatGPT 5.1 hallucinates API's that don't exist, though it's a step forward in that it also hallucinates the non existence of APIs that exist.
But I reckon that every time that humans have been able to improve their information processing in any way, the world has changed. Even if all we get is to have an LLM be right more times than it is wrong, the world will change again.

by vessenes

4 subcomments

```
  > "If I could short MCP, I would"
```
I mean, MCP is hard to work with. But there's a very large set of things that we want a hardened interface to out there - if not MCP, it will be something very like it. In particular, MCP was probably overly complicated at the design phase to deal with the realities of streaming text / tokens back and forth live. That is, it chose not to abstract these realities in exchange for some nice features, and we got a lot of implementation complexity early.
To quote the Systems Bible, any working complex system is only the result of the growth of a working simple system -- MCP seems to me to be right on the edge of what you'd define as a "working simple system" -- but to the extent it's all torn down for something simpler, that thing will inevitably evolve to allow API specifications, API calls, and streaming interaction modes.
Anyway, I'm "neutral" on MCP, which is to say I don't love it. But I don't have a better system in mind, and crucially, because these models still need fine-tuning to deal properly with agent setups, I think it's likely here to stay.

by zby

0 subcomment

I believe that what we need is treating prompts as stochastic programs and using a special shell for calling them. Claude Code and Codex and other coding agents are like that - now everybody understands that they are not just coding assistants they are a general shell that can use LLM for executing specs. I would like to have this extracted from IDE tools - this is what I am working on in llm-do.

by WatchDog

1 subcomments

The most useful LLM "extension" isn't even mentioned in this article, and that is shell use.
An LLM with a shell integration can do anything you need it to.

by bloppe

2 subcomments

How is this related to the bitter lesson?

by ttkciar

3 subcomments

Well, that's just great.
The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence.
Now OpenAI has usurped the term to refer to these inference-guiding .md files.
I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the OpenAI feature. Semantic overload sucks.
How do we deal with this? Start using "competencies" (or similar) in academic papers? Or just resign ourselves to suffering the ambiguity?
Or maybe the OpenAI feature will fall flat and nobody will talk about it at all. That would frankly be the best outcome.

by jdblair

0 subcomment

a funny thing happened why i added and emacs eval mcp tool to claude code
https://hachyderm.io/@jdblair/115605988820465712

by lerp-io

1 subcomments

can someone explain to me the difference between MCP and calling a cli tool eg curl or whatever i still don’t understand i’ve been using ai for years now.

by Der_Einzige

1 subcomments

Zero discussion around LLM sampling. How do you leave such a gaping hole in such a written piece? I know it's not AI cus AI wouldn't be that sloppy.

by OBELISK_ASI

0 subcomment

[dead]