- (former employee here) congrats Michel! so glad to see you guys adapting to the AI age so well (and using the crap out of Devin!)
hmm so airbyte agents could serve as a form of MCP gateway, or a key building block of an MCP gateway, which btw is how anthropic uses mcp themselves for all their internal apps https://www.youtube.com/watch?v=CD6R4Wf3jnY&t=1s&pp=0gcJCd4K...
i think my most sad/interesting observation about ai engineers is that many ai apps are super data hungry, but many dont have the necessary data engineering background to even know they need an airbyte or what tradeoffs to make in an etl pipeline. would love a "data engineering for ai engineers" type braindump session from someone from airbyte at AIE (https://ai.engineer/cfp )
- Your billing support email forwards to a google group which rejects the email entirely. So i embedded my question inside the websites sales enquiry form and received multiple rounds of emails that couldn’t be further from human.
It’s not why we started using posthog but it definitely sealed the deal when you see how simple and reliable that experience is
- I feel like we've been working in parallel here :) We are using PyAirbyte (hi
aaronsteers) for our users to connect their data sources to our agents. We originally wanted to use the airbyte white-label platform, but the team said that it was being deprecated. I think this really drives home just how crucial it is to have a clear model for accessing your data, and Airbyte has been great at that for quite a while.
- We built something similar an A/B testing framework that measures Unblocked's impact on real AI coding agents.
It spawns agent CLIs (Claude Code, Codex, Cursor, GitHub Copilot) with and without Unblocked's MCP server attached, then statistically compares the results: https://github.com/unblocked/unblocked-harness-compare
We likewise measured token savings, (wall clock) time, # tool calls, and # turns.
- Looks interesting!
If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?
Have you done any testing on guided indexing, or metadata layers on top of the data? My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time. I.e. _what_ you index, and how you signpost for agents, becomes really important (unless your data is super clean I guess). This does look like a good foundation for that kind of tooling though!
- The prompts you mentioned here sound like SQL. Is there any way to run actual SQL on these systems? Is "agents need to poke around endlessly" a symptom of the fact that there isn't a way to run an actual query?
(I'd guess there is actually SQL at the bottom layer, but there's no way to talk to it?)
- Super interesting idea! Congrats on the launch. Context is definitely something that is lacking in my experience. Im always frustrated when an agent cannot answer business-related questions, and i compare them to coding agents which seem to be able to answer everything. The difference is that coding agent has the context right there at the fingertips, while for business its gated behind a bunch of services and custom data models. Context is king :)
How do you handle encryption and confidentiality? Im building in this space too (MCP gateway https://www.gatana.ai/) which already have semantic search for tool outputs, and ensuring encryption and confidentiality is not trivial.
by nerdright
1 subcomments
- This is such a great direction airbyte is taking and congrats to the lunch! I think you're very well-positioned for this opportunity than most people realize, given your reputable brand and your uncanny expertise in etl. It's honestly a natural progression of airbyte as far as the current AI landscape goes. Kudos to you and the team!
(We use airbyte at my company, although we self-host it.)
- More and more SaaS companies like ServiceNow or Hubspot are creating new tollgates for agent api calls. How do you think will this impact Airbyte Agents? I guess that replicating data locally will be harder since the platforms will try to protect it or charge for it.
- What are the main differences to Glean? My company is evaluating Glean and I feel like Airbyte is a strong alternative (at least for some use cases).
How does Airbyte handle data authorization?
- Just want to call out a couple of nuances in our methodology. In general, we tried our best to do apples-to-apples comparisons where we could, and gave ourselves a discount where we couldn’t. Unsurprisingly, it’s a challenge to find MCPs for various vendors (which is another reason we are trying to solve this). Here’s a video walkthrough of the benchmark harness:https://www.loom.com/share/9d96c8c64c1a4b7fad0356774fc54acc
Where the comparison wasn't valid or not apples-to-apples:
Gong and Zendesk: no official native MCP exists, so we used the most popular community implementations we could find. We were only able to benchmark Gong Search as the Gong MCP does not have a Get tool call.
While our Search testing yielded the same number of records on either path, vendor-specific search implementations means results aren’t identical. Contents are similar in general, so the ratios remain directionally correct.
The general test set:
2 scenarios (Retrieval and Search) across 4 connectors isn’t a huge test set. While we hope to extend this over time, we’ve made the harness public so anyone can contribute in the meantime. Let us know if you find any MCP with better results!
Where the vendor MCP wins or ties:
Salesforce showed the smallest win at 16%. This is primarily because Salesforce, unlike many vendors, uniquely provides great search support out of the box with their SOQL.
We see identical records for Get. As noted, Search returns different sets of identical counts. Airbyte uses fewer tokens because the Salesforce records contain mandatory metadata (type and url).
Where the vendor MCP is costly to context:
Zendesk is a great example of this. The extreme gap is because the Zendesk MCP (reminder - a community alternative) returns the entire API response in search results. This averages to 9KB per record against our production Zendesk account!
Airbyte’s implementation provides filtering, which allows agents to retrieve the minimal data needed to achieve the outcome, explaining the drastic gap.
- Congrats, you built an ETL pipeline and called it an agent. The industry has come full circle.
- Did you find that some data model patterns were easier to detect for some LLM ? I am curious on how training might have made some agents better at graph navigation for instance?
by xcf_seetan
0 subcomment
- Shameless plug: I have written a paper about using the MCP server architecture to enable agents to overcome the knowledge cutoff, to work with software released after the training stop.
[https://zenodo.org/records/19925469]
by ritonlajoie
1 subcomments
- Hi Michel, congrats and I have nice memories of working with you in lafayette street !!
Keep up the good work on airbyte ! :)
- Doesn't Skills solve all of this?
OpenClaw, Hermes and other agents have already made skill adoption mainstream?
Are you guys still seeing a future where people are dumping entire MCP tool defs into context?
- sounds very familiar to what I ended up doing on my internal system - especially anything to do with search - much better to just sync everything to a DB and give the agent access to the DB
- What actions does agents enable that weren't already available from Airbyte?
by Nation3Labs
0 subcomment
- [dead]
by aayushkumar121
0 subcomment
- [flagged]
by clauderodriguez
0 subcomment
- [flagged]
- [flagged]
- [flagged]
- [flagged]
by dubovskiyIM
0 subcomment
- [flagged]
Hacker News