by DeveloperErrata
3 subcomments
- Seems neat - I'm not sure if you do anything like this but one thing that would be useful with RAG apps (esp at big scales) is vector based search over cache contents. What I mean is that, users can phrase the same question (which has the same answer) in tons of different ways. If I could pass a raw user query into your cache and get back the end result for a previously computed query (even if the current phrasing is a bit different than the current phrasing) then not only would I avoid having to submit a new OpenAI call, but I could also avoid having to run my entire RAG pipeline. So kind of like a "meta-RAG" system that avoids having to run the actual RAG system for queries that are sufficiently similar to a cached query, or like a "approximate" cache.
by OutOfHere
2 subcomments
- A cache is better when it's local rather than on the web. And I certainly don't need to pay anyone to cache local request responses.
by phillipcarter
1 subcomments
- Congrats on the launch! I love the devex here and things you're focusing on.
Have you had thoughts on how to you might integrate data from an upstream RAG pipeline, say as a part of a distributed trace, to aid in debugging the core "am I talking to the LLM the right way" use case?
by angoragoats
2 subcomments
- I don't understand the problem that's being solved here. At the scale you're talking about (e.g. millions of requests per day with FindAI), why would I want to house immutable log data inside a relational database, presumably alongside actual relational data that's critical to my app? It's only going to bog down the app for my users.
There are plenty of other solutions (examples include Presto, Athena, Redshift, or straight up jq over raw log files on disk) which are better suited for this use case. Storing log data in a relational DB is pretty much always an anti-pattern, in my experience.
- Looks cool. Just out of curiosity, how does this compare to other OpenLLMetry-type observation tools like Arize, Traceloop, LangSmith, LlamaTrace, etc.?
From personal experience, they're all pretty simple to install and use. Then mileage varies in analyzing and taking action on the logs. Does Velvet offer something the others do not?
For my client projects, I've been leaning towards open source platforms like Arize so clients have the option of pulling it inhouse if needed. Most often for HIPAA requirements.
RAG support would be great to add to Velvet. Specifically pgvector and pinecone traces. But maybe Velvet already supports it and I missed it in the quick read of the docs.
- > we were frustrated by the lack of LLM infrastructure
May I ask what you specifically were frustrated about? Seems like there are more than enough solutions
by reichertjalex
1 subcomments
- Very nice! I really like the design of the whole product, very clean and simple. Out of curiosity, do you have a designer, or did you take inspiration from any other products (for the landing page, dashboard, etc) when you were building this? I'm always curious how founders approach design these days.
by TripleChecker
1 subcomments
- Does it support MySQL for queries/storage - or only PostgreSQL?
Also, caught a few typos on the site: https://triplechecker.com/s/o2d2iR/usevelvet.com?v=qv9Qk
- Nice! Sort of like Langsmith without the Langchain, which will be an attractive value proposition to many developers.
- Neat! I'd love to play with this, but site doesn't open (403: Forbidden).
by codegladiator
0 subcomment
- Error: Forbidden
403: Forbidden ID: bom1::k5dng-1727242244208-0aa02a53f334
- This seems to require sharing our data we provide to OpenAI with yet another party. I don't see any zero-retention offering.
- interesting, seems more of an enterprise offering. its OpenAI only for and you plan to expand to other vendors? anything opensource?
Hacker News