- Reading the whitepaper, the inference provider still has the ability to access the prompt and response plaintext. This scheme does seem to guarantee that plaintext cannot be read for all other parties (e.g. the API router), and that the client's identity is hidden and cannot be associated with their request. Perhaps the precise privacy guarantees and allowances should be summarized in the readme.
With that in mind, does this scheme offer any advantage over the much simpler setup of a user sending an inference request:
- directly to an inference provider (no API router middleman)
- that accepts anonymous crypto payments (I believe such things exist)
- using a VPN to mask their IP?
- I was part of a team that does the same thing. Arguably as a paid service, but source availability and meaningful attestation.
Service: https://www.privatemode.ai/
Code: https://github.com/edgelesssys/privatemode-public
- Yes: "provably" private... unless you have $1000 for a logic analyzer and a steady hand to solder together a fake DDR module.
https://news.ycombinator.com/item?id=45746753
by kiwicopple
1 subcomments
- impressive work jmo - thanks for open sourcing this (and OSI-compliant)
we are working on a challenge which is somewhat like a homomorphic encryption problem - I'm wondering if OpenPCC could help in some way? :
When developing websites/apps, developers generally use logs to debug production issues. However with wearables, logs can be privacy issue: imagine some AR glasses logging visual data (like someone's face). Would OpenPCC help to extract/clean/anonymize this sort of data for developers to help with their debugging?
- Really nice release. Excited to see this out in the wild and hopeful more companies leverage this for better end user privacy.
by sublimefire
1 subcomments
- Quite similar to what Azure with conf ai inference did [1].
[1] https://techcommunity.microsoft.com/blog/azureconfidentialco...
by DeveloperOne
0 subcomment
- Glad to see Golang here. Go will surpass Python in the AI field, mark my words.
- Where is the compute node source code?
- That's nice... in theory. Like it could be cool, and useful... but like what would I actually run on it if I'm not a spammer?
Edit : reminds me of federated learning and FlowerLLM (training only AFAIR, not inference), like... yes, nice, I ALWAYS applaud any way to disentangle from proprieaty software and wall gardens... but like what for? What actual usage?
- Thought this was going to be about Orchard from the title.
by mr_windfrog
0 subcomment
- [dead]
by MangoToupe
1 subcomments
- @dang can we modify the title to acknowledge that it's specific to chatbots? The title reads like this is about generic compute, and the content is emphatically not about generic compute.
I realize this is just bad branding by apple but it's still hella confusing.
by okelahbos28
0 subcomment
- [flagged]
- [flagged]
Hacker News