The protection of private and other confidential data in Hermes Agent and AgentZero vis-à-vis LLMs is essential to me.
Configuring and using the built-in tools is, at the very least, cumbersome, opaque and unpredictable – mistakes do happen.
Particularly in multi-user environments, a high level of technical expertise and a willingness to take the long way round are required to ensure a minimum level of protection. But woe betide you if you’re in the zone, under pressure, or if nobody cares about the outcome.
“Yes. The detection and redaction engine is supplied as a single binary that you can run in your VPC, with a Kubernetes Helm chart available. The audit log can write to your own S3, Splunk or Datadog. This keeps regulated data within your perimeter, which external text-redaction APIs cannot offer. Contact sales for the deployment package.”
Is porteden OSS, the fact that a solution like this is not OSS, and are they offering SaaS.
Which means, generally, you’re not giving your data to one party, and sharing it with another different party.
at the end of the day, the task execution must share some data somehow.
in this case, i am better off trusting the model provider, than some third party potentially data broker.
99% of my infra and tooling runs on Oss, and recently, including even the Models. Well i don’t have compute to selfhost, but if i did, i would.
in this scenarios, youd rather work an inference provider, with no data retention, which, as per the terms, you could sue if it happens, as opposed to other options.
Opencode go, provides similarly, among others.
the cost of hosting a Model like GLM 5.2 is roughly $80 per day.
If data is very sensitive and its for organization, then its probably the best option, as you manage everything end to end.
Havin AI to help detects and redact PII is definitely one way to sift through the large volumes of data quickly and efficiently and consistently.
My concern is:
How do we know that the AI and it’s agent are doing it properly?
How do we know that the AI and it’s agents and logs are not storing some of the PII in it’s own logs and other such like trails which is used for audibility and compliance?
You’re absolutely right. I’m really surprised that, given all the current hype around AI, you don’t actually see anything about this anywhere. Either we laypeople are making a mountain out of a molehill, and the experts have had this on their radar for ages and have viable solutions up their sleeves.
In any case, you have to manually store the required credentials on the agent servers so that they can read them from the credential stores there. However, this is very time-consuming and quickly becomes complex when dealing with multiple systems, perhaps involving multiple credentials, especially as these are plain text files (.env) that only support the variable=value mapping.
I’ve developed and implemented a concept specifically for Hermes to prevent data leakage. The agent-based AI servers act as a sort of gatekeeper in this process.
However, I get the impression that they’re also rather forgetful when it comes to the specifications. Presumably a question of the size of the context window.
When I then consider where the LLMs are being operated around the world…
A nightmare, even in non-production use.
I’d be happy to share the concept. I just didn’t get the impression that anyone here was interested, or that anyone wanted to spend time analysing and discussing something like this.
Sure, you can discuss it with ‘your AI’ too – that’s what I did, after all. But will that actually produce a high-quality solution?
At some point, people here in the forum were speculating about why there’s no activity here anymore. If fundamental topics like this don’t interest anyone, then what does?
As laypeople, I think we should be making a mountain out of the molehill if we feel that there is lack openess and transparency on the whole topic regarding the protection of the data we feed AI, having AI respect the GDPR laws and does it’s job properly when it assists during the Right To Be Forgotten process.
Although I am only at the begining stages, I can definitely see this as a problem and I am unsure whether I should be surprised or not that it’s so difficult/complicated. Which makes me wonder how the sys admins of the complex systems grant access to some of their automated systems and why they don’t reuse some of their existing processes for granting access.
I am happy to discuss this more, although I am still new to the technical aspects of it all
But if it’s just the two of us racking our brains over this, it’s like a blind man and a lame man trying to hobble towards a destination.
We definitely need more expertise here. Surely we can’t be the first two people to have a problem with this.
Of course, Hermes Agent and AgentZero are still in development and are nowhere near finished. But the fact that such fundamental architectural issues remain unresolved is a mystery to me.
In my company, we wouldn’t use something like that. I wouldn’t trust any service provider who did use it either.
In a tender, I would ask for a very detailed explanation of how data leakage can be prevented.
Nevertheless, the potential of agent servers to act as gatekeepers is enormous.