I split my assistant in two

I installed a personal agent on a server, with the classic idea: an assistant that reads my email, follows my calendar, tracks my training and keeps watch on tech news for me. One agent, all my access, reachable from my phone.

I changed my mind before the installation was over. Here is why, and above all what I found afterwards, because the interesting part is not the design. It is what broke.

Three ingredients that must not be mixed

An agent becomes dangerous when three things come together in it: access to private data, exposure to content I did not write, and a way to send something out.

Taken separately, each is ordinary. Together, they form a complete chain. A web page or an email can contain a disguised instruction. The agent reads it as text to process, but nothing in what a model receives truly separates data from an instruction. If it also has my email and a way to send, the hidden instruction becomes an action.

That is why I stopped looking for the right wording of a defensive prompt. A rule written in a configuration file remains a suggestion. What I wanted was a separation the agent could not cross even if it decided to.

Separate accounts, one direction of flow

So I have two profiles, running under two distinct Unix accounts on the same machine.

The first has my personal access: email, calendar, notes, training data. It has no web access. The second does the watching: it reads repositories, security advisories, business publications. It has access to nothing about me, and it can neither send nor publish.

Each has its own messaging bot and its own API key. If I have to revoke the profile exposed to the web, I do not cut off the one that reads my email.

Between the two, an exchange directory, in one direction only. The watch profile drops dated files. The personal profile reads them as data, never as instructions, and that sentence is written in its configuration in plain words. It is not a guarantee, it is one more layer. The watch profile holds no key that can reach anything of mine.

I added a third account, which runs no agent. It reads the profiles' configuration and pushes it every hour to a private Git repository. The agents have neither Git nor a repository key: that would give them a way out. This account exists for one reason only: an agent that rewrites its own instructions must leave a trace I can review.

The failures

The design above held. What did not hold was everything else. I give the failures in the order I found them, because the order tells something.

I wrote system services the tool did not need. My units pointed to a wrong binary path and to a subcommand that does not exist. They restarted in a loop 6,000 times while the service shipped with the tool ran perfectly well next to them. Lesson: before writing a unit, I check whether the software already installs one.

An ACL mask cut off my monitoring without a word. I had given the audit account read access through an ACL. The agent periodically resets the permissions of its configuration folder, and a plain permission change recalculates the ACL mask and sets it to zero. The entry stayed visible in getfacl, but had no effect anymore. The monitoring failed every hour for two days. I only saw it when I noticed there were no new commits.

The output of scheduled jobs went nowhere. On a server, the output of a cron job goes into a local mailbox nobody reads. A job that fails every hour is therefore perfectly silent. I redirected the output to a log file and gave each job a destination in case of failure, to my messaging app.

A monitoring loop woke up 288 times a day for one actual delivery. It polled a directory every five minutes. Yet there was a unit that triggers on a file drop, written but disabled, with the same path error as mine. It also watched for the condition "directory not empty" on a directory that contained an archive subdirectory, so it was never empty. Rewritten to watch for changes, it fires once per drop.

SSH password authentication was still active. The main configuration file did say no. But server images drop an extra file that turns the option back on, and the daemon keeps the first value it reads, so the one from the file with the lowest number. I had edited the wrong file and checked by rereading what I had just written, instead of querying the effective configuration.

The backup did not contain the schedules. It ran under the audit account, which is not allowed to read the jobs folder. A restore would have given me back a complete agent without any of its jobs. While moving it under the administrator account, I also found that it carried two gigabytes of reinstallable runtime. After adjusting the exclusions: 85 megabytes, schedules included.

What I take away

Of six failures, five were silent. None would have produced a visible error for weeks. That is the real difficulty with this kind of setup: an agent that answers correctly when I talk to it may well have stopped doing everything else.

So now I check effects, not intentions. I do not reread a configuration file to know whether an option is active, I query the service. I do not look at whether a job exists, I look at whether it produced something. I do not assume a right is granted because it is written down, I try to read the file with the account concerned.

And I give every automatic thing a way to shout. A backup that fails must tell me. A monitor that no longer monitors must tell me. It is less elegant than a clean architecture, and it is what saved me the most time.

What I have not solved yet

The exchange directory remains a bridge between the two profiles. A file dropped by the watch agent ends up read by the one that holds my access. Its configuration tells it to treat those files as data. I would prefer a technical barrier to an instruction.

At the time of these tests, I also had no measure of cost. The price of my model was unknown to the tool, so the daily log could not say what the day had cost. For a machine meant to be quick to audit, that was a gap.