Let’s talk
Guides / Give an AI agent a memory and limits

GUIDE 03

Give an AI agent
a memory and limits.

A step-by-step method for setting up a personal AI agent that owns one area of your life or work and gets sharper the longer it runs. The agent keeps its memory in notes you can read and edit, carries an identity file you control, reaches out when something matters and stays silent when nothing does, and never books, pays, or sends without your yes in the moment. The pattern is the one behind the three agents on the Hermes page of this site.

STEPS
10
TIME
An afternoon to a working agent, a few weeks for it to be worth keeping
WHO IT’S FOR
Anyone with a recurring area of admin, health, family, or business they would like an assistant to hold. Comfortable with a terminal, or willing to follow a setup guide once.
BEFORE YOU START

What you need.

  • An agent runtime that can hold a long-running conversation, run scheduled jobs, and read and write files. This guide was built on Hermes Agent, an open-source runtime by Nous Research; Claude Code with scheduled routines, or a hosted agent product, works the same way.
  • A notes app whose files live on disk as plain text and sync between your machine and the agent’s. Obsidian with Obsidian Sync is the reference setup; any folder of Markdown files that both of you can reach will do.
  • A chat app you already use every day that the runtime can connect to, such as Telegram or Slack.
  • Optionally, an email address of the agent’s own. AgentMail provides inboxes made for agents; a spare Google account works too.
  • One domain to start with, and a name for the agent. Not a general assistant: an operations agent for the household, a coach for your health, a chief of staff for your business.
Start with step one
STEP 01 OF 10

Pick one domain and give the agent a name

One agent, one domain, one person. An agent that owns household operations, or your health, or your parenting, or your business, can be held accountable for outcomes in that area. A general assistant cannot, because nothing is ever quite its job.

Give it a human name and write one sentence about what it is for. “Scout carries the household’s operational load so nothing time-sensitive gets missed.” That sentence becomes the first line of its identity file in step three.

When a side task grows into real work, the rule is to propose a second agent with its own domain, never to widen the first one. A household agent that starts filing business content is a signal that a business agent is needed, not that the household agent should become one.

YOU SHOULD NOW HAVE
  • A name, a domain, and a one-sentence job.
Next: Give it a folder it owns
STEP 02 OF 10

Give it a folder it owns

The agent’s memory is a folder in your notes vault that it owns and maintains: not a chat history it cannot see, and not a hidden memory store inside a product. Everything durable it learns goes into notes you can open, correct, and move.

Create the folder, and inside it an ownership note that records what the agent owns, what belongs to other agents or to you, the live surfaces it should keep current, and where its local data lives. That note is the contract between you.

Sync the vault to the machine the agent runs on, so it reads and writes the same files you do. If the agent runs on a server, a sync client on that server is enough.

PROMPT TO COPY
# [Folder] Ownership - [Agent]

[Agent] is the primary owner and maintainer of this folder.

## What [Agent] owns
- [the index note for this folder]
- [the active plan or tracker it keeps current]
- [the kinds of notes it may create here, one per line]

## Boundary with [other agents or me]
- [Other agent] may capture quick notes and route them here, but ongoing organisation,
  maintenance, and follow-through belong to [Agent].
- Structural changes to this folder need my approval first.

## Safety boundary
- Notes here support tracking, reminders, and preparation. They are not [medical / legal / financial] advice.
- [Anything that stays with a professional or with me, named.]

## Canonical live surfaces
- [[Index note]] · [[Active plan]] · [dashboard URL if any]

## Local source and data locations
- [path on the agent's machine for raw data, which never gets copied into the vault]
YOU SHOULD NOW HAVE
  • A folder the agent owns, synced to where it runs.
  • An ownership note at the top of it.
Next: Write the identity file
STEP 03 OF 10

Write the identity file

The identity file is the agent’s standing instructions, and it has a fixed shape that is worth keeping across every agent you make. It is written to the agent in the second person. The sections that matter most:

  • The standing objective, in priority order, with the hard constraint first. A health agent’s first objective is to reduce dangerous lows; everything else is optimised inside that. Stating the order means the agent says so out loud when goals conflict instead of picking one on its own.
  • The line it does not cross. Step four.
  • A tiered knowledge base. Sources it defers to, frameworks worth borrowing, and your own data and preferences as the highest tier. If your preference differs from the textbook default, yours wins, and the agent notes it so the rejected option is not suggested again.
  • When to reach out and when to stay silent. Reach out when something is worth acting on; if nothing is, say nothing.
  • Communication style with a right and a wrong example. One concrete pair teaches more than a paragraph of adjectives.
  • Output discipline. You only see the finished message. No narration, no tool calls, no “here is what I am doing”.
PROMPT TO COPY
# Identity

## Who you are
You are [Name], [my household operations assistant / my health coach / my chief of staff]. Your job is
[one sentence on the outcome you are accountable for]. You are the reason nothing in this area falls
through the cracks, not a chatbot that answers questions when asked.
Use my notes vault as your durable memory. You own the folder [path].

## Your standing objective, in priority order
1. [The hard constraint. Nothing else matters if this fails.]
2. [The primary outcome, with the number or signal you judge it by.]
3. [The secondary outcome.]
4. [The input that makes 1 to 3 possible, such as sleep, cash, or time.]
When these conflict, say so out loud rather than picking one on your own.

## The line you do not cross
You research, shortlist, draft, and prepare. You do not book, pay, confirm, send, or sign on my behalf
without my explicit go-ahead in that moment. Present the options, state the trade-off, wait for a yes.
You do not give [medical / legal / tax / financial] advice; you surface the question for the professional.
Sensitive details (addresses, confirmation numbers, payment details) stay where I put them.

## Your knowledge base
- Tier 1, defer to: [official policy pages, clinician guidance, the contract]
- Tier 2, borrow from: [a framework or two, named]
- Tier 3, highest value, only you have it: my own data, preferences, and what has already been tried
  and rejected. If my preference differs from a best-practice default, mine wins; note the divergence.

## Being proactive
Reach out when there is something worth acting on: a deadline approaching, a decision ready,
a recurring task overdue, a pattern worth flagging. If there is nothing actionable, stay silent.

## The plan
Keep one current tracker in your folder: time-sensitive items first, open threads with what is
blocking a decision, then recurring items with next-due dates.

## Communication style
Plain, practical, direct. Lead with the recommendation and the one trade-off that matters.
Right: "Two options for the Denver flight: nonstop at 6am for $410, or a layover at 9am for $340.
I'd take the nonstop unless the $70 matters more than two hours. Book it?"
Wrong: "Here are six options with full details on each, let me know which you prefer."

## Output discipline
I only see your message. Never paste code, tool calls, raw data, or narration. Do the work, then
send the finished result, a short status, or one direct question if you are blocked.

## When something is not actionable by you
Say so plainly and give the minimum next step. Never leave it half done without telling me.
YOU SHOULD NOW HAVE
  • An identity file with every section filled in for your agent.
Next: Write the line it does not cross
STEP 04 OF 10

Write the line it does not cross

Limits work best as a pattern rather than a list of forbidden tools: research, shortlist, draft, and prepare on its own; book, pay, confirm, send, or sign only after your yes in that moment. The pattern covers situations you did not think of.

Add the domain-specific lines: no medical, legal, or financial advice; questions for the professional get surfaced, not answered. Then add where sensitive data may and may not go.

Where a limit has to hold every time, write it as a protocol the agent must follow step by step, in its own file, and refer to it from the identity file. A health agent helping with insulin, for example, follows a fixed checklist (current reading and trend first, show the arithmetic, the pump’s calculator is the reference, log every suggestion) rather than a general instruction to be careful.

TIPIf this agent will run scheduled jobs that touch other people (email, bookings, messages), read the companion guide on hiring an AI employee for the three-tier permission line and the three-run probation. The two guides are meant to be used together.

YOU SHOULD NOW HAVE
  • The “line you do not cross” section written, plus a protocol file for any hard constraint.
Next: Keep the identity file in the vault, not in the runtime
STEP 05 OF 10

Keep the identity file in the vault, not in the runtime

Most runtimes expect the identity file in their own folder. Keep the real copy in the agent’s vault folder instead, and link it into the runtime’s location (on a Unix machine, a symbolic link). Two things follow.

You can edit the agent’s identity from your notes app on any device, and the change is live the next time the agent loads. And the identity is not locked into one product’s file layout: if you change runtimes, the file moves with you.

PROMPT TO COPY
ln -s "/path/to/vault/[agent folder]/Identity.md" "/path/to/runtime/profiles/[agent]/SOUL.md"
YOU SHOULD NOW HAVE
  • The identity file living in the vault, linked into the runtime, and a test edit that showed up on the agent’s next load.
Next: Connect a channel you already use, and give it an address
STEP 06 OF 10

Connect a channel you already use, and give it an address

The agent should be where you already are. Connecting it to the chat app you use every day turns it from a tool you open into an assistant you text. That is also where it reaches you when something matters.

Give it an email address of its own. Then it can email you and your partner a shortlist, send calendar invites, receive work you forward to it, and sign up for research newsletters without any of that landing in your inbox. Put the address’s outbound mail behind your approval until it has earned otherwise.

YOU SHOULD NOW HAVE
  • The agent answering in your chat app.
  • An address of its own, with sending to anyone but you gated behind your yes.
Next: Teach it to file directly and capture raw
STEP 07 OF 10

Teach it to file directly and capture raw

When you send the agent a note, a link, or a fact, it should infer the right place in the vault and file it there, asking only when the destination is genuinely ambiguous. No inbox and no holding folder: capture-then-sort only works if the sort happens, and it does not.

Tell it to record the fact and not the interpretation. The model that captures a note is usually cheaper than the one that will reason over it later, and a guess written down at capture time reads as settled fact a month on. Classifying (which folder, which person, which topic) is its job; explaining why it matters is not.

Every note it creates gets its metadata from the agent at creation time, in the same format you use. Fixing metadata in a later cleanup pass always loses to a script writing one untagged note a day.

PROMPT TO COPY
## Filing rules
When I send you a note or a link, infer the right location in the vault and file it there without
over-questioning me. Ask only when the destination is genuinely ambiguous. There is no inbox or
"sort later" folder: file directly into the real destination every time.
Record the fact, not an interpretation. Classify (folder, person, topic) but do not write up why it
matters, a thesis, or analysis. That work happens later, when someone needs it.
Every note you create gets its metadata block at creation time, in the vault's standard format.
YOU SHOULD NOW HAVE
  • Filing rules added to the identity file, and one test capture that landed in the right place with correct metadata.
Next: Add scheduled routines one at a time, each with a job card
STEP 08 OF 10

Add scheduled routines one at a time, each with a job card

Scheduled jobs are where the agent earns its keep: a morning briefing, a rental watch, a medication reminder, a weekly sweep. Add them one at a time, and give each a job card so you can see at a glance what runs, when, on which model, and where the result goes.

Two delivery rules keep routines from becoming noise. Deliver only when there is something new; a watch that found nothing says nothing, and a check that found nothing writes no log file. And deduplicate: a routine that looks for new things keeps a seen-list file and reports only what is not already on it.

Keep every job card in one reference note, with a status. A job that stops earning its place is paused and recorded as paused, not deleted.

PROMPT TO COPY
### [Job name]
- Job ID: [from the runtime]
- Schedule: [days and times, with the time zone stated]
- Model and provider: [pinned, so a model change is a decision you made]
- Skills: [named skills the job may use, or none]
- Delivery: [chat message to me / email / written to the vault at [path] / local only]
- Silence rule: [post only when there is something new / always post a one-line heartbeat]
- Dedupe file: [path to the seen-list it checks before reporting]
- What it does: [one paragraph, specific enough that someone else could judge a run]
YOU SHOULD NOW HAVE
  • One routine running, its job card in a jobs reference note, and a silence rule that you have seen hold.
Next: Correct the rule, not the output
STEP 09 OF 10

Correct the rule, not the output

When the agent does something you did not want, resist fixing the output. Fix the rule in the identity file or the ownership note, and record the correction as a dated entry with the reason: what changed, on what date, at your request, and why. Reasons generalise to situations you did not anticipate; bare rules get pattern-matched.

Examples of the kind of entry that earns its place: “no log file for a check that found nothing”; “this field is factual, from the export, never a guess”; “any script that references a folder that no longer exists is a bug: tell me rather than running it”.

Any correction you find yourself giving twice becomes an entry.

PROMPT TO COPY
Changed [YYYY-MM-DD], per [me]: [the new rule, in one sentence].
Why: [the reason, in one or two sentences, including what went wrong].
Replaces: [the old behaviour, so the history is visible].
YOU SHOULD NOW HAVE
  • A dated corrections section in the operating notes, with at least one entry.
Next: Keep a control center, and prune
STEP 10 OF 10

Keep a control center, and prune

Once there are two or three agents and a handful of routines, nobody can hold the picture in their head. Have one script regenerate a simple dashboard every day: which agents exist, where each identity file lives, which jobs run on which schedule, which pipelines write where. Secrets and raw credentials stay out of it.

Add a daily sweep of the vault that looks for drift (stale files, missing metadata, broken links) and reports only when it finds some. And once a week, look at the dashboard and ask of each routine: did it run, was the output correct or merely present, and would you notice if it disappeared? Pause what fails the last question.

YOU SHOULD NOW HAVE
  • A dashboard regenerated on a schedule.
  • A sweep that reports drift.
  • A weekly habit of pausing routines that no longer earn their place.
Credits and sources
CREDITS AND SOURCES

Where this comes from.

This is the pattern behind Scout, Heath, and Paula, the three personal agents described on this site, which have run on Hermes Agent by Nous Research with an Obsidian vault as memory since mid-2026. The identity file shape, the ownership note, the filing rules, and the job card are the real ones, with names and paths generalised.

Two ideas are borrowed with thanks: the GTD capture, clarify, organise loop and the PARA folder structure inform how captured notes move through the vault, and the companion guide on hiring an AI employee, adapted from Linas Beliūnas, supplies the probation and promotion steps this pattern was missing.

LET’S TALK

Would rather have it built
with you?

Bring the idea, or the job you want off your desk. We can run the same steps together, or I can build it and hand it over.

Tell me what you have in mind