Blog ·

My #aicore setup, 10 months later

Back in January, I wrote about my "#aicore stack" in a fit of early AI Psychosis, err, I mean shiny new thing excitement. A couple new tools have made it into the setup... .

The core is still the same, still logging into a tmux session over mosh / the blink app (iOS) into my raspberry pi5. The Claude app now allows you to connect to a session running somewhere else, but I still prefer tmux, mostly because tmux also allows me to start neovim, or run shell commands, not just the chat session. And I have truly seamless handover between devices.

The biggest addition is kbmcp, my version of a knowledge base with markdown files. The initial idea was very simple: I was tired of having to copy files from the Claude app to Claude Code. I'd often riff of some idea, have Claude write a spec in markdown, copy that into the raspberry pi via tailscale, and so on. I thought "what if I had an MCP that's essentially just a bunch of markdown files, then both sides can read/write to it?"

Getting there was a bit more complex, writing an MCP that's just read_file, write_file is straightforward, but then how would I access it while on the road? Tailscale helped a bit, but the central Claude instance could still not reach it. Eventually, I built a "minimal OAuth" wrapper based on fosite so I could talk to the knowledge base from anywhere.

In addition, I also created a web editor for the knowledge base with which I could watch the files from a browser, also using OAuth, but then talking through a REST API to the kbmcp.

So now I could discuss an idea with Claude on the app, tell it to write it into the knowledge base, and then switch to the Blink app and have Claude Code on the raspberry pi pick it up.

And I realized this is also doing something else, suddenly I had notes on my projects (that I fully owned). If I came across a topic I already discussed somewhere else, I could tell Claude, "read that file for context". Sure, Claude has project artefacts, but somehow the context I need is often not in the project I'm in right now. Over time, the knowledge base didn't become just a file exchange, but really a collection of all kinds of ideas and more a personal log that started to redefine how I discussed ideas with Claude.

YAML frontmatter search and projections

The next stage was to make the data more searchable. So I added tools that essentially called ripgrep or fd to search for texts or filenames. That already was good, but I went one step further and introduced YAML searchmatter into the files together with a search_frontmatter tool. This turned a folder into something that was almost like a database table.

I don't exactly recall where I had that idea, but I still had a script that would poll my emails, and store them into a folder. The original script already did some massaging, but what if I essentially pulled my email, turned it into YAML frontmatter + markdown and stored it in the knowledge base?

I immediately got a searchable index of my mails. I could also take different accounts, and project them all into the same folder.

You might wonder can't I just use the Google Mail MCP? You could, but what this gave me was a unified search space for different sources, instead of an agent having to make multiple calls to each of the MCPs. I added another projection from my calendar entries, now I could search both my calendar and my emails. Pretty cool!

My first loop

A while back I had this idea for closedloops.app (one of my many domains), where an agent would look at your emails and track shipments, remind you to answer your tax accountant, etc. That's where I had my IMAP polling script from. But back then my idea was something like "process each email with a small local LLM to extract the information, then have some code that tracks entries in a database, then a stronger agent that you can give the brief to." This didn't really work well because the first stage would make mistakes which I couldn't fix later.

But now I had given an agent a way to query and sort through my emails, and the whole analysis and brief could be done in one session.

In addition, since I had connected kbmcp with Claude, I could use scheduled tasks to run a session every morning. Fully automated, but working on my own data, served from the raspberry pi in my living room.

With the frontmatter I could also store tasks in a folder in the knowledge base. Having a "status:" field in the frontmatter allowed the agent to easily poll for open tasks, etc.

More than a knowledge base

People right now are usually a bit skeptical of knowledge bases which are "just a bunch of markdown files", but while playing around with one, I realized that it gives you the perfect storage layer for not just knowledge but also state. YAML frontmatter gives you the structure, search_frontmatter the ability to run simple queries like on a database table. Enough to maintain state from one agent loop to the next.

I threw in a couple other things, most notably git, so that every edit becomes its own git commit. I could in principle reverse any edit, although I never did that.

The editor became like a debugger to inspect the state of my agents.

And by putting most of the instructions for a scheduled task on the knowledge base (and have the scheduled task essentially be "read that file on kb and follow the instructions there"), I had everything in one place, and could even talk with the agent at the end of the session and iterate and refine the prompt.

So yeah, my stack grew:

  • kbmcp as a knowledge base / durable state / scratchpad / instruction store for agents

  • kbeditor to view and edit the knowledge base

  • projections to put information into the knowledge base (currently I have: mail, calendars, rss feed readers)

  • using scheduled tasks on Claude to run autonomous agent loops

React to this post

Back to all posts