AI How-To

A Headless Bridge: Live-Syncing an AI Agent’s Workspace into CouchDB


The architecture, the concept, and the one-line bug that cost me four hours

Most Obsidian sync stories end at “install the plugin, point it at CouchDB, done.” This one starts where that leaves off: there is no Obsidian. The vault I needed to sync lives on a headless Linux box, written to by an AI agent, with no GUI, no plugin, and no human hitting Ctrl-S. The goal was to take that agent’s working directory and mirror it — live — into the same CouchDB database my real Obsidian clients already use, so everything the agent produces shows up in my vault on every device.

It worked. Eventually. But the last mile was a four-hour debugging session that a well-known AI assistant walked me straight into and could not walk me out of. The fix was a single misnamed environment variable. Here’s the whole thing — concept, architecture, and the war story — because the war story is where the actual lesson lives.


The concept

I run an AI agent (I call the host colonelclaw) that produces Markdown continuously: notes, research, scratch files, task logs. That output is only useful if it lands where I actually read things — my Obsidian vault, which syncs across my Mac and iPhone through a self-hosted CouchDB instance using the excellent Self-hosted LiveSync plugin.

The problem: the agent’s box has no Obsidian and never will. So I needed a headless writer — something that watches the agent’s workspace, and every time a .md file changes, pushes it into CouchDB in exactly the format LiveSync expects, so the plugin on my other devices treats it as just another synced note.

The building blocks:

  • obsidian-vault-cli — a Node.js tool that speaks the LiveSync wire format to CouchDB from the command line. This is the key that makes a headless writer possible.
  • inotify — Linux kernel file-system events, so I react to changes instantly instead of polling.
  • Tailscale — so colonelclawd can reach the database host over an encrypted overlay without exposing anything to the public internet.

The architecture

Three separate paths converge on one CouchDB database. That convergence is the whole design, so it’s worth seeing it laid out:

Path 1 — public clients over HTTPS. My Mac and iPhone reach CouchDB the normal way: a public hostname resolved by Cloudflare, in over the WAN on 443, terminated by Nginx Proxy Manager, which reverse-proxies to CouchDB on 5984. Standard, TLS-protected, works from anywhere.

Path 2 — direct over Tailscale. When a device is on my tailnet, it can skip the public round-trip entirely and hit CouchDB directly over the encrypted overlay. Lower latency, no dependence on the public edge.

Path 3 — the headless pipeline. This is the new part. On colonelclaw, a sync daemon watches the agent’s workspace with inotify. On every write it invokes the Vault CLI, which loads its credentials from a .env file and writes the changed note into CouchDB — also over Tailscale, straight to port 5984, bypassing Nginx entirely.

All three land in the same database (obsidian-vault). CouchDB’s replication does the rest: whatever the agent writes propagates out to my Mac and phone as if I’d typed it myself.

One clarification worth making, because it tripped up the first draft of this diagram: the CLI’s “direct API call” to 100.x.y.z:5984 is not bypassing the network — that’s a Tailscale CGNAT address. The direct path is the overlay. Same transport my Mac uses when it’s on the tailnet.


The build

The daemon itself is intentionally boring — a bash script doing a bulk pass, then dropping into a live-watch loop:

#!/bin/bash
WATCHED_DIR="/home/agent/.workspace"

sync_file() {
    local full_path="$1"
    local relative_path="${full_path#$WATCHED_DIR/}"
    if [[ "$full_path" == *.md ]] && [ -f "$full_path" ]; then
        echo "[+] Processing: $relative_path"
        # Pipe content so YAML frontmatter (---) doesn't get parsed as flags,
        # and use '--' to force the CLI to stop reading flags.
        cat "$full_path" | obsidian-vault write -- "$relative_path"
    fi
}

echo "[*] Bulk migration..."
find "$WATCHED_DIR" -type f -name "*.md" | while read -r f; do
    sync_file "$f"
done

echo "[*] Live monitoring..."
inotifywait -m -r -e close_write --format '%w%f' "$WATCHED_DIR" |
while read -r changed; do
    sync_file "$changed"
done

Credentials live in a .env next to the CLI (sanitized here):

COUCHDB_URL=http://<tailscale-ip>:5984
COUCHDB_USER=admin
COUCHDB_PASSWORD=<password>
DB_NAME=obsidian-vault
E2EE_PASSPHRASE=<passphrase>

That file is where the entire four hours went. Look at it again after you read the next section.


The war story

I fired up the daemon and got one error, over and over:

failed to fetch milestone doc: 404 Object Not Found

I handed this to an AI assistant and we started chasing it. It confidently proposed a database that didn’t exist, then a fresh/uninitialized database, then a mode mismatch between two replicator implementations. Each hypothesis was plausible and each one was wrong, and every wrong turn ate another twenty minutes. Four hours in, I was no closer.

Here’s the diagnostic ladder that actually mattered — run from the client box, so it also tests the Tailscale path end to end:

1. Is CouchDB answering, and does the database exist?

curl -s -u admin:'<password>' http://<tailscale-ip>:5984/_all_dbs
# ["_replicator","_users","obsidian-vault"]

There it is — obsidian-vault, right where it should be. So the database is not missing. Auth works (a bad password returns 401, not 404). Scratch the first two theories.

2. Is the database actually empty, or already initialized?

curl -s -u admin:'<password>' \
  http://<tailscale-ip>:5984/obsidian-vault/_all_docs
# {"total_rows":3, ... "obsydian_livesync_version" ... "test note.md" ...}

Not empty. It has real content and the LiveSync version marker. (And yes, the doc ID is spelled obsydian — that’s LiveSync’s own historical typo, not mine.) So the plugin had clearly initialized this database at some point.

3. Where does the milestone actually live?

The milestone isn’t a normal document — it’s a CouchDB local doc, which never shows up in _all_docs:

curl -s -u admin:'<password>' \
  http://<tailscale-ip>:5984/obsidian-vault/_local/obsydian_livesync_milestone
# {"_id":"_local/obsydian_livesync_milestone", "_rev":"0-3", ...}

It was right there. The milestone existed. The database existed. The credentials worked. So why was the CLI getting a 404 fetching a document I could pull by hand?

4. Ask the source, not the assistant.

grep -rniE "DB_NAME|COUCHDB_DATABASE" ./src
# src/lib/connection.ts:76:
#   database: process.env.DB_NAME || fileEnv.DB_NAME || "obsidiannotes",

And there it is. The CLI reads the database name from DB_NAME, falling back to a default of obsidiannotes. My .env — following a config the assistant had generated — set COUCHDB_DATABASE=obsidian-vaultThat variable is never read.The CLI ignored my setting, fell back to the default obsidiannotes, queried a database that doesn’t exist, and returned 404 on every milestone fetch.

Four hours. One wrong environment-variable name. A variable the assistant had invented and then never suspected.

The fix:

sed -i 's/^COUCHDB_DATABASE=/DB_NAME=/' .env
echo "test" | obsidian-vault write -- test-from-cli.md
# Written: test-from-cli.md (4 bytes)

First write in four hours. The pipeline came up immediately after.


What I’d actually take away from this

The error message told the truth the whole time. 404 Object Not Found — not 401, not a connection refused. It reachedthe server, authenticated, and asked for something in a place that didn’t exist. A 404 after a successful auth is almost never a credentials or network problem; it’s a “you’re looking in the wrong drawer” problem. I let a confident assistant talk me past what the status code was plainly saying.

Read the source before you theorize. Two greps answered a question that four hours of hypotheses couldn’t. When a tool has its config-parsing logic sitting right there on disk, the fastest path to truth is grep -rn for the variable name — not a plausible-sounding story about replicator modes.

AI assistants hallucinate config, and then trust their own hallucination. The invented COUCHDB_DATABASE key looked exactly like something that should exist. That’s what made it dangerous. The generated code was never re-examined as a suspect, because the assistant “knew” it had written it correctly. Treat AI-generated config the way you’d treat a junior engineer’s first PR: assume the boring, dumb bug before the exotic one.

One env-var away from working is the most expensive kind of broken. Everything downstream was correct. The database, the credentials, the milestone, the Tailscale path, the CLI itself — all fine. A single misnamed key made a perfectly good system look catastrophically broken. That’s the trap: when 95% is right, you go hunting in the 95% instead of the 5%.


Where it stands

The bridge runs as a systemd service now. The agent writes Markdown; inotify catches the close_write; the CLI pushes it over Tailscale into CouchDB; LiveSync fans it out to my Mac and phone. The agent’s brain and my vault are now the same vault.

Total moving parts: a 20-line bash script, one CLI, one overlay network, and one database. The hard part was never the architecture. It was one word in a config file — and being willing to stop trusting the confident wrong answer long enough to go read the code.

Credentials, hostnames, and tailnet addresses in this post are sanitized. If you’re building something similar and want the systemd unit or the full daemon, reach out.


Discover more from CyberCloudAI.tech

Subscribe to get the latest posts sent to your email.

Scroll to Top