Skip to content

iMessage / SMS

The live iMessage/SMS connector. Unlike every other connector, it runs on the host Mac, not in the container scheduler — read this whole page before turning it on.

What it does

An incremental poll over ~/Library/Messages/chat.db, the local SQLite database the macOS Messages app maintains for iMessage and (via Continuity/SMS relay) text messages. The connector opens the database read-only and walks forward from a {rowid, guid} cursor, normalizing new rows into the same cap_items shape as every other connector. No credentials, no network calls.

Prereqs

Full Disk Access required

chat.db lives behind macOS's Full Disk Access (FDA) protection. Grant FDA to whichever process runs the poll — the terminal app, or the python3 binary itself: System Settings → Privacy & Security → Full Disk Access → add it. Without FDA, opening chat.db fails with authorization denied.

  • macOS only, run on the same Mac whose Messages app owns chat.db.
  • No IMAP-style account or app password — this connector reads a local file.

Why host-only

The connector declares host_only = True. capture_agent.poll partitions connectors by that flag: the containerized scheduler (poll.py without --host) skips any host_only row silently — no last_error noise — and only the owner's --host invocation polls it. There's deliberately no module.yaml schedule for it; it's driven by an owner-installed launchd job instead (see below).

The cap_connectors row

{
  "kind": "imessage",
  "enabled": true,
  "secret_ref": "",
  "config": {
    "db_path": "~/Library/Messages/chat.db"
  },
  "owner_addresses": ["<your iMessage handles>"]
}

secret_ref is empty — chat.db needs no credentials. owner_addresses should list your own iMessage handles (phone numbers / email aliases); if left empty, the connector falls back to the comma-separated JARVIS_OWNER_EMAIL env var, same as every other connector.

Turn it on (host)

Create the cap_connectors row above via the PocketBase admin UI, then run the poll once by hand from the repo root to confirm it works:

PYTHONPATH="agent-runtime/src:modules/capture/agent" POCKETBASE_URL=http://127.0.0.1:8091 DEV_USER_EMAIL=user@jarvis.local DEV_USER_PASSWORD=userpassword123 python3 -m capture_agent.poll --host

Expect capture poll: {'connectors_polled': 1, 'ingested': N, 'errors': 0}.

LaunchAgent (recurring)

For it to run on its own every 10 minutes, install the LaunchAgent template at modules/capture/agent/host/com.jarvis.capture.imessage-poll.plist:

  1. Edit the plist, replacing every REPLACE_ME_* placeholder (repo root, dev credentials, owner email).
  2. Install and load it:
cp modules/capture/agent/host/com.jarvis.capture.imessage-poll.plist ~/Library/LaunchAgents/com.jarvis.capture.imessage-poll.plist

launchctl load ~/Library/LaunchAgents/com.jarvis.capture.imessage-poll.plist

The job runs python3 -m capture_agent.poll --host every 10 minutes against PocketBase at http://127.0.0.1:8091. Logs go to /tmp/jarvis-capture-imessage-poll.log (stdout) and /tmp/jarvis-capture-imessage-poll.err (stderr). To stop it:

launchctl unload ~/Library/LaunchAgents/com.jarvis.capture.imessage-poll.plist

Full details, including how the LaunchAgent is wired up, live in the host runbook: modules/capture/agent/host/HOST_POLL.md.

Backfill

To import history that predates turning the live poll on:

python -m capture_agent.backfill imessage --db ~/Library/Messages/chat.db --since 2026-01-01

Voice notes: local transcription

A voice note is a message with no text and one audio attachment (Audio Message.caf, Opus in a CAF container) whose file lives under ~/Library/Messages/Attachments. Without transcription it is captured as an empty body, the app shows "Voice message", and a distilled note can only say a voice message was sent. With a local Whisper model on this Mac, the host poll turns each new voice note into text before it is written — nothing leaves the machine, and the audio file itself is never copied anywhere but a temp dir.

Prereqs

  • whisper-cli from whisper.cpp: brew install whisper-cpp (already present on a Mac that has run it once). afconvert (built into macOS) decodes the CAF; ffmpeg is the fallback decoder if you have it.
  • A ggml model file. The default configuration expects large-v3-turbo (1.6 GB, the best quality/speed balance on Apple silicon):
mkdir -p ~/.whisper-models && curl -L -o ~/.whisper-models/ggml-large-v3-turbo.bin https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin

Configure

Two env vars, read by the host poll (the LaunchAgent template already carries them):

JARVIS_WHISPER_MODEL=~/.whisper-models/ggml-large-v3-turbo.bin   # any path works
JARVIS_WHISPER_LANGUAGE=auto   # or a code such as fr

Leave JARVIS_WHISPER_MODEL empty to keep the previous behaviour (empty bodies). A configured-but-missing model prints one WARNING: voice-note transcription disabled: … line per run and the poll continues without it. The LaunchAgent also sets PATH to include /opt/homebrew/bin, where whisper-cli lives — launchd's default PATH does not. Its command runs the poll with python3.11 when present (the Homebrew Python that has the agent's dependencies): under a login shell macOS's path_helper puts /usr/bin first, so a bare python3 would be the system 3.9, which cannot import this code. After editing the installed plist, reload it — a file edit alone changes nothing launchd runs:

launchctl bootout gui/$(id -u)/com.jarvis.capture.imessage-poll; launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.jarvis.capture.imessage-poll.plist

What changes downstream

The transcript becomes the message body; the audio attachment ref stays, so the reader shows the words with a mic icon and the brain routine reads the row as [voice message, transcribed] …. A source file over 20 MB (well over an hour of Opus), a file Messages has already purged, or a transcription that yields nothing leaves the body empty as before. Whisper invents words on silence ("you", "Thank you."); the poll runs it with non-speech tokens suppressed and drops those fillers, so a pocket recording stays empty rather than becoming a fabricated sentence.

Backfill existing voice notes

Voice notes captured before the model was configured have empty bodies. A one-off host command finds every iMessage item with an empty body and an audio attachment whose file still exists, transcribes it, fills the body and re-runs the sensitivity tag on the new words. Idempotent: a filled body is never touched again, and a note whose audio yielded nothing is marked (links.voice_note = "no-speech") so later runs skip it; clear that key in the admin to retry one. --limit counts Whisper runs. Run it from the repo root on this Mac (the audio files are here), against the backend the app uses — the Hetzner box through its public URL, or the local stack:

JARVIS_WHISPER_MODEL=~/.whisper-models/ggml-large-v3-turbo.bin PYTHONPATH="agent-runtime/src:modules/capture/agent" POCKETBASE_URL=https://jarvis.example.com DEV_USER_EMAIL=… DEV_USER_PASSWORD=… python3 -m capture_agent.transcribe_backfill --limit 200

It prints transcribe backfill (whisper.cpp:ggml-large-v3-turbo): {'scanned': N, 'transcribed': N, 'no_audio_file': N, 'already_marked': N, 'failed': N} (no_audio_file also counts empty photo and reaction rows the filter matches).

Verify

In the PocketBase admin: cap_items grows by roughly the number of new messages, the row's last_poll_at updates each run, and last_error is empty. The row's cursor becomes {"rowid": <last ROWID>, "guid": <that row's guid>}.

Errors

  • Full Disk Access denied — opening chat.db raises authorization denied; grant FDA as described above and re-run.
  • chat.db rebuilt — a Messages restore or migration can reset SQLite ROWIDs. Before trusting its cursor, the connector checks that the row still at the cursor's rowid still has the recorded guid; if it doesn't, the cursor resets to 0 and the whole database is re-scanned. This is safe: cap_items dedupes by external_id (the message guid), so already-ingested messages are skipped and only genuinely new ones get written.