Lore Vault - Self-Hosted Memory Service for Coding Agents

Project Introduction

Lore Vault is a self-hosted memory service for coding agents (Claude Code and other MCP clients). Agents write architecture decisions, pitfalls they hit and handoff notes as notes, then find them again within the same project through hybrid keyword + semantic search. It replaces the project-memory system I previously built on Open Notebook, and a single docker compose brings up the HTTP API, MCP endpoint, Web UI and backups.

Core Philosophy

• Project-Scoped: Vaults map to git remotes automatically, scope is enforced in the storage layer, and cross-project queries must be explicit

• Index First, Full Text on Demand: Queries return titles and summaries only, so the context window is not flooded

• Separate Spaces: Development (dev), worldbuilding (lore) and personal never mix

• Graceful Degradation: When the service is unreachable, the local shell still offers keyword search over a read-only snapshot

My Responsibilities

As the sole developer of the project, I am responsible for:

System Architecture Design

• Designed the layered data model (space → vault → note / document chunk) and the git-remote mapping rules

• Planned the hybrid retrieval pipeline combining SQLite FTS5 keywords with embedding vectors

• Designed HTTP and stdio MCP access that share one set of tool definitions

Core Feature Implementation

• Retrieval: FTS5 and bge-m3 vectors fused with RRF ranking

• ask: Hands retrieved notes to a model for a point-by-point answer, with cited note ids checked mechanically

• Documents: Uploaded PDF, Word and PowerPoint files are chunked in the background and searched alongside notes

• Memory Pipeline: Hooks collect agent work episodes, distilled daily into concepts injected before tool calls

• Operations: Doctor reconciliation checks, verified VACUUM INTO backups, schema migrations on startup

• Web UI: Search, browse and edit notes, manage documents and vaults, health checks and maintenance

Key Features

1. Project-Scoped Memory

• Vault keys are normalized from git remotes, with rename aliases

• Every read and write carries a vault; cross-vault queries must be explicit

2. Hybrid Search and ask

• SQLite FTS5 keywords plus Ollama bge-m3 vectors, with output bounded by a character budget

• ask produces sourced answers, positioned as a reference rather than the only source of truth

3. MCP Access

• A built-in Streamable HTTP endpoint at /mcp, connected with a single claude mcp add

• A local stdio shell that infers the vault from the working directory and falls back to a local snapshot when offline

4. Self-Hosting and Distribution

• One-command Docker Compose startup with optional Ollama and cloudflared tunnel profiles

• An installer kit with --update, --dry-run and --rollback for connecting other machines

Technologies Used

Backend

• Python: Main language (managed with uv)

• FastAPI + Uvicorn: HTTP API

• SQLite FTS5 + NumPy: Keyword and vector retrieval

• MCP SDK: HTTP and stdio access

AI / Retrieval

• Ollama (bge-m3): 1024-dimension embeddings

• OpenAI API: Summaries and ask answers

Frontend and Deployment

• Vite + TypeScript: Web UI, with Playwright end-to-end tests

• Docker Compose: Self-hosted deployment

Project Status

Current Version: v0.1.1

• ✅ Replaced the previous project-memory system; 1,455 legacy notes imported and reconciled

• ✅ Hybrid search, ask, document search and Web UI live

• ✅ Daily memory-pipeline distillation and concept injection running

• ✅ Docker self-hosted release and installer kit

• 🔄 Ongoing work on retrieval quality and multi-machine setup

Development Challenges and Achievements

1. Full-Text Search for Chinese Queries

Challenge: FTS turned whole Chinese questions into a single phrase, so keywords almost never matched.

Solution: Fixed how queries are assembled and built a fixed question set for regression testing.

Result: Top-10 hit rate on the 59-question regression rose from 72.5% to 85%.

2. A Cutover Without Interruption

Challenge: Replace a memory system in daily use without losing a single note.

Solution: Import first, reconcile record by record, and keep the old system through an observation period as a fallback.

Result: All 1,455 notes migrated, with every agent and project configuration pointed at the new service.

3. Protecting the Context Window

Challenge: The more memory there is, the more context a single agent query consumes.

Solution: Retrieval returns titles and summaries under a character budget, with full text fetched on demand.

Future Plans

• Extend retrieval support for the concept layer

• Bring worldbuilding and creative material into the lore space

• Generalize the multi-machine installer

Project Highlights

Technical Innovation

• ✅ Project scope enforced at the storage layer, with explicit cross-project queries

• ✅ Hybrid retrieval fusing FTS5 and vectors with RRF

• ✅ Read-only snapshot fallback when the service is offline

Development Quality

• ✅ Retrieval regression tests on a fixed question set

• ✅ Verified backups and startup schema migrations

Tech Stack: Python, FastAPI, SQLite FTS5, Ollama (bge-m3), MCP, Vite + TypeScript, Docker License: MIT Current Stage: v0.1.1, in daily use