graph-of-claims word processor · agent fleet behind a writing surface
Vellum
The essay exports as prose. The argument underneath stays queryable.
status liverole designed + built end to end
The architecture as drawn in the repo: prose and claim graph in one projection, two model tiers by frequency and stakes, verification pushed off the request path. The repository is named vellum and the product ships as Penstroke, so the code reads Penstroke throughout.
What it solves
Every AI writing tool on the market improves sentences, which is the part a careful writer least needs help with. The part that actually fails is structural: a claim on page nine quietly contradicting page two, an assertion nobody ever sourced, a conclusion resting on a premise that was never written down. Prose polish cannot see any of that, because it treats writing as text instead of as an argument with load-bearing parts.
Who it serves
Researchers, journalists, and essayists whose credibility depends on claims holding up.
Who it is for
The writer whose professional liability is being wrong in public.
How it works
- Detect claims while typingHaiku classifies every sentence as factual, opinion, speculation, evidence, or question in roughly 200ms, so marks appear as you write rather than in a batch review later. The cheap model does the high-frequency work.
- Hold prose and graph in one documentA custom Tiptap schema over a Yjs CRDT keeps the text and the claim graph in sync through a deterministic projection, mergeable in both directions. Edit the prose, the graph follows; edit the graph, the prose follows.
- Verify in the backgroundSonnet with tool use searches the writer's own bibliography through pgvector and the open web, and never blocks typing while it works. Graph traversal, relational queries, and vector retrieval all run in one Postgres with Apache AGE, so there is a single consistency boundary instead of two databases drifting apart.
- Reconcile contradictionsWhen two claims conflict, an agent drafts a unified rewrite the writer can accept or reject. The rewrite is a proposal, never an automatic edit.
- Score the classifier in CIA 30-case hand-labeled gold set scores claim detection through Braintrust, and GitHub Actions runs it automatically on any pull request that touches the agent code, against the real model and the real database rather than fixtures. The regression gate had a hole worth naming: establishing a baseline on a first run is right on my machine and wrong in CI, where every runner is a fresh checkout and therefore always a first run. It wrote a baseline to a disk that gets thrown away and passed against it, so it was structurally incapable of failing. Two fixes: it now refuses to pass in CI without a committed baseline, and the baseline is committed, from a real run of the detector against a real database, 30-case gold set on Haiku 4.5, at 96.67 percent claim-count and type match. Proven to enforce: a second run drifted a few points on the LLM-scored metrics, the tolerance band absorbed it, and a real regression past the band would fail it. The Azure database it was built on had been deleted, so the run was measured against a local Postgres with the same Apache AGE and pgvector setup.
Built with
Next.js 16 Tiptap + Yjs CRDT Apache AGE on Postgres pgvector Haiku 4.5 + Sonnet 4.6 Braintrust evals in CI