Soma

Appearance

V2 Clarity Plan ​

This document is a clarity-first snapshot of the current Soma repo. It combines:

  • the current architecture as implemented today
  • the main sources of confusion and drift
  • a documentation cleanup plan
  • a split-readiness plan for backend vs desktop repos

The goal is not to describe the ideal future system. The goal is to describe the system that actually exists now, identify what is transitional, and define how to make the repo legible again.

Executive Summary ​

The repo is not random chaos. It is a real, multi-surface system carrying several active transitions at once.

Today the project is effectively three things in one repository:

  • a backend/distributed systems platform under backend/
  • a substantial Electron desktop app under desktop/soma
  • a second, less mature desktop app under desktop/tapia

The main clarity problem is not absence of structure. The main problem is that current docs, plans, historical notes, and unfinished migrations are mixed together and presented with similar weight.

The result is that a reader cannot quickly tell:

  • what is implemented
  • what is transitional
  • what is historical
  • what is only planned

That is the core problem this plan addresses.

Current Architecture ​

Top-Level Shape ​

The repo is physically organized in a mostly sensible way:

  • backend/: Rust workspace for peer, daemon, server, infra, storage, and networking
  • desktop/: Electron apps and shared desktop packages
  • proto/: shared contracts for daemon, agent, and membership concepts
  • docs/: user-facing and developer-facing documentation
  • deploy/: infrastructure manifests
  • prd/: product requirement placeholders and drafts

The problem is not top-level shape. The problem is that the repo root and docs mix code-adjacent truth with planning and migration artifacts.

Backend Reality ​

The backend is real and already substantial.

Implemented backend binaries include:

  • soma-daemon: the desktop backend; owns UDS gRPC, local peer identity, SQLite, blobs, spaces, pages, documents, and join flows
  • soma-botd: a hosted peer/bot with read-only bot mode and admin mode
  • soma-agentd: a local UDS gRPC worker for chat, embeddings, rerank, and Yjs merge/drift handling
  • soma-relayd: relay infrastructure service
  • soma-rendezvousd: rendezvous discovery service
  • soma-bffd: HTTP backend-for-frontend for model/provider access
  • soma-serverd: currently a thin wrapper runner, not a true unified backend

Important backend ownership lines:

  • soma-peer owns shared libp2p runtime and peer protocols
  • soma-membership owns join/membership/issuer/mailbox logic
  • soma-storage owns persistence for memberships, requests, decisions, documents, pages, blobs, and peer keys
  • soma-vdfs owns content-addressed blob primitives and filesystem blob storage

Implemented backend features that appear genuinely end-to-end:

  • join request and join decision flows
  • daemon-managed spaces, memberships, pages, and documents
  • daemon-managed blob upload/read/list/metadata
  • peer blob fetch by CID
  • bot manual approval and mailbox retry paths
  • agent chat/embed/rerank/drift flows
  • relay/rendezvous connectivity services

Desktop Reality ​

The desktop side is uneven.

desktop/soma is the real product UI today. It includes:

  • startup gating on daemon readiness
  • Electron main/preload/renderer boundary
  • space list/settings/member flows
  • page tree and editor flows
  • blob upload and custom soma-blob:// rendering
  • settings and local AI UI
  • daemon event consumption and cache invalidation

desktop/tapia is real as a desktop shell, but its feature depth is far behind Soma. It still uses local/stubbed exercise and leaderboard logic in places where Soma already relies on daemon-backed state.

Shared desktop packages exist, but several still carry migration residue:

  • desktop/desktp-proto: real and important
  • desktop/desktp-config: useful, but docs and package scripts are drifted
  • desktop/desktp-data: real but still part of an unfinished persistence story
  • desktop/desktp-editor: real shared editor package
  • desktop/desktp-ui: useful, but still contains Tauri-era leftovers

Shared Contract Reality ​

The strongest coupling between backend and desktop is not accidental code import. It is the shared runtime contract:

  • proto/daemon/v1/daemon.proto
  • proto/agent/v1/agent.proto
  • daemon event stream semantics
  • socket naming and stage conventions
  • blob URL shape and daemon read semantics

That shared contract layer is currently real, but it is not yet treated as a clean, versioned product boundary.

What Is Making the Repo Feel Chaotic ​

The repo is carrying multiple transitions at the same time.

1. Yoopta -> TipTap Is Incomplete ​

There is a real editor transition in progress. New editor work is TipTap-based, but names in proto, events, and some storage surfaces still refer to Yoopta.

Examples:

  • proto/daemon/v1/daemon.proto
  • backend/bins/daemon/src/grpc.rs
  • desktop/soma/src/main/services/daemon-client.ts
  • planning/active/plan-02-tiptap-editor.md
  • planning/active/weekend-task-list-tiptap-cutover.md

This creates a semantic mismatch between current UI reality and contract naming.

2. Frontend Persistence Is Mid-Migration ​

Persistence still spans multiple concepts:

  • daemon-owned canonical state
  • Electron main electron-store
  • React DB projections and local UI state
  • some historical or compatibility layers

This is documented in planning/active/plan-03-tanstack-db-migration.md and visible in desktop/soma/src/main/services/app-data-store.ts.

3. Tauri -> Electron History Still Leaks Into Current Code ​

Tauri-era concepts are still visible in code and docs, especially in shared UI and historical docs.

Examples:

  • desktop/desktp-ui/package.json
  • desktop/desktp-ui/src/hooks/use-tauri-store.ts
  • desktop/desktp-ui/src/components/layout/window-chrome.tsx
  • docs/src/archive/tauri-commands.md

Even if they no longer define the runtime, they still consume attention and erode trust.

4. Docs Present Different Time Horizons As If They Were Equal ​

The current docs mix:

  • current implementation truth
  • migration notes
  • planning docs
  • TODO-only PRDs
  • stale references

This is the biggest documentation problem in the repo.

5. Names and Ownership Boundaries Drift ​

Several names require historical explanation before they make sense:

  • desktp-*
  • VDF vs VDFS
  • server-daemon
  • lingering Yoopta* event names

None of these are fatal individually. Together they create constant low-grade uncertainty.

Current Weak Points and Transitional Seams ​

The following areas should be considered transitional or suspicious until cleaned up.

Contracts Ahead of Implementations ​

Some proto surfaces describe APIs that are not implemented or not fully real yet.

Examples include:

  • IssueIssuerCapability
  • DiscoverSpaces
  • service definitions in proto/space/v1/membership.proto that do not appear to be bound as live servers

This creates false confidence when reading contracts.

Partial or Uneven Enforcement ​

Some security and lifecycle behavior appears asymmetric or partial:

  • signature verification differs across surfaces
  • revocation is incomplete as a distributed concept
  • deletion cleanup appears incomplete for pages/blobs/blob refs
  • local socket permissions are broader than expected

This does not mean the system is fake. It means the contracts and operational guarantees need a sharper distinction between implemented and planned.

Desktop Shell and Placeholder Features ​

Soma still includes visible shell-stage areas and placeholders:

  • placeholder join screen
  • sparse home screen
  • command palette filler/demo items
  • TODO comments in feature-adjacent UI

Tapia is even more transitional in scope.

Docs and Code Drift ​

Some docs already describe behavior that is no longer current, especially around:

  • bot auto-approval behavior
  • agentd access topology
  • desktop config build assumptions
  • Tapia scope
  • blob behavior history

The Documentation Strategy Going Forward ​

The repo needs one rule:

Visible docs should describe only current, working, or finished behavior.

Anything else should be moved out of the main docs surface.

What Should Stay In Docs ​

Docs should contain only:

  • current architecture that matches implementation
  • current developer workflows that work today
  • current operations/deployment guidance
  • current contracts that desktop/backend depend on
  • ADRs that still explain live constraints

What Should Not Stay In Docs ​

The main docs surface should not contain:

  • speculative future architecture
  • migration notes presented as current truth
  • TODO-only PRDs
  • abandoned or stale design notes
  • historical Tauri or cutover guidance unless clearly archived

New Documentation Model ​

Recommended structure:

  • docs/src/00-overview.md: short repo and product overview
  • docs/src/01-glossary.md: shared language only
  • docs/src/02-v2.md: this clarity map and restructuring plan
  • docs/src/architecture/: only current, implementation-matching architecture docs
  • docs/src/development/: only current development workflows
  • docs/src/operations/ or docs/src/deployment/: current operational guidance
  • docs/src/archive/: optional; only if historical material is still useful
  • planning/: active plans, migrations, spikes, and cutover notes

File-by-File Documentation Disposition ​

This section is the concrete cleanup table.

Keep ​

Keep, but verify accuracy and tighten language where needed:

  • docs/src/00-overview.md
  • docs/src/01-glossary.md
  • docs/src/index.md
  • docs/src/security/threat-model.md
  • architecture docs that still reflect implementation, especially peer connectivity, relay/rendezvous, and deployed backend/runtime behavior
  • ADRs that explain still-visible design choices

Rewrite ​

These should remain but be rewritten to match current truth:

  • README.md
  • AGENTS.md
  • docs/src/getting-started/index.md
  • docs/src/development/daemon-grpcurl.md
  • docs/src/development/agentd-ipc.md
  • docs/src/development/desktop-config.md
  • any docs that still present Tapia as backend-integrated to the same degree as Soma
  • any docs that present Yoopta terms as if they are still the UI/editor truth rather than contract residue

Move To Planning ​

These should no longer be treated as docs. They are planning artifacts:

  • planning/active/plan-01.md
  • planning/active/plan-02-tiptap-editor.md
  • planning/active/plan-03-tanstack-db-migration.md
  • planning/active/plan-04-command-palette-cmdk.md
  • planning/active/plan-05-space-enrollment-settings-mvp.md
  • planning/active/weekend-task-list-tiptap-cutover.md

Recommended destination:

  • planning/active/ for still-relevant work
  • planning/done/ for completed plans worth retaining briefly

Archive ​

Archive only if the document still explains a strange current constraint or migration legacy:

  • docs/src/archive/tauri-commands.md
  • docs/src/archive/backend-refactor-notes.md
  • old cutover notes that explain naming or packaging compatibility

Recommended destination:

  • docs/src/archive/

If a historical doc has no ongoing explanatory value, delete it instead of archiving it.

Delete ​

Delete or replace these unless they are actively being filled with real product truth:

  • prd/000-overview.md
  • prd/010-spaces-topics.md
  • prd/020-docs-realtime.md
  • prd/030-chat.md
  • prd/040-agent.md
  • prd/050-permissions-sharing.md
  • prd/900-nonfunctional.md

As they currently stand, these files mostly create the impression that product truth exists somewhere else when it does not.

Adopt the following rules for all docs work.

Rule 1: Current Docs Must Match Running Code ​

If a feature is not implemented or not operable today, it should not appear in current docs as a normal documented feature.

Rule 2: Plans Are Not Docs ​

Plans belong in planning/, not in docs nav and not at the repo root.

Rule 3: Historical Notes Must Be Explicitly Archived ​

If a document describes prior architecture, it must live under archive/ and say so in the title.

Rule 4: Contracts Need Their Own Section ​

Daemon proto, agent proto, event streams, socket conventions, and packaging/release conventions should be documented as shared contracts, separate from product prose.

Rule 5: Delete Aggressively ​

If a document is stale, misleading, TODO-only, or duplicated, delete it unless it explains a live compatibility constraint.

Split Readiness: Backend vs Desktop ​

The repo can probably split in the future, but it should not split yet.

Why Not Yet ​

The split blockers are not physical directory layout. The blockers are shared contracts and release coupling.

Current hard coupling points:

  • shared proto/ source used by both Rust and TypeScript
  • direct desktop use of daemon and agent gRPC contracts
  • socket/stage naming assumptions shared across repos
  • daemon event stream semantics consumed by desktop
  • bundle packaging that depends on both backend and desktop release outputs
  • root tooling and docs that assume one checkout

Preconditions Before Splitting ​

Before any repo split, do the following:

  1. make proto/ a versioned contract surface
  2. decide whether desktop talks to both daemon and agentd, or only daemon
  3. define artifact naming and manifest contracts for packaging
  4. isolate desktop-only tooling from backend workspace assumptions
  5. split docs into backend-owned, desktop-owned, and shared-contract docs
  6. ensure desktop can test against released backend artifacts instead of same-commit source

Recommendation ​

Recommended sequence:

  • first: clean docs and contracts
  • second: stabilize shared interfaces
  • third: simplify ownership and release boundaries
  • fourth: evaluate repo split

So the recommendation is:

split later, after a contract-extraction phase; do not split now

Concrete Cleanup Phases ​

Phase 1: Truth Inventory ​

Create a single tracked inventory of:

  • implemented features
  • transitional features
  • planned-only surfaces
  • stale docs
  • ownership boundaries

This file is the starting point for that inventory.

Phase 2: Root Cleanup ​

Reduce visual noise at the repo root.

Actions:

  • move plan files into planning/
  • move local runtime state into one ignored runtime directory convention
  • keep root focused on code roots, top-level config, and a small number of canonical entry docs

Phase 3: Documentation Cleanup ​

Actions:

  • remove TODO-only or misleading docs
  • archive truly historical notes
  • rewrite current docs to match implementation
  • ensure nav points only to trusted, current material

Phase 4: Contract Cleanup ​

Actions:

  • document what parts of proto are implemented vs declared-only
  • document daemon event semantics as a contract
  • document socket/stage conventions as a contract
  • document packaging artifact expectations as a contract

Phase 5: Naming Cleanup ​

Actions:

  • decide the long-term strategy for desktp-*
  • decide the long-term strategy for VDF vs VDFS
  • reduce lingering Yoopta naming once contract migration is safe

Phase 6: Split Readiness Review ​

Only after the previous phases, reassess:

  • backend repo viability
  • desktop repo viability
  • whether proto/ should live in a shared contracts package/repo

Suggested Immediate Actions ​

If the goal is maximum clarity with minimum disruption, do these first:

  1. add this file and make it part of the docs entry flow
  2. move all root plan-*.md files into planning/
  3. remove or archive historical docs from docs/src/development/
  4. delete TODO-only PRD files unless they are about to be actively maintained
  5. rewrite README.md, docs/src/index.md, and docs/src/getting-started/index.md so they only describe current working behavior
  6. create one follow-up contract doc for shared backend/desktop interfaces

Final Position ​

The repo is complex, but not fundamentally incoherent.

Its main problem is that implementation truth, migration state, and historical residue are not clearly separated. The documentation cleanup should therefore be treated as architecture work, not cosmetic work.

Once the docs reflect only current and working reality, the codebase will feel dramatically less chaotic even before any structural split happens.