feat(notes): operation log, block leases and sync API for collaborative editing
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 50s

Phase 1 of the collaboration plan (server side):
- internal/collab: block tree with parent IDs and fractional order keys,
  operations with conflict rules (version check, block leases, conflict
  copies instead of lost text, restore on late edits, cycle checks),
  in-memory leases/sessions and long-poll notifier.
- Storage: per-note lock + transaction, append-only effect log (doc_ops),
  idempotent results per opId (doc_op_results), snapshot, log paging,
  central DocRole. Old flat block data is converted to the tree.
- The existing whole-document save (PUT/PATCH) runs as an adapter through
  the same engine, so the current editor keeps working and writes to the log.
- API: GET /docs/{id}/snapshot, POST /docs/{id}/sync, GET /docs/{id}/changes
  (long poll), POST /docs/{id}/leave; DOC_SEQ poke for WebSocket/SSE rooms.
- Backend accepts editor (write) shares for notes and folders; the forced
  read-only migration is gone. The share dialog still offers read only.
- Tests: conflict matrix, order keys, leases, storage adapter round trip,
  idempotency, API protocol incl. long poll, and a randomized simulation
  (4 clients, lost/duplicated requests and responses, offline phases,
  lease expiry) checking convergence, no lost text and exactly-once results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
sandrodachandClaude Opus 5.5 committed 2026-10-28 11:47:56 +01:00
1 parent bcc06da3ee
commit 1ea0c78b41
23 files changed
+3267 -288

No files matched your search

+37 -4
View File
@@ -67,17 +67,50 @@ Schreibzugriffe von Lesern auf Listen oder deren Aufgaben liefern `403` („Kein
| GET/POST | `/workspace/folders/shared` | Geteilte Ordner; anlegen (`name`, `mountPath`) | | GET/POST | `/workspace/folders/shared` | Geteilte Ordner; anlegen (`name`, `mountPath`) |
| GET/DELETE | `/workspace/folders/shared/{id}` | Abruf; Löschen | | GET/DELETE | `/workspace/folders/shared/{id}` | Abruf; Löschen |
| PUT | `/workspace/folders/shared/{id}/mount` | Eigenen Einhängepfad ändern (`mountPath`) | | PUT | `/workspace/folders/shared/{id}/mount` | Eigenen Einhängepfad ändern (`mountPath`) |
| GET/POST/DELETE | `/workspace/folders/shared/{id}/shares[/{shareId}]` | Freigaben (nur Lesen wirksam) | | GET/POST/DELETE | `/workspace/folders/shared/{id}/shares[/{shareId}]` | Freigaben (`permission`: `read`/`write`) |
| GET | `/workspace/tags`, `/workspace/tags/{tag}` | Tag-Übersicht und -Details | | GET | `/workspace/tags`, `/workspace/tags/{tag}` | Tag-Übersicht und -Details |
| PUT | `/workspace/tags/{tag}/note` | Tag-Seite speichern (`content`) | | PUT | `/workspace/tags/{tag}/note` | Tag-Seite speichern (`content`) |
| GET/POST | `/documents` | Notizen auflisten; anlegen (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`) | | GET/POST | `/documents` | Notizen auflisten; anlegen (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`) |
| GET/PUT/DELETE | `/documents/{id}` | Abruf; Ändern (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`); Löschen | | GET/PUT/DELETE | `/documents/{id}` | Abruf; Ändern (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`); Löschen |
| PATCH | `/documents/{id}/blocks` | Teil-Update der Blöcke (`upserted`, `deletedIds`, Reihenfolge); löst `DOC_BLOCKS_PATCHED` aus | | PATCH | `/documents/{id}/blocks` | Teil-Update der Blöcke (`upserted`, `deletedIds`, `blockOrder`); läuft intern über das Änderungs-Log; löst `DOC_BLOCKS_PATCHED` aus |
| PUT | `/documents/{id}/move` | Ordner wechseln (`folderPath`) | | PUT | `/documents/{id}/move` | Ordner wechseln (`folderPath`) |
| POST | `/documents/{id}/blocks/{blockId}/enhanced-task` | Aus einem Block eine verknüpfte Aufgabe machen (`content`, `checked`) | | POST | `/documents/{id}/blocks/{blockId}/enhanced-task` | Aus einem Block eine verknüpfte Aufgabe machen (`content`, `checked`) |
| GET/POST/DELETE | `/documents/{id}/shares[/{userId}]` | Notiz-Freigaben (`userId`/`email`/`teamId`; wirksam nur Lesen) | | GET/POST/DELETE | `/documents/{id}/shares[/{userId}]` | Notiz-Freigaben (`userId`/`email`/`teamId`, `permission`: `read`/`write`) |
Nur der Eigentümer schreibt in eine Notiz; Empfänger erhalten `403`/„nur Lesezugriff“ ([FLAW-17](../flaws.md)). Die Endpunkte `PUT /documents/{id}` und `PATCH /documents/{id}/blocks` sind der bisherige Speicherweg (ganzes Dokument). Sie werden intern in Operationen umgerechnet und landen im selben Log wie `/sync`; der letzte Schreiber gewinnt. Sie bleiben, bis der Editor auf `/sync` umgestellt ist.
## Gemeinsames Bearbeiten von Notizen
Grundlage: [plans/kollaboration-notizen.md](../plans/kollaboration-notizen.md). Zugriff über die Rolle `owner`/`editor`/`viewer` (`Store.DocRole`); ohne Rolle `404`.
| Methode | Pfad | Zweck |
|---|---|---|
| GET | `/docs/{id}/snapshot` | Kompletter Blockbaum inkl. Grabsteinen, `title`, `titleVersion`, `headSeq`, `role`, `shared` |
| POST | `/docs/{id}/sync` | Ein Rundlauf: eigene Operationen senden, Leases halten/abgeben, Log ab `lastSeq` abholen |
| GET | `/docs/{id}/changes?since=N&wait=S` | Long-Polling: antwortet sofort, wenn `headSeq > N`, sonst bei der nächsten Änderung (auch Leases/Präsenz) oder nach höchstens 25 s. Antwort `{headSeq, changed}`; Daten holt der Client danach per `/sync` |
| POST | `/docs/{id}/leave` | Sitzung beenden, alle Leases des Clients freigeben (`{clientId}`) |
`/sync`-Anfrage:
```json
{
"clientId": "tab-7f3…",
"lastSeq": 412,
"ops": [
{ "opId": "c1a…", "kind": "set_content", "blockId": "b9…", "baseVersion": 7, "content": "Milch und Brot" },
{ "opId": "c1b…", "kind": "insert", "blockId": "bA…", "parentId": "b2…", "orderKey": "a0V", "type": "bullet", "content": "" }
],
"leases": { "hold": ["b9…", "bA…"], "release": [] }
}
```
- Operationsarten: `insert`, `set_content`, `set_props` (`patch`: `type`, `checked`, `collapsed`, `lang`), `move`, `delete`, `restore` (`ids`), `set_title` (`baseVersion`, `title`).
- `leases.hold` ist die vollständige Liste der Blöcke, die der Client halten will; alle anderen Leases des Clients werden freigegeben. Neu eingefügte Blöcke und Konfliktkopien bekommt der Autor automatisch.
- Höchstens 500 Operationen pro Anfrage. Leser dürfen keine Operationen senden (`403`), aber abfragen.
Antwort: `results` (je Operation `status` `applied`/`conflict`/`rejected`, ggf. `conflictBlockId`, `version`, `seq`), `ops` (Log-Einträge `{seq, opId, userId, clientId, effect}` ab `lastSeq`, höchstens 1000; `hasMore` bei mehr), `headSeq`, `resync` (Client ist dem Server voraus, Snapshot neu laden), `role`, `shared`, `sessions`, `presence`, `leases` (`granted`, `denied`, `others`).
Wiederholte Operationen (gleiche `opId`) werden nicht erneut angewendet; die Antwort enthält das gespeicherte Ergebnis.
## Geteilte Listen ## Geteilte Listen
+1
View File
@@ -43,6 +43,7 @@ Client → Server versteht der Server nur: `{"event":"JOIN_ROOM","room":"<id>"}`
| `TASK_REORDERED` | Aufgaben umsortieren | Benutzer aller betroffenen Listen | | `TASK_REORDERED` | Aufgaben umsortieren | Benutzer aller betroffenen Listen |
| `TASK_LIST_UPDATED` / `TASK_LIST_DELETED` | Liste ändern, (un)teilen, löschen; Payload nur `{id}`, Clients laden ihre Listen neu | Benutzer vor und nach der Änderung | | `TASK_LIST_UPDATED` / `TASK_LIST_DELETED` | Liste ändern, (un)teilen, löschen; Payload nur `{id}`, Clients laden ihre Listen neu | Benutzer vor und nach der Änderung |
| `DOC_CREATED`, `DOC_UPDATED`, `DOC_DELETED` | Notiz anlegen/ändern/löschen, Freigaben | berechtigte Benutzer | | `DOC_CREATED`, `DOC_UPDATED`, `DOC_DELETED` | Notiz anlegen/ändern/löschen, Freigaben | berechtigte Benutzer |
| `DOC_SEQ` | Änderung an Notiz, Leases oder Präsenz (`docId`, `headSeq`); Clients holen die Daten per `/docs/{id}/sync` | Raum `doc:<id>` |
| `DOC_BLOCKS_PATCHED` | `PATCH /documents/{id}/blocks` (`docId`, `updatedBy`, `upserted`, `deletedIds`, `blockOrder`) | Raum `doc:<id>` und berechtigte Benutzer | | `DOC_BLOCKS_PATCHED` | `PATCH /documents/{id}/blocks` (`docId`, `updatedBy`, `upserted`, `deletedIds`, `blockOrder`) | Raum `doc:<id>` und berechtigte Benutzer |
| `WORKSPACE_FOLDERS_UPDATED` | Ordner/Mounts ändern | Benutzer selbst; bei geteilten Ordnern Eigentümer und alle Beteiligten | | `WORKSPACE_FOLDERS_UPDATED` | Ordner/Mounts ändern | Benutzer selbst; bei geteilten Ordnern Eigentümer und alle Beteiligten |
| `LIST_CREATED`, `LIST_UPDATED`, `LIST_DELETED`, `LIST_SHARED`, `LIST_UNSHARED`, `LIST_CLEARED` | Listenverwaltung | Benutzer bzw. Raum der Liste | | `LIST_CREATED`, `LIST_UPDATED`, `LIST_DELETED`, `LIST_SHARED`, `LIST_UNSHARED`, `LIST_CLEARED` | Listenverwaltung | Benutzer bzw. Raum der Liste |
+1
View File
@@ -34,6 +34,7 @@ Das Frontend liegt per `go:embed` im Binary ([cmd/server/main.go](../cmd/server/
| `internal/storage/` | `sqlite.go` (Schema, Migrationen, gesamte Datenlogik) | | `internal/storage/` | `sqlite.go` (Schema, Migrationen, gesamte Datenlogik) |
| `internal/domain/` | Strukturen (`models.go`), ID-Erzeugung, Tag-Extraktion | | `internal/domain/` | Strukturen (`models.go`), ID-Erzeugung, Tag-Extraktion |
| `internal/ws/` | `hub.go` | | `internal/ws/` | `hub.go` |
| `internal/collab/` | Kern der Notiz-Synchronisation ohne Datenbank: Blockbaum, Operationen und Konfliktregeln, Sortierschlüssel, Leases, Long-Poll-Benachrichtigung |
| `internal/auth/` | Tokens und Passwörter | | `internal/auth/` | Tokens und Passwörter |
| `internal/config/` | Umgebungsvariablen | | `internal/config/` | Umgebungsvariablen |
| `frontend/src/features/` | `triage`, `workspace`, `lists`, `teams`, `settings`, `capture`, `auth`, `help` | | `frontend/src/features/` | `triage`, `workspace`, `lists`, `teams`, `settings`, `capture`, `auth`, `help` |
+5 -3
View File
@@ -28,11 +28,13 @@ Eine Aufgabe ist ein `thought` mit `status='kept'` oder `status='converted' AND
| Tabelle | Zweck | | Tabelle | Zweck |
|---|---| |---|---|
| `documents` | Notizen: `title`, `icon`, `folder_path`, `tags` (JSON), `is_archived`, `markdown_cache`, `shared_folder_id`, `sub_folder_path`, Altspalten `content`, `blocks_json` | | `documents` | Notizen: `title`, `icon`, `folder_path`, `tags` (JSON), `is_archived`, `markdown_cache`, `shared_folder_id`, `sub_folder_path`, Altspalten `content`, `blocks_json` |
| `blocks` | Outliner-Blöcke: `document_id`, `type`, `content`, `level`, `position` (REAL), `checked`, `collapsed`, `lang` | | `blocks` | Outliner-Blöcke als Baum: `document_id`, `parent_id` (`''` = oberste Ebene), `order_key` (Sortierschlüssel unter Geschwistern), `type`, `content`, `checked`, `collapsed`, `lang`, `version` (Textversion), `conflict_of`/`conflict_author` (Konfliktkopien), `updated_by`, `deleted` (Grabstein). `level`/`position` sind Altspalten und werden nicht mehr gepflegt; Ebene und Reihenfolge berechnet der Server beim Lesen aus dem Baum. |
| `doc_ops` | Änderungs-Log je Notiz: `seq` (fortlaufend pro Notiz), `op_id`, `user_id`, `client_id`, `kind`, `block_id`, `payload` (Effekt als JSON). Abspielen aller Einträge ergibt den Stand. |
| `doc_op_results` | Ergebnis je Client-Operation (`op_id` eindeutig) für idempotente Wiederholungen |
| `block_tags` | `#Hashtags` je Block (`tag`, `document_id`, `block_id`) | | `block_tags` | `#Hashtags` je Block (`tag`, `document_id`, `block_id`) |
| `tag_pages` | Freitext-Notiz je Benutzer und Tag | | `tag_pages` | Freitext-Notiz je Benutzer und Tag |
| `doc_pins` | Favoriten je Benutzer | | `doc_pins` | Favoriten je Benutzer |
| `doc_shares`, `doc_team_shares` | Notiz-Freigaben an Benutzer bzw. Gruppen (`permission` effektiv immer `read`) | | `doc_shares`, `doc_team_shares` | Notiz-Freigaben an Benutzer bzw. Gruppen (`permission`: `read` = Leser, `write` = Bearbeiter) |
| `workspace_folders` | Explizit angelegte Ordnerpfade | | `workspace_folders` | Explizit angelegte Ordnerpfade |
| `workspace_shared_folders` | Geteilte Ordner (Entität mit Eigentümer) | | `workspace_shared_folders` | Geteilte Ordner (Entität mit Eigentümer) |
| `workspace_folder_shares` | Freigaben geteilter Ordner | | `workspace_folder_shares` | Freigaben geteilter Ordner |
@@ -74,7 +76,7 @@ teams 1─n team_members n─1 users
- Spalten/Tabellen/Indizes werden nachgezogen (`ALTER TABLE … ADD COLUMN`, Fehler werden ignoriert). - Spalten/Tabellen/Indizes werden nachgezogen (`ALTER TABLE … ADD COLUMN`, Fehler werden ignoriert).
- Veraltete Tabellen werden gelöscht: `spaces`, `space_members`, `space_team_shares`, `document_crdt_state`, `document_crdt_updates`, `document_blocks`, `doc_block_tags`. - Veraltete Tabellen werden gelöscht: `spaces`, `space_members`, `space_team_shares`, `document_crdt_state`, `document_crdt_updates`, `document_blocks`, `doc_block_tags`.
- `users.username` wird aus dem E-Mail-Präfix aufgefüllt. - `users.username` wird aus dem E-Mail-Präfix aufgefüllt.
- Alle Dokument-, Team- und Ordnerfreigaben werden auf `read` gesetzt (bei **jedem** Start, siehe [FLAW-17](flaws.md)). - Notiz-Synchronisation (`migrateDocSync`): neue Spalten an `blocks` und `documents` (`head_seq`, `title_version`), Tabellen `doc_ops` und `doc_op_results`. Blöcke im alten flachen Format (nur `level`/`position`) werden einmalig in den Baum umgewandelt.
- Standardwerte in `system_settings` (`INSERT OR IGNORE`): `sharing_mode=teams_only`, `allow_registration=false`, `instance_name=MindSift`, SMTP leer. - Standardwerte in `system_settings` (`INSERT OR IGNORE`): `sharing_mode=teams_only`, `allow_registration=false`, `instance_name=MindSift`, SMTP leer.
- Die Tabelle `teams` wird einmalig ohne Fremdschlüssel auf `organizations` neu aufgebaut. - Die Tabelle `teams` wird einmalig ohne Fremdschlüssel auf `organizations` neu aufgebaut.
+4 -2
View File
@@ -20,13 +20,13 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
| FLAW-08 | Auth | Webhook-Ingest mit JWT prüft weder Sperrliste noch `token_version` noch `is_active` | Mittel | | FLAW-08 | Auth | Webhook-Ingest mit JWT prüft weder Sperrliste noch `token_version` noch `is_active` | Mittel |
| FLAW-09 | Auth | Kein Rate-Limit, keine Sperre bei Fehlversuchen, Mindestpasswort 4 Zeichen | Mittel | | FLAW-09 | Auth | Kein Rate-Limit, keine Sperre bei Fehlversuchen, Mindestpasswort 4 Zeichen | Mittel |
| FLAW-10 | Freigaben | `sharing_mode` wird bei Freigaben nicht durchgesetzt; Benutzerverzeichnis leakt alle Konten | Mittel | | FLAW-10 | Freigaben | `sharing_mode` wird bei Freigaben nicht durchgesetzt; Benutzerverzeichnis leakt alle Konten | Mittel |
| FLAW-11 | Workspace | Block-Upsert (`ON CONFLICT(id)`) prüft das Dokument nicht | Mittel | | FLAW-11 | Workspace | ~~Block-Upsert (`ON CONFLICT(id)`) prüft das Dokument nicht~~ | **Behoben** |
| FLAW-12 | Benachrichtigungen | `notify_before_minutes` wirkungslos; Alarm feuert nach Umplanung nie wieder | Mittel | | FLAW-12 | Benachrichtigungen | `notify_before_minutes` wirkungslos; Alarm feuert nach Umplanung nie wieder | Mittel |
| FLAW-13 | Datenbank | Keine Schema-Versionierung; Migrationsfehler werden verschluckt; `migrations/001_init.sql` ist toter Code | Mittel | | FLAW-13 | Datenbank | Keine Schema-Versionierung; Migrationsfehler werden verschluckt; `migrations/001_init.sql` ist toter Code | Mittel |
| FLAW-14 | Backend | Fehler von Schreiboperationen werden häufig ignoriert (`_, _ =`) | Mittel | | FLAW-14 | Backend | Fehler von Schreiboperationen werden häufig ignoriert (`_, _ =`) | Mittel |
| FLAW-15 | Realtime | ~~`WriteTimeout` von 30 s beendet SSE-Streams~~ | **Behoben** | | FLAW-15 | Realtime | ~~`WriteTimeout` von 30 s beendet SSE-Streams~~ | **Behoben** |
| FLAW-16 | Setup | `/auth/setup` ist bis zum ersten Konto offen; Demo-Modus legt `admin/admin` an | Mittel | | FLAW-16 | Setup | `/auth/setup` ist bis zum ersten Konto offen; Demo-Modus legt `admin/admin` an | Mittel |
| FLAW-17 | Workspace | Dokument- und Ordnerfreigaben sind per Migration erzwungen schreibgeschützt; tote Schreibpfade | Niedrig | | FLAW-17 | Workspace | Bearbeiter-Freigaben für Notizen erst im Backend aktiv, nicht im UI | Niedrig |
| FLAW-18 | Suche | Nur eigene Gedanken und Dokumente, `LIKE` ohne Escaping, keine FTS | Niedrig | | FLAW-18 | Suche | Nur eigene Gedanken und Dokumente, `LIKE` ohne Escaping, keine FTS | Niedrig |
| FLAW-19 | Qualität | Frontend-Tests nur für den Outliner, `lint` ist ein Platzhalter | Niedrig | | FLAW-19 | Qualität | Frontend-Tests nur für den Outliner, `lint` ist ein Platzhalter | Niedrig |
| FLAW-20 | Qualität | Sehr große Dateien, duplizierte Zugriffslogik | Niedrig | | FLAW-20 | Qualität | Sehr große Dateien, duplizierte Zugriffslogik | Niedrig |
@@ -96,6 +96,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
- **Maßnahme:** Eine zentrale Funktion `canShareWith(owner, target)` und in allen Share-Handlern verwenden; Verzeichnisse darauf einschränken. - **Maßnahme:** Eine zentrale Funktion `canShareWith(owner, target)` und in allen Share-Handlern verwenden; Verzeichnisse darauf einschränken.
### FLAW-11: Block-Upsert prüft das Dokument nicht ### FLAW-11: Block-Upsert prüft das Dokument nicht
- **Status: Behoben.** Alle Blockänderungen laufen über `Store.applyDocChange`: Einfügen mit einer Block-ID aus einer anderen Notiz wird abgelehnt (`exists`), und der Upsert ändert nur Zeilen derselben Notiz (`WHERE blocks.document_id = excluded.document_id`). Test: `TestInsertCannotReuseBlockIDOfAnotherNote`.
- **Fundstelle:** `Store.PatchDocumentBlocks`, `INSERT INTO blocks … ON CONFLICT(id) DO UPDATE`. - **Fundstelle:** `Store.PatchDocumentBlocks`, `INSERT INTO blocks … ON CONFLICT(id) DO UPDATE`.
- **Problem:** `blocks.id` ist global eindeutig. Existiert die ID bereits in einem **anderen** Dokument, überschreibt der Konflikt-Zweig dessen Inhalt, ohne `document_id` zu prüfen. Voraussetzung: Schreibrecht im eigenen Dokument und Kenntnis einer fremden Block-ID. - **Problem:** `blocks.id` ist global eindeutig. Existiert die ID bereits in einem **anderen** Dokument, überschreibt der Konflikt-Zweig dessen Inhalt, ohne `document_id` zu prüfen. Voraussetzung: Schreibrecht im eigenen Dokument und Kenntnis einer fremden Block-ID.
- **Maßnahme:** `DO UPDATE … WHERE blocks.document_id = excluded.document_id`; fremde IDs ablehnen. - **Maßnahme:** `DO UPDATE … WHERE blocks.document_id = excluded.document_id`; fremde IDs ablehnen.
@@ -134,6 +135,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
- **Maßnahme:** Setup per einmaligem Token aus dem Server-Log absichern; Demo-Modus nur zusammen mit expliziter Warnung/Bestätigung. - **Maßnahme:** Setup per einmaligem Token aus dem Server-Log absichern; Demo-Modus nur zusammen mit expliziter Warnung/Bestätigung.
### FLAW-17: Schreibgeschützte Dokumentfreigaben ### FLAW-17: Schreibgeschützte Dokumentfreigaben
- **Stand:** Die Zwangsmigration auf `read` ist entfernt, das Backend speichert und prüft Bearbeiter-Freigaben (`DocRole`, `/docs/{id}/sync`). Offen bis Phase 5 des [Kollaborationsplans](plans/kollaboration-notizen.md): Freigabe-Dialog bietet nur „Nur Lesen“, der alte Editor (`PUT /documents/{id}`) schreibt nur für den Eigentümer.
- **Fundstellen:** Migration in `Store.migrate` (`UPDATE doc_shares SET permission = 'read' …`, ebenso `doc_team_shares`, `workspace_folder_shares`), `PatchDocumentBlocks`/`UpdateDocument` (nur Eigentümer). - **Fundstellen:** Migration in `Store.migrate` (`UPDATE doc_shares SET permission = 'read' …`, ebenso `doc_team_shares`, `workspace_folder_shares`), `PatchDocumentBlocks`/`UpdateDocument` (nur Eigentümer).
- **Hinweis:** Das ist offenbar gewollt: Nur der Eigentümer schreibt in ein Dokument, Empfänger lesen. Es gibt daher **kein** kollaboratives Bearbeiten. Das Schema (`CHECK(permission IN ('read','write'))`), die Zweige für `write` im Code und frühere Dokumentation suggerieren etwas anderes. Es gibt weder Sperren noch Konfliktauflösung, noch Cursor-Presence. - **Hinweis:** Das ist offenbar gewollt: Nur der Eigentümer schreibt in ein Dokument, Empfänger lesen. Es gibt daher **kein** kollaboratives Bearbeiten. Das Schema (`CHECK(permission IN ('read','write'))`), die Zweige für `write` im Code und frühere Dokumentation suggerieren etwas anderes. Es gibt weder Sperren noch Konfliktauflösung, noch Cursor-Presence.
- **Maßnahme:** Entweder Schreibfreigaben mit Konfliktbehandlung ergänzen oder den toten Code und das Schema bereinigen. - **Maßnahme:** Entweder Schreibfreigaben mit Konfliktbehandlung ergänzen oder den toten Code und das Schema bereinigen.
+1 -1
View File
@@ -4,7 +4,7 @@ Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.ts
## Datenhaltung ## Datenhaltung
- Eine Notiz ist ein Datensatz in `documents` plus geordnete Zeilen in `blocks` (`position` als Kommazahl, `level` für die Einrückung). Alles liegt in SQLite; es gibt keine Dateien auf der Platte. - Eine Notiz ist ein Datensatz in `documents` plus ein Baum von Blöcken in `blocks` (`parent_id`, Sortierschlüssel `order_key`). Jede Änderung läuft über `internal/collab` und wird im Log `doc_ops` festgehalten. Ebene und Position für die Oberfläche berechnet der Server beim Lesen. Alles liegt in SQLite; es gibt keine Dateien auf der Platte.
- Bei jeder Änderung wird `documents.markdown_cache` neu erzeugt; `block_tags` wird aus `#Hashtags` der Blöcke neu aufgebaut. - Bei jeder Änderung wird `documents.markdown_cache` neu erzeugt; `block_tags` wird aus `#Hashtags` der Blöcke neu aufgebaut.
- Der Export als `.md` erfolgt im Browser (Button im Editor, Blob-Download). - Der Export als `.md` erfolgt im Browser (Button im Editor, Blob-Download).
- Blocktypen: Überschriften `h1`–`h3`, `paragraph`/`text`, `bullet`, `todo`, `code`, `callout`, `divider`, `table`, `progress`. - Blocktypen: Überschriften `h1`–`h3`, `paragraph`/`text`, `bullet`, `todo`, `code`, `callout`, `divider`, `table`, `progress`.
+1 -1
View File
@@ -278,7 +278,7 @@ Damit private Notizen keinen unnötigen Aufwand erzeugen, liefert der Server im
| Phase | Inhalt | Ergebnis | | Phase | Inhalt | Ergebnis |
|---|---|---| |---|---|---|
| 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Umsortieren (FLAW-03), Stream-Timeout (FLAW-15). **Erledigt.** Die zentrale `docRole` wandert in Phase 1, weil sie auf dem neuen Freigabemodell (`workspace_access`) aufbaut. | Grundlage dicht | | 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Umsortieren (FLAW-03), Stream-Timeout (FLAW-15). **Erledigt.** Die zentrale `docRole` wandert in Phase 1, weil sie auf dem neuen Freigabemodell (`workspace_access`) aufbaut. | Grundlage dicht |
| 1 | Neues Schema inkl. `workspace_access` und zentraler `docRole`, Operations-Log, `/snapshot`, `/sync`, Konfliktregeln, Simulationstests | Server fertig und abgesichert | | 1 | Operations-Log, `/snapshot`, `/sync`, `/changes`, `/leave`, Konfliktregeln, Leases, zentrale `DocRole`, Simulationstests. **Erledigt.** Abweichungen: Die bestehenden Tabellen `blocks`/`documents` wurden erweitert statt ersetzt (Baum per `parent_id`/`order_key`, alte Daten werden umgewandelt); der bisherige Speicherweg (`PUT`/`PATCH`) läuft als Adapter durch dieselbe Engine. `DocRole` arbeitet auf den vorhandenen Freigabetabellen; deren Zusammenlegung zu `workspace_access` erfolgt mit dem Freigabe-Dialog in Phase 5. `/changes` liefert nur `{headSeq, changed}`, die Daten kommen immer über `/sync`. | Server fertig und abgesichert |
| 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt (Baum statt `level`) | Bearbeiten funktioniert überall, auch offline | | 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt (Baum statt `level`) | Bearbeiten funktioniert überall, auch offline |
| 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | | 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z |
| 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt | | 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt |
+3 -3
View File
@@ -14,10 +14,10 @@
|---|---|---|---|---| |---|---|---|---|---|
| **Aufgabenliste** (`task_list_shares.permission`) | alles, löschen, (un)teilen | Aufgaben anlegen/ändern/abhaken/verschieben/löschen, Liste umbenennen, Spalten ändern | nur ansehen | Benutzer, Gruppen | | **Aufgabenliste** (`task_list_shares.permission`) | alles, löschen, (un)teilen | Aufgaben anlegen/ändern/abhaken/verschieben/löschen, Liste umbenennen, Spalten ändern | nur ansehen | Benutzer, Gruppen |
| **Geteilte Liste** (`list_shares.role`) | alles, löschen, (un)teilen | Einträge anlegen/ändern/abhaken/löschen | nur ansehen | Benutzer, Gruppen | | **Geteilte Liste** (`list_shares.role`) | alles, löschen, (un)teilen | Einträge anlegen/ändern/abhaken/löschen | nur ansehen | Benutzer, Gruppen |
| **Notiz** (`doc_shares`, `doc_team_shares`) | schreiben | – | nur lesen | Benutzer, Gruppen | | **Notiz** (`doc_shares`, `doc_team_shares`) | alles, Freigaben, löschen | Inhalt, Struktur, Titel | nur lesen | Benutzer, Gruppen |
| **Geteilter Ordner** (`workspace_folder_shares`) | schreiben | – | nur lesen | Benutzer, Gruppen | | **Geteilter Ordner** (`workspace_folder_shares`) | alles | Notizen darin bearbeiten | nur lesen | Benutzer, Gruppen |
- **Notizen und Ordner sind absichtlich schreibgeschützt für Empfänger.** Das erzwingt eine Migration beim Start (`permission = 'read'`); Schreibpfade im Code sind tot ([FLAW-17](flaws.md)). - **Notizen und Ordner:** Die effektive Rolle berechnet `Store.DocRole` (Eigentümer, Eigentümer des geteilten Ordners = Bearbeiter, direkte Freigabe, Gruppenfreigabe, Ordnerfreigabe; höchste gewinnt). Das Backend akzeptiert Bearbeiter-Freigaben (`write`). Der Freigabe-Dialog bietet bis zur Umstellung des Editors (Phase 5 im [Plan](plans/kollaboration-notizen.md)) noch nur „Nur Lesen“ an, und der alte Editor schreibt weiterhin nur für den Eigentümer.
- Bei Aufgabenlisten und Listen gilt: Hat jemand mehrere Zugriffswege (direkt und über Gruppen), zählt die **höchste** Stufe. - Bei Aufgabenlisten und Listen gilt: Hat jemand mehrere Zugriffswege (direkt und über Gruppen), zählt die **höchste** Stufe.
- Für Gruppen gilt die Freigabe für alle aktuellen und künftigen Mitglieder. - Für Gruppen gilt die Freigabe für alle aktuellen und künftigen Mitglieder.
- Aufgabenlisten besitzen zusätzlich das Altfeld `team_id`: Mitglieder dieser Gruppe haben ohne Freigabezeile Bearbeiterrechte. Eine Gruppenfreigabe über den Dialog setzt `team_id`, falls es noch leer ist. - Aufgabenlisten besitzen zusätzlich das Altfeld `team_id`: Mitglieder dieser Gruppe haben ohne Freigabezeile Bearbeiterrechte. Eine Gruppenfreigabe über den Dialog setzt `team_id`, falls es noch leer ist.
+228
View File
@@ -0,0 +1,228 @@
package api
import (
"encoding/json"
"net/http"
"strconv"
"time"
"github.com/go-chi/chi/v5"
"github.com/sandrodach/mindsift/internal/auth"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/storage"
)
const (
syncMaxOps = 500
syncMaxLog = 1000
syncMaxBody = 8 << 20
longPollMax = 25 * time.Second
maxClientIDLen = 64
)
type syncLeases struct {
Granted []string `json:"granted"`
Denied []string `json:"denied"`
Others []collab.LeaseInfo `json:"others"`
}
type syncResponse struct {
Results []collab.Result `json:"results"`
Ops []storage.DocLogEntry `json:"ops"`
HeadSeq int64 `json:"headSeq"`
HasMore bool `json:"hasMore"`
Resync bool `json:"resync,omitempty"` // client state is ahead of the server: reload the snapshot
Role string `json:"role"`
Shared bool `json:"shared"`
Sessions int `json:"sessions"`
Presence []collab.Presence `json:"presence"`
Leases syncLeases `json:"leases"`
}
// docAccess resolves the caller's role; it writes 404 and returns "" without access.
func (s *Server) docAccess(w http.ResponseWriter, r *http.Request) (userID, docID, role string) {
userID = auth.GetUserID(r.Context())
docID = chi.URLParam(r, "id")
role = s.store.DocRole(userID, docID)
if role == "" {
writeError(w, http.StatusNotFound, storage.ErrDocNotFound.Error())
}
return userID, docID, role
}
func (s *Server) handleDocSnapshot(w http.ResponseWriter, r *http.Request) {
_, docID, role := s.docAccess(w, r)
if role == "" {
return
}
snap, err := s.store.GetDocSnapshot(docID)
if err != nil {
writeError(w, http.StatusNotFound, err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"docId": snap.DocID,
"title": snap.Title,
"titleVersion": snap.TitleVersion,
"headSeq": snap.HeadSeq,
"blocks": snap.Blocks,
"role": role,
"shared": s.store.IsDocShared(docID),
})
}
// handleDocSync is the single round trip of the sync protocol: send own operations, hold or release
// block leases, and receive every log entry after lastSeq.
func (s *Server) handleDocSync(w http.ResponseWriter, r *http.Request) {
userID, docID, role := s.docAccess(w, r)
if role == "" {
return
}
var req struct {
ClientID string `json:"clientId"`
LastSeq int64 `json:"lastSeq"`
Ops []collab.Op `json:"ops"`
Leases *struct {
Hold []string `json:"hold"`
Release []string `json:"release"`
} `json:"leases"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, syncMaxBody)).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "Ungültiges JSON")
return
}
if req.ClientID == "" || len(req.ClientID) > maxClientIDLen {
writeError(w, http.StatusBadRequest, "clientId fehlt oder ist zu lang")
return
}
if len(req.Ops) > syncMaxOps {
writeError(w, http.StatusRequestEntityTooLarge, "Zu viele Änderungen in einer Anfrage")
return
}
canEdit := storage.CanEditDoc(role)
if len(req.Ops) > 0 && !canEdit {
writeError(w, http.StatusForbidden, "Keine Schreibrechte für diese Notiz (nur Lesen)")
return
}
actor := collab.Actor{UserID: userID, ClientID: req.ClientID}
if u, err := s.store.GetUserByID(userID); err == nil {
actor.Name = u.DisplayName
}
s.leases.Touch(docID, actor)
resp := syncResponse{Results: []collab.Result{}, Role: role, Leases: syncLeases{Granted: []string{}, Denied: []string{}}}
changed := false
if req.Leases != nil && canEdit {
s.leases.ReleaseAllExcept(docID, req.ClientID, req.Leases.Hold)
s.leases.Release(docID, req.ClientID, req.Leases.Release)
resp.Leases.Granted, resp.Leases.Denied = s.leases.Hold(docID, actor, req.Leases.Hold)
changed = true
}
if len(req.Ops) > 0 {
results, _, err := s.store.ApplyDocOps(docID, actor, req.Ops, s.leases.View(docID))
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
resp.Results = results
// New blocks (and conflict copies) belong to their author right away
var own []string
for i, res := range results {
if res.Status == collab.StatusApplied && req.Ops[i].Kind == collab.OpInsert {
own = append(own, req.Ops[i].BlockID)
}
if res.ConflictBlockID != "" {
own = append(own, res.ConflictBlockID)
}
if res.Seq > 0 {
changed = true
}
}
if len(own) > 0 {
granted, _ := s.leases.Hold(docID, actor, own)
resp.Leases.Granted = append(resp.Leases.Granted, granted...)
}
}
entries, head, err := s.store.GetDocOpsSince(docID, req.LastSeq, syncMaxLog)
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
if req.LastSeq > head {
resp.Resync = true
entries = []storage.DocLogEntry{}
}
resp.Ops = entries
resp.HeadSeq = head
resp.HasMore = len(entries) == syncMaxLog && entries[len(entries)-1].Seq < head
resp.Shared = s.store.IsDocShared(docID)
resp.Sessions, resp.Presence = s.leases.Sessions(docID)
resp.Leases.Others = s.leases.Others(docID, req.ClientID)
if changed {
s.notifyDocChanged(docID, head)
}
writeJSON(w, http.StatusOK, resp)
}
// notifyDocChanged wakes long-poll requests and pokes WebSocket/SSE clients in the note's room.
func (s *Server) notifyDocChanged(docID string, head int64) {
s.notifier.Notify(docID)
s.wsHub.BroadcastDocEvent(docID, "DOC_SEQ", map[string]interface{}{"docId": docID, "headSeq": head})
}
// handleDocChanges is the long-poll fallback: it returns as soon as the note changes (or leases/presence
// change), at the latest after `wait` seconds. Clients then call /sync to fetch the data.
func (s *Server) handleDocChanges(w http.ResponseWriter, r *http.Request) {
_, docID, role := s.docAccess(w, r)
if role == "" {
return
}
since, _ := strconv.ParseInt(r.URL.Query().Get("since"), 10, 64)
wait := longPollMax
if sec, err := strconv.Atoi(r.URL.Query().Get("wait")); err == nil && sec >= 0 && time.Duration(sec)*time.Second < wait {
wait = time.Duration(sec) * time.Second
}
_ = http.NewResponseController(w).SetWriteDeadline(time.Now().Add(wait + 10*time.Second))
woken := s.notifier.Changed(docID) // subscribe before reading, so no change can slip through
_, head, err := s.store.GetDocOpsSince(docID, since, 0)
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
changed := head > since
if !changed {
select {
case <-woken:
changed = true
case <-time.After(wait):
case <-r.Context().Done():
return
}
_, head, _ = s.store.GetDocOpsSince(docID, since, 0)
}
writeJSON(w, http.StatusOK, map[string]interface{}{"headSeq": head, "changed": changed})
}
// handleDocLeave ends a client's session on a note and frees its leases (tab closed, app in background).
func (s *Server) handleDocLeave(w http.ResponseWriter, r *http.Request) {
_, docID, role := s.docAccess(w, r)
if role == "" {
return
}
var req struct {
ClientID string `json:"clientId"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&req); err != nil || req.ClientID == "" {
writeError(w, http.StatusBadRequest, "clientId fehlt")
return
}
s.leases.Leave(docID, req.ClientID)
_, head, _ := s.store.GetDocOpsSince(docID, 0, 0)
s.notifyDocChanged(docID, head)
writeJSON(w, http.StatusOK, map[string]bool{"success": true})
}
+198
View File
@@ -0,0 +1,198 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain"
)
func syncDoc(t *testing.T, ts *httptest.Server, token, docID string, body map[string]interface{}) (int, syncResponse) {
t.Helper()
resp := doJSON(t, ts, http.MethodPost, "/api/v1/docs/"+docID+"/sync", token, body)
var out syncResponse
if resp.StatusCode == http.StatusOK {
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatalf("decode sync response: %v", err)
}
}
return resp.StatusCode, out
}
func strp(s string) *string { return &s }
type collabFixture struct {
ts *httptest.Server
docID string
ownerTok, editorTok, viewerTok string
outsiderTok string
}
func newCollabFixture(t *testing.T) collabFixture {
t.Helper()
_, store, ts, owner, ownerTok := startRealtimeServer(t)
editor, editorTok := newTestUser(t, store, "editor@example.com")
viewer, viewerTok := newTestUser(t, store, "viewer@example.com")
_, outsiderTok := newTestUser(t, store, "outsider2@example.com")
doc, err := store.CreateDocument(owner.ID, "", "Gemeinsam", "📄", "", []domain.DocumentBlock{{ID: "b1", Type: "bullet", Content: "Erster Punkt"}}, nil, false)
if err != nil {
t.Fatalf("create doc: %v", err)
}
if _, err := store.CreateDocShare(owner.ID, doc.ID, editor.ID, "write"); err != nil {
t.Fatalf("share editor: %v", err)
}
if _, err := store.CreateDocShare(owner.ID, doc.ID, viewer.ID, "read"); err != nil {
t.Fatalf("share viewer: %v", err)
}
return collabFixture{ts: ts, docID: doc.ID, ownerTok: ownerTok, editorTok: editorTok, viewerTok: viewerTok, outsiderTok: outsiderTok}
}
func TestDocSyncBetweenTwoEditors(t *testing.T) {
f := newCollabFixture(t)
// Owner inserts a block and edits the existing one in one round trip
code, a := syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{
"clientId": "owner-tab", "lastSeq": 0,
"ops": []collab.Op{
{OpID: "o1", Kind: collab.OpInsert, BlockID: "b2", ParentID: strp(""), OrderKey: "a5", Content: strp("Zweiter Punkt")},
{OpID: "o2", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Erster Punkt (bearbeitet)")},
},
})
if code != http.StatusOK || a.Results[0].Status != collab.StatusApplied || a.Results[1].Status != collab.StatusApplied {
t.Fatalf("owner sync: %d %+v", code, a.Results)
}
if len(a.Leases.Granted) == 0 || a.Leases.Granted[0] != "b2" {
t.Fatalf("inserted block must be leased to its author: %+v", a.Leases)
}
// Editor pulls everything since 0 and replays it
code, b := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{"clientId": "editor-tab", "lastSeq": 0})
if code != http.StatusOK || b.HeadSeq != a.HeadSeq || b.Role != "editor" || !b.Shared {
t.Fatalf("editor pull: %d head=%d role=%s shared=%v", code, b.HeadSeq, b.Role, b.Shared)
}
replay := collab.NewDoc()
for _, e := range b.Ops {
replay.ApplyEffect(e.Effect)
}
flat := replay.Flatten()
if len(flat) != 2 || flat[0].Content != "Erster Punkt (bearbeitet)" || flat[1].Content != "Zweiter Punkt" {
t.Fatalf("replayed state wrong: %+v", flat)
}
if len(b.Leases.Others) == 0 {
t.Fatal("editor must see the owner's lease on b2")
}
// Editor writes into the leased block: owner keeps it, editor's text becomes a conflict copy
_, c := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{
"clientId": "editor-tab", "lastSeq": b.HeadSeq,
"ops": []collab.Op{{OpID: "e1", Kind: collab.OpSetContent, BlockID: "b2", BaseVersion: 1, Content: strp("Editor war hier")}},
})
if c.Results[0].Status != collab.StatusConflict || c.Results[0].ConflictBlockID == "" {
t.Fatalf("write into foreign lease must conflict: %+v", c.Results)
}
}
func TestDocSyncRetryIsIdempotent(t *testing.T) {
f := newCollabFixture(t)
body := map[string]interface{}{
"clientId": "tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "same-op", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Einmal")}},
}
_, first := syncDoc(t, f.ts, f.ownerTok, f.docID, body)
_, second := syncDoc(t, f.ts, f.ownerTok, f.docID, body) // response of the first was "lost"
if first.Results[0] != second.Results[0] || first.HeadSeq != second.HeadSeq {
t.Fatalf("retry applied twice: %+v / %+v", first, second)
}
}
func TestDocSyncPermissions(t *testing.T) {
f := newCollabFixture(t)
op := map[string]interface{}{
"clientId": "tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "v1", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Leser schreibt")}},
}
if code, _ := syncDoc(t, f.ts, f.viewerTok, f.docID, op); code != http.StatusForbidden {
t.Errorf("viewer sending ops: want 403, got %d", code)
}
code, pull := syncDoc(t, f.ts, f.viewerTok, f.docID, map[string]interface{}{"clientId": "tab", "lastSeq": 0, "leases": map[string]interface{}{"hold": []string{"b1"}}})
if code != http.StatusOK || pull.Role != "viewer" || len(pull.Leases.Granted) != 0 {
t.Errorf("viewer pull: %d role=%s granted=%v", code, pull.Role, pull.Leases.Granted)
}
if code, _ := syncDoc(t, f.ts, f.outsiderTok, f.docID, map[string]interface{}{"clientId": "tab"}); code != http.StatusNotFound {
t.Errorf("outsider: want 404, got %d", code)
}
for _, path := range []string{"/snapshot", "/changes?wait=0"} {
if resp := doJSON(t, f.ts, http.MethodGet, "/api/v1/docs/"+f.docID+path, f.outsiderTok, nil); resp.StatusCode != http.StatusNotFound {
t.Errorf("outsider %s: want 404, got %d", path, resp.StatusCode)
}
}
}
func TestDocSnapshotMatchesLog(t *testing.T) {
f := newCollabFixture(t)
syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{
"clientId": "tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "t1", Kind: collab.OpSetTitle, BaseVersion: 0, Title: strp("Neuer Titel")}},
})
resp := doJSON(t, f.ts, http.MethodGet, "/api/v1/docs/"+f.docID+"/snapshot", f.editorTok, nil)
var snap struct {
Title string `json:"title"`
HeadSeq int64 `json:"headSeq"`
Blocks []collab.Block `json:"blocks"`
Role string `json:"role"`
}
if err := json.NewDecoder(resp.Body).Decode(&snap); err != nil {
t.Fatal(err)
}
if snap.Title != "Neuer Titel" || snap.HeadSeq == 0 || len(snap.Blocks) != 1 || snap.Role != "editor" {
t.Fatalf("snapshot: %+v", snap)
}
}
func TestDocChangesLongPollWakesOnChange(t *testing.T) {
f := newCollabFixture(t)
_, base := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{"clientId": "editor-tab", "lastSeq": 0})
type pollResult struct {
Changed bool `json:"changed"`
HeadSeq int64 `json:"headSeq"`
}
done := make(chan pollResult, 1)
start := time.Now()
go func() {
req, _ := http.NewRequest(http.MethodGet, f.ts.URL+"/api/v1/docs/"+f.docID+"/changes?wait=20&since="+itoa(base.HeadSeq), nil)
req.Header.Set("Authorization", "Bearer "+f.editorTok)
resp, err := http.DefaultClient.Do(req)
if err != nil {
done <- pollResult{}
return
}
defer resp.Body.Close()
var r pollResult
_ = json.NewDecoder(resp.Body).Decode(&r)
done <- r
}()
time.Sleep(300 * time.Millisecond)
syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{
"clientId": "owner-tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "lp1", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Weckruf")}},
})
select {
case r := <-done:
if !r.Changed || r.HeadSeq <= base.HeadSeq || time.Since(start) > 5*time.Second {
t.Fatalf("long poll result: %+v after %v", r, time.Since(start))
}
case <-time.After(10 * time.Second):
t.Fatal("long poll did not wake up")
}
}
func itoa(n int64) string {
b, _ := json.Marshal(n)
return string(b)
}
+13 -2
View File
@@ -18,6 +18,7 @@ import (
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/cors" "github.com/go-chi/cors"
"github.com/sandrodach/mindsift/internal/auth" "github.com/sandrodach/mindsift/internal/auth"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/config" "github.com/sandrodach/mindsift/internal/config"
"github.com/sandrodach/mindsift/internal/domain" "github.com/sandrodach/mindsift/internal/domain"
"github.com/sandrodach/mindsift/internal/storage" "github.com/sandrodach/mindsift/internal/storage"
@@ -29,6 +30,9 @@ type Server struct {
store *storage.Store store *storage.Store
wsHub *ws.Hub wsHub *ws.Hub
staticFS fs.FS staticFS fs.FS
leases *collab.LeaseManager // block leases and open sessions of notes
notifier *collab.Notifier // wakes long-poll requests on note changes
} }
func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS fs.FS) *Server { func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS fs.FS) *Server {
@@ -37,6 +41,8 @@ func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS
store: store, store: store,
wsHub: wsHub, wsHub: wsHub,
staticFS: staticFS, staticFS: staticFS,
leases: collab.NewLeaseManager(nil),
notifier: collab.NewNotifier(),
} }
wsHub.SetRoomAuthorizer(srv.canJoinRoom) wsHub.SetRoomAuthorizer(srv.canJoinRoom)
go srv.startAlarmTicker() go srv.startAlarmTicker()
@@ -46,8 +52,7 @@ func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS
// canJoinRoom allows realtime rooms only to users with read access: "doc:<id>" for notes, otherwise a shared list ID. // canJoinRoom allows realtime rooms only to users with read access: "doc:<id>" for notes, otherwise a shared list ID.
func (s *Server) canJoinRoom(userID, room string) bool { func (s *Server) canJoinRoom(userID, room string) bool {
if docID, ok := strings.CutPrefix(room, "doc:"); ok { if docID, ok := strings.CutPrefix(room, "doc:"); ok {
_, err := s.store.GetDocumentByID(userID, docID) return s.store.DocRole(userID, docID) != ""
return err == nil
} }
_, err := s.store.GetListByID(userID, room) _, err := s.store.GetListByID(userID, room)
return err == nil return err == nil
@@ -159,6 +164,12 @@ func (s *Server) Router() http.Handler {
r.Delete("/api/v1/documents/{id}", s.handleDeleteDocument) r.Delete("/api/v1/documents/{id}", s.handleDeleteDocument)
r.Post("/api/v1/documents/{id}/blocks/{blockId}/enhanced-task", s.handleCreateEnhancedTask) r.Post("/api/v1/documents/{id}/blocks/{blockId}/enhanced-task", s.handleCreateEnhancedTask)
// Collaborative note editing (operation log)
r.Get("/api/v1/docs/{id}/snapshot", s.handleDocSnapshot)
r.Post("/api/v1/docs/{id}/sync", s.handleDocSync)
r.Get("/api/v1/docs/{id}/changes", s.handleDocChanges)
r.Post("/api/v1/docs/{id}/leave", s.handleDocLeave)
// Document Sharing & Collaboration // Document Sharing & Collaboration
r.Get("/api/v1/documents/{id}/shares", s.handleGetDocShares) r.Get("/api/v1/documents/{id}/shares", s.handleGetDocShares)
r.Post("/api/v1/documents/{id}/shares", s.handleCreateDocShare) r.Post("/api/v1/documents/{id}/shares", s.handleCreateDocShare)
+201
View File
@@ -0,0 +1,201 @@
// Package collab contains the database-independent core of collaborative note editing: the block tree,
// operations with their conflict rules, block leases and change notification. The same rules run on the
// server for every incoming operation; clients replay the resulting effects to stay in sync.
package collab
import (
"fmt"
"sort"
)
// Props are per-block settings that change independently of the text.
type Props struct {
Checked bool `json:"checked,omitempty"`
Collapsed bool `json:"collapsed,omitempty"`
Lang string `json:"lang,omitempty"`
ConflictOf string `json:"conflictOf,omitempty"` // ID of the block (or "title") this conflict copy belongs to
ConflictAuthor string `json:"conflictAuthor,omitempty"` // display name of the author whose text lost
}
type Block struct {
ID string `json:"id"`
ParentID string `json:"parentId"` // "" = top level
OrderKey string `json:"orderKey"`
Type string `json:"type"`
Content string `json:"content"`
Props Props `json:"props"`
Version int `json:"version"` // content version; props changes do not bump it
Deleted bool `json:"deleted,omitempty"`
UpdatedBy string `json:"updatedBy,omitempty"`
}
// Doc is the complete state of one note including deleted blocks (tombstones).
type Doc struct {
Title string
TitleVersion int
Blocks map[string]*Block
}
func NewDoc() *Doc {
return &Doc{Blocks: make(map[string]*Block)}
}
func (d *Doc) Clone() *Doc {
c := &Doc{Title: d.Title, TitleVersion: d.TitleVersion, Blocks: make(map[string]*Block, len(d.Blocks))}
for id, b := range d.Blocks {
cp := *b
c.Blocks[id] = &cp
}
return c
}
func siblingLess(a, b *Block) bool {
if a.OrderKey != b.OrderKey {
return a.OrderKey < b.OrderKey
}
return a.ID < b.ID
}
// Children returns the live children of parentID in display order.
func (d *Doc) Children(parentID string) []*Block {
out := make([]*Block, 0)
for _, b := range d.Blocks {
if !b.Deleted && b.ParentID == parentID {
out = append(out, b)
}
}
sort.Slice(out, func(i, j int) bool { return siblingLess(out[i], out[j]) })
return out
}
// FlatBlock is a live block with its depth, in document (depth-first) order.
type FlatBlock struct {
*Block
Level int
}
// Flatten returns all live blocks in display order with their nesting level.
func (d *Doc) Flatten() []FlatBlock {
children := make(map[string][]*Block)
for _, b := range d.Blocks {
if !b.Deleted {
children[b.ParentID] = append(children[b.ParentID], b)
}
}
for _, list := range children {
sort.Slice(list, func(i, j int) bool { return siblingLess(list[i], list[j]) })
}
out := make([]FlatBlock, 0, len(d.Blocks))
var walk func(parent string, level int)
walk = func(parent string, level int) {
for _, b := range children[parent] {
out = append(out, FlatBlock{Block: b, Level: level})
walk(b.ID, level+1)
}
}
walk("", 0)
return out
}
// subtree returns id and all descendants; with liveOnly, deleted blocks are skipped.
func (d *Doc) subtree(id string, liveOnly bool) []string {
children := make(map[string][]string)
for _, b := range d.Blocks {
if liveOnly && b.Deleted {
continue
}
children[b.ParentID] = append(children[b.ParentID], b.ID)
}
out := []string{id}
for i := 0; i < len(out); i++ {
out = append(out, children[out[i]]...)
}
sort.Strings(out[1:])
return out
}
// liveAncestor walks up from id until it finds a live block; "" means top level.
func (d *Doc) liveAncestor(id string) string {
seen := map[string]bool{}
for id != "" && !seen[id] {
seen[id] = true
b, ok := d.Blocks[id]
if !ok {
return ""
}
if !b.Deleted {
return id
}
id = b.ParentID
}
return ""
}
// isAncestor reports whether ancestorID is ancestorID itself or above id in the tree.
func (d *Doc) isAncestor(ancestorID, id string) bool {
seen := map[string]bool{}
for id != "" && !seen[id] {
if id == ancestorID {
return true
}
seen[id] = true
b, ok := d.Blocks[id]
if !ok {
return false
}
id = b.ParentID
}
return false
}
// keyAfter returns a key directly after block b among its live siblings.
func (d *Doc) keyAfter(b *Block) string {
next := ""
for _, s := range d.Children(b.ParentID) {
if siblingLess(b, s) {
next = s.OrderKey
break
}
}
if next == b.OrderKey {
// Equal keys are only ordered by ID; append after the whole run instead
next = ""
for _, s := range d.Children(b.ParentID) {
if s.OrderKey > b.OrderKey {
next = s.OrderKey
break
}
}
}
k, err := KeyBetween(b.OrderKey, next)
if err != nil {
k, _ = KeyBetween(b.OrderKey, "")
}
return k
}
// Validate checks the tree invariants: parents exist, live blocks have live parents, no cycles, valid keys.
func (d *Doc) Validate() error {
for id, b := range d.Blocks {
if b.ID != id {
return fmt.Errorf("block %s stored under key %s", b.ID, id)
}
if !ValidOrderKey(b.OrderKey) {
return fmt.Errorf("block %s has invalid order key %q", id, b.OrderKey)
}
if b.ParentID == "" {
continue
}
p, ok := d.Blocks[b.ParentID]
if !ok {
return fmt.Errorf("block %s has missing parent %s", id, b.ParentID)
}
if !b.Deleted && p.Deleted {
return fmt.Errorf("live block %s has deleted parent %s", id, p.ID)
}
if d.isAncestor(id, b.ParentID) {
return fmt.Errorf("block %s is part of a cycle", id)
}
}
return nil
}
+197
View File
@@ -0,0 +1,197 @@
package collab
import (
"sort"
"sync"
"time"
)
const (
LeaseTTL = 20 * time.Second
SessionTimeout = 30 * time.Second
)
type lease struct {
clientID string
userID string
name string
expires time.Time
}
type session struct {
userID string
name string
lastSeen time.Time
}
// LeaseInfo describes a block held by another client, for display ("Anna bearbeitet …").
type LeaseInfo struct {
BlockID string `json:"blockId"`
UserID string `json:"userId"`
Name string `json:"name"`
}
// Presence is one client that synced a document recently.
type Presence struct {
UserID string `json:"userId"`
Name string `json:"name"`
}
// LeaseManager keeps block leases and active sessions in memory. Leases are short-lived; after a server
// restart clients simply request them again with their next sync.
type LeaseManager struct {
mu sync.Mutex
now func() time.Time
leases map[string]map[string]lease // docID -> blockID -> lease
sessions map[string]map[string]session // docID -> clientID -> session
}
func NewLeaseManager(now func() time.Time) *LeaseManager {
if now == nil {
now = time.Now
}
return &LeaseManager{now: now, leases: map[string]map[string]lease{}, sessions: map[string]map[string]session{}}
}
func (m *LeaseManager) docLeases(docID string) map[string]lease {
l := m.leases[docID]
if l == nil {
l = map[string]lease{}
m.leases[docID] = l
}
return l
}
func (m *LeaseManager) prune(docID string, now time.Time) {
for id, l := range m.leases[docID] {
if !now.Before(l.expires) {
delete(m.leases[docID], id)
}
}
for cid, s := range m.sessions[docID] {
if now.Sub(s.lastSeen) > SessionTimeout {
delete(m.sessions[docID], cid)
}
}
}
// Hold acquires or renews leases for a client. A block held by another live client is denied.
func (m *LeaseManager) Hold(docID string, a Actor, blockIDs []string) (granted, denied []string) {
m.mu.Lock()
defer m.mu.Unlock()
now := m.now()
m.prune(docID, now)
l := m.docLeases(docID)
for _, id := range blockIDs {
if cur, ok := l[id]; ok && cur.clientID != a.ClientID {
denied = append(denied, id)
continue
}
l[id] = lease{clientID: a.ClientID, userID: a.UserID, name: a.Name, expires: now.Add(LeaseTTL)}
granted = append(granted, id)
}
return granted, denied
}
// Release gives up leases held by this client.
func (m *LeaseManager) Release(docID, clientID string, blockIDs []string) {
m.mu.Lock()
defer m.mu.Unlock()
l := m.leases[docID]
for _, id := range blockIDs {
if cur, ok := l[id]; ok && cur.clientID == clientID {
delete(l, id)
}
}
}
// ReleaseAllExcept drops every lease of the client in this document that is not in keep.
func (m *LeaseManager) ReleaseAllExcept(docID, clientID string, keep []string) {
m.mu.Lock()
defer m.mu.Unlock()
keepSet := make(map[string]bool, len(keep))
for _, id := range keep {
keepSet[id] = true
}
for id, cur := range m.leases[docID] {
if cur.clientID == clientID && !keepSet[id] {
delete(m.leases[docID], id)
}
}
}
// View returns a snapshot of the current leases of a document for operation validation.
func (m *LeaseManager) View(docID string) LeaseView {
m.mu.Lock()
defer m.mu.Unlock()
now := m.now()
m.prune(docID, now)
snap := make(leaseSnapshot, len(m.leases[docID]))
for id, l := range m.leases[docID] {
snap[id] = l.clientID
}
return snap
}
// Others lists blocks held by clients other than clientID.
func (m *LeaseManager) Others(docID, clientID string) []LeaseInfo {
m.mu.Lock()
defer m.mu.Unlock()
m.prune(docID, m.now())
out := make([]LeaseInfo, 0)
for id, l := range m.leases[docID] {
if l.clientID != clientID {
out = append(out, LeaseInfo{BlockID: id, UserID: l.userID, Name: l.name})
}
}
sort.Slice(out, func(i, j int) bool { return out[i].BlockID < out[j].BlockID })
return out
}
// Touch records that a client synced the document; it counts as an open session for SessionTimeout.
func (m *LeaseManager) Touch(docID string, a Actor) {
m.mu.Lock()
defer m.mu.Unlock()
s := m.sessions[docID]
if s == nil {
s = map[string]session{}
m.sessions[docID] = s
}
s[a.ClientID] = session{userID: a.UserID, name: a.Name, lastSeen: m.now()}
}
// Leave removes a client's session and all its leases (tab closed, app sent to background).
func (m *LeaseManager) Leave(docID, clientID string) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.sessions[docID], clientID)
for id, l := range m.leases[docID] {
if l.clientID == clientID {
delete(m.leases[docID], id)
}
}
}
// Sessions returns how many clients have the document open and who they are.
func (m *LeaseManager) Sessions(docID string) (int, []Presence) {
m.mu.Lock()
defer m.mu.Unlock()
m.prune(docID, m.now())
seen := map[string]bool{}
people := make([]Presence, 0)
for _, s := range m.sessions[docID] {
if !seen[s.userID] {
seen[s.userID] = true
people = append(people, Presence{UserID: s.userID, Name: s.name})
}
}
sort.Slice(people, func(i, j int) bool { return people[i].UserID < people[j].UserID })
return len(m.sessions[docID]), people
}
type leaseSnapshot map[string]string
func (s leaseSnapshot) HolderOf(blockID string) (string, bool) {
c, ok := s[blockID]
return c, ok
}
+34
View File
@@ -0,0 +1,34 @@
package collab
import "sync"
// Notifier wakes up long-poll requests waiting for changes of a document.
type Notifier struct {
mu sync.Mutex
wait map[string]chan struct{}
}
func NewNotifier() *Notifier {
return &Notifier{wait: map[string]chan struct{}{}}
}
// Changed returns a channel that is closed on the next Notify for docID.
func (n *Notifier) Changed(docID string) <-chan struct{} {
n.mu.Lock()
defer n.mu.Unlock()
ch, ok := n.wait[docID]
if !ok {
ch = make(chan struct{})
n.wait[docID] = ch
}
return ch
}
func (n *Notifier) Notify(docID string) {
n.mu.Lock()
defer n.mu.Unlock()
if ch, ok := n.wait[docID]; ok {
close(ch)
delete(n.wait, docID)
}
}
+396
View File
@@ -0,0 +1,396 @@
package collab
import (
"crypto/sha256"
"encoding/hex"
"sort"
)
const (
OpInsert = "insert"
OpSetContent = "set_content"
OpSetProps = "set_props"
OpMove = "move"
OpDelete = "delete"
OpRestore = "restore"
OpSetTitle = "set_title"
StatusApplied = "applied"
StatusConflict = "conflict"
StatusRejected = "rejected"
maxContentLength = 100_000
maxTypeLength = 20
)
// PropsPatch changes selected block settings; nil fields stay as they are.
type PropsPatch struct {
Type *string `json:"type,omitempty"`
Checked *bool `json:"checked,omitempty"`
Collapsed *bool `json:"collapsed,omitempty"`
Lang *string `json:"lang,omitempty"`
}
// Op is a change requested by a client. OpID is chosen by the client and makes retries idempotent.
type Op struct {
OpID string `json:"opId"`
Kind string `json:"kind"`
BlockID string `json:"blockId,omitempty"`
BaseVersion int `json:"baseVersion,omitempty"` // set_content, set_title
ParentID *string `json:"parentId,omitempty"` // insert, move ("" = top level)
OrderKey string `json:"orderKey,omitempty"` // insert, move
Type string `json:"type,omitempty"` // insert
Content *string `json:"content,omitempty"` // insert, set_content
Props *Props `json:"props,omitempty"` // insert
Patch *PropsPatch `json:"patch,omitempty"` // set_props
IDs []string `json:"ids,omitempty"` // restore
Title *string `json:"title,omitempty"` // set_title
}
// Effect is a normalized, already validated change. The server log stores effects; replaying them in
// order on an empty Doc reproduces the server state exactly.
type Effect struct {
Kind string `json:"kind"`
BlockID string `json:"blockId,omitempty"`
ParentID string `json:"parentId,omitempty"`
OrderKey string `json:"orderKey,omitempty"`
Type string `json:"type,omitempty"`
Content string `json:"content,omitempty"`
Props *Props `json:"props,omitempty"`
Version int `json:"version,omitempty"`
IDs []string `json:"ids,omitempty"`
Title string `json:"title,omitempty"`
TitleVersion int `json:"titleVersion,omitempty"`
By string `json:"by,omitempty"`
}
type Result struct {
OpID string `json:"opId"`
Status string `json:"status"`
Reason string `json:"reason,omitempty"`
Version int `json:"version,omitempty"`
ConflictBlockID string `json:"conflictBlockId,omitempty"`
Seq int64 `json:"seq,omitempty"`
}
// Actor identifies who sends an operation. ClientID is one browser tab / sync engine instance.
type Actor struct {
UserID string
ClientID string
Name string
}
// LeaseView answers which client currently holds the lease on a block.
type LeaseView interface {
HolderOf(blockID string) (clientID string, held bool)
}
// Env provides outside knowledge needed to validate operations.
type Env struct {
Leases LeaseView // nil = no leases (e.g. server-internal writes)
IDTaken func(id string) bool // true if the block ID exists in another document
}
// ApplyEffect mutates the document with one effect. It never fails; effects are validated by Apply.
func (d *Doc) ApplyEffect(e Effect) {
switch e.Kind {
case OpInsert:
b := &Block{ID: e.BlockID, ParentID: e.ParentID, OrderKey: e.OrderKey, Type: e.Type, Content: e.Content, Version: e.Version, UpdatedBy: e.By}
if e.Props != nil {
b.Props = *e.Props
}
d.Blocks[e.BlockID] = b
case OpSetContent:
if b, ok := d.Blocks[e.BlockID]; ok {
b.Content, b.Version, b.UpdatedBy = e.Content, e.Version, e.By
}
case OpSetProps:
if b, ok := d.Blocks[e.BlockID]; ok {
b.Type = e.Type
if e.Props != nil {
b.Props = *e.Props
}
}
case OpMove:
if b, ok := d.Blocks[e.BlockID]; ok {
b.ParentID, b.OrderKey = e.ParentID, e.OrderKey
}
case OpDelete:
for _, id := range e.IDs {
if b, ok := d.Blocks[id]; ok {
b.Deleted = true
}
}
case OpRestore:
for _, id := range e.IDs {
if b, ok := d.Blocks[id]; ok {
b.Deleted = false
}
}
case OpSetTitle:
d.Title, d.TitleVersion = e.Title, e.TitleVersion
}
}
// ConflictBlockID derives the ID of a conflict copy from the losing operation, so retries never duplicate it.
func ConflictBlockID(opID string) string {
sum := sha256.Sum256([]byte("conflict:" + opID))
return "blk_cf_" + hex.EncodeToString(sum[:8])
}
type applier struct {
d *Doc
a Actor
env Env
effects []Effect
}
func (ap *applier) emit(e Effect) {
e.By = ap.a.UserID
ap.d.ApplyEffect(e)
ap.effects = append(ap.effects, e)
}
func (ap *applier) foreignLease(blockID string) bool {
if ap.env.Leases == nil {
return false
}
holder, held := ap.env.Leases.HolderOf(blockID)
return held && holder != ap.a.ClientID
}
func rejected(op Op, reason string) Result {
return Result{OpID: op.OpID, Status: StatusRejected, Reason: reason}
}
// Apply validates op against d, mutates d and returns the produced effects. Rejected operations
// produce no effects; conflicting text is kept as a separate conflict block.
func Apply(d *Doc, op Op, a Actor, env Env) ([]Effect, Result) {
ap := &applier{d: d, a: a, env: env}
res := ap.apply(op)
res.OpID = op.OpID
return ap.effects, res
}
func (ap *applier) apply(op Op) Result {
d := ap.d
switch op.Kind {
case OpInsert:
if op.BlockID == "" || len(op.BlockID) > 64 {
return rejected(op, "invalid_id")
}
if _, exists := d.Blocks[op.BlockID]; exists || (ap.env.IDTaken != nil && ap.env.IDTaken(op.BlockID)) {
return rejected(op, "exists")
}
parent := ""
if op.ParentID != nil {
parent = *op.ParentID
}
if parent != "" {
if _, ok := d.Blocks[parent]; !ok {
return rejected(op, "parent_missing")
}
parent = d.liveAncestor(parent)
}
if !ValidOrderKey(op.OrderKey) {
return rejected(op, "invalid_order_key")
}
typ := op.Type
if typ == "" {
typ = "bullet"
}
if len(typ) > maxTypeLength {
return rejected(op, "invalid_type")
}
content := ""
if op.Content != nil {
content = *op.Content
}
if len(content) > maxContentLength {
return rejected(op, "too_long")
}
props := Props{}
if op.Props != nil {
props = *op.Props
}
ap.emit(Effect{Kind: OpInsert, BlockID: op.BlockID, ParentID: parent, OrderKey: op.OrderKey, Type: typ, Content: content, Props: &props, Version: 1})
return Result{Status: StatusApplied, Version: 1}
case OpSetContent:
b, ok := d.Blocks[op.BlockID]
if !ok {
return rejected(op, "missing")
}
if op.Content == nil {
return rejected(op, "no_content")
}
if len(*op.Content) > maxContentLength {
return rejected(op, "too_long")
}
if b.Deleted {
// A late edit on a deleted block brings it back instead of losing the text
ap.restore([]string{b.ID})
}
if *op.Content == b.Content {
return Result{Status: StatusApplied, Version: b.Version}
}
if ap.foreignLease(b.ID) || op.BaseVersion != b.Version {
return ap.conflictCopy(op, b, *op.Content)
}
ap.emit(Effect{Kind: OpSetContent, BlockID: b.ID, Content: *op.Content, Version: b.Version + 1})
return Result{Status: StatusApplied, Version: b.Version}
case OpSetProps:
b, ok := d.Blocks[op.BlockID]
if !ok || b.Deleted {
return rejected(op, "missing")
}
if op.Patch == nil {
return rejected(op, "no_patch")
}
typ, props := b.Type, b.Props
if p := op.Patch; p != nil {
if p.Type != nil && *p.Type != "" && len(*p.Type) <= maxTypeLength {
typ = *p.Type
}
if p.Checked != nil {
props.Checked = *p.Checked
}
if p.Collapsed != nil {
props.Collapsed = *p.Collapsed
}
if p.Lang != nil {
props.Lang = *p.Lang
}
}
if typ != b.Type || props != b.Props {
ap.emit(Effect{Kind: OpSetProps, BlockID: b.ID, Type: typ, Props: &props})
}
return Result{Status: StatusApplied}
case OpMove:
b, ok := d.Blocks[op.BlockID]
if !ok || b.Deleted {
return rejected(op, "missing")
}
if ap.foreignLease(b.ID) {
return rejected(op, "locked")
}
parent := ""
if op.ParentID != nil {
parent = *op.ParentID
}
if parent != "" {
p, ok := d.Blocks[parent]
if !ok || p.Deleted {
return rejected(op, "parent_missing")
}
if d.isAncestor(b.ID, parent) {
return rejected(op, "cycle")
}
}
if !ValidOrderKey(op.OrderKey) {
return rejected(op, "invalid_order_key")
}
if parent != b.ParentID || op.OrderKey != b.OrderKey {
ap.emit(Effect{Kind: OpMove, BlockID: b.ID, ParentID: parent, OrderKey: op.OrderKey})
}
return Result{Status: StatusApplied}
case OpDelete:
b, ok := d.Blocks[op.BlockID]
if !ok {
return rejected(op, "missing")
}
if b.Deleted {
return Result{Status: StatusApplied}
}
ids := d.subtree(b.ID, true)
for _, id := range ids {
if ap.foreignLease(id) {
return rejected(op, "locked")
}
}
ap.emit(Effect{Kind: OpDelete, IDs: ids})
return Result{Status: StatusApplied}
case OpRestore:
ids := make([]string, 0, len(op.IDs))
for _, id := range op.IDs {
if b, ok := d.Blocks[id]; ok && b.Deleted {
ids = append(ids, id)
}
}
if len(ids) > 0 {
ap.restore(ids)
}
return Result{Status: StatusApplied}
case OpSetTitle:
if op.Title == nil {
return rejected(op, "no_title")
}
if len(*op.Title) > 1000 {
return rejected(op, "too_long")
}
if *op.Title == d.Title {
return Result{Status: StatusApplied, Version: d.TitleVersion}
}
if op.BaseVersion != d.TitleVersion {
return ap.titleConflict(op, *op.Title)
}
ap.emit(Effect{Kind: OpSetTitle, Title: *op.Title, TitleVersion: d.TitleVersion + 1})
return Result{Status: StatusApplied, Version: d.TitleVersion}
}
return rejected(op, "unknown_kind")
}
// restore brings back deleted blocks; blocks whose parent stays deleted move to the nearest live ancestor.
func (ap *applier) restore(ids []string) {
d := ap.d
set := make(map[string]bool, len(ids))
for _, id := range ids {
set[id] = true
}
sorted := append([]string(nil), ids...)
sort.Strings(sorted)
ap.emit(Effect{Kind: OpRestore, IDs: sorted})
for _, id := range sorted {
b := d.Blocks[id]
if b.ParentID == "" || set[b.ParentID] {
continue
}
if p, ok := d.Blocks[b.ParentID]; ok && !p.Deleted {
continue
}
ap.emit(Effect{Kind: OpMove, BlockID: b.ID, ParentID: d.liveAncestor(b.ParentID), OrderKey: b.OrderKey})
}
}
// conflictCopy keeps the losing text as a new block directly below the contested one.
func (ap *applier) conflictCopy(op Op, b *Block, content string) Result {
id := ConflictBlockID(op.OpID)
if _, exists := ap.d.Blocks[id]; !exists {
props := Props{ConflictOf: b.ID, ConflictAuthor: ap.a.Name}
ap.emit(Effect{Kind: OpInsert, BlockID: id, ParentID: b.ParentID, OrderKey: ap.d.keyAfter(b), Type: b.Type, Content: content, Props: &props, Version: 1})
}
return Result{Status: StatusConflict, Version: b.Version, ConflictBlockID: id}
}
// titleConflict keeps a losing title as a conflict block at the very top of the note.
func (ap *applier) titleConflict(op Op, title string) Result {
id := ConflictBlockID(op.OpID)
if _, exists := ap.d.Blocks[id]; !exists {
first := ""
if roots := ap.d.Children(""); len(roots) > 0 {
first = roots[0].OrderKey
}
key, err := KeyBetween("", first)
if err != nil {
key, _ = KeyBetween("", "")
}
props := Props{ConflictOf: "title", ConflictAuthor: ap.a.Name}
ap.emit(Effect{Kind: OpInsert, BlockID: id, OrderKey: key, Type: "text", Content: title, Props: &props, Version: 1})
}
return Result{Status: StatusConflict, Version: ap.d.TitleVersion, ConflictBlockID: id}
}
+263
View File
@@ -0,0 +1,263 @@
package collab
import (
"testing"
"time"
)
var (
anna = Actor{UserID: "u_anna", ClientID: "c_anna", Name: "Anna"}
ben = Actor{UserID: "u_ben", ClientID: "c_ben", Name: "Ben"}
)
func str(s string) *string { return &s }
type fixedLeases map[string]string
func (f fixedLeases) HolderOf(id string) (string, bool) { c, ok := f[id]; return c, ok }
// docWith builds a doc via real operations so versions and keys are realistic.
func docWith(t *testing.T, blocks ...[3]string) *Doc { // {id, parent, content}
t.Helper()
d := NewDoc()
keys := map[string]string{}
for i, b := range blocks {
k, _ := KeyBetween(keys[b[1]], "")
keys[b[1]] = k
_, res := Apply(d, Op{OpID: "seed" + string(rune('a'+i)), Kind: OpInsert, BlockID: b[0], ParentID: str(b[1]), OrderKey: k, Content: str(b[2])}, anna, Env{})
if res.Status != StatusApplied {
t.Fatalf("seed %v: %+v", b, res)
}
}
return d
}
func mustValid(t *testing.T, d *Doc) {
t.Helper()
if err := d.Validate(); err != nil {
t.Fatal(err)
}
}
func TestSetContentAppliesOnMatchingVersion(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Hallo"})
_, res := Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Hallo Welt")}, anna, Env{})
if res.Status != StatusApplied || d.Blocks["a"].Content != "Hallo Welt" || d.Blocks["a"].Version != 2 {
t.Fatalf("got %+v, block %+v", res, d.Blocks["a"])
}
}
func TestStaleVersionCreatesConflictCopy(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Start"}, [3]string{"b", "", "Danach"})
Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Von Anna")}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Von Ben")}, ben, Env{})
if res.Status != StatusConflict || res.ConflictBlockID == "" {
t.Fatalf("want conflict, got %+v", res)
}
if d.Blocks["a"].Content != "Von Anna" {
t.Fatal("first writer must keep the block")
}
flat := d.Flatten()
if len(flat) != 3 || flat[1].ID != res.ConflictBlockID || flat[1].Content != "Von Ben" || flat[2].ID != "b" {
t.Fatalf("conflict copy must sit directly below the block: %+v", flat)
}
if c := d.Blocks[res.ConflictBlockID].Props; c.ConflictOf != "a" || c.ConflictAuthor != "Ben" {
t.Fatalf("conflict props wrong: %+v", c)
}
mustValid(t, d)
}
func TestForeignLeaseCreatesConflictCopy(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Start"})
_, res := Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Ben tippt")}, ben, Env{Leases: fixedLeases{"a": "c_anna"}})
if res.Status != StatusConflict || d.Blocks["a"].Content != "Start" {
t.Fatalf("lease holder must win: %+v", res)
}
// The lease holder itself may write
_, res = Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Anna tippt")}, anna, Env{Leases: fixedLeases{"a": "c_anna"}})
if res.Status != StatusApplied {
t.Fatalf("lease holder rejected: %+v", res)
}
}
func TestIdenticalContentIsNoConflict(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Gleich"})
Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Neu")}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Neu")}, ben, Env{})
if res.Status != StatusApplied || len(d.Blocks) != 1 {
t.Fatalf("same text must not produce a conflict copy: %+v", res)
}
}
func TestEditOnDeletedBlockRestoresIt(t *testing.T) {
d := docWith(t, [3]string{"p", "", "Eltern"}, [3]string{"c", "p", "Kind"})
Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "p"}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "c", BaseVersion: 1, Content: str("Spät getippt")}, ben, Env{})
if res.Status != StatusApplied {
t.Fatalf("got %+v", res)
}
c := d.Blocks["c"]
if c.Deleted || c.Content != "Spät getippt" || c.ParentID != "" {
t.Fatalf("block must be restored at the nearest live ancestor: %+v", c)
}
mustValid(t, d)
}
func TestDeleteIsBlockedByForeignLeaseInSubtree(t *testing.T) {
d := docWith(t, [3]string{"p", "", "Ast"}, [3]string{"c", "p", "Unterpunkt"})
_, res := Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "p"}, anna, Env{Leases: fixedLeases{"c": "c_ben"}})
if res.Status != StatusRejected || res.Reason != "locked" || d.Blocks["p"].Deleted {
t.Fatalf("got %+v", res)
}
_, res = Apply(d, Op{OpID: "2", Kind: OpDelete, BlockID: "p"}, anna, Env{})
if res.Status != StatusApplied || !d.Blocks["c"].Deleted {
t.Fatal("delete must tombstone the whole subtree")
}
}
func TestMoveRejectsCyclesAndLockedBlocks(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"})
if _, res := Apply(d, Op{OpID: "1", Kind: OpMove, BlockID: "a", ParentID: str("b"), OrderKey: "a0"}, anna, Env{}); res.Reason != "cycle" {
t.Fatalf("want cycle, got %+v", res)
}
if _, res := Apply(d, Op{OpID: "2", Kind: OpMove, BlockID: "b", ParentID: str(""), OrderKey: "a5"}, anna, Env{Leases: fixedLeases{"b": "c_ben"}}); res.Reason != "locked" {
t.Fatalf("want locked, got %+v", res)
}
if _, res := Apply(d, Op{OpID: "3", Kind: OpMove, BlockID: "b", ParentID: str(""), OrderKey: "a5"}, anna, Env{}); res.Status != StatusApplied {
t.Fatalf("got %+v", res)
}
mustValid(t, d)
}
func TestInsertIntoDeletedParentGoesToLiveAncestor(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"})
Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "b"}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpInsert, BlockID: "n", ParentID: str("b"), OrderKey: "a0", Content: str("neu")}, ben, Env{})
if res.Status != StatusApplied || d.Blocks["n"].ParentID != "a" {
t.Fatalf("got %+v parent=%q", res, d.Blocks["n"].ParentID)
}
}
func TestInsertRejectsTakenIDs(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"})
if _, res := Apply(d, Op{OpID: "1", Kind: OpInsert, BlockID: "a", OrderKey: "a5"}, anna, Env{}); res.Reason != "exists" {
t.Fatalf("got %+v", res)
}
env := Env{IDTaken: func(id string) bool { return id == "other_doc_block" }}
if _, res := Apply(d, Op{OpID: "2", Kind: OpInsert, BlockID: "other_doc_block", OrderKey: "a5"}, anna, env); res.Reason != "exists" {
t.Fatalf("block IDs of other documents must be rejected: %+v", res)
}
}
func TestTitleConflictBecomesTopBlock(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Erster"})
Apply(d, Op{OpID: "1", Kind: OpSetTitle, BaseVersion: 0, Title: str("Titel Anna")}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetTitle, BaseVersion: 0, Title: str("Titel Ben")}, ben, Env{})
if res.Status != StatusConflict || d.Title != "Titel Anna" {
t.Fatalf("got %+v title=%q", res, d.Title)
}
if flat := d.Flatten(); flat[0].ID != res.ConflictBlockID || flat[0].Props.ConflictOf != "title" {
t.Fatalf("title conflict must be the first block: %+v", flat[0])
}
}
func TestSetPropsDoesNotBumpContentVersion(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Todo"})
Apply(d, Op{OpID: "1", Kind: OpSetProps, BlockID: "a", Patch: &PropsPatch{Checked: new(bool)}}, anna, Env{})
yes := true
Apply(d, Op{OpID: "2", Kind: OpSetProps, BlockID: "a", Patch: &PropsPatch{Checked: &yes}}, ben, Env{Leases: fixedLeases{"a": "c_anna"}})
if !d.Blocks["a"].Props.Checked || d.Blocks["a"].Version != 1 {
t.Fatalf("props must apply independently of text and leases: %+v", d.Blocks["a"])
}
}
// Replaying the effects of every operation on an empty doc must give exactly the same state.
func TestEffectsReplayToSameState(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"})
replay := d.Clone()
ops := []struct {
op Op
a Actor
}{
{Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("A2")}, anna},
{Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("A3")}, ben},
{Op{OpID: "3", Kind: OpDelete, BlockID: "a"}, anna},
{Op{OpID: "4", Kind: OpSetContent, BlockID: "b", BaseVersion: 1, Content: str("B2")}, ben},
{Op{OpID: "5", Kind: OpSetTitle, Title: str("T")}, anna},
}
for _, o := range ops {
effects, _ := Apply(d, o.op, o.a, Env{})
for _, e := range effects {
replay.ApplyEffect(e)
}
}
if !docsEqual(d, replay) {
t.Fatal("replayed effects differ from server state")
}
}
func docsEqual(a, b *Doc) bool {
if a.Title != b.Title || a.TitleVersion != b.TitleVersion || len(a.Blocks) != len(b.Blocks) {
return false
}
for id, x := range a.Blocks {
y, ok := b.Blocks[id]
if !ok || *x != *y {
return false
}
}
return true
}
func TestLeaseManager(t *testing.T) {
now := time.Unix(1000, 0)
m := NewLeaseManager(func() time.Time { return now })
g, den := m.Hold("d", anna, []string{"a", "b"})
if len(g) != 2 || len(den) != 0 {
t.Fatalf("anna: granted=%v denied=%v", g, den)
}
if g, den = m.Hold("d", ben, []string{"a", "c"}); len(g) != 1 || den[0] != "a" {
t.Fatalf("ben: granted=%v denied=%v", g, den)
}
if others := m.Others("d", "c_ben"); len(others) != 2 || others[0].Name != "Anna" {
t.Fatalf("others: %+v", others)
}
now = now.Add(LeaseTTL + time.Second)
if g, _ = m.Hold("d", ben, []string{"a"}); len(g) != 1 {
t.Fatal("expired lease must be available")
}
m.ReleaseAllExcept("d", "c_ben", []string{"a"})
if _, held := m.View("d").HolderOf("c"); held {
t.Fatal("lease not in keep set must be released")
}
m.Touch("d", anna)
m.Touch("d", ben)
if n, people := m.Sessions("d"); n != 2 || len(people) != 2 {
t.Fatalf("sessions: %d %+v", n, people)
}
m.Leave("d", "c_ben")
if _, held := m.View("d").HolderOf("a"); held {
t.Fatal("leaving must drop the client's leases")
}
}
func TestNotifier(t *testing.T) {
n := NewNotifier()
ch := n.Changed("d")
select {
case <-ch:
t.Fatal("must not fire before Notify")
default:
}
n.Notify("d")
select {
case <-ch:
case <-time.After(time.Second):
t.Fatal("Notify must wake waiters")
}
}
+267
View File
@@ -0,0 +1,267 @@
package collab
import (
"errors"
"strings"
)
// Order keys sort siblings lexicographically (ties broken by block ID) and allow inserting between any two
// keys without renumbering. Format as in the "fractional-indexing" scheme: an integer part whose first
// character encodes its length ('a'..'z' positive, 'A'..'Z' negative) followed by base-62 digits, then an
// optional fraction without trailing '0'. Appending stays short ("a0", "a1", …, "az", "b00", …).
const keyDigits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
const smallestInteger = "A00000000000000000000000000"
var errInvalidOrderKey = errors.New("ungültiger Sortierschlüssel")
func digitIndex(c byte) int {
return strings.IndexByte(keyDigits, c)
}
func integerLength(head byte) int {
switch {
case head >= 'a' && head <= 'z':
return int(head-'a') + 2
case head >= 'A' && head <= 'Z':
return int('Z'-head) + 2
}
return 0
}
func integerPart(key string) (string, bool) {
if key == "" {
return "", false
}
n := integerLength(key[0])
if n == 0 || n > len(key) {
return "", false
}
return key[:n], true
}
// ValidOrderKey reports whether k is a well-formed key.
func ValidOrderKey(k string) bool {
if k == "" || len(k) > 256 || k == smallestInteger {
return false
}
ip, ok := integerPart(k)
if !ok {
return false
}
for i := 1; i < len(k); i++ {
if digitIndex(k[i]) < 0 {
return false
}
}
frac := k[len(ip):]
return frac == "" || frac[len(frac)-1] != '0'
}
func incrementInteger(x string) (string, bool) {
head, digs := x[0], []byte(x[1:])
carry := true
for i := len(digs) - 1; carry && i >= 0; i-- {
d := digitIndex(digs[i]) + 1
if d == len(keyDigits) {
digs[i] = '0'
} else {
digs[i] = keyDigits[d]
carry = false
}
}
if !carry {
return string(head) + string(digs), true
}
if head == 'Z' {
return "a0", true
}
if head == 'z' {
return "", false
}
h := head + 1
if h > 'a' {
digs = append(digs, '0')
} else {
digs = digs[:len(digs)-1]
}
return string(h) + string(digs), true
}
func decrementInteger(x string) (string, bool) {
head, digs := x[0], []byte(x[1:])
borrow := true
for i := len(digs) - 1; borrow && i >= 0; i-- {
d := digitIndex(digs[i]) - 1
if d == -1 {
digs[i] = 'z'
} else {
digs[i] = keyDigits[d]
borrow = false
}
}
if !borrow {
return string(head) + string(digs), true
}
if head == 'a' {
return "Z" + string(keyDigits[len(keyDigits)-1]), true
}
if head == 'A' {
return "", false
}
h := head - 1
if h < 'Z' {
digs = append(digs, 'z')
} else {
digs = digs[:len(digs)-1]
}
return string(h) + string(digs), true
}
// midpoint returns a fraction between a and b (a may be "" = 0, b may be "" = 1).
func midpoint(a, b string) string {
if b != "" {
n := 0
for n < len(b) {
ca := byte('0')
if n < len(a) {
ca = a[n]
}
if ca != b[n] {
break
}
n++
}
if n > 0 {
rest := ""
if n < len(a) {
rest = a[n:]
}
return b[:n] + midpoint(rest, b[n:])
}
}
da := 0
if a != "" {
da = digitIndex(a[0])
}
db := len(keyDigits)
if b != "" {
db = digitIndex(b[0])
}
if db-da > 1 {
return string(keyDigits[(da+db)/2])
}
if b != "" && len(b) > 1 {
return b[:1]
}
rest := ""
if len(a) > 1 {
rest = a[1:]
}
return string(keyDigits[da]) + midpoint(rest, "")
}
// KeyBetween returns a key strictly between a and b. An empty a means "before everything",
// an empty b means "after everything".
func KeyBetween(a, b string) (string, error) {
if (a != "" && !ValidOrderKey(a)) || (b != "" && !ValidOrderKey(b)) {
return "", errInvalidOrderKey
}
if a != "" && b != "" && a >= b {
return "", errInvalidOrderKey
}
switch {
case a == "" && b == "":
return "a0", nil
case a == "":
ib, _ := integerPart(b)
fb := b[len(ib):]
if ib == smallestInteger {
return ib + midpoint("", fb), nil
}
if ib < b {
return ib, nil
}
res, ok := decrementInteger(ib)
if !ok {
return "", errInvalidOrderKey
}
return res, nil
case b == "":
ia, _ := integerPart(a)
fa := a[len(ia):]
if i, ok := incrementInteger(ia); ok {
return i, nil
}
return ia + midpoint(fa, ""), nil
}
ia, _ := integerPart(a)
fa := a[len(ia):]
ib, _ := integerPart(b)
fb := b[len(ib):]
if ia == ib {
return ia + midpoint(fa, fb), nil
}
i, ok := incrementInteger(ia)
if !ok {
return "", errInvalidOrderKey
}
if i < b {
return i, nil
}
return ia + midpoint(fa, ""), nil
}
// KeysBetween returns n increasing keys between a and b, spread so that bulk inserts stay short.
func KeysBetween(a, b string, n int) ([]string, error) {
if n <= 0 {
return nil, nil
}
if n == 1 {
k, err := KeyBetween(a, b)
if err != nil {
return nil, err
}
return []string{k}, nil
}
if b == "" {
keys := make([]string, 0, n)
prev := a
for i := 0; i < n; i++ {
k, err := KeyBetween(prev, "")
if err != nil {
return nil, err
}
keys = append(keys, k)
prev = k
}
return keys, nil
}
if a == "" {
keys := make([]string, n)
next := b
for i := n - 1; i >= 0; i-- {
k, err := KeyBetween("", next)
if err != nil {
return nil, err
}
keys[i] = k
next = k
}
return keys, nil
}
mid := n / 2
c, err := KeyBetween(a, b)
if err != nil {
return nil, err
}
left, err := KeysBetween(a, c, mid)
if err != nil {
return nil, err
}
right, err := KeysBetween(c, b, n-mid-1)
if err != nil {
return nil, err
}
return append(append(left, c), right...), nil
}
+90
View File
@@ -0,0 +1,90 @@
package collab
import (
"math/rand"
"sort"
"testing"
)
func TestKeyBetweenBasics(t *testing.T) {
cases := [][2]string{{"", ""}, {"", "a0"}, {"a0", ""}, {"a0", "a1"}, {"a0", "a0V"}, {"a1", "a2"}, {"a0V", "a1"}, {"Zz", "a0"}, {"az", ""}, {"", "Zz"}}
for _, c := range cases {
k, err := KeyBetween(c[0], c[1])
if err != nil {
t.Fatalf("KeyBetween(%q,%q): %v", c[0], c[1], err)
}
if !ValidOrderKey(k) {
t.Fatalf("KeyBetween(%q,%q) = %q is not a valid key", c[0], c[1], k)
}
if (c[0] != "" && k <= c[0]) || (c[1] != "" && k >= c[1]) {
t.Fatalf("KeyBetween(%q,%q) = %q is not between", c[0], c[1], k)
}
}
if _, err := KeyBetween("a1", "a0"); err == nil {
t.Fatal("reversed range must fail")
}
if _, err := KeyBetween("a0V0", ""); err == nil {
t.Fatal("trailing zero key must be rejected")
}
}
// Random inserts between neighbours must always produce a strictly increasing sequence.
func TestKeyBetweenRandomInserts(t *testing.T) {
r := rand.New(rand.NewSource(42))
keys := []string{}
for i := 0; i < 3000; i++ {
pos := r.Intn(len(keys) + 1)
a, b := "", ""
if pos > 0 {
a = keys[pos-1]
}
if pos < len(keys) {
b = keys[pos]
}
k, err := KeyBetween(a, b)
if err != nil {
t.Fatalf("step %d: %v", i, err)
}
keys = append(keys[:pos], append([]string{k}, keys[pos:]...)...)
}
if !sort.StringsAreSorted(keys) {
t.Fatal("keys are not sorted")
}
for i := 1; i < len(keys); i++ {
if keys[i] == keys[i-1] {
t.Fatalf("duplicate key %q", keys[i])
}
}
}
// Appending at the end, the most common case, must keep keys short.
func TestKeyBetweenAppendStaysShort(t *testing.T) {
last := ""
for i := 0; i < 1000; i++ {
k, _ := KeyBetween(last, "")
last = k
}
if len(last) > 4 {
t.Fatalf("key after 1000 appends too long: %d chars", len(last))
}
}
func TestKeysBetweenBulk(t *testing.T) {
for _, r := range [][2]string{{"", ""}, {"a0", ""}, {"", "a0"}, {"a0", "a1"}} {
keys, err := KeysBetween(r[0], r[1], 500)
if err != nil {
t.Fatalf("%v: %v", r, err)
}
if len(keys) != 500 || !sort.StringsAreSorted(keys) {
t.Fatalf("%v: wrong or unsorted result", r)
}
for i, k := range keys {
if !ValidOrderKey(k) || (i > 0 && keys[i-1] == k) {
t.Fatalf("%v: bad key %q", r, k)
}
if (r[0] != "" && k <= r[0]) || (r[1] != "" && k >= r[1]) {
t.Fatalf("%v: key %q out of range", r, k)
}
}
}
}
+663
View File
@@ -0,0 +1,663 @@
package storage
import (
"database/sql"
"encoding/json"
"errors"
"fmt"
"log/slog"
"sort"
"strings"
"sync"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain"
)
// Collaborative note editing: every change to a note's blocks or title runs through collab.Apply under a
// per-document lock, is appended to doc_ops (the log) and recorded in doc_op_results (idempotency).
// ErrDocNotFound is returned when a note does not exist or the user has no access to it.
var ErrDocNotFound = errors.New("Notiz nicht gefunden oder kein Zugriff")
// Roles a user can have on a note.
const (
DocRoleOwner = "owner"
DocRoleEditor = "editor"
DocRoleViewer = "viewer"
)
// DocLogEntry is one effect from the log, as sent to clients.
type DocLogEntry struct {
Seq int64 `json:"seq"`
OpID string `json:"opId,omitempty"`
UserID string `json:"userId"`
ClientID string `json:"clientId"`
Effect collab.Effect `json:"effect"`
}
// DocSnapshot is the complete state of a note including tombstones, needed to replay later effects.
type DocSnapshot struct {
DocID string `json:"docId"`
Title string `json:"title"`
TitleVersion int `json:"titleVersion"`
HeadSeq int64 `json:"headSeq"`
Blocks []collab.Block `json:"blocks"`
}
var docLocks sync.Map // docID -> *sync.Mutex
func lockDoc(docID string) func() {
m, _ := docLocks.LoadOrStore(docID, &sync.Mutex{})
mu := m.(*sync.Mutex)
mu.Lock()
return mu.Unlock
}
type docQueryer interface {
QueryRow(query string, args ...interface{}) *sql.Row
Query(query string, args ...interface{}) (*sql.Rows, error)
}
// migrateDocSync adds the columns and tables of the collaboration model and converts old flat block lists.
func (s *Store) migrateDocSync() {
stmts := []string{
"ALTER TABLE blocks ADD COLUMN parent_id TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN order_key TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN version INTEGER NOT NULL DEFAULT 1;",
"ALTER TABLE blocks ADD COLUMN conflict_of TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN conflict_author TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN updated_by TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN deleted INTEGER NOT NULL DEFAULT 0;",
"CREATE INDEX IF NOT EXISTS idx_blocks_doc_parent ON blocks(document_id, parent_id);",
"ALTER TABLE documents ADD COLUMN head_seq INTEGER NOT NULL DEFAULT 0;",
"ALTER TABLE documents ADD COLUMN title_version INTEGER NOT NULL DEFAULT 0;",
`CREATE TABLE IF NOT EXISTS doc_ops (
doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
seq INTEGER NOT NULL,
op_id TEXT NOT NULL DEFAULT '',
user_id TEXT NOT NULL DEFAULT '',
client_id TEXT NOT NULL DEFAULT '',
kind TEXT NOT NULL,
block_id TEXT NOT NULL DEFAULT '',
payload TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (doc_id, seq)
);`,
`CREATE TABLE IF NOT EXISTS doc_op_results (
op_id TEXT PRIMARY KEY,
doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
user_id TEXT NOT NULL DEFAULT '',
result TEXT NOT NULL,
created_at TEXT NOT NULL
);`,
"CREATE INDEX IF NOT EXISTS idx_doc_op_results_doc ON doc_op_results(doc_id);",
}
for _, q := range stmts {
if _, err := s.db.Exec(q); err != nil && !strings.Contains(err.Error(), "duplicate column") {
slog.Error("Migration der Notiz-Synchronisation fehlgeschlagen", "error", err, "statement", q)
}
}
s.backfillBlockTree()
}
// backfillBlockTree converts blocks stored as a flat list (level + position) into parent_id + order_key.
func (s *Store) backfillBlockTree() {
rows, err := s.db.Query(`SELECT DISTINCT document_id FROM blocks WHERE order_key = ''`)
if err != nil {
return
}
var docIDs []string
for rows.Next() {
var id string
if rows.Scan(&id) == nil {
docIDs = append(docIDs, id)
}
}
_ = rows.Close()
for _, docID := range docIDs {
bRows, err := s.db.Query(`SELECT id, level FROM blocks WHERE document_id = ? ORDER BY position ASC, rowid ASC`, docID)
if err != nil {
continue
}
type flat struct {
id string
level int
}
var list []flat
for bRows.Next() {
var f flat
if bRows.Scan(&f.id, &f.level) == nil {
list = append(list, f)
}
}
_ = bRows.Close()
parents, keys := treeFromLevels(len(list), func(i int) int { return list[i].level })
tx, err := s.db.Begin()
if err != nil {
continue
}
for i, f := range list {
parent := ""
if parents[i] >= 0 {
parent = list[parents[i]].id
}
_, _ = tx.Exec(`UPDATE blocks SET parent_id = ?, order_key = ? WHERE id = ?`, parent, keys[i], f.id)
}
if err := tx.Commit(); err != nil {
slog.Error("Umwandlung der Blockstruktur fehlgeschlagen", "doc_id", docID, "error", err)
}
}
}
// treeFromLevels turns a flat list with nesting levels into parent indices (-1 = top level) and
// sequential order keys per parent. Levels deeper than possible are clamped.
func treeFromLevels(n int, level func(int) int) ([]int, []string) {
parents := make([]int, n)
keys := make([]string, n)
lastKey := map[int]string{}
var stack []int // stack[l] = index of the last block at level l
for i := 0; i < n; i++ {
lvl := level(i)
if lvl < 0 {
lvl = 0
}
if lvl > len(stack) {
lvl = len(stack)
}
stack = stack[:lvl]
parent := -1
if lvl > 0 {
parent = stack[lvl-1]
}
stack = append(stack, i)
parents[i] = parent
k, _ := collab.KeyBetween(lastKey[parent], "")
lastKey[parent] = k
keys[i] = k
}
return parents, keys
}
func loadCollabDoc(q docQueryer, docID string) (*collab.Doc, int64, error) {
d := collab.NewDoc()
var head int64
err := q.QueryRow(`SELECT title, title_version, head_seq FROM documents WHERE id = ?`, docID).Scan(&d.Title, &d.TitleVersion, &head)
if err != nil {
return nil, 0, err
}
rows, err := q.Query(
`SELECT id, parent_id, order_key, type, content, checked, collapsed, lang, conflict_of, conflict_author, version, deleted, updated_by
FROM blocks WHERE document_id = ?`, docID)
if err != nil {
return nil, 0, err
}
defer rows.Close()
for rows.Next() {
b := &collab.Block{}
var chk, col, del int
if err := rows.Scan(&b.ID, &b.ParentID, &b.OrderKey, &b.Type, &b.Content, &chk, &col, &b.Props.Lang,
&b.Props.ConflictOf, &b.Props.ConflictAuthor, &b.Version, &del, &b.UpdatedBy); err != nil {
return nil, 0, err
}
b.Props.Checked, b.Props.Collapsed, b.Deleted = chk != 0, col != 0, del != 0
d.Blocks[b.ID] = b
}
return d, head, rows.Err()
}
// flatDocumentBlocks converts the live tree into the flat list (level + position) the existing API returns.
func flatDocumentBlocks(docID string, d *collab.Doc) []domain.DocumentBlock {
flat := d.Flatten()
out := make([]domain.DocumentBlock, len(flat))
for i, f := range flat {
out[i] = domain.DocumentBlock{
ID: f.ID, DocumentID: docID, Type: f.Type, Content: f.Content, Checked: f.Props.Checked,
Lang: f.Props.Lang, Level: f.Level, Position: float64((i + 1) * 1000), Collapsed: f.Props.Collapsed,
}
}
return out
}
func boolInt(b bool) int {
if b {
return 1
}
return 0
}
// ApplyDocOps applies client operations to a note. Operations whose opId was already processed return
// the stored result without being applied again, so clients can safely resend after lost responses.
func (s *Store) ApplyDocOps(docID string, a collab.Actor, ops []collab.Op, leases collab.LeaseView) ([]collab.Result, int64, error) {
return s.applyDocChange(docID, a, leases, func(*collab.Doc) []collab.Op { return ops })
}
// applyDocChange builds operations from the current state (under the document lock) and applies them.
func (s *Store) applyDocChange(docID string, a collab.Actor, leases collab.LeaseView, build func(*collab.Doc) []collab.Op) ([]collab.Result, int64, error) {
unlock := lockDoc(docID)
defer unlock()
tx, err := s.db.Begin()
if err != nil {
return nil, 0, err
}
defer tx.Rollback()
doc, head, err := loadCollabDoc(tx, docID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, 0, ErrDocNotFound
}
return nil, 0, err
}
before := doc.Clone()
startHead := head
env := collab.Env{
Leases: leases,
IDTaken: func(id string) bool {
var owner string
return tx.QueryRow(`SELECT document_id FROM blocks WHERE id = ?`, id).Scan(&owner) == nil && owner != docID
},
}
now := domain.NowISO()
ops := build(doc)
results := make([]collab.Result, 0, len(ops))
for _, op := range ops {
if op.OpID == "" || len(op.OpID) > 100 {
results = append(results, collab.Result{OpID: op.OpID, Status: collab.StatusRejected, Reason: "invalid_op_id"})
continue
}
var stored, storedDoc string
err := tx.QueryRow(`SELECT result, doc_id FROM doc_op_results WHERE op_id = ?`, op.OpID).Scan(&stored, &storedDoc)
if err == nil {
var res collab.Result
if storedDoc != docID || json.Unmarshal([]byte(stored), &res) != nil {
res = collab.Result{OpID: op.OpID, Status: collab.StatusRejected, Reason: "op_id_reused"}
}
results = append(results, res)
continue
}
if !errors.Is(err, sql.ErrNoRows) {
return nil, 0, err
}
effects, res := collab.Apply(doc, op, a, env)
for _, e := range effects {
head++
payload, _ := json.Marshal(e)
if _, err := tx.Exec(
`INSERT INTO doc_ops (doc_id, seq, op_id, user_id, client_id, kind, block_id, payload, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
docID, head, op.OpID, a.UserID, a.ClientID, e.Kind, e.BlockID, string(payload), now,
); err != nil {
return nil, 0, err
}
}
if len(effects) > 0 {
res.Seq = head
}
resJSON, _ := json.Marshal(res)
if _, err := tx.Exec(`INSERT INTO doc_op_results (op_id, doc_id, user_id, result, created_at) VALUES (?, ?, ?, ?, ?)`,
op.OpID, docID, a.UserID, string(resJSON), now); err != nil {
return nil, 0, err
}
results = append(results, res)
}
if head != startHead {
if err := persistDocState(tx, docID, before, doc, head, now); err != nil {
return nil, 0, err
}
}
if err := tx.Commit(); err != nil {
return nil, 0, err
}
return results, head, nil
}
// persistDocState writes changed blocks and refreshes data derived from the note (markdown, search text, tags).
func persistDocState(tx *sql.Tx, docID string, before, after *collab.Doc, head int64, now string) error {
stmt, err := tx.Prepare(`
INSERT INTO blocks (id, document_id, parent_id, order_key, type, content, checked, collapsed, lang,
conflict_of, conflict_author, version, deleted, updated_by, level, position)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, 0)
ON CONFLICT(id) DO UPDATE SET
parent_id = excluded.parent_id, order_key = excluded.order_key, type = excluded.type,
content = excluded.content, checked = excluded.checked, collapsed = excluded.collapsed,
lang = excluded.lang, conflict_of = excluded.conflict_of, conflict_author = excluded.conflict_author,
version = excluded.version, deleted = excluded.deleted, updated_by = excluded.updated_by
WHERE blocks.document_id = excluded.document_id`)
if err != nil {
return err
}
defer stmt.Close()
tagsChanged := false
for id, b := range after.Blocks {
if old, ok := before.Blocks[id]; ok && *old == *b {
continue
}
tagsChanged = true
if _, err := stmt.Exec(b.ID, docID, b.ParentID, b.OrderKey, b.Type, b.Content, boolInt(b.Props.Checked),
boolInt(b.Props.Collapsed), b.Props.Lang, b.Props.ConflictOf, b.Props.ConflictAuthor, b.Version,
boolInt(b.Deleted), b.UpdatedBy); err != nil {
return err
}
}
flat := flatDocumentBlocks(docID, after)
var plain strings.Builder
for _, b := range flat {
if b.Content != "" {
plain.WriteString(b.Content)
plain.WriteString("\n")
}
}
blocksJSON, _ := json.Marshal(flat)
if _, err := tx.Exec(
`UPDATE documents SET title = ?, title_version = ?, head_seq = ?, markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? WHERE id = ?`,
after.Title, after.TitleVersion, head, domain.CompileBlocksToMarkdown(flat), plain.String(), string(blocksJSON), now, docID,
); err != nil {
return err
}
if tagsChanged {
if _, err := tx.Exec(`DELETE FROM block_tags WHERE document_id = ?`, docID); err != nil {
return err
}
for _, b := range flat {
for _, tag := range domain.ExtractTags(b.Content) {
if _, err := tx.Exec(`INSERT OR IGNORE INTO block_tags (tag, document_id, block_id) VALUES (?, ?, ?)`, tag, docID, b.ID); err != nil {
return err
}
}
}
}
return nil
}
// GetDocOpsSince returns up to limit log entries after seq, plus the current head.
func (s *Store) GetDocOpsSince(docID string, since int64, limit int) ([]DocLogEntry, int64, error) {
var head int64
if err := s.db.QueryRow(`SELECT head_seq FROM documents WHERE id = ?`, docID).Scan(&head); err != nil {
return nil, 0, err
}
rows, err := s.db.Query(
`SELECT seq, op_id, user_id, client_id, payload FROM doc_ops WHERE doc_id = ? AND seq > ? ORDER BY seq ASC LIMIT ?`,
docID, since, limit)
if err != nil {
return nil, 0, err
}
defer rows.Close()
out := make([]DocLogEntry, 0)
for rows.Next() {
var e DocLogEntry
var payload string
if err := rows.Scan(&e.Seq, &e.OpID, &e.UserID, &e.ClientID, &payload); err != nil {
return nil, 0, err
}
if err := json.Unmarshal([]byte(payload), &e.Effect); err != nil {
return nil, 0, fmt.Errorf("log entry %d unreadable: %w", e.Seq, err)
}
out = append(out, e)
}
return out, head, rows.Err()
}
// GetDocSnapshot returns the full block tree (including tombstones) and the log position it corresponds to.
func (s *Store) GetDocSnapshot(docID string) (*DocSnapshot, error) {
unlock := lockDoc(docID)
defer unlock()
d, head, err := loadCollabDoc(s.db, docID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrDocNotFound
}
return nil, err
}
snap := &DocSnapshot{DocID: docID, Title: d.Title, TitleVersion: d.TitleVersion, HeadSeq: head, Blocks: make([]collab.Block, 0, len(d.Blocks))}
for _, b := range d.Blocks {
snap.Blocks = append(snap.Blocks, *b)
}
sort.Slice(snap.Blocks, func(i, j int) bool { return snap.Blocks[i].ID < snap.Blocks[j].ID })
return snap, nil
}
// DocRole returns the user's role on a note ("owner", "editor", "viewer") or "" without access.
// Sources: ownership, direct shares, team shares and shares of the shared folder the note lives in.
func (s *Store) DocRole(userID, docID string) string {
if userID == "" {
return ""
}
var ownerID, sharedFolderID, folderOwner string
err := s.db.QueryRow(
`SELECT d.user_id, d.shared_folder_id, COALESCE(wsf.owner_id, '')
FROM documents d LEFT JOIN workspace_shared_folders wsf ON wsf.id = d.shared_folder_id
WHERE d.id = ?`, docID).Scan(&ownerID, &sharedFolderID, &folderOwner)
if err != nil {
return ""
}
if ownerID == userID {
return DocRoleOwner
}
if folderOwner == userID {
return DocRoleEditor
}
rows, err := s.db.Query(
`SELECT permission FROM doc_shares WHERE doc_id = ? AND user_id = ?
UNION ALL SELECT dts.permission FROM doc_team_shares dts JOIN team_members tm ON tm.team_id = dts.team_id
WHERE dts.doc_id = ? AND tm.user_id = ?
UNION ALL SELECT wfs.permission FROM workspace_folder_shares wfs
WHERE wfs.folder_id = ? AND wfs.folder_id != '' AND (
(wfs.target_type = 'user' AND wfs.target_id = ?)
OR (wfs.target_type = 'team' AND wfs.target_id IN (SELECT team_id FROM team_members WHERE user_id = ?)))`,
docID, userID, docID, userID, sharedFolderID, userID, userID)
if err != nil {
return ""
}
defer rows.Close()
role := ""
for rows.Next() {
var p string
if rows.Scan(&p) != nil {
continue
}
if p == "write" {
return DocRoleEditor
}
role = DocRoleViewer
}
return role
}
// CanEditDoc reports whether the role allows changing a note's content.
func CanEditDoc(role string) bool {
return role == DocRoleOwner || role == DocRoleEditor
}
// IsDocShared reports whether anyone besides the owner can access the note.
func (s *Store) IsDocShared(docID string) bool {
var n int
_ = s.db.QueryRow(
`SELECT (SELECT COUNT(*) FROM doc_shares WHERE doc_id = ?)
+ (SELECT COUNT(*) FROM doc_team_shares WHERE doc_id = ?)
+ (SELECT COUNT(*) FROM documents d JOIN workspace_folder_shares wfs ON wfs.folder_id = d.shared_folder_id
WHERE d.id = ? AND d.shared_folder_id != '')`,
docID, docID, docID).Scan(&n)
return n > 0
}
// --- Adapter for the existing whole-document API (PUT /documents/{id}) ---
const legacyClientID = "legacy"
// replaceDocBlocksInDB stores a complete flat block list by diffing it against the current tree and
// applying the difference as regular operations, so it shows up in the log like any other change.
func (s *Store) replaceDocBlocksInDB(docID, userID string, blocks []domain.DocumentBlock, _ string) {
a := collab.Actor{UserID: userID, ClientID: legacyClientID}
if _, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op { return legacyOps(d, blocks) }); err != nil {
slog.Error("Speichern der Notiz fehlgeschlagen", "doc_id", docID, "error", err)
}
}
// applyLegacyTitle records a title change from the whole-document API in the log.
func (s *Store) applyLegacyTitle(docID, userID, title string) {
a := collab.Actor{UserID: userID, ClientID: legacyClientID}
if _, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op {
if d.Title == title {
return nil
}
return []collab.Op{{OpID: domain.NewID("op"), Kind: collab.OpSetTitle, BaseVersion: d.TitleVersion, Title: &title}}
}); err != nil {
slog.Error("Speichern des Notiztitels fehlgeschlagen", "doc_id", docID, "error", err)
}
}
// legacyOps computes the operations that turn the current tree into the given flat list.
// Last writer wins, exactly like the old whole-document save.
func legacyOps(d *collab.Doc, blocks []domain.DocumentBlock) []collab.Op {
// Normalize IDs (missing or duplicated IDs get new ones)
seen := make(map[string]bool, len(blocks))
list := make([]domain.DocumentBlock, len(blocks))
for i, b := range blocks {
id := strings.TrimSpace(b.ID)
if id == "" || seen[id] {
id = domain.NewID("blk")
}
seen[id] = true
b.ID = id
if b.Type == "" {
b.Type = "bullet"
}
list[i] = b
}
parentIdx, _ := treeFromLevels(len(list), func(i int) int { return list[i].Level })
parentOf := make([]string, len(list))
children := map[string][]int{}
for i := range list {
if parentIdx[i] >= 0 {
parentOf[i] = list[parentIdx[i]].ID
}
children[parentOf[i]] = append(children[parentOf[i]], i)
}
keys := make([]string, len(list))
for parent, idxs := range children {
assignKeys(d, parent, idxs, list, keys)
}
var ops []collab.Op
newOp := func(op collab.Op) {
op.OpID = domain.NewID("op")
ops = append(ops, op)
}
wanted := make(map[string]bool, len(list))
for i, b := range list {
wanted[b.ID] = true
parent := parentOf[i]
patch := &collab.PropsPatch{Type: &list[i].Type, Checked: &list[i].Checked, Collapsed: &list[i].Collapsed, Lang: &list[i].Lang}
cur, exists := d.Blocks[b.ID]
if !exists {
content := b.Content
props := collab.Props{Checked: b.Checked, Collapsed: b.Collapsed, Lang: b.Lang}
newOp(collab.Op{Kind: collab.OpInsert, BlockID: b.ID, ParentID: &parent, OrderKey: keys[i], Type: b.Type, Content: &content, Props: &props})
continue
}
if cur.Deleted {
newOp(collab.Op{Kind: collab.OpRestore, IDs: []string{b.ID}})
}
if cur.ParentID != parent || cur.OrderKey != keys[i] || cur.Deleted {
newOp(collab.Op{Kind: collab.OpMove, BlockID: b.ID, ParentID: &parent, OrderKey: keys[i]})
}
if cur.Content != b.Content {
content := b.Content
newOp(collab.Op{Kind: collab.OpSetContent, BlockID: b.ID, BaseVersion: cur.Version, Content: &content})
}
if cur.Type != b.Type || cur.Props.Checked != b.Checked || cur.Props.Collapsed != b.Collapsed || cur.Props.Lang != b.Lang {
newOp(collab.Op{Kind: collab.OpSetProps, BlockID: b.ID, Patch: patch})
}
}
// Deletes last, after moves have rescued wanted children out of deleted branches
for _, fb := range d.Flatten() {
if !wanted[fb.ID] {
newOp(collab.Op{Kind: collab.OpDelete, BlockID: fb.ID})
}
}
return ops
}
// assignKeys picks order keys for the children idxs of parent. Existing keys that are already in the right
// relative order (longest increasing subsequence) are kept, so unchanged blocks do not show up as moves.
func assignKeys(d *collab.Doc, parent string, idxs []int, list []domain.DocumentBlock, keys []string) {
n := len(idxs)
existing := make([]string, n)
for j, i := range idxs {
if b, ok := d.Blocks[list[i].ID]; ok && !b.Deleted && b.ParentID == parent {
existing[j] = b.OrderKey
}
}
keep := longestIncreasing(existing)
for j := 0; j < n; {
if keep[j] {
keys[idxs[j]] = existing[j]
j++
continue
}
start := j
for j < n && !keep[j] {
j++
}
left, right := "", ""
if start > 0 {
left = keys[idxs[start-1]]
}
if j < n {
right = existing[j]
}
fresh, err := collab.KeysBetween(left, right, j-start)
if err != nil {
fresh, _ = collab.KeysBetween(left, "", j-start)
}
for k := start; k < j; k++ {
keys[idxs[k]] = fresh[k-start]
}
}
}
// longestIncreasing marks a longest strictly increasing subsequence of the non-empty keys.
func longestIncreasing(keys []string) []bool {
n := len(keys)
keep := make([]bool, n)
tails := []int{} // indices of the smallest tail for each length
prev := make([]int, n) // predecessor in the subsequence
for i, k := range keys {
prev[i] = -1
if k == "" {
continue
}
lo, hi := 0, len(tails)
for lo < hi {
mid := (lo + hi) / 2
if keys[tails[mid]] < k {
lo = mid + 1
} else {
hi = mid
}
}
if lo > 0 {
prev[i] = tails[lo-1]
}
if lo == len(tails) {
tails = append(tails, i)
} else {
tails[lo] = i
}
}
if len(tails) > 0 {
for i := tails[len(tails)-1]; i >= 0; i = prev[i] {
keep[i] = true
}
}
return keep
}
+402
View File
@@ -0,0 +1,402 @@
package storage
import (
"fmt"
"math/rand"
"sort"
"testing"
"time"
"github.com/sandrodach/mindsift/internal/collab"
)
// The simulation drives several clients against the real store over an unreliable network and checks
// that everyone converges to the server state without losing text or applying anything twice.
type simClient struct {
actor collab.Actor
confirmed *collab.Doc
lastSeq int64
pending []collab.Op // not sent yet; base versions are filled in when a batch is formed
inflight []collab.Op // sent, response not received; resent unchanged (same op IDs)
holds map[string]bool
offline bool
counter int
}
type simWorld struct {
t *testing.T
r *rand.Rand
store *Store
docID string
leases *collab.LeaseManager
now time.Time
clients []*simClient
sent map[string]collab.Op // every op ever sent, by ID
results map[string]collab.Result // first result received per op ID
}
func (w *simWorld) newOpID(c *simClient) string {
c.counter++
return fmt.Sprintf("%s-%d", c.actor.ClientID, c.counter)
}
// localView is what the client's editor shows: confirmed state plus its own unconfirmed operations.
func (c *simClient) localView() *collab.Doc {
v := c.confirmed.Clone()
for _, op := range append(append([]collab.Op{}, c.inflight...), c.pending...) {
if op.Kind == collab.OpSetContent {
if b, ok := v.Blocks[op.BlockID]; ok {
op.BaseVersion = b.Version
}
}
if op.Kind == collab.OpSetTitle {
op.BaseVersion = v.TitleVersion
}
collab.Apply(v, op, c.actor, collab.Env{})
}
return v
}
func pick[T any](r *rand.Rand, list []T) (T, bool) {
var zero T
if len(list) == 0 {
return zero, false
}
return list[r.Intn(len(list))], true
}
// edit lets a client perform one random editor action on its local view.
func (w *simWorld) edit(c *simClient) {
v := c.localView()
flat := v.Flatten()
var target *collab.Block
if fb, ok := pick(w.r, flat); ok {
target = fb.Block
}
switch n := w.r.Intn(100); {
case n < 45 && target != nil: // typing
content := target.Content + fmt.Sprintf(" %s%d", c.actor.ClientID, c.counter)
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetContent, BlockID: target.ID, Content: &content})
c.holds[target.ID] = true
case n < 65: // Enter: new block after target (or first block)
parent, after := "", ""
if target != nil {
parent, after = target.ParentID, target.OrderKey
}
next := ""
for _, s := range v.Children(parent) {
if s.OrderKey > after {
next = s.OrderKey
break
}
}
key, err := collab.KeyBetween(after, next)
if err != nil {
return
}
id := fmt.Sprintf("blk_%s_%d", c.actor.ClientID, c.counter)
content := "neu von " + c.actor.ClientID
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpInsert, BlockID: id, ParentID: &parent, OrderKey: key, Content: &content})
c.holds[id] = true
case n < 73 && target != nil: // indent under previous sibling
siblings := v.Children(target.ParentID)
for i, s := range siblings {
if s.ID == target.ID && i > 0 {
prev := siblings[i-1]
last := ""
if kids := v.Children(prev.ID); len(kids) > 0 {
last = kids[len(kids)-1].OrderKey
}
key, _ := collab.KeyBetween(last, "")
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpMove, BlockID: target.ID, ParentID: &prev.ID, OrderKey: key})
}
}
case n < 80 && target != nil: // outdent
if p, ok := v.Blocks[target.ParentID]; ok {
key, err := collab.KeyBetween(p.OrderKey, "")
if err == nil {
for _, s := range v.Children(p.ParentID) {
if s.OrderKey > p.OrderKey {
key, _ = collab.KeyBetween(p.OrderKey, s.OrderKey)
break
}
}
grand := p.ParentID
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpMove, BlockID: target.ID, ParentID: &grand, OrderKey: key})
}
}
case n < 87 && target != nil: // delete branch
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpDelete, BlockID: target.ID})
case n < 93 && target != nil: // tick checkbox
checked := !target.Props.Checked
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetProps, BlockID: target.ID, Patch: &collab.PropsPatch{Checked: &checked}})
case n < 96: // rename note
title := fmt.Sprintf("Titel %s%d", c.actor.ClientID, c.counter)
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetTitle, Title: &title})
default: // stop editing some block
for id := range c.holds {
delete(c.holds, id)
break
}
}
}
// formBatch takes pending operations, keeps only the last text change per block and fills in base versions.
func (c *simClient) formBatch() []collab.Op {
lastText := map[string]int{}
lastTitle := -1
for i, op := range c.pending {
switch op.Kind {
case collab.OpSetContent:
lastText[op.BlockID] = i
case collab.OpSetTitle:
lastTitle = i
}
}
var batch []collab.Op
for i, op := range c.pending {
switch op.Kind {
case collab.OpSetContent:
if lastText[op.BlockID] != i {
continue
}
op.BaseVersion = 1
if b, ok := c.confirmed.Blocks[op.BlockID]; ok {
op.BaseVersion = b.Version
}
case collab.OpSetTitle:
if i != lastTitle {
continue
}
op.BaseVersion = c.confirmed.TitleVersion
}
batch = append(batch, op)
}
c.pending = nil
return batch
}
// serverSync is what POST /sync does, minus HTTP.
func (w *simWorld) serverSync(c *simClient, batch []collab.Op, holds []string) []collab.Result {
w.leases.Touch(w.docID, c.actor)
w.leases.ReleaseAllExcept(w.docID, c.actor.ClientID, holds)
w.leases.Hold(w.docID, c.actor, holds)
results, _, err := w.store.ApplyDocOps(w.docID, c.actor, batch, w.leases.View(w.docID))
if err != nil {
w.t.Fatalf("ApplyDocOps: %v", err)
}
var own []string
for i, res := range results {
if res.Status == collab.StatusApplied && batch[i].Kind == collab.OpInsert {
own = append(own, batch[i].BlockID)
}
if res.ConflictBlockID != "" {
own = append(own, res.ConflictBlockID)
}
}
w.leases.Hold(w.docID, c.actor, own)
return results
}
// pull fetches the log after lastSeq in small pages (to exercise paging) and replays it.
func (w *simWorld) pull(c *simClient) {
for {
entries, head, err := w.store.GetDocOpsSince(w.docID, c.lastSeq, 7)
if err != nil {
w.t.Fatalf("GetDocOpsSince: %v", err)
}
for _, e := range entries {
if e.Seq != c.lastSeq+1 {
w.t.Fatalf("%s: log gap, expected seq %d got %d", c.actor.ClientID, c.lastSeq+1, e.Seq)
}
c.confirmed.ApplyEffect(e.Effect)
c.lastSeq = e.Seq
}
if c.lastSeq >= head {
return
}
}
}
// sync runs one round trip over the unreliable (or, with reliable=true, a perfect) network.
func (w *simWorld) sync(c *simClient, reliable bool) {
if c.inflight == nil {
c.inflight = c.formBatch()
for _, op := range c.inflight {
w.sent[op.OpID] = op
}
}
holds := make([]string, 0, len(c.holds))
for id := range c.holds {
holds = append(holds, id)
}
sort.Strings(holds)
fate := 100
if !reliable {
fate = w.r.Intn(100)
}
switch {
case fate < 10: // request lost
return
case fate < 20: // server applied it, response lost
w.serverSync(c, c.inflight, holds)
return
case fate < 25: // request duplicated by a retrying proxy
w.serverSync(c, c.inflight, holds)
}
results := w.serverSync(c, c.inflight, holds)
for _, res := range results {
if prev, ok := w.results[res.OpID]; ok && prev != res {
w.t.Fatalf("op %s answered differently on retry: %+v vs %+v", res.OpID, prev, res)
}
w.results[res.OpID] = res
}
c.inflight = nil
w.pull(c)
}
func runSimulation(t *testing.T, seed int64, steps int) {
store, u := newSyncTestStore(t)
doc, err := store.CreateDocument(u.ID, "", "Sim", "📄", "", flatBlocks([3]interface{}{"root", 0, "Start"}), nil, false)
if err != nil {
t.Fatal(err)
}
w := &simWorld{t: t, r: rand.New(rand.NewSource(seed)), store: store, docID: doc.ID,
now: time.Unix(1_700_000_000, 0), sent: map[string]collab.Op{}, results: map[string]collab.Result{}}
w.leases = collab.NewLeaseManager(func() time.Time { return w.now })
for i := 0; i < 4; i++ {
snap, err := store.GetDocSnapshot(doc.ID)
if err != nil {
t.Fatal(err)
}
c := &simClient{actor: collab.Actor{UserID: fmt.Sprintf("u%d", i%2), ClientID: fmt.Sprintf("c%d", i), Name: fmt.Sprintf("Client %d", i)},
confirmed: collab.NewDoc(), lastSeq: snap.HeadSeq, holds: map[string]bool{}}
c.confirmed.Title, c.confirmed.TitleVersion = snap.Title, snap.TitleVersion
for _, b := range snap.Blocks {
b := b
c.confirmed.Blocks[b.ID] = &b
}
w.clients = append(w.clients, c)
}
for step := 0; step < steps; step++ {
c := w.clients[w.r.Intn(len(w.clients))]
switch n := w.r.Intn(100); {
case n < 55:
w.edit(c)
case n < 90:
if !c.offline {
w.sync(c, false)
}
case n < 95:
c.offline = !c.offline
default:
w.now = w.now.Add(time.Duration(w.r.Intn(25)) * time.Second) // leases may expire
}
}
// Everyone comes back online and syncs until nothing is left
for round := 0; round < 10; round++ {
for _, c := range w.clients {
c.offline = false
w.sync(c, true)
}
}
for _, c := range w.clients {
if len(c.pending) > 0 || len(c.inflight) > 0 {
t.Fatalf("%s still has unsent operations", c.actor.ClientID)
}
w.pull(c)
}
server, head, err := loadCollabDoc(store.db, doc.ID)
if err != nil {
t.Fatal(err)
}
if err := server.Validate(); err != nil {
t.Fatalf("seed %d: invalid tree: %v", seed, err)
}
for _, c := range w.clients {
if c.lastSeq != head || !docsEqual(c.confirmed, server) {
t.Fatalf("seed %d: %s diverged from the server (seq %d vs %d)", seed, c.actor.ClientID, c.lastSeq, head)
}
}
// Every operation got exactly one stored result, and no text was lost
var stored int
if err := store.db.QueryRow(`SELECT COUNT(*) FROM doc_op_results WHERE doc_id = ? AND user_id IN ('u0', 'u1')`, doc.ID).Scan(&stored); err != nil {
t.Fatal(err)
}
if stored != len(w.sent) {
t.Fatalf("seed %d: %d operations sent but %d results stored", seed, len(w.sent), stored)
}
log, _, _ := store.GetDocOpsSince(doc.ID, 0, 1_000_000)
bySeq := map[int64]collab.Effect{}
inserted := map[string]string{}
for _, e := range log {
bySeq[e.Seq] = e.Effect
if e.Effect.Kind == collab.OpInsert {
inserted[e.Effect.BlockID] = e.Effect.Content
}
}
conflicts := 0
for id, op := range w.sent {
res, ok := w.results[id]
if !ok {
continue // result only seen by the server (response lost, never retried after success) is covered by the count above
}
switch op.Kind {
case collab.OpInsert, collab.OpSetContent, collab.OpSetTitle:
if res.Status == collab.StatusRejected {
t.Fatalf("seed %d: text-carrying op %s was rejected: %+v", seed, id, res)
}
}
if op.Kind == collab.OpSetContent && res.Status == collab.StatusApplied && res.Seq > 0 {
if e := bySeq[res.Seq]; e.Kind != collab.OpSetContent || e.Content != *op.Content {
t.Fatalf("seed %d: applied text of %s not in the log", seed, id)
}
}
if res.Status == collab.StatusConflict {
conflicts++
text := ""
if op.Content != nil {
text = *op.Content
} else if op.Title != nil {
text = *op.Title
}
if got, ok := inserted[res.ConflictBlockID]; !ok || got != text {
t.Fatalf("seed %d: conflict copy of %s missing or wrong", seed, id)
}
}
}
t.Logf("seed %d: %d ops, %d log entries, %d conflict copies, %d blocks", seed, len(w.sent), len(log), conflicts, len(server.Blocks))
}
func docsEqual(a, b *collab.Doc) bool {
if a.Title != b.Title || a.TitleVersion != b.TitleVersion || len(a.Blocks) != len(b.Blocks) {
return false
}
for id, x := range a.Blocks {
y, ok := b.Blocks[id]
if !ok || *x != *y {
return false
}
}
return true
}
func TestSyncSimulationUnreliableNetwork(t *testing.T) {
seeds := 25
if testing.Short() {
seeds = 3
}
for seed := int64(1); seed <= int64(seeds); seed++ {
seed := seed
t.Run(fmt.Sprintf("seed-%d", seed), func(t *testing.T) {
runSimulation(t, seed, 600)
})
}
}
+186
View File
@@ -0,0 +1,186 @@
package storage
import (
"path/filepath"
"testing"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain"
)
func newSyncTestStore(t *testing.T) (*Store, *domain.User) {
t.Helper()
store, err := NewStore(filepath.Join(t.TempDir(), "sync.db"))
if err != nil {
t.Fatalf("store: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
u, err := store.CreateUser("owner@example.com", "hash", "Owner", "member")
if err != nil {
t.Fatalf("user: %v", err)
}
return store, u
}
func flatBlocks(spec ...[3]interface{}) []domain.DocumentBlock { // {id, level, content}
out := make([]domain.DocumentBlock, len(spec))
for i, s := range spec {
out[i] = domain.DocumentBlock{ID: s[0].(string), Level: s[1].(int), Content: s[2].(string), Type: "bullet"}
}
return out
}
func outline(blocks []domain.DocumentBlock) [][2]interface{} {
out := make([][2]interface{}, len(blocks))
for i, b := range blocks {
out[i] = [2]interface{}{b.ID, b.Level}
}
return out
}
func sameOutline(t *testing.T, got []domain.DocumentBlock, want []domain.DocumentBlock) {
t.Helper()
if len(got) != len(want) {
t.Fatalf("got %v, want %v", outline(got), outline(want))
}
for i := range got {
if got[i].ID != want[i].ID || got[i].Level != want[i].Level || got[i].Content != want[i].Content {
t.Fatalf("block %d: got %v, want %v", i, outline(got), outline(want))
}
}
}
func headSeq(t *testing.T, s *Store, docID string) int64 {
t.Helper()
_, head, err := s.GetDocOpsSince(docID, 0, 1)
if err != nil {
t.Fatal(err)
}
return head
}
func TestLegacySaveRoundTripsThroughTheLog(t *testing.T) {
s, u := newSyncTestStore(t)
initial := flatBlocks(
[3]interface{}{"a", 0, "Projekt"},
[3]interface{}{"b", 1, "Aufgabe 1"},
[3]interface{}{"c", 2, "Detail"},
[3]interface{}{"d", 1, "Aufgabe 2"},
[3]interface{}{"e", 0, "Anderes"},
)
doc, err := s.CreateDocument(u.ID, "", "Notiz", "📄", "", initial, nil, false)
if err != nil {
t.Fatal(err)
}
got, _ := s.GetDocumentByID(u.ID, doc.ID)
sameOutline(t, got.Blocks, initial)
// Saving the same content again must not produce any log entries
head := headSeq(t, s, doc.ID)
if _, err := s.UpdateDocument(u.ID, doc.ID, nil, nil, nil, &initial, nil, nil); err != nil {
t.Fatal(err)
}
if h := headSeq(t, s, doc.ID); h != head {
t.Fatalf("unchanged save wrote %d log entries", h-head)
}
// Reorder, outdent, edit and delete in one save
changed := flatBlocks(
[3]interface{}{"e", 0, "Anderes"},
[3]interface{}{"a", 0, "Projekt (neu)"},
[3]interface{}{"d", 1, "Aufgabe 2"},
[3]interface{}{"c", 1, "Detail"},
[3]interface{}{"n", 2, "Neu"},
)
if _, err := s.UpdateDocument(u.ID, doc.ID, nil, nil, nil, &changed, nil, nil); err != nil {
t.Fatal(err)
}
got, _ = s.GetDocumentByID(u.ID, doc.ID)
sameOutline(t, got.Blocks, changed)
// The log alone must rebuild exactly this state
entries, _, err := s.GetDocOpsSince(doc.ID, 0, 10_000)
if err != nil {
t.Fatal(err)
}
replay := collab.NewDoc()
for _, e := range entries {
replay.ApplyEffect(e.Effect)
}
sameOutline(t, flatDocumentBlocks(doc.ID, replay), changed)
}
func TestApplyDocOpsIsIdempotent(t *testing.T) {
s, u := newSyncTestStore(t)
doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", flatBlocks([3]interface{}{"a", 0, "Start"}), nil, false)
a := collab.Actor{UserID: u.ID, ClientID: "tab1", Name: "Owner"}
text := "Geändert"
op := collab.Op{OpID: "op-1", Kind: collab.OpSetContent, BlockID: "a", BaseVersion: 1, Content: &text}
first, head1, err := s.ApplyDocOps(doc.ID, a, []collab.Op{op}, nil)
if err != nil || first[0].Status != collab.StatusApplied {
t.Fatalf("first: %+v %v", first, err)
}
// Resend after a lost response: same result, nothing applied twice
again, head2, err := s.ApplyDocOps(doc.ID, a, []collab.Op{op}, nil)
if err != nil || again[0] != first[0] || head2 != head1 {
t.Fatalf("retry changed something: %+v head %d -> %d", again, head1, head2)
}
}
func TestInsertCannotReuseBlockIDOfAnotherNote(t *testing.T) {
s, u := newSyncTestStore(t)
other, _ := s.CreateDocument(u.ID, "", "Andere", "📄", "", flatBlocks([3]interface{}{"shared_id", 0, "Fremd"}), nil, false)
doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", nil, nil, false)
a := collab.Actor{UserID: u.ID, ClientID: "tab1"}
content, root := "Übernahme", ""
res, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "x", Kind: collab.OpInsert, BlockID: "shared_id", ParentID: &root, OrderKey: "a0", Content: &content}}, nil)
if err != nil || res[0].Reason != "exists" {
t.Fatalf("got %+v %v", res, err)
}
got, _ := s.GetDocumentByID(u.ID, other.ID)
if got.Blocks[0].Content != "Fremd" {
t.Fatal("block of another note was modified")
}
}
func TestDocRole(t *testing.T) {
s, owner := newSyncTestStore(t)
reader, _ := s.CreateUser("reader@example.com", "hash", "Reader", "member")
writer, _ := s.CreateUser("writer@example.com", "hash", "Writer", "member")
stranger, _ := s.CreateUser("stranger@example.com", "hash", "Stranger", "member")
doc, _ := s.CreateDocument(owner.ID, "", "Notiz", "📄", "", nil, nil, false)
now := domain.NowISO()
if _, err := s.db.Exec(`INSERT INTO doc_shares (id, doc_id, user_id, permission, created_at, created_by) VALUES ('s1', ?, ?, 'read', ?, ?), ('s2', ?, ?, 'write', ?, ?)`,
doc.ID, reader.ID, now, owner.ID, doc.ID, writer.ID, now, owner.ID); err != nil {
t.Fatal(err)
}
for user, want := range map[string]string{owner.ID: DocRoleOwner, writer.ID: DocRoleEditor, reader.ID: DocRoleViewer, stranger.ID: ""} {
if got := s.DocRole(user, doc.ID); got != want {
t.Errorf("role of %s: got %q, want %q", user, got, want)
}
}
if !s.IsDocShared(doc.ID) {
t.Error("note with shares must count as shared")
}
}
func TestBackfillConvertsFlatBlocksToTree(t *testing.T) {
s, u := newSyncTestStore(t)
doc, _ := s.CreateDocument(u.ID, "", "Alt", "📄", "", nil, nil, false)
if _, err := s.db.Exec(`DELETE FROM blocks WHERE document_id = ?`, doc.ID); err != nil {
t.Fatal(err)
}
// Rows as written by the old schema: only level and position
for i, row := range [][2]interface{}{{"x", 0}, {"y", 1}, {"z", 1}, {"w", 0}} {
if _, err := s.db.Exec(`INSERT INTO blocks (id, document_id, content, level, position, type) VALUES (?, ?, ?, ?, ?, 'bullet')`,
row[0], doc.ID, row[0], row[1], (i+1)*1000); err != nil {
t.Fatal(err)
}
}
s.backfillBlockTree()
got, _ := s.GetDocumentByID(u.ID, doc.ID)
want := flatBlocks([3]interface{}{"x", 0, "x"}, [3]interface{}{"y", 1, "y"}, [3]interface{}{"z", 1, "z"}, [3]interface{}{"w", 0, "w"})
sameOutline(t, got.Blocks, want)
}
+75 -271
View File
@@ -415,11 +415,6 @@ func (s *Store) migrate() error {
// Backfill username for existing accounts where username is empty // Backfill username for existing accounts where username is empty
_, _ = s.db.Exec(`UPDATE users SET username = LOWER(SUBSTR(email, 1, INSTR(email, '@') - 1)) WHERE (username IS NULL OR username = '') AND INSTR(email, '@') > 0`) _, _ = s.db.Exec(`UPDATE users SET username = LOWER(SUBSTR(email, 1, INSTR(email, '@') - 1)) WHERE (username IS NULL OR username = '') AND INSTR(email, '@') > 0`)
// Ensure all existing document & folder shares are strictly read-only
_, _ = s.db.Exec(`UPDATE doc_shares SET permission = 'read' WHERE permission != 'read'`)
_, _ = s.db.Exec(`UPDATE doc_team_shares SET permission = 'read' WHERE permission != 'read'`)
_, _ = s.db.Exec(`UPDATE workspace_folder_shares SET permission = 'read' WHERE permission != 'read'`)
// Default system settings (public registration disabled by default; invite or admin creation required) // Default system settings (public registration disabled by default; invite or admin creation required)
_, _ = s.db.Exec(`INSERT OR IGNORE INTO system_settings (key, value) VALUES _, _ = s.db.Exec(`INSERT OR IGNORE INTO system_settings (key, value) VALUES
('sharing_mode', 'teams_only'), ('sharing_mode', 'teams_only'),
@@ -463,6 +458,9 @@ func (s *Store) migrate() error {
_, _ = s.db.Exec(`UPDATE team_members SET role = 'member' WHERE role NOT IN ('admin', 'member')`) _, _ = s.db.Exec(`UPDATE team_members SET role = 'member' WHERE role NOT IN ('admin', 'member')`)
_, _ = s.db.Exec(`UPDATE users SET role = 'member' WHERE role = 'team_admin'`) _, _ = s.db.Exec(`UPDATE users SET role = 'member' WHERE role = 'team_admin'`)
// Block tree, operation log and idempotency tables for collaborative editing
s.migrateDocSync()
// Migrate workspace documents, blocks, folders, and strip legacy folder:* tags into pure DB tables // Migrate workspace documents, blocks, folders, and strip legacy folder:* tags into pure DB tables
s.migrateWorkspaceToPureDB() s.migrateWorkspaceToPureDB()
@@ -1449,88 +1447,24 @@ func (s *Store) migrateWorkspaceToPureDB() {
} }
func (s *Store) loadDocBlocks(docID string, fallbackJSON string) []domain.DocumentBlock { func (s *Store) loadDocBlocks(docID string, fallbackJSON string) []domain.DocumentBlock {
rows, err := s.db.Query(` if d, _, err := loadCollabDoc(s.db, docID); err == nil && len(d.Blocks) > 0 {
SELECT id, document_id, type, content, level, position, checked, collapsed, lang return flatDocumentBlocks(docID, d)
FROM blocks
WHERE document_id = ?
ORDER BY position ASC, rowid ASC
`, docID)
if err == nil {
blocks := make([]domain.DocumentBlock, 0)
for rows.Next() {
var b domain.DocumentBlock
var chk, col int
if scanErr := rows.Scan(&b.ID, &b.DocumentID, &b.Type, &b.Content, &b.Level, &b.Position, &chk, &col, &b.Lang); scanErr == nil {
b.Checked = chk != 0
b.Collapsed = col != 0
blocks = append(blocks, b)
}
}
_ = rows.Close()
if len(blocks) > 0 {
return blocks
}
} }
// Very old notes only stored blocks_json; convert them once
blocks := make([]domain.DocumentBlock, 0) blocks := make([]domain.DocumentBlock, 0)
if fallbackJSON != "" && fallbackJSON != "[]" { if fallbackJSON != "" && fallbackJSON != "[]" {
_ = json.Unmarshal([]byte(fallbackJSON), &blocks) _ = json.Unmarshal([]byte(fallbackJSON), &blocks)
if len(blocks) > 0 { if len(blocks) > 0 {
s.replaceDocBlocksInDB(docID, "", blocks, domain.NowISO()) s.replaceDocBlocksInDB(docID, "", blocks, domain.NowISO())
if d, _, err := loadCollabDoc(s.db, docID); err == nil {
return flatDocumentBlocks(docID, d)
}
} }
} }
return blocks return blocks
} }
func (s *Store) replaceDocBlocksInDB(docID, _ string, blocks []domain.DocumentBlock, now string) {
_, _ = s.db.Exec(`DELETE FROM blocks WHERE document_id = ?`, docID)
if len(blocks) > 0 {
stmt, err := s.db.Prepare(`
INSERT INTO blocks (id, document_id, content, level, position, type, checked, collapsed, lang)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
document_id = excluded.document_id,
content = excluded.content,
level = excluded.level,
position = excluded.position,
type = excluded.type,
checked = excluded.checked,
collapsed = excluded.collapsed,
lang = excluded.lang
`)
if err == nil {
defer stmt.Close()
for i, b := range blocks {
bID := strings.TrimSpace(b.ID)
if bID == "" {
bID = domain.NewID("blk")
}
bType := b.Type
if bType == "" {
bType = "bullet"
}
chk := 0
if b.Checked {
chk = 1
}
col := 0
if b.Collapsed {
col = 1
}
pos := b.Position
if pos <= 0 {
pos = float64((i + 1) * 1000)
}
_, _ = stmt.Exec(bID, docID, b.Content, b.Level, pos, bType, chk, col, b.Lang)
}
}
}
// Update markdown_cache on documents
mdCache := domain.CompileBlocksToMarkdown(blocks)
_, _ = s.db.Exec(`UPDATE documents SET markdown_cache = ?, updated_at = ? WHERE id = ?`, mdCache, now, docID)
}
func (s *Store) LoadDocumentState(docID string) (string, []domain.DocumentBlock, error) { func (s *Store) LoadDocumentState(docID string) (string, []domain.DocumentBlock, error) {
var title, blocksJSON string var title, blocksJSON string
err := s.db.QueryRow(`SELECT title, blocks_json FROM documents WHERE id = ?`, docID).Scan(&title, &blocksJSON) err := s.db.QueryRow(`SELECT title, blocks_json FROM documents WHERE id = ?`, docID).Scan(&title, &blocksJSON)
@@ -1541,222 +1475,85 @@ func (s *Store) LoadDocumentState(docID string) (string, []domain.DocumentBlock,
return title, blocks, nil return title, blocks, nil
} }
// FlushDocumentState persists the in-memory blocks into `blocks`, extracts all #tags into `block_tags`, // FlushDocumentState stores a complete block list (and optionally the title) through the operation log.
// and compiles the hierarchical Markdown into `documents.markdown_cache`.
func (s *Store) FlushDocumentState(docID string, title string, blocks []domain.DocumentBlock) error { func (s *Store) FlushDocumentState(docID string, title string, blocks []domain.DocumentBlock) error {
now := domain.NowISO() s.replaceDocBlocksInDB(docID, "", blocks, domain.NowISO())
// Normalize positions sequentially
normalized := make([]domain.DocumentBlock, len(blocks))
var plainBuilder strings.Builder
for i, b := range blocks {
if strings.TrimSpace(b.ID) == "" {
b.ID = domain.NewID("blk")
}
if b.Type == "" {
b.Type = "bullet"
}
b.DocumentID = docID
b.Position = float64((i + 1) * 1000)
normalized[i] = b
if b.Content != "" {
plainBuilder.WriteString(b.Content)
plainBuilder.WriteString("\n")
}
}
s.replaceDocBlocksInDB(docID, "", normalized, now)
_ = s.indexDocBlockTags(docID, "", normalized)
mdCache := domain.CompileBlocksToMarkdown(normalized)
plainContent := plainBuilder.String()
blocksJSON, _ := json.Marshal(normalized)
if strings.TrimSpace(title) != "" { if strings.TrimSpace(title) != "" {
_, err := s.db.Exec(` s.applyLegacyTitle(docID, "", title)
UPDATE documents }
SET title = ?, markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? return nil
WHERE id = ?
`, title, mdCache, plainContent, string(blocksJSON), now, docID)
return err
} }
_, err := s.db.Exec(` // PatchDocumentBlocks applies block upserts, deletions and an optional order to a note through the
UPDATE documents // operation log. Blocks not mentioned keep their position relative to each other.
SET markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ?
WHERE id = ?
`, mdCache, plainContent, string(blocksJSON), now, docID)
return err
}
// PatchDocumentBlocks applies atomic block-level upserts, deletions, and order reconciliation without overwriting concurrent edits from other sessions.
func (s *Store) PatchDocumentBlocks(userID, docID string, req domain.DocBlocksPatchRequest) (*domain.Document, []string, error) { func (s *Store) PatchDocumentBlocks(userID, docID string, req domain.DocBlocksPatchRequest) (*domain.Document, []string, error) {
if !CanEditDoc(s.DocRole(userID, docID)) {
return nil, nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)")
}
doc, err := s.GetDocumentByID(userID, docID) doc, err := s.GetDocumentByID(userID, docID)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
if !doc.IsOwner {
return nil, nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)") deleted := make(map[string]bool, len(req.DeletedIDs))
for _, id := range req.DeletedIDs {
deleted[strings.TrimSpace(id)] = true
}
upserts := make(map[string]domain.DocumentBlock, len(req.Upserted))
for _, b := range req.Upserted {
if id := strings.TrimSpace(b.ID); id != "" && !deleted[id] {
upserts[id] = b
}
} }
now := domain.NowISO() merged := make([]domain.DocumentBlock, 0, len(doc.Blocks)+len(upserts))
used := make(map[string]bool, len(upserts))
// Ensure blocks table is seeded if previously only in blocks_json for _, b := range doc.Blocks {
var existingCount int if deleted[b.ID] {
_ = s.db.QueryRow(`SELECT COUNT(*) FROM blocks WHERE document_id = ?`, docID).Scan(&existingCount)
if existingCount == 0 && len(doc.Blocks) > 0 {
s.replaceDocBlocksInDB(docID, doc.UserID, doc.Blocks, doc.UpdatedAt)
}
// Read current ordered block IDs from DB before patch
var existingOrder []string
ordRows, err := s.db.Query(`SELECT id FROM blocks WHERE document_id = ? ORDER BY position ASC, rowid ASC`, docID)
if err == nil {
for ordRows.Next() {
var bid string
if err := ordRows.Scan(&bid); err == nil {
existingOrder = append(existingOrder, bid)
}
}
_ = ordRows.Close()
}
// 1. Delete explicitly deleted block IDs
deletedSet := make(map[string]bool, len(req.DeletedIDs))
for _, delID := range req.DeletedIDs {
delID = strings.TrimSpace(delID)
if delID == "" {
continue continue
} }
deletedSet[delID] = true if up, ok := upserts[b.ID]; ok {
_, _ = s.db.Exec(`DELETE FROM blocks WHERE document_id = ? AND id = ?`, docID, delID) merged = append(merged, up)
} used[b.ID] = true
// Filter existingOrder to exclude deleted blocks
survivingExisting := make([]string, 0, len(existingOrder))
survivingSet := make(map[string]bool, len(existingOrder))
for _, bid := range existingOrder {
if !deletedSet[bid] {
survivingExisting = append(survivingExisting, bid)
survivingSet[bid] = true
}
}
// 2. Upsert modified or newly created blocks
upsertedIDs := make([]string, 0, len(req.Upserted))
for idx, b := range req.Upserted {
bID := strings.TrimSpace(b.ID)
if bID == "" || deletedSet[bID] {
continue continue
} }
bType := b.Type merged = append(merged, b)
if bType == "" {
bType = "bullet"
} }
chk := 0 for _, b := range req.Upserted {
if b.Checked { if id := strings.TrimSpace(b.ID); id != "" && !used[id] && !deleted[id] {
chk = 1 merged = append(merged, b)
used[id] = true
} }
col := 0
if b.Collapsed {
col = 1
} }
defaultSort := float64((len(survivingExisting) + idx + 1) * 1000)
_, _ = s.db.Exec(`
INSERT INTO blocks (id, document_id, content, level, position, type, checked, collapsed, lang)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
type = excluded.type,
content = excluded.content,
level = excluded.level,
checked = excluded.checked,
collapsed = excluded.collapsed,
lang = excluded.lang
`, bID, docID, b.Content, b.Level, defaultSort, bType, chk, col, b.Lang)
upsertedIDs = append(upsertedIDs, bID)
survivingSet[bID] = true
}
// 3. Reconcile block ordering without dropping blocks concurrently created by other users
var mergedOrder []string
inMerged := make(map[string]bool)
if len(req.BlockOrder) > 0 { if len(req.BlockOrder) > 0 {
for _, bid := range req.BlockOrder { rank := make(map[string]int, len(req.BlockOrder))
if !deletedSet[bid] && survivingSet[bid] && !inMerged[bid] { for i, id := range req.BlockOrder {
mergedOrder = append(mergedOrder, bid) rank[id] = i
inMerged[bid] = true
}
}
for i, bid := range survivingExisting {
if inMerged[bid] {
continue
}
insertPos := 0
for j := i - 1; j >= 0; j-- {
predID := survivingExisting[j]
if inMerged[predID] {
for k, mID := range mergedOrder {
if mID == predID {
insertPos = k + 1
break
}
}
break
}
}
mergedOrder = append(mergedOrder[:insertPos], append([]string{bid}, mergedOrder[insertPos:]...)...)
inMerged[bid] = true
}
for _, bid := range upsertedIDs {
if !inMerged[bid] {
mergedOrder = append(mergedOrder, bid)
inMerged[bid] = true
}
}
} else {
for _, bid := range survivingExisting {
if !inMerged[bid] {
mergedOrder = append(mergedOrder, bid)
inMerged[bid] = true
}
}
for _, bid := range upsertedIDs {
if !inMerged[bid] {
mergedOrder = append(mergedOrder, bid)
inMerged[bid] = true
} }
sort.SliceStable(merged, func(i, j int) bool {
ri, okI := rank[merged[i].ID]
rj, okJ := rank[merged[j].ID]
if okI && okJ {
return ri < rj
} }
return okI && !okJ
})
} }
// Update position in blocks s.replaceDocBlocksInDB(docID, userID, merged, domain.NowISO())
for idx, bid := range mergedOrder { if req.Title != nil {
_, _ = s.db.Exec(`UPDATE blocks SET position = ? WHERE document_id = ? AND id = ?`, float64((idx+1)*1000), docID, bid) s.applyLegacyTitle(docID, userID, strings.TrimSpace(*req.Title))
} }
// Reload authoritative block list and update document snapshot updated, err := s.GetDocumentByID(userID, docID)
finalBlocks := s.loadDocBlocks(docID, "") if err != nil {
var plainBuilder strings.Builder return nil, nil, err
for _, b := range finalBlocks {
if b.Content != "" {
plainBuilder.WriteString(b.Content)
plainBuilder.WriteString("\n")
} }
order := make([]string, len(updated.Blocks))
for i, b := range updated.Blocks {
order[i] = b.ID
} }
plainContent := plainBuilder.String() return updated, order, nil
mdCache := domain.CompileBlocksToMarkdown(finalBlocks)
blocksJSON, _ := json.Marshal(finalBlocks)
_, _ = s.db.Exec(`UPDATE documents SET markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? WHERE id = ?`,
mdCache, plainContent, string(blocksJSON), now, docID)
doc.Blocks = finalBlocks
doc.MarkdownCache = mdCache
doc.Content = plainContent
doc.UpdatedAt = now
_ = s.indexDocBlockTags(doc.ID, doc.UserID, finalBlocks)
return doc, mergedOrder, nil
} }
// GetAuthorizedDocUsers returns all user IDs who have read/write/owner access to docID. // GetAuthorizedDocUsers returns all user IDs who have read/write/owner access to docID.
@@ -2383,6 +2180,7 @@ func (s *Store) UpdateDocument(userID, docID string, title, icon, folderPath *st
if title != nil { if title != nil {
doc.Title = strings.TrimSpace(*title) doc.Title = strings.TrimSpace(*title)
s.applyLegacyTitle(docID, userID, doc.Title)
} }
if icon != nil { if icon != nil {
doc.Icon = *icon doc.Icon = *icon
@@ -3445,8 +3243,7 @@ func (s *Store) CreateFolderShare(currentUserID, folderID, targetType, targetIde
} }
} }
// Geteilte Ordner können ausschließlich mit Leserberechtigung geteilt werden (wie Dokumente) permission = normalizeSharePermission(permission)
permission = "read"
var targetID, targetName, email, avatarURL string var targetID, targetName, email, avatarURL string
if cleanType == "team" { if cleanType == "team" {
@@ -3671,8 +3468,7 @@ func (s *Store) CreateDocTeamShare(currentUserID, docID, teamID, permission stri
return nil, fmt.Errorf("team nicht gefunden") return nil, fmt.Errorf("team nicht gefunden")
} }
// Document sharing is strictly read-only permission = normalizeSharePermission(permission)
permission = "read"
now := domain.NowISO() now := domain.NowISO()
shareID := domain.NewID("share") shareID := domain.NewID("share")
@@ -3699,6 +3495,15 @@ func (s *Store) CreateDocTeamShare(currentUserID, docID, teamID, permission stri
}, nil }, nil
} }
// normalizeSharePermission maps share roles from the API to the stored values: "write" (editor) or "read" (viewer).
func normalizeSharePermission(p string) string {
switch strings.ToLower(strings.TrimSpace(p)) {
case "write", "editor":
return "write"
}
return "read"
}
func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permission string) (*domain.DocShare, error) { func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permission string) (*domain.DocShare, error) {
doc, err := s.GetDocumentByID(currentUserID, docID) doc, err := s.GetDocumentByID(currentUserID, docID)
if err != nil { if err != nil {
@@ -3712,11 +3517,10 @@ func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permi
// If identifier looks like a team or is prefixed with team_, route to CreateDocTeamShare // If identifier looks like a team or is prefixed with team_, route to CreateDocTeamShare
if strings.HasPrefix(cleanTarget, "team_") { if strings.HasPrefix(cleanTarget, "team_") {
return s.CreateDocTeamShare(currentUserID, docID, cleanTarget, "read") return s.CreateDocTeamShare(currentUserID, docID, cleanTarget, permission)
} }
// Document sharing is strictly read-only permission = normalizeSharePermission(permission)
permission = "read"
// Resolve target user by ID, username, or Email // Resolve target user by ID, username, or Email
var targetUser *domain.User var targetUser *domain.User
+1 -1
View File
@@ -574,7 +574,7 @@ func TestDatabaseFirstWorkspaceArchitecture(t *testing.T) {
} }
} }
_, err = store.CreateDocShare(userA.ID, doc.ID, userB.ID, "write") _, err = store.CreateDocShare(userA.ID, doc.ID, userB.ID, "read")
if err != nil { if err != nil {
t.Fatalf("CreateDocShare failed: %v", err) t.Fatalf("CreateDocShare failed: %v", err)
} }