From af578d8237808e4a35862f3c8d11a0bce2520cbc Mon Sep 17 00:00:00 2001 From: Sandro Date: Wed, 28 Oct 2026 11:03:45 +0100 Subject: [PATCH 01/11] docs: add plan for collaborative note editing Server-authoritative operation log with per-block versions and 20s block leases, conflict copies instead of data loss, IndexedDB queue for offline use, HTTP-based sync with WebSocket/SSE/long-poll wakeups, unified note/folder permissions, and per-user undo. Co-Authored-By: Claude Opus 5.5 --- Doku/README.md | 1 + Doku/plans/kollaboration-notizen.md | 293 ++++++++++++++++++++++++++++ 2 files changed, 294 insertions(+) create mode 100644 Doku/plans/kollaboration-notizen.md diff --git a/Doku/README.md b/Doku/README.md index f9be5aa..d599a4a 100644 --- a/Doku/README.md +++ b/Doku/README.md @@ -18,6 +18,7 @@ Referenz für Entwickler und Administratoren. Alle Angaben wurden gegen den Code | [reverse-proxy.md](reverse-proxy.md) | Nginx, Traefik, Caddy, Cloudflare | | [theming.md](theming.md) | Themes und Design-Variablen | | [flaws.md](flaws.md) | Bekannte Schwachstellen und technische Schulden | +| [plans/kollaboration-notizen.md](plans/kollaboration-notizen.md) | Plan: gemeinsames Bearbeiten von Notizen | ## Überblick in 60 Sekunden diff --git a/Doku/plans/kollaboration-notizen.md b/Doku/plans/kollaboration-notizen.md new file mode 100644 index 0000000..8d161e5 --- /dev/null +++ b/Doku/plans/kollaboration-notizen.md @@ -0,0 +1,293 @@ +# Plan: Gemeinsames Bearbeiten von Notizen + +Status: Entschieden, bereit zur Umsetzung · Stand 28.10.2026 + +## 1. Ziel und Anforderungen + +Mehrere Personen bearbeiten dieselbe Notiz gleichzeitig. Verbindlich: + +1. **Kein Datenverlust.** Was jemand getippt hat, steht danach in der Notiz, auch nach Verbindungsabbruch, Neuladen oder gleichzeitigem Bearbeiten desselben Blocks. +2. **Funktioniert ohne WebSocket.** Hinter Firmen-Proxys, die WebSockets blocken oder Streams puffern, muss die Bearbeitung voll funktionieren, nur mit etwas mehr Verzögerung. +3. **Funktioniert bei schlechtem Netz.** Mobil mit Abbrüchen, Wechsel zwischen WLAN und Mobilfunk, App im Hintergrund: Eingaben werden lokal gesichert und später zuverlässig übertragen. +4. **Klare Regel bei Konflikten.** Bearbeiten zwei Personen denselben Bullet, bekommt eine das Recht. Der Text der anderen geht nicht verloren. +5. **Berechtigungen.** Notizen und Ordner lassen sich als *Bearbeiter* oder *Leser* teilen. Der Server setzt das bei jeder Änderung durch. + +Ausgangslage: Der Editor sendet alle 800 ms das ganze Dokument (`PUT /documents/{id}`), der Server ersetzt alle Blöcke. Zwei gleichzeitige Bearbeiter überschreiben sich dabei vollständig. Die vorhandenen Ansätze (CRDT-Reste, `PATCH /blocks`, erzwungenes Nur-Lesen) werden **nicht** weiterverwendet. Es gibt keine Produktivdaten, das Datenmodell wird neu aufgesetzt. + +## 2. Entscheidung + +**Server-autoritatives Operations-Log mit Versionen pro Block und Block-Sperren (Leases).** Die Übertragung läuft immer über gewöhnliche HTTP-Anfragen; WebSocket, SSE und Long-Polling dienen nur als „Weckruf“. + +| Baustein | Aufgabe | +|---|---| +| **Operations-Log** | Jede Änderung ist eine kleine Operation (Text eines Blocks ändern, Block einfügen, verschieben, löschen). Der Server prüft, wendet an und vergibt eine fortlaufende Nummer pro Dokument (`seq`). Wer `seq` kennt, holt sich einfach alles danach. | +| **Version pro Block** | Jede Operation nennt die Blockversion, auf der sie basiert. Passt sie nicht mehr, liegt ein Konflikt vor, und der Server erkennt ihn sicher. | +| **Lease (Sperre) pro Block** | Wer in einen Bullet klickt, bekommt ihn für 20 Sekunden reserviert und verlängert das, solange er tippt. Andere sehen „Anna bearbeitet …“ und können diesen Bullet so lange nicht ändern, alle anderen aber schon. Das verhindert fast alle Konflikte. | +| **Konfliktkopie** | Kommt es trotzdem zum Konflikt (z. B. Lease abgelaufen, weil jemand offline war), behält der Block die zuerst gespeicherte Fassung. Der andere Text wird als eigener Block direkt darunter eingefügt und als Konflikt markiert. Nichts geht verloren. | +| **Lokale Warteschlange** | Jede Änderung landet sofort in IndexedDB und wird von dort gesendet. Jede Operation hat eine eindeutige ID; doppeltes Senden ist harmlos (idempotent). | +| **Transport-Stufen** | Daten fließen immer über `POST /sync`. Benachrichtigung über WebSocket → SSE → Long-Polling → Intervall-Abfrage, je nachdem, was durchkommt. | + +### Warum nicht CRDT (Yjs/Automerge) oder OT + +- **CRDT** führt Zeichen für Zeichen zusammen, auch gleichzeitiges Tippen im selben Satz. Dafür braucht das Go-Backend eine vollständige Yjs-Implementierung, um Rechte pro Block, Hashtags und Markdown serverseitig auszuwerten; die gibt es in Go nicht ausgereift. Ohne sie wäre der Server nur ein Speicher für Binärdaten, den er nicht prüfen kann. Der frühere Versuch ist genau daran gescheitert. +- **OT** braucht einen fehlerfreien Transformations-Kern für jede Operationsart; das ist schwer korrekt zu bekommen und schwer zu testen. +- Du willst ausdrücklich, dass bei einem Bullet **einer das Recht bekommt**. Das passt zu Sperren plus Versionsprüfung: einfach zu verstehen, vollständig serverseitig prüfbar und mit Tests gegen Netzfehler absicherbar. +- Bewusster Verzicht: Zwei Personen können nicht gleichzeitig im **selben** Bullet tippen. Gleichzeitig in **verschiedenen** Bullets, auch im selben Ast, funktioniert. + +## 3. Datenmodell (neu) + +Die bisherigen Tabellen `documents`, `blocks`, `block_tags`, `doc_shares`, `doc_team_shares`, `workspace_folder_shares` und die Spalten `blocks_json`, `content` werden ersetzt. + +```sql +CREATE TABLE documents ( + id TEXT PRIMARY KEY, + owner_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + title TEXT NOT NULL DEFAULT '', + title_version INTEGER NOT NULL DEFAULT 0, + icon TEXT NOT NULL DEFAULT '📄', + shared_folder_id TEXT, -- falls in einem geteilten Ordner + head_seq INTEGER NOT NULL DEFAULT 0, -- letzte vergebene Operationsnummer + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL +); + +CREATE TABLE blocks ( + id TEXT PRIMARY KEY, -- vom Client erzeugt (UUID), damit offline einfügbar + doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE, + parent_id TEXT, -- NULL = oberste Ebene; Baum statt "level" + order_key TEXT NOT NULL, -- Fractional Index, sortiert Geschwister + type TEXT NOT NULL DEFAULT 'bullet', + content TEXT NOT NULL DEFAULT '', + props TEXT NOT NULL DEFAULT '{}', -- checked, collapsed, lang, conflictOf … + version INTEGER NOT NULL DEFAULT 1, + updated_by TEXT NOT NULL, + updated_at TEXT NOT NULL, + deleted_at TEXT -- Grabstein statt hartem Löschen +); +CREATE INDEX idx_blocks_doc ON blocks(doc_id, parent_id, order_key); + +CREATE TABLE doc_ops ( -- das Log + doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE, + seq INTEGER NOT NULL, + op_id TEXT NOT NULL, -- vom Client, eindeutig → Idempotenz + user_id TEXT NOT NULL, + client_id TEXT NOT NULL, + kind TEXT NOT NULL, -- set_content | set_props | insert | move | delete | set_title + block_id TEXT, + base_version INTEGER, + payload TEXT NOT NULL, -- JSON inkl. alter und neuer Werte + status TEXT NOT NULL, -- applied | conflict | rejected + result TEXT NOT NULL DEFAULT '{}', -- z. B. ID der Konfliktkopie + created_at TEXT NOT NULL, + PRIMARY KEY (doc_id, seq) +); +CREATE UNIQUE INDEX idx_doc_ops_op ON doc_ops(op_id); + +CREATE TABLE block_tags (tag TEXT, doc_id TEXT, block_id TEXT, PRIMARY KEY(tag, block_id)); + +CREATE TABLE workspace_access ( -- ein Freigabemodell für Notizen und Ordner + id TEXT PRIMARY KEY, + resource_type TEXT NOT NULL CHECK(resource_type IN ('document','folder')), + resource_id TEXT NOT NULL, + principal_type TEXT NOT NULL CHECK(principal_type IN ('user','team')), + principal_id TEXT NOT NULL, + role TEXT NOT NULL CHECK(role IN ('viewer','editor')), + created_by TEXT NOT NULL, + created_at TEXT NOT NULL, + UNIQUE(resource_type, resource_id, principal_type, principal_id) +); +``` + +Entscheidungen dazu: + +- **Baum mit `parent_id` statt flacher Liste mit `level`.** Einrücken oder Verschieben eines Astes ist dann *eine* Operation auf *einem* Block; die Unterpunkte wandern automatisch mit. Mit der flachen Liste ergeben gleichzeitige Einrückungen leicht kaputte Strukturen. +- **Fractional Index (`order_key`).** Ein neuer Block zwischen `a0` und `a1` bekommt `a0V`; es muss nie umnummeriert werden. Gleiche Schlüssel (zwei gleichzeitige Einfügungen) werden über die Block-ID eindeutig sortiert. +- **Grabsteine.** Gelöschte Blöcke bleiben mit `deleted_at` erhalten. So kann eine verspätete Textänderung an einem gelöschten Block den Block wiederherstellen statt verloren zu gehen. Aufräumen nach 30 Tagen. +- **Leases liegen im Arbeitsspeicher**, nicht in der Datenbank. Sie sind kurzlebig (20 s), und nach einem Neustart fordern Clients sie automatisch neu an. MindSift läuft als eine Instanz, das passt. +- **Abgeleitete Daten** (`block_tags`, Markdown für Export und Suche) werden aus den geänderten Blöcken in derselben Transaktion bzw. bei Bedarf erzeugt. + +## 4. Protokoll + +### 4.1 Endpunkte + +| Methode | Pfad | Zweck | +|---|---|---| +| GET | `/api/v1/docs/{id}/snapshot` | Alle lebenden Blöcke, Titel, `headSeq`, eigene Rolle, aktuelle Leases | +| POST | `/api/v1/docs/{id}/sync` | **Ein Aufruf für alles**: eigene Operationen senden, Leases halten/abgeben, neue Operationen abholen | +| GET | `/api/v1/docs/{id}/changes?since=N&wait=25` | Long-Polling: antwortet sofort, wenn `headSeq > N`, sonst nach höchstens 25 s mit leerem Ergebnis | + +Anfrage an `/sync`: + +```json +{ + "clientId": "tab-7f3…", + "lastSeq": 412, + "ops": [ + { "opId": "c1a…", "kind": "set_content", "blockId": "b9…", "baseVersion": 7, "content": "Milch und Brot" }, + { "opId": "c1b…", "kind": "insert", "blockId": "bA…", "parentId": "b2…", "orderKey": "a0V", "type": "bullet", "content": "" } + ], + "leases": { "hold": ["b9…"], "release": ["b4…"] } +} +``` + +Antwort: + +```json +{ + "results": [ + { "opId": "c1a…", "status": "applied", "seq": 413, "version": 8 }, + { "opId": "c1b…", "status": "applied", "seq": 414 } + ], + "ops": [ /* alle Operationen mit seq > 412, auch fremde */ ], + "headSeq": 415, + "leases": { "granted": ["b9…"], "denied": [], "others": [{ "blockId": "b3…", "user": "Anna" }] }, + "presence": [{ "userId": "…", "name": "Anna", "blockId": "b3…" }] +} +``` + +### 4.2 Regeln auf dem Server + +Alle Operationen eines Dokuments laufen nacheinander: ein Mutex pro Dokument im Prozess und eine SQLite-Transaktion pro `/sync`-Aufruf. Dadurch gibt es eine eindeutige Reihenfolge, und „wer zuerst kommt“ ist immer eindeutig. + +1. **Idempotenz:** Ist `opId` schon im Log, wird das gespeicherte Ergebnis zurückgegeben, ohne erneut anzuwenden. Verlorene Antworten sind dadurch harmlos. +2. **Berechtigung:** Ohne Rolle `editor` oder `owner` wird jede Operation abgelehnt (`403`). +3. **Zugehörigkeit:** Block und Elternblock müssen zu diesem Dokument gehören (behebt FLAW-11). +4. **`set_content`:** + - Hält ein *anderer* Client eine gültige Lease → **Konflikt**. + - `baseVersion == version` → anwenden, `version + 1`. + - `baseVersion < version` → **Konflikt**. + - Block gelöscht → wiederherstellen (am alten Platz; ist der Elternblock auch gelöscht, unter dem nächsten lebenden Vorfahren) und anwenden. +5. **Konflikt:** Der Block behält seinen Inhalt. Der Server fügt direkt danach einen neuen Block mit dem abgelehnten Text ein, `props.conflictOf = `, `props.conflictAuthor = `. Ergebnis `status: "conflict"` mit der neuen Block-ID. +6. **`move`:** Wird abgelehnt, wenn ein Zyklus entstünde (Block unter seinen eigenen Nachfahren) oder ein anderer Client eine Lease auf dem Block hält. Der Client lädt dann den Stand neu. +7. **`delete`:** Wird abgelehnt, wenn ein anderer Client gerade einen Block in diesem Ast bearbeitet (Lease). Sonst Grabstein für den ganzen Ast. +8. **`insert`:** Ist der Elternblock gelöscht, wird unter dem nächsten lebenden Vorfahren eingefügt. +9. **Leases:** Vergabe nur, wenn frei, abgelaufen oder schon im Besitz desselben Clients; Gültigkeit 20 s ab letzter Bestätigung. Der Client sendet alle 7 s einen `/sync` mit seinen gehaltenen Blöcken (Herzschlag). + +### 4.3 Ablauf beim Tippen + +```mermaid +sequenceDiagram + participant A as Client Anna + participant S as Server + participant B as Client Ben + A->>S: /sync leases.hold=[b9] + S-->>A: granted b9 + S--)B: Weckruf headSeq=413 + B->>S: /sync lastSeq=412 + S-->>B: leases.others: b9 = Anna (Bullet gesperrt) + A->>S: /sync op set_content b9 base=7 + S-->>A: applied v8, seq 414 + S--)B: Weckruf headSeq=414 + B->>S: /sync lastSeq=413 + S-->>B: ops [set_content b9 v8] +``` + +## 5. Client + +Neues Modul `frontend/src/lib/docSync/` (ersetzt das Autosave in `DocumentEditor.tsx`): + +- **`SyncEngine` pro geöffnetem Dokument:** + - *Bestätigter Stand*: Blöcke wie vom Server zuletzt gemeldet, plus `lastSeq`. + - *Warteschlange*: eigene, noch nicht bestätigte Operationen. + - *Angezeigter Stand* = bestätigter Stand + Warteschlange darüber angewendet. Kommen fremde Operationen herein, wird der bestätigte Stand aktualisiert und die Warteschlange erneut darübergelegt. +- **Persistenz:** Warteschlange und bestätigter Stand liegen in IndexedDB. Jede Eingabe wird dort gesichert, bevor sie gesendet wird. Nach Neuladen, Absturz oder vom Handy beendetem Tab wird die Warteschlange erneut gesendet; die Operations-IDs verhindern doppelte Anwendung. +- **Tippen:** Textänderungen eines Blocks werden 300 ms gesammelt. Pro Block ist höchstens eine Textoperation unterwegs; weitere Änderungen werden in die nächste zusammengefasst. So bleibt `baseVersion` immer eindeutig. +- **Senden:** immer nur ein `/sync` gleichzeitig. Das hält die Reihenfolge der eigenen Operationen stabil. Wiederholung mit Backoff (1 s, 2 s, 4 s … max. 30 s). +- **Leases:** + - Beim Fokussieren eines Bullets wird die Lease angefordert. Bis zur Antwort darf getippt werden. + - Wird sie verweigert: Der Bullet wird schreibgeschützt („Ben bearbeitet gerade“), schon Getipptes wird als neuer Block darunter eingefügt. + - Nach 60 s ohne Eingabe oder beim Verlassen des Bullets gibt der Client die Lease frei. +- **Mobil und Hintergrund:** Bei `visibilitychange` (Seite verborgen) und `pagehide` sendet der Client die Warteschlange mit `fetch(…, { keepalive: true })` und gibt Leases frei. Bei `online` und beim Zurückkehren in die App synchronisiert er sofort. +- **Anzeige:** + - Gesperrte Bullets zeigen dezent Avatar und Namen. + - Konfliktblöcke sind farbig markiert, mit den Aktionen „Übernehmen“ (ersetzt den Originaltext) und „Verwerfen“. + - Ein Statushinweis zeigt „Offline – 3 Änderungen warten“ bzw. „Gespeichert“. + - Leser sehen alles schreibgeschützt und live. + +## 6. Transport: robust auch hinter Proxys + +Grundsatz: **Daten fließen nur über normale HTTP-Anfragen** (`/sync`, `/snapshot`). Push-Kanäle melden nur „Dokument X hat jetzt `headSeq` N“. Fällt jeder Push-Kanal aus, funktioniert die Bearbeitung trotzdem, nur mit Verzögerung. + +| Stufe | Erkennung eines Ausfalls | Wechsel zu | +|---|---|---| +| 1. WebSocket (`/api/v1/ws`) | Kein Aufbau in 3 s oder 2 verpasste Pongs | SSE | +| 2. SSE (`/api/v1/events`) | Erstes Ereignis kommt nicht innerhalb von 5 s an (Proxy puffert) | Long-Polling | +| 3. Long-Polling (`/changes?wait=25`) | Wiederholte Fehler oder Zeitüberschreitungen | Intervall-Abfrage | +| 4. Intervall-Abfrage | – | alle 3 s bei offener Notiz, alle 30 s sonst | + +Unabhängig von der Stufe: +- Herzschlag-`/sync` alle 7 s, solange eine Lease gehalten wird. Das ist zugleich eine Abfrage. +- Sicherheitsabfrage alle 30 s, auch wenn der Push-Kanal angeblich steht. +- Der Client merkt sich die zuletzt funktionierende Stufe und versucht WebSocket alle 5 Minuten erneut. + +Serverseitig nötig: +- Räume nur nach Berechtigungsprüfung betreten (behebt FLAW-01 und FLAW-02). +- Schreib-Timeout für Streams aufheben (behebt FLAW-15). Long-Polling bleibt mit 25 s unter üblichen Proxy-Grenzen (Nginx 60 s, Cloudflare 100 s). + +## 7. Berechtigungen + +- Rollen je Notiz: `owner` (Eigentümer), `editor`, `viewer`. Freigaben über `workspace_access` an Benutzer oder Gruppen, für eine Notiz oder einen geteilten Ordner. Eine Ordnerfreigabe gilt für alle Notizen darin. +- **Effektive Rolle** = höchste aus Eigentümer, direkter Freigabe, Gruppenfreigaben und Ordnerfreigaben. Eine zentrale Funktion `docRole(userID, docID)` im Backend, benutzt von Snapshot, Sync, Long-Polling, Raumbeitritt und Export. +- Nur der Eigentümer verwaltet Freigaben und löscht die Notiz. Bearbeiter haben sonst volle Rechte: Inhalt, Struktur, Titel und Symbol. Der Titel wird wie ein Block behandelt (eigene Version, Lease, Konfliktkopie als Block am Anfang der Notiz). +- Im Freigabe-Dialog wird „Bearbeiten“ für Notizen und Ordner wieder auswählbar (`allowRoleSelection` statt `fixedPermission="viewer"` in `WorkspaceView.tsx`). +- Rechteentzug wirkt sofort: Der nächste `/sync` liefert `403`. Der Client verwirft die Warteschlange nicht, sondern bietet an, die Änderungen als neue eigene Notiz zu sichern. + +## 8. Nicht geteilte Notizen + +Es gibt nur ein Modell; das alte `PUT` mit Komplettersatz entfällt auch für private Notizen. Gründe: kein zweiter Speicherweg, Schutz vor gegenseitigem Überschreiben zwischen eigenen Geräten und Tabs, Offline-Sicherung über die lokale Warteschlange. + +Damit private Notizen keinen unnötigen Aufwand erzeugen, liefert der Server im Snapshot und in jeder `/sync`-Antwort `shared` (gibt es Freigaben?) und `sessions` (wie viele Clients haben die Notiz gerade offen). Solange `shared = false` und `sessions = 1`: + +- keine Leases, also auch kein Herzschlag, +- kein Live-Raum, keine regelmäßige Abfrage außer der 30-s-Sicherheitsabfrage, +- kürzere Aufbewahrung im Log (7 Tage statt 30), weil niemand alte Einträge nachholen muss. + +Öffnet ein zweites Gerät die Notiz oder wird sie geteilt, bekommen alle offenen Clients das mit der nächsten Antwort mit und schalten auf den vollen Ablauf um (gleicher Code, nur Leases und Push aktiv). Im Vergleich zum heutigen `PUT` (ganzes Dokument alle 800 ms) überträgt und schreibt das neue Modell nur geänderte Blöcke und ist damit bei längeren Notizen sparsamer. + +## 9. Rückgängig und Wiederholen (Strg+Z) + +- **Jeder macht nur seine eigenen Änderungen rückgängig.** `Strg+Z` / `Cmd+Z` macht die letzte eigene Änderung rückgängig, `Strg+Umschalt+Z` bzw. `Strg+Y` wiederholt. Fremde Änderungen bleiben unberührt. +- **Umsetzung im Client:** Zu jeder eigenen Operation merkt sich der Client die Gegenoperation (alter Text, alter Platz im Baum, Wiederherstellen statt Löschen). Rückgängig sendet die Gegenoperation als ganz normale Operation über `/sync`. Server, Log und Konfliktregeln brauchen dafür nichts Neues. +- **Schritte:** Tippen wird zu Schritten gebündelt (neuer Schritt nach 1 s Pause, beim Wechsel des Bullets oder nach Satzzeichen). Einfügen, Löschen, Einrücken, Verschieben und Abhaken sind jeweils ein Schritt. Ein Einfügen mehrerer Zeilen ist ein Schritt. +- **Wenn inzwischen jemand anders den Block geändert hat:** Passt die Version nicht mehr, wird dieser Schritt nicht blind überschrieben. Der Client zeigt „Bullet wurde inzwischen von Anna geändert“ und überspringt ihn. Kein fremder Text geht durch Rückgängig verloren. +- **Gesperrte Bullets:** Hält gerade jemand anders die Lease, wartet der Schritt nicht, sondern wird mit demselben Hinweis übersprungen. +- **Reichweite:** Der Stapel gilt pro Tab und Notiz, bis zu 200 Schritte, und bleibt beim Wechsel zwischen Notizen erhalten. Nach Neuladen beginnt er leer. +- **Native Textfeld-Undo** wird abgefangen (`keydown` und `beforeinput` mit `historyUndo`), sonst würden Browser-Undo und eigenes Undo gegeneinander arbeiten. +- **Später möglich:** Eine Verlaufsansicht („Wer hat wann was geändert“, ältere Fassung eines Bullets wiederherstellen) lässt sich direkt aus dem Log bauen. Sie ist nicht Teil dieses Features. + +## 10. Tests (Pflicht für „super robust“) + +1. **Simulation im Go-Backend** (Kern der Absicherung): N simulierte Clients erzeugen zufällige Operationen. Ein simuliertes Netz verliert, verdoppelt, verzögert und vertauscht Anfragen und Antworten; Clients gehen offline und kommen zurück. Nach jedem Lauf wird geprüft: + - Alle Clients haben denselben Stand wie der Server (Konvergenz). + - Jeder je getippte Endtext eines Clients steht im Dokument, im Block selbst oder in einer Konfliktkopie (kein Verlust). + - Der Baum ist gültig: keine Zyklen, keine verwaisten Blöcke. + + Tausende Läufe mit festem Seed in CI. +2. **Konfliktmatrix als Unit-Tests:** jede Kombination aus Lease, Version, Grabstein, Verschieben und Löschen. +3. **Frontend-Engine mit Vitest:** gefälschter Transport, `fake-indexeddb`, Neuladen mitten im Senden, verlorene Antworten. +4. **Rückgängig:** eigene Schritte zurücknehmen und wiederholen, Überspringen bei fremder Änderung, gebündeltes Tippen, Einfügen mehrerer Zeilen als ein Schritt. +5. **Ende-zu-Ende mit Playwright:** zwei Browser-Kontexte im selben Dokument. WebSocket per Routing blockiert → Fallback auf SSE und Long-Polling; Offline-Schalter mitten im Tippen; gedrosseltes Netz. + +## 11. Umsetzung in Phasen + +| Phase | Inhalt | Ergebnis | +|---|---|---| +| 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Stream-Timeout (FLAW-15), zentrale `docRole` | Grundlage dicht | +| 1 | Neues Schema, Operations-Log, `/snapshot`, `/sync`, Konfliktregeln, Simulationstests | Server fertig und abgesichert | +| 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt (Baum statt `level`) | Bearbeiten funktioniert überall, auch offline | +| 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | +| 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt | +| 5 | Freigabe-Dialog mit Bearbeiter-Rolle, Ordnerfreigaben, Altlasten entfernen (`PUT`-Blockersatz, `PATCH /blocks`, `doc_shares`, `doc_team_shares`, Zwangsmigration auf „read“) | Feature komplett | +| 6 | Härtung: Playwright-Tests, Log-Kompaktierung (Snapshot statt Log für Clients mit sehr altem `lastSeq`), Aufräumen der Grabsteine | Langzeitbetrieb | + +Phase 2 ist bewusst vor den Push-Kanälen: Steht die Bearbeitung über reine HTTP-Abfragen, ist der schwierigste Fall (Proxy blockt alles außer HTTP) von Anfang an abgedeckt. WebSocket und SSE machen es danach nur schneller. + +## 12. Getroffene Entscheidungen + +| Frage | Entscheidung | +|---|---| +| Lease-Dauer | 20 s, Verlängerung alle 7 s beim Tippen | +| Konflikte | Konfliktkopie als eigener Block darunter, mit Übernehmen/Verwerfen. Kein automatisches Zusammenführen. | +| Titel | Jeder Bearbeiter darf ihn ändern; Bearbeiter haben außer Freigaben und Löschen volle Rechte. | +| Verlauf/Undo | Strg+Z/Wiederholen für eigene Änderungen ist Teil des Features (Phase 3). Eine Verlaufsansicht kommt später. | +| Private Notizen | Gleiches Modell, ohne Leases und Live-Kanal, solange nur eine Sitzung offen ist | From 74797e3f443a0994270769a0e46a8d795eaea25a Mon Sep 17 00:00:00 2001 From: Sandro Date: Wed, 28 Oct 2026 11:10:15 +0100 Subject: [PATCH 02/11] docs(plan): block leases with auto-lease on insert and release grace period Co-Authored-By: Claude Opus 5.5 --- Doku/plans/kollaboration-notizen.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/Doku/plans/kollaboration-notizen.md b/Doku/plans/kollaboration-notizen.md index 8d161e5..0ce21ed 100644 --- a/Doku/plans/kollaboration-notizen.md +++ b/Doku/plans/kollaboration-notizen.md @@ -22,7 +22,7 @@ Ausgangslage: Der Editor sendet alle 800 ms das ganze Dokument (`PUT /documents/ |---|---| | **Operations-Log** | Jede Änderung ist eine kleine Operation (Text eines Blocks ändern, Block einfügen, verschieben, löschen). Der Server prüft, wendet an und vergibt eine fortlaufende Nummer pro Dokument (`seq`). Wer `seq` kennt, holt sich einfach alles danach. | | **Version pro Block** | Jede Operation nennt die Blockversion, auf der sie basiert. Passt sie nicht mehr, liegt ein Konflikt vor, und der Server erkennt ihn sicher. | -| **Lease (Sperre) pro Block** | Wer in einen Bullet klickt, bekommt ihn für 20 Sekunden reserviert und verlängert das, solange er tippt. Andere sehen „Anna bearbeitet …“ und können diesen Bullet so lange nicht ändern, alle anderen aber schon. Das verhindert fast alle Konflikte. | +| **Lease (Sperre) pro Block** | Wer in einen Bullet klickt, bekommt ihn für 20 Sekunden reserviert und verlängert das, solange er tippt. Neu erzeugte Bullets gehören automatisch ihrem Ersteller. Andere sehen „Anna bearbeitet …“ und können diesen Bullet so lange nicht ändern, alle anderen aber schon, auch im selben Ast. Das verhindert fast alle Konflikte. | | **Konfliktkopie** | Kommt es trotzdem zum Konflikt (z. B. Lease abgelaufen, weil jemand offline war), behält der Block die zuerst gespeicherte Fassung. Der andere Text wird als eigener Block direkt darunter eingefügt und als Konflikt markiert. Nichts geht verloren. | | **Lokale Warteschlange** | Jede Änderung landet sofort in IndexedDB und wird von dort gesendet. Jede Operation hat eine eindeutige ID; doppeltes Senden ist harmlos (idempotent). | | **Transport-Stufen** | Daten fließen immer über `POST /sync`. Benachrichtigung über WebSocket → SSE → Long-Polling → Intervall-Abfrage, je nachdem, was durchkommt. | @@ -161,7 +161,9 @@ Alle Operationen eines Dokuments laufen nacheinander: ein Mutex pro Dokument im 6. **`move`:** Wird abgelehnt, wenn ein Zyklus entstünde (Block unter seinen eigenen Nachfahren) oder ein anderer Client eine Lease auf dem Block hält. Der Client lädt dann den Stand neu. 7. **`delete`:** Wird abgelehnt, wenn ein anderer Client gerade einen Block in diesem Ast bearbeitet (Lease). Sonst Grabstein für den ganzen Ast. 8. **`insert`:** Ist der Elternblock gelöscht, wird unter dem nächsten lebenden Vorfahren eingefügt. -9. **Leases:** Vergabe nur, wenn frei, abgelaufen oder schon im Besitz desselben Clients; Gültigkeit 20 s ab letzter Bestätigung. Der Client sendet alle 7 s einen `/sync` mit seinen gehaltenen Blöcken (Herzschlag). +9. **Leases:** Vergabe nur, wenn frei, abgelaufen oder schon im Besitz desselben Clients; Gültigkeit 20 s ab letzter Bestätigung. Ein Client darf beliebig viele Blöcke gleichzeitig halten. Der Client sendet alle 7 s einen `/sync` mit allen gehaltenen Blöcken (ein Herzschlag für alle). +10. **Automatische Lease beim Einfügen:** Ein `insert` vergibt dem einfügenden Client die Lease auf den neuen Block, ohne eigene Anfrage. Der Block ist vorher niemandem bekannt, es kann also keinen Wettlauf geben. +11. **Keine Lease auf Elternblöcke:** Einfügen, Einrücken und Ausrücken unter einem Elternblock brauchen keine Lease auf diesen. Gleichzeitiges Einfügen zweier Personen in denselben Ast ergibt zwei Blöcke (Sortierung über `order_key`). Gesperrt wird nur der Inhalt einzelner Blöcke sowie Löschen/Verschieben von Ästen, in denen jemand anders einen Block hält (Regeln 6 und 7). ### 4.3 Ablauf beim Tippen @@ -194,9 +196,12 @@ Neues Modul `frontend/src/lib/docSync/` (ersetzt das Autosave in `DocumentEditor - **Tippen:** Textänderungen eines Blocks werden 300 ms gesammelt. Pro Block ist höchstens eine Textoperation unterwegs; weitere Änderungen werden in die nächste zusammengefasst. So bleibt `baseVersion` immer eindeutig. - **Senden:** immer nur ein `/sync` gleichzeitig. Das hält die Reihenfolge der eigenen Operationen stabil. Wiederholung mit Backoff (1 s, 2 s, 4 s … max. 30 s). - **Leases:** - - Beim Fokussieren eines Bullets wird die Lease angefordert. Bis zur Antwort darf getippt werden. - - Wird sie verweigert: Der Bullet wird schreibgeschützt („Ben bearbeitet gerade“), schon Getipptes wird als neuer Block darunter eingefügt. - - Nach 60 s ohne Eingabe oder beim Verlassen des Bullets gibt der Client die Lease frei. + - **Tippen wartet nie auf eine Lease.** Beim Fokussieren eines bestehenden Bullets wird die Lease mit dem nächsten `/sync` angefordert; getippt werden darf sofort. + - **Neue Bullets (Enter, Einfügen) sind sofort eigene**, ohne Anfrage (Regel 10). Bullet für Bullet zu schreiben erzeugt keinen einzigen Sperrwechsel. + - **Nachlauf:** Verlassene Bullets bleiben noch 20 s gesperrt und werden erst dann freigegeben, wenn man nicht zurückgekehrt ist. Springen mit Pfeiltasten oder Ergänzen im Bullet davor braucht keine neue Anfrage. + - **Verweigert** (nur bei bestehenden Bullets möglich, die jemand anders schon hält): Der Bullet wird schreibgeschützt („Ben bearbeitet gerade“), schon Getipptes wird als neuer, eigener Block darunter eingefügt. + - Nach 60 s ohne Eingabe gibt der Client alle Leases frei, auch beim fokussierten Bullet. + - **Bewusst nicht:** Sperre auf Elternblock oder ganzen Ast. Bei Bullets auf oberster Ebene wäre das die ganze Notiz, und Arbeit tief im Ast würde Kollegen im selben Ast ausschließen. - **Mobil und Hintergrund:** Bei `visibilitychange` (Seite verborgen) und `pagehide` sendet der Client die Warteschlange mit `fetch(…, { keepalive: true })` und gibt Leases frei. Bei `online` und beim Zurückkehren in die App synchronisiert er sofort. - **Anzeige:** - Gesperrte Bullets zeigen dezent Avatar und Namen. @@ -287,6 +292,7 @@ Phase 2 ist bewusst vor den Push-Kanälen: Steht die Bearbeitung über reine HTT | Frage | Entscheidung | |---|---| | Lease-Dauer | 20 s, Verlängerung alle 7 s beim Tippen | +| Lease-Umfang | Pro Block, nicht pro Ast. Neue Bullets automatisch gesperrt für den Ersteller, 20 s Nachlauf nach Verlassen, Tippen wartet nie auf die Sperre. | | Konflikte | Konfliktkopie als eigener Block darunter, mit Übernehmen/Verwerfen. Kein automatisches Zusammenführen. | | Titel | Jeder Bearbeiter darf ihn ändern; Bearbeiter haben außer Freigaben und Löschen volle Rechte. | | Verlauf/Undo | Strg+Z/Wiederholen für eigene Änderungen ist Teil des Features (Phase 3). Eine Verlaufsansicht kommt später. | From bcc06da3ee2bb2d8ea2c7570cbbb47d5fcb4582b Mon Sep 17 00:00:00 2001 From: Sandro Date: Wed, 28 Oct 2026 11:19:14 +0100 Subject: [PATCH 03/11] fix(realtime): scope events to authorized users, authorize rooms, keep SSE alive Phase 0 of the collaboration plan: - Remove BroadcastToAll; task, task list and workspace folder events now only reach users who can see the resource. Task list events carry only the list ID, clients reload with their own permissions. - Reject unauthenticated WebSocket/SSE connections (401). Every room join (URL room, JOIN_ROOM, SSE rooms) is checked via a room authorizer; /events/rooms requires login and only accepts the caller's own sseId. - Reorder rejects tasks the caller cannot access or edit (IDOR). - Lift the server WriteTimeout for SSE streams so they are not cut after 30s. - Fix folder share removal sending DOC_DELETED to the share ID. - Tests for all of the above; docs and flaw list updated. Co-Authored-By: Claude Opus 5.5 --- Doku/api/rest.md | 6 +- Doku/api/websocket.md | 18 +- Doku/flaws.md | 14 +- Doku/plans/kollaboration-notizen.md | 4 +- frontend/src/features/triage/TasksView.tsx | 3 + internal/api/realtime_test.go | 268 +++++++++++++++++++++ internal/api/server.go | 116 ++++++--- internal/storage/sqlite.go | 49 ++++ internal/ws/hub.go | 96 +++++--- internal/ws/hub_test.go | 38 ++- 10 files changed, 526 insertions(+), 86 deletions(-) create mode 100644 internal/api/realtime_test.go diff --git a/Doku/api/rest.md b/Doku/api/rest.md index e14f675..db53683 100644 --- a/Doku/api/rest.md +++ b/Doku/api/rest.md @@ -13,15 +13,15 @@ Basis: `/api/v1`. Antworten sind JSON; Fehler haben die Form `{"error": "Text"}` | POST | `/auth/login` | Anmeldung (`login`/`email`/`username`, `password`) → `{token, user}` | | POST | `/auth/quickstart` | Immer `403` (Gastzugang deaktiviert) | | POST | `/webhooks/ingest` | Aufgabe per Webhook anlegen, Token als `?token=`, `X-API-Key` oder Bearer. Body: `title`/`content`/`text`/`message`, `priority`, `source`, `tags`, `dueDate`; Klartext-Body wird als Inhalt genommen | -| GET | `/ws`, `/ws/spaces/{id}`, `/ws/lists/{id}` | WebSocket, siehe [websocket.md](websocket.md) | -| GET | `/events` | SSE-Fallback | -| POST | `/events/rooms` | SSE-Räume beitreten/verlassen ([FLAW-02](../flaws.md)) | +| GET | `/ws`, `/ws/spaces/{id}`, `/ws/lists/{id}` | WebSocket, siehe [websocket.md](websocket.md); Token als `?token=` nötig | +| GET | `/events` | SSE-Fallback; Token als `?token=` nötig | ## Konto und Allgemeines (angemeldet) | Methode | Pfad | Zweck | |---|---|---| | GET | `/auth/me` | Benutzer und Kennzahlen | +| POST | `/events/rooms` | SSE-Raum beitreten/verlassen (`sseId`, `room`, `action`); nur eigene `sseId`, nur Räume mit Lesezugriff | | POST | `/auth/logout` | Token sperren | | POST | `/auth/revoke-all` | Alle Sitzungen beenden (`token_version` +1) | | PATCH | `/auth/settings` | `displayName`, `email`, `password`, `avatarUrl`, `alarmChannels`, `alarmEmail`, `discordWebhookUrl`, `customWebhookUrl`, `notifyBeforeMinutes` | diff --git a/Doku/api/websocket.md b/Doku/api/websocket.md index c2fe7e7..16a88ea 100644 --- a/Doku/api/websocket.md +++ b/Doku/api/websocket.md @@ -13,7 +13,9 @@ Der Server verteilt Änderungen als JSON-Ereignisse an verbundene Clients. Die E Der Client versucht zuerst WebSocket. Gelingt der Aufbau nicht innerhalb von 2,5 s oder bricht die Verbindung unsauber ab, wechselt er auf SSE und bleibt dabei (Firmen-Proxys). Dort liefert der Server `SSE_CONNECTED` mit einer `sseId`; Räume werden über `POST /api/v1/events/rooms` (`sseId`, `room`, `action`: `join`/`leave`) abonniert. -**Authentifizierung:** Token als `?token=` oder `Authorization`-Header. Ohne gültiges Token wird die Verbindung als `anonymous` akzeptiert ([FLAW-01](../flaws.md), [FLAW-02](../flaws.md)). Das Token wird nur beim Verbinden geprüft. +**Authentifizierung:** Token als `?token=` oder `Authorization`-Header. Ohne gültiges Token antworten `/ws` und `/events` mit `401`. Das Token wird nur beim Verbinden geprüft ([FLAW-22](../flaws.md)). + +**Räume:** Jeder Beitritt (Raum in der URL, `JOIN_ROOM`, SSE-Startparameter `rooms`/`docId`, `POST /events/rooms`) wird geprüft: `doc:` nur mit Lesezugriff auf die Notiz, sonst muss die ID eine lesbare geteilte Liste sein. Nicht erlaubte Räume werden ignoriert, `POST /events/rooms` antwortet dann mit `403`. SSE-Räume kann nur der Besitzer der jeweiligen `sseId` ändern. ## Nachrichtenformat @@ -29,20 +31,20 @@ Client → Server versteht der Server nur: `{"event":"JOIN_ROOM","room":""}` |---|---| | Benutzer-Kanal (`BroadcastToUser`) | alle Verbindungen eines Benutzers | | Raum (`BroadcastRoom`, `Broadcast`) | Verbindungen im Raum | -| Alle (`BroadcastToAll`) | **jede** Verbindung, auch unangemeldete | +| Mehrere Benutzer (`BroadcastToUsers`) | alle Verbindungen einer berechneten Benutzergruppe, z. B. Eigentümer und Mitglieder einer Aufgabenliste | ## Ereignisse | Ereignis | Auslöser | Kanal | |---|---|---| -| `TASK_CREATED` | `POST /inbox` | Benutzer | -| `TASK_UPDATED` | `PATCH /thoughts/{id}` (Triage: Benutzer) | alle | -| `TASK_DELETED` | `DELETE /thoughts/{id}` | alle | -| `TASK_REORDERED` | Aufgaben umsortieren | alle | -| `TASK_LIST_UPDATED` / `TASK_LIST_DELETED` | Liste ändern, (un)teilen, löschen | alle | +| `TASK_CREATED` | `POST /inbox`, Webhook | Eigentümer und Benutzer der Liste | +| `TASK_UPDATED` | `PATCH /thoughts/{id}`, Triage | Eigentümer und Benutzer der alten und neuen Liste | +| `TASK_DELETED` | `DELETE /thoughts/{id}` | Eigentümer und Benutzer der Liste | +| `TASK_REORDERED` | Aufgaben umsortieren | Benutzer aller betroffenen Listen | +| `TASK_LIST_UPDATED` / `TASK_LIST_DELETED` | Liste ändern, (un)teilen, löschen; Payload nur `{id}`, Clients laden ihre Listen neu | Benutzer vor und nach der Änderung | | `DOC_CREATED`, `DOC_UPDATED`, `DOC_DELETED` | Notiz anlegen/ändern/löschen, Freigaben | berechtigte Benutzer | | `DOC_BLOCKS_PATCHED` | `PATCH /documents/{id}/blocks` (`docId`, `updatedBy`, `upserted`, `deletedIds`, `blockOrder`) | Raum `doc:` und berechtigte Benutzer | -| `WORKSPACE_FOLDERS_UPDATED` | Ordner/Mounts ändern | Benutzer | +| `WORKSPACE_FOLDERS_UPDATED` | Ordner/Mounts ändern | Benutzer selbst; bei geteilten Ordnern Eigentümer und alle Beteiligten | | `LIST_CREATED`, `LIST_UPDATED`, `LIST_DELETED`, `LIST_SHARED`, `LIST_UNSHARED`, `LIST_CLEARED` | Listenverwaltung | Benutzer bzw. Raum der Liste | | `ITEM_CREATED`, `ITEM_UPDATED`, `ITEM_TOGGLED`, `ITEM_DELETED`, `ITEM_REORDERED` | Listeneinträge | Raum der Liste | | `USER_PRESENCE` | Verbindung auf-/abgebaut (`onlineCount`) | Raum | diff --git a/Doku/flaws.md b/Doku/flaws.md index b7a4112..7a6043d 100644 --- a/Doku/flaws.md +++ b/Doku/flaws.md @@ -10,9 +10,9 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( | ID | Bereich | Kurzbeschreibung | Schwere | |---|---|---|---| -| FLAW-01 | Realtime | `BroadcastToAll` sendet Aufgaben- und Listendaten an alle Verbindungen, auch unauthentifizierte | **Hoch** | -| FLAW-02 | Realtime | Räume (WS `JOIN_ROOM`, SSE `/events/rooms`, `?listId=`) ohne Berechtigungsprüfung | **Hoch** | -| FLAW-03 | Aufgaben | `ReorderTasks` ändert beliebige Aufgaben per ID, ohne Zugriffsprüfung (IDOR) | **Hoch** | +| FLAW-01 | Realtime | ~~`BroadcastToAll` sendet Aufgaben- und Listendaten an alle Verbindungen~~ | **Behoben** | +| FLAW-02 | Realtime | ~~Räume ohne Berechtigungsprüfung~~ | **Behoben** | +| FLAW-03 | Aufgaben | ~~`ReorderTasks` ändert beliebige Aufgaben per ID (IDOR)~~ | **Behoben** | | FLAW-04 | Auth | Fest eincodiertes Fallback-`JWT_SECRET` | **Hoch** | | FLAW-05 | Benachrichtigungen | Discord-/Webhook-URLs ungeprüft (SSRF) | Mittel | | FLAW-06 | Aufgaben | Jeder Editor einer Aufgabenliste kann sie per `teamId` an weitere Gruppen freigeben | Mittel | @@ -24,7 +24,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( | FLAW-12 | Benachrichtigungen | `notify_before_minutes` wirkungslos; Alarm feuert nach Umplanung nie wieder | Mittel | | FLAW-13 | Datenbank | Keine Schema-Versionierung; Migrationsfehler werden verschluckt; `migrations/001_init.sql` ist toter Code | Mittel | | FLAW-14 | Backend | Fehler von Schreiboperationen werden häufig ignoriert (`_, _ =`) | Mittel | -| FLAW-15 | Realtime | `WriteTimeout` von 30 s trifft vermutlich auch SSE-Streams *(nicht live verifiziert)* | Mittel | +| FLAW-15 | Realtime | ~~`WriteTimeout` von 30 s beendet SSE-Streams~~ | **Behoben** | | FLAW-16 | Setup | `/auth/setup` ist bis zum ersten Konto offen; Demo-Modus legt `admin/admin` an | Mittel | | FLAW-17 | Workspace | Dokument- und Ordnerfreigaben sind per Migration erzwungen schreibgeschützt; tote Schreibpfade | Niedrig | | FLAW-18 | Suche | Nur eigene Gedanken und Dokumente, `LIKE` ohne Escaping, keine FTS | Niedrig | @@ -40,16 +40,19 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( ## Details ### FLAW-01: Broadcast an alle Verbindungen (Datenabfluss) +- **Status: Behoben.** `BroadcastToAll` existiert nicht mehr. Aufgabenereignisse gehen an Eigentümer und Benutzer der Liste (`GetTaskListUserIDs`), Listenereignisse tragen nur noch die ID, Ordnerereignisse gehen an Benutzer bzw. Ordnerbeteiligte (`GetSharedFolderUserIDs`). Test: `TestTaskEventsOnlyReachAuthorizedUsers`. - **Fundstellen:** [`internal/ws/hub.go`](../internal/ws/hub.go) `BroadcastToAll`; Aufrufer in [`internal/api/server.go`](../internal/api/server.go): `handleUpdateThought` (`TASK_UPDATED`), `handleDeleteThought` (`TASK_DELETED`), `handleReorderTasks` (`TASK_REORDERED`), `handleUpdateTaskList`, `handleCreateTaskListShare`, `handleDeleteTaskListShare` (`TASK_LIST_UPDATED`), `handleDeleteTaskList`. - **Problem:** `BroadcastToAll` schreibt an `allClients` und `sseClients`. Dazu zählen auch Verbindungen mit `userID = "anonymous"`, denn `handleWebSocket` und `handleSSE` liegen außerhalb der Auth-Gruppe und lassen jede Verbindung zu. Der vollständige Aufgabeninhalt (Titel, Notizen, Unteraufgaben) und die Listen samt Mitgliedern gehen so an jeden, der `/api/v1/events` öffnet. - **Maßnahme:** Ereignisse an die berechtigten Nutzer adressieren (wie bei `DOC_*` über `GetAuthorizedDocUsers`), unauthentifizierte Verbindungen ablehnen. ### FLAW-02: Räume ohne Berechtigungsprüfung +- **Status: Behoben.** `/ws` und `/events` verlangen ein gültiges Token (sonst `401`). Jeder Raumbeitritt läuft über `Hub.SetRoomAuthorizer` → `Server.canJoinRoom` (Notiz bzw. Liste muss lesbar sein). `/events/rooms` liegt hinter der Anmeldung und akzeptiert nur die eigene `sseId`. Tests: `TestRealtimeEndpointsRequireAuth`, `TestSSERoomJoinRequiresAccess`, `TestRoomJoinsRequireAuthorization`. - **Fundstellen:** `Hub.readPump` (`JOIN_ROOM`), `handleSSERooms` (Route `/api/v1/events/rooms` liegt **außerhalb** der Auth-Gruppe), `Hub.Register` (Raum aus `?listId=`/`?spaceId=`). - **Problem:** Wer eine Raum-ID kennt (`doc:`, Listen-ID), tritt bei und erhält Inhalte wie `DOC_BLOCKS_PATCHED` oder `ITEM_*`. Raum-IDs tauchen in Ereignissen auf und sind nicht geheim. Für den SSE-Beitritt reicht die `sseId`, die der Server selbst im Stream mitsendet. - **Maßnahme:** Beitritt nur nach Zugriffsprüfung (`GetDocumentByID`, `GetListByID`); `/events/rooms` hinter `authMiddleware`. ### FLAW-03: `ReorderTasks` ohne Zugriffsprüfung (IDOR) +- **Status: Behoben.** `handleReorderTasks` lehnt die ganze Anfrage ab, sobald eine Aufgabe nicht zugreifbar (`404`) oder schreibgeschützt (`403`) ist. Test: `TestReorderRejectsForeignTasks`. - **Fundstelle:** `Store.ReorderTasks` (`UPDATE thoughts SET sort_order … WHERE id = ?`) und `handleReorderTasks`. - **Problem:** Die SQL-Anweisung ist weder auf `user_id` noch auf die Liste eingeschränkt. Jeder angemeldete Nutzer kann `sortOrder`, `kanbanStatus` und damit `isCompleted` fremder Aufgaben ändern, sofern er die ID kennt. Die Leser-Prüfung für Aufgabenlisten überspringt Aufgaben, auf die der Nutzer gar keinen Zugriff hat (`GetThoughtByID` liefert dann einen Fehler), und schützt davor nicht. - **Maßnahme:** Jede ID gegen `accessibleTaskListIDsQuery` bzw. `user_id` prüfen und bei Verstoß ablehnen. @@ -119,7 +122,8 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( - **Problem:** Scheitert ein Schreibzugriff (z. B. `SQLITE_BUSY`), antwortet die API trotzdem mit Erfolg und broadcastet den Stand, der nie gespeichert wurde. Fehler lassen sich nur schwer nachvollziehen. - **Maßnahme:** Mehrstufige Änderungen in Transaktionen bündeln und Fehler zurückgeben. -### FLAW-15: WriteTimeout und SSE *(nicht live verifiziert)* +### FLAW-15: WriteTimeout und SSE +- **Status: Behoben.** `ServeSSE` hebt die Schreibfrist für die eigene Antwort auf (`http.ResponseController.SetWriteDeadline`). Test mit 1-s-Timeout: `TestSSEStreamOutlivesWriteTimeout`. Der Test hat den Abbruch im alten Code bestätigt. - **Fundstelle:** [`cmd/server/main.go`](../cmd/server/main.go) `WriteTimeout: 30 * time.Second`. - **Problem:** Go beendet nach diesem Zeitraum das Schreiben der Antwort. Für den SSE-Fallback (`ServeSSE`) bedeutet das voraussichtlich einen Verbindungsabbruch alle 30 s. WebSockets sind durch Hijacking nicht betroffen. Der Client verbindet sich neu, daher fällt es kaum auf. - **Maßnahme:** Im SSE-Handler `http.ResponseController.SetWriteDeadline(time.Time{})` setzen oder `WriteTimeout` für diese Route aufheben. diff --git a/Doku/plans/kollaboration-notizen.md b/Doku/plans/kollaboration-notizen.md index 0ce21ed..a1ee83e 100644 --- a/Doku/plans/kollaboration-notizen.md +++ b/Doku/plans/kollaboration-notizen.md @@ -277,8 +277,8 @@ Damit private Notizen keinen unnötigen Aufwand erzeugen, liefert der Server im | Phase | Inhalt | Ergebnis | |---|---|---| -| 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Stream-Timeout (FLAW-15), zentrale `docRole` | Grundlage dicht | -| 1 | Neues Schema, Operations-Log, `/snapshot`, `/sync`, Konfliktregeln, Simulationstests | Server fertig und abgesichert | +| 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Umsortieren (FLAW-03), Stream-Timeout (FLAW-15). **Erledigt.** Die zentrale `docRole` wandert in Phase 1, weil sie auf dem neuen Freigabemodell (`workspace_access`) aufbaut. | Grundlage dicht | +| 1 | Neues Schema inkl. `workspace_access` und zentraler `docRole`, Operations-Log, `/snapshot`, `/sync`, Konfliktregeln, Simulationstests | Server fertig und abgesichert | | 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt (Baum statt `level`) | Bearbeiten funktioniert überall, auch offline | | 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | | 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt | diff --git a/frontend/src/features/triage/TasksView.tsx b/frontend/src/features/triage/TasksView.tsx index 5dd7628..2ee010e 100644 --- a/frontend/src/features/triage/TasksView.tsx +++ b/frontend/src/features/triage/TasksView.tsx @@ -220,6 +220,9 @@ export const TasksView: React.FC = ({ } } else if (msg.event === 'TASK_REORDERED') { loadData(); + } else if (msg.event === 'TASK_LIST_UPDATED' || msg.event === 'TASK_LIST_DELETED') { + // Payload only carries the list ID; reload so permissions reflect this user's view + api.getTaskLists().then((lists) => setTaskLists(Array.isArray(lists) ? lists : [])).catch(() => {}); } }; diff --git a/internal/api/realtime_test.go b/internal/api/realtime_test.go new file mode 100644 index 0000000..51520a0 --- /dev/null +++ b/internal/api/realtime_test.go @@ -0,0 +1,268 @@ +package api + +import ( + "bufio" + "bytes" + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" + + "github.com/sandrodach/mindsift/internal/auth" + "github.com/sandrodach/mindsift/internal/domain" + "github.com/sandrodach/mindsift/internal/storage" +) + +const testJWTSecret = "test-jwt-secret-key" + +type sseMsg struct { + Event string `json:"event"` + Payload json.RawMessage `json:"payload"` +} + +type sseStream struct { + id string + events chan sseMsg +} + +// startRealtimeServer serves the router over real HTTP with a short WriteTimeout, like production but faster. +func startRealtimeServer(t *testing.T) (*Server, *storage.Store, *httptest.Server, *domain.User, string) { + t.Helper() + srv, store, token := setupTestServer(t) + claims, err := auth.ValidateToken(testJWTSecret, token) + if err != nil { + t.Fatalf("token: %v", err) + } + owner, _ := store.GetUserByID(claims.UserID) + ts := httptest.NewUnstartedServer(srv.Router()) + ts.Config.WriteTimeout = time.Second + ts.Start() + t.Cleanup(ts.Close) + return srv, store, ts, owner, token +} + +func newTestUser(t *testing.T, store *storage.Store, email string) (*domain.User, string) { + t.Helper() + u, err := store.CreateUser(email, "hash", strings.Split(email, "@")[0], "member") + if err != nil { + t.Fatalf("create user: %v", err) + } + tok, err := auth.GenerateToken(testJWTSecret, u.ID, u.Email, u.DisplayName, u.TokenVersion) + if err != nil { + t.Fatalf("token: %v", err) + } + return u, tok +} + +func openSSE(t *testing.T, ts *httptest.Server, token string) *sseStream { + t.Helper() + ctx, cancel := context.WithCancel(context.Background()) + req, _ := http.NewRequestWithContext(ctx, http.MethodGet, ts.URL+"/api/v1/events?token="+token, nil) + resp, err := http.DefaultClient.Do(req) + if err != nil { + cancel() + t.Fatalf("open sse: %v", err) + } + if resp.StatusCode != http.StatusOK { + cancel() + t.Fatalf("open sse: status %d", resp.StatusCode) + } + st := &sseStream{events: make(chan sseMsg, 64)} + go func() { + defer close(st.events) + sc := bufio.NewScanner(resp.Body) + for sc.Scan() { + if line, ok := strings.CutPrefix(sc.Text(), "data: "); ok { + var m sseMsg + if json.Unmarshal([]byte(line), &m) == nil { + st.events <- m + } + } + } + }() + t.Cleanup(func() { + cancel() + _ = resp.Body.Close() + }) + connected := st.waitFor(t, "SSE_CONNECTED", 2*time.Second) + if connected == nil { + t.Fatal("no SSE_CONNECTED event") + } + var p struct { + SSEID string `json:"sseId"` + } + _ = json.Unmarshal(connected.Payload, &p) + st.id = p.SSEID + return st +} + +func (st *sseStream) waitFor(t *testing.T, event string, timeout time.Duration) *sseMsg { + t.Helper() + deadline := time.After(timeout) + for { + select { + case m, ok := <-st.events: + if !ok { + return nil + } + if m.Event == event { + return &m + } + case <-deadline: + return nil + } + } +} + +func doJSON(t *testing.T, ts *httptest.Server, method, path, token string, body interface{}) *http.Response { + t.Helper() + var buf bytes.Buffer + if body != nil { + _ = json.NewEncoder(&buf).Encode(body) + } + req, _ := http.NewRequest(method, ts.URL+path, &buf) + req.Header.Set("Authorization", "Bearer "+token) + req.Header.Set("Content-Type", "application/json") + resp, err := http.DefaultClient.Do(req) + if err != nil { + t.Fatalf("%s %s: %v", method, path, err) + } + t.Cleanup(func() { _ = resp.Body.Close() }) + return resp +} + +func TestRealtimeEndpointsRequireAuth(t *testing.T) { + _, _, ts, _, _ := startRealtimeServer(t) + for _, path := range []string{"/api/v1/events", "/api/v1/ws", "/api/v1/events/rooms"} { + method := http.MethodGet + if strings.HasSuffix(path, "/rooms") { + method = http.MethodPost + } + req, _ := http.NewRequest(method, ts.URL+path, strings.NewReader(`{}`)) + resp, err := http.DefaultClient.Do(req) + if err != nil { + t.Fatalf("%s: %v", path, err) + } + _ = resp.Body.Close() + if resp.StatusCode != http.StatusUnauthorized { + t.Errorf("%s without token: want 401, got %d", path, resp.StatusCode) + } + } +} + +func TestSSEStreamOutlivesWriteTimeout(t *testing.T) { + srv, _, ts, owner, token := startRealtimeServer(t) + st := openSSE(t, ts, token) + + time.Sleep(1500 * time.Millisecond) // longer than the server's WriteTimeout of 1s + srv.wsHub.BroadcastToUser(owner.ID, "LATE_EVENT", map[string]string{"ok": "1"}) + + if st.waitFor(t, "LATE_EVENT", 2*time.Second) == nil { + t.Fatal("SSE stream was cut by the WriteTimeout") + } +} + +func TestTaskEventsOnlyReachAuthorizedUsers(t *testing.T) { + _, store, ts, owner, ownerToken := startRealtimeServer(t) + member, memberToken := newTestUser(t, store, "member@example.com") + _, outsiderToken := newTestUser(t, store, "outsider@example.com") + + list, err := store.CreateTaskList(owner.ID, "", "", "Haushalt", "🏠", "#000000") + if err != nil { + t.Fatalf("create list: %v", err) + } + if _, err := store.AddTaskListShare(list.ID, "user", member.ID, "viewer", owner.ID); err != nil { + t.Fatalf("share: %v", err) + } + task, err := store.CreateThoughtWithDetails(owner.ID, "Fenster putzen", "normal", "manual", "", list.ID, "todo", "") + if err != nil { + t.Fatalf("create task: %v", err) + } + + memberStream := openSSE(t, ts, memberToken) + outsiderStream := openSSE(t, ts, outsiderToken) + + resp := doJSON(t, ts, http.MethodPatch, "/api/v1/thoughts/"+task.ID, ownerToken, map[string]string{"content": "Fenster putzen (geheim)"}) + if resp.StatusCode != http.StatusOK { + t.Fatalf("update task: status %d", resp.StatusCode) + } + if memberStream.waitFor(t, "TASK_UPDATED", 2*time.Second) == nil { + t.Error("list member did not receive TASK_UPDATED") + } + if outsiderStream.waitFor(t, "TASK_UPDATED", 500*time.Millisecond) != nil { + t.Error("outsider received TASK_UPDATED of a foreign list") + } + + resp = doJSON(t, ts, http.MethodPut, "/api/v1/task-lists/"+list.ID, ownerToken, map[string]string{"title": "Haushalt 2"}) + if resp.StatusCode != http.StatusOK { + t.Fatalf("update list: status %d", resp.StatusCode) + } + msg := memberStream.waitFor(t, "TASK_LIST_UPDATED", 2*time.Second) + if msg == nil { + t.Fatal("list member did not receive TASK_LIST_UPDATED") + } + if strings.Contains(string(msg.Payload), "permission") || !strings.Contains(string(msg.Payload), list.ID) { + t.Errorf("TASK_LIST_UPDATED must only carry the list ID, got %s", msg.Payload) + } + if outsiderStream.waitFor(t, "TASK_LIST_UPDATED", 500*time.Millisecond) != nil { + t.Error("outsider received TASK_LIST_UPDATED of a foreign list") + } +} + +func TestSSERoomJoinRequiresAccess(t *testing.T) { + _, store, ts, owner, _ := startRealtimeServer(t) + friend, friendToken := newTestUser(t, store, "friend@example.com") + _, strangerToken := newTestUser(t, store, "stranger@example.com") + + list, err := store.CreateList(owner.ID, "Einkauf", "🛒", "shopping") + if err != nil { + t.Fatalf("create list: %v", err) + } + friendStream := openSSE(t, ts, friendToken) + strangerStream := openSSE(t, ts, strangerToken) + + join := func(token, sseID string) int { + resp := doJSON(t, ts, http.MethodPost, "/api/v1/events/rooms", token, map[string]string{"sseId": sseID, "room": list.ID}) + return resp.StatusCode + } + + if code := join(strangerToken, strangerStream.id); code != http.StatusForbidden { + t.Errorf("stranger joining a foreign list room: want 403, got %d", code) + } + if code := join(friendToken, friendStream.id); code != http.StatusForbidden { + t.Errorf("friend joining before being shared: want 403, got %d", code) + } + if _, err := store.AddListShare(owner.ID, list.ID, "user", friend.ID, "viewer"); err != nil { + t.Fatalf("share list: %v", err) + } + if code := join(strangerToken, friendStream.id); code != http.StatusForbidden { + t.Errorf("joining with someone else's sseId: want 403, got %d", code) + } + if code := join(friendToken, friendStream.id); code != http.StatusOK { + t.Errorf("friend joining shared list room: want 200, got %d", code) + } +} + +func TestReorderRejectsForeignTasks(t *testing.T) { + _, store, ts, owner, _ := startRealtimeServer(t) + _, attackerToken := newTestUser(t, store, "attacker@example.com") + + task, err := store.CreateThoughtWithDetails(owner.ID, "Privat", "normal", "manual", "", "", "todo", "") + if err != nil { + t.Fatalf("create task: %v", err) + } + done := "done" + resp := doJSON(t, ts, http.MethodPost, "/api/v1/tasks/reorder", attackerToken, map[string]interface{}{ + "items": []domain.ThoughtReorderItem{{ID: task.ID, SortOrder: 99, KanbanStatus: &done}}, + }) + if resp.StatusCode == http.StatusOK { + t.Fatal("reordering a foreign task must be rejected") + } + after, _ := store.GetThoughtByID(owner.ID, task.ID) + if after.SortOrder == 99 || after.IsCompleted { + t.Fatalf("foreign task was modified: sortOrder=%d completed=%v", after.SortOrder, after.IsCompleted) + } +} diff --git a/internal/api/server.go b/internal/api/server.go index 0fcc16b..0b9c0dc 100644 --- a/internal/api/server.go +++ b/internal/api/server.go @@ -38,10 +38,29 @@ func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS wsHub: wsHub, staticFS: staticFS, } + wsHub.SetRoomAuthorizer(srv.canJoinRoom) go srv.startAlarmTicker() return srv } +// canJoinRoom allows realtime rooms only to users with read access: "doc:" for notes, otherwise a shared list ID. +func (s *Server) canJoinRoom(userID, room string) bool { + if docID, ok := strings.CutPrefix(room, "doc:"); ok { + _, err := s.store.GetDocumentByID(userID, docID) + return err == nil + } + _, err := s.store.GetListByID(userID, room) + return err == nil +} + +// taskAudience returns everyone who can see a task: its owner plus all users of its task list. +func (s *Server) taskAudience(th *domain.Thought) []string { + if th == nil { + return nil + } + return append(s.store.GetTaskListUserIDs(th.TaskListID), th.UserID) +} + func (s *Server) Router() http.Handler { r := chi.NewRouter() @@ -69,13 +88,13 @@ func (s *Server) Router() http.Handler { r.Get("/api/v1/ws/spaces/{spaceId}", s.handleWebSocket) r.Get("/api/v1/ws/lists/{listId}", s.handleWebSocket) r.Get("/api/v1/events", s.handleSSE) - r.Post("/api/v1/events/rooms", s.handleSSERooms) // Authenticated API Routes r.Group(func(r chi.Router) { r.Use(s.authMiddleware) r.Get("/api/v1/auth/me", s.handleMe) + r.Post("/api/v1/events/rooms", s.handleSSERooms) r.Post("/api/v1/auth/logout", s.handleLogout) r.Post("/api/v1/auth/revoke-all", s.handleRevokeAllSessions) r.Patch("/api/v1/auth/settings", s.handleUpdateUserSettings) @@ -699,7 +718,7 @@ func (s *Server) handleCaptureInbox(w http.ResponseWriter, r *http.Request) { } } - s.wsHub.BroadcastTaskEvent(thought.UserID, "TASK_CREATED", thought) + s.wsHub.BroadcastToUsers(s.taskAudience(thought), "TASK_CREATED", thought) writeJSON(w, http.StatusCreated, thought) } @@ -785,7 +804,7 @@ func (s *Server) handleTriageAction(w http.ResponseWriter, r *http.Request) { } } - s.wsHub.BroadcastTaskEvent(updated.UserID, "TASK_UPDATED", updated) + s.wsHub.BroadcastToUsers(s.taskAudience(updated), "TASK_UPDATED", updated) writeJSON(w, http.StatusOK, updated) } @@ -819,11 +838,19 @@ func (s *Server) handleReorderTasks(w http.ResponseWriter, r *http.Request) { writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error()) return } + // ReorderTasks updates by ID only, so every item must be a task the caller may edit + audience := []string{userID} for _, it := range req.Items { - if th, err := s.store.GetThoughtByID(userID, it.ID); err == nil && th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) { + th, err := s.store.GetThoughtByID(userID, it.ID) + if err != nil { + writeError(w, http.StatusNotFound, "Aufgabe nicht gefunden") + return + } + if th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) { writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error()) return } + audience = append(audience, s.taskAudience(th)...) } if err := s.store.ReorderTasks(taskListID, req.Items); err != nil { @@ -831,7 +858,7 @@ func (s *Server) handleReorderTasks(w http.ResponseWriter, r *http.Request) { return } - s.wsHub.BroadcastToAll("TASK_REORDERED", map[string]interface{}{ + s.wsHub.BroadcastToUsers(audience, "TASK_REORDERED", map[string]interface{}{ "taskListId": taskListID, "items": req.Items, "userId": userID, @@ -859,13 +886,14 @@ func (s *Server) handleUpdateThought(w http.ResponseWriter, r *http.Request) { return } + before, _ := s.store.GetThoughtByID(userID, thoughtID) updated, err := s.store.UpdateThoughtDetails(userID, thoughtID, req.Content, req.IsCompleted, req.DueDate, req.Priority, req.TaskListID, req.KanbanStatus, req.Notes, req.Subtasks, req.Tags) if err != nil { writeStoreError(w, http.StatusBadRequest, err) return } s.store.SyncDocBlockAndTask(userID, thoughtID, updated.IsCompleted) - s.wsHub.BroadcastToAll("TASK_UPDATED", updated) + s.wsHub.BroadcastToUsers(append(s.taskAudience(before), s.taskAudience(updated)...), "TASK_UPDATED", updated) writeJSON(w, http.StatusOK, updated) } @@ -879,16 +907,18 @@ func (s *Server) handleUnlinkEnhancedTask(w http.ResponseWriter, r *http.Request func (s *Server) handleDeleteThought(w http.ResponseWriter, r *http.Request) { userID := auth.GetUserID(r.Context()) thoughtID := chi.URLParam(r, "id") - if th, err := s.store.GetThoughtByID(userID, thoughtID); err == nil && th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) { + th, err := s.store.GetThoughtByID(userID, thoughtID) + if err == nil && th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) { writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error()) return } + audience := s.taskAudience(th) s.store.UnlinkTaskFromDocBlock(userID, thoughtID) if err := s.store.DeleteThought(userID, thoughtID); err != nil { writeStoreError(w, http.StatusInternalServerError, err) return } - s.wsHub.BroadcastToAll("TASK_DELETED", map[string]string{"id": thoughtID}) + s.wsHub.BroadcastToUsers(audience, "TASK_DELETED", map[string]string{"id": thoughtID}) writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) } @@ -1096,7 +1126,7 @@ func (s *Server) handleCreateFolder(w http.ResponseWriter, r *http.Request) { writeError(w, http.StatusBadRequest, err.Error()) return } - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": cleanPath}) + s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": cleanPath}) writeJSON(w, http.StatusCreated, map[string]string{"path": cleanPath}) } @@ -1117,7 +1147,7 @@ func (s *Server) handleDeleteFolder(w http.ResponseWriter, r *http.Request) { for _, docID := range deletedDocIDs { s.wsHub.BroadcastDocEvent(docID, "DOC_DELETED", map[string]string{"id": docID}) } - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": folderPath}) + s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": folderPath}) writeJSON(w, http.StatusOK, map[string]interface{}{ "deleted": true, @@ -1146,7 +1176,7 @@ func (s *Server) handleMoveFolder(w http.ResponseWriter, r *http.Request) { s.broadcastDocToAuthorizedUsers(id, "DOC_UPDATED") } } - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{ + s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", map[string]string{ "sourcePath": req.SourcePath, "newPath": newPath, }) @@ -1183,7 +1213,7 @@ func (s *Server) handleCreateSharedFolder(w http.ResponseWriter, r *http.Request writeError(w, http.StatusBadRequest, err.Error()) return } - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", folder) + s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", folder) writeJSON(w, http.StatusCreated, folder) } @@ -1201,11 +1231,12 @@ func (s *Server) handleGetSharedFolderByID(w http.ResponseWriter, r *http.Reques func (s *Server) handleDeleteSharedFolder(w http.ResponseWriter, r *http.Request) { userID := auth.GetUserID(r.Context()) folderID := chi.URLParam(r, "id") + audience := s.store.GetSharedFolderUserIDs(folderID) if err := s.store.DeleteSharedFolder(userID, folderID); err != nil { writeError(w, http.StatusInternalServerError, err.Error()) return } - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID}) + s.wsHub.BroadcastToUsers(append(audience, userID), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID}) writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) } @@ -1273,7 +1304,7 @@ func (s *Server) handleCreateFolderShare(w http.ResponseWriter, r *http.Request) writeError(w, http.StatusBadRequest, err.Error()) return } - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", share) + s.wsHub.BroadcastToUsers(s.store.GetSharedFolderUserIDs(folderID), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID}) if docIDs, err := s.store.GetSharedFolderDocIDs(folderID); err == nil { for _, docID := range docIDs { s.broadcastDocToAuthorizedUsers(docID, "DOC_CREATED") @@ -1287,13 +1318,24 @@ func (s *Server) handleDeleteFolderShare(w http.ResponseWriter, r *http.Request) folderID := chi.URLParam(r, "id") shareID := chi.URLParam(r, "shareId") docIDs, _ := s.store.GetSharedFolderDocIDs(folderID) + audienceBefore := s.store.GetSharedFolderUserIDs(folderID) if err := s.store.DeleteFolderShare(userID, folderID, shareID); err != nil { writeError(w, http.StatusInternalServerError, err.Error()) return } - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID, "shareId": shareID}) + audienceAfter := s.store.GetSharedFolderUserIDs(folderID) + stillAllowed := make(map[string]bool, len(audienceAfter)) + for _, uid := range audienceAfter { + stillAllowed[uid] = true + } + s.wsHub.BroadcastToUsers(append(audienceBefore, audienceAfter...), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID}) for _, docID := range docIDs { - s.wsHub.BroadcastToUser(shareID, "DOC_DELETED", map[string]string{"id": docID}) + // Users who lost access remove the notes; the rest refresh them + for _, uid := range audienceBefore { + if !stillAllowed[uid] { + s.wsHub.BroadcastToUser(uid, "DOC_DELETED", map[string]string{"id": docID}) + } + } s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED") } writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) @@ -1316,7 +1358,8 @@ func (s *Server) handleMoveDocument(w http.ResponseWriter, r *http.Request) { } s.wsHub.BroadcastDocEvent(docID, "DOC_UPDATED", doc) s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED") - s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", nil) + docUsers, _ := s.store.GetAuthorizedDocUsers(docID) + s.wsHub.BroadcastToUsers(append(docUsers, userID), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"docId": docID}) writeJSON(w, http.StatusOK, doc) } @@ -1751,7 +1794,8 @@ func (s *Server) handleWebSocket(w http.ResponseWriter, r *http.Request) { userID := s.authenticateUserFromReq(r) if userID == "" { - userID = "anonymous" + writeError(w, http.StatusUnauthorized, "Nicht autorisiert – bitte anmelden") + return } s.wsHub.ServeWS(w, r, roomID, userID) } @@ -1764,7 +1808,8 @@ func (s *Server) handleSSE(w http.ResponseWriter, r *http.Request) { userID := s.authenticateUserFromReq(r) if userID == "" { - userID = "anonymous" + writeError(w, http.StatusUnauthorized, "Nicht autorisiert – bitte anmelden") + return } var initialRooms []string @@ -1784,6 +1829,7 @@ func (s *Server) handleSSE(w http.ResponseWriter, r *http.Request) { } func (s *Server) handleSSERooms(w http.ResponseWriter, r *http.Request) { + userID := auth.GetUserID(r.Context()) var req struct { SSEID string `json:"sseId"` Room string `json:"room"` @@ -1803,9 +1849,13 @@ func (s *Server) handleSSERooms(w http.ResponseWriter, r *http.Request) { } if req.Action == "leave" { - s.wsHub.LeaveSSERoomByID(req.SSEID, req.Room) - } else { - s.wsHub.JoinSSERoomByID(req.SSEID, req.Room) + s.wsHub.LeaveSSERoomByID(req.SSEID, userID, req.Room) + writeJSON(w, http.StatusOK, map[string]bool{"success": true}) + return + } + if !s.wsHub.JoinSSERoomByID(req.SSEID, userID, req.Room) { + writeError(w, http.StatusForbidden, "Kein Zugriff auf diesen Raum") + return } writeJSON(w, http.StatusOK, map[string]bool{"success": true}) } @@ -1943,7 +1993,7 @@ func (s *Server) handleWebhookIngest(w http.ResponseWriter, r *http.Request) { return } - s.wsHub.BroadcastTaskEvent(thought.UserID, "TASK_CREATED", thought) + s.wsHub.BroadcastToUsers(s.taskAudience(thought), "TASK_CREATED", thought) writeJSON(w, http.StatusCreated, map[string]interface{}{ "status": "ingested", @@ -2180,23 +2230,25 @@ func (s *Server) handleUpdateTaskList(w http.ResponseWriter, r *http.Request) { writeError(w, http.StatusBadRequest, "Ungültiges JSON") return } + audienceBefore := s.store.GetTaskListUserIDs(listID) tl, err := s.store.UpdateTaskList(userID, listID, req.Title, req.Icon, req.Color, req.TeamID, req.SpaceID, req.Columns) if err != nil { writeStoreError(w, http.StatusBadRequest, err) return } - s.wsHub.BroadcastToAll("TASK_LIST_UPDATED", tl) + s.broadcastTaskListChange(listID, audienceBefore, "TASK_LIST_UPDATED") writeJSON(w, http.StatusOK, tl) } func (s *Server) handleDeleteTaskList(w http.ResponseWriter, r *http.Request) { userID := auth.GetUserID(r.Context()) listID := chi.URLParam(r, "id") + audienceBefore := s.store.GetTaskListUserIDs(listID) if err := s.store.DeleteTaskList(userID, listID); err != nil { writeError(w, http.StatusInternalServerError, err.Error()) return } - s.wsHub.BroadcastToAll("TASK_LIST_DELETED", map[string]string{"id": listID}) + s.broadcastTaskListChange(listID, audienceBefore, "TASK_LIST_DELETED") writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) } @@ -2245,8 +2297,7 @@ func (s *Server) handleCreateTaskListShare(w http.ResponseWriter, r *http.Reques writeError(w, http.StatusBadRequest, err.Error()) return } - updatedList, _ := s.store.GetTaskListByID(userID, listID) - s.wsHub.BroadcastToAll("TASK_LIST_UPDATED", updatedList) + s.broadcastTaskListChange(listID, nil, "TASK_LIST_UPDATED") writeJSON(w, http.StatusCreated, share) } @@ -2263,15 +2314,22 @@ func (s *Server) handleDeleteTaskListShare(w http.ResponseWriter, r *http.Reques writeError(w, http.StatusForbidden, "Nur der Eigentümer kann Freigaben entfernen") return } + audienceBefore := s.store.GetTaskListUserIDs(listID) if err := s.store.RemoveTaskListShare(listID, shareID); err != nil { writeError(w, http.StatusInternalServerError, err.Error()) return } - updatedList, _ := s.store.GetTaskListByID(userID, listID) - s.wsHub.BroadcastToAll("TASK_LIST_UPDATED", updatedList) + s.broadcastTaskListChange(listID, audienceBefore, "TASK_LIST_UPDATED") writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) } +// broadcastTaskListChange notifies everyone who could see the list before or can see it now. Only the ID is sent, +// because the list payload depends on the receiver's permission; clients reload their lists. +func (s *Server) broadcastTaskListChange(listID string, audienceBefore []string, event string) { + users := append(audienceBefore, s.store.GetTaskListUserIDs(listID)...) + s.wsHub.BroadcastToUsers(users, event, map[string]string{"id": listID}) +} + // --- ORGANIZATIONS & TEAMS --- func (s *Server) handleGetOrganizations(w http.ResponseWriter, r *http.Request) { diff --git a/internal/storage/sqlite.go b/internal/storage/sqlite.go index 3ac0158..12cf1c6 100644 --- a/internal/storage/sqlite.go +++ b/internal/storage/sqlite.go @@ -4747,6 +4747,55 @@ func (s *Store) GetTaskListShares(taskListID string) ([]domain.TaskListShare, er return shares, nil } +// queryUserIDs runs a query returning a single user_id column and collects the distinct IDs. +func (s *Store) queryUserIDs(query string, args ...interface{}) []string { + rows, err := s.db.Query(query, args...) + if err != nil { + return []string{} + } + defer rows.Close() + ids := make([]string, 0) + seen := make(map[string]bool) + for rows.Next() { + var id string + if rows.Scan(&id) == nil && id != "" && !seen[id] { + seen[id] = true + ids = append(ids, id) + } + } + return ids +} + +// GetTaskListUserIDs returns every user who can see the task list: owner, direct shares, members of shared teams +// and members of the legacy team_id. +func (s *Store) GetTaskListUserIDs(listID string) []string { + if listID == "" { + return []string{} + } + return s.queryUserIDs( + `SELECT user_id FROM task_lists WHERE id = ? + UNION SELECT target_id FROM task_list_shares WHERE task_list_id = ? AND target_type = 'user' + UNION SELECT tm.user_id FROM team_members tm + WHERE tm.team_id IN (SELECT target_id FROM task_list_shares WHERE task_list_id = ? AND target_type = 'team') + OR tm.team_id IN (SELECT team_id FROM task_lists WHERE id = ? AND team_id != '')`, + listID, listID, listID, listID, + ) +} + +// GetSharedFolderUserIDs returns the owner of a shared workspace folder and everyone it is shared with. +func (s *Store) GetSharedFolderUserIDs(folderID string) []string { + if folderID == "" { + return []string{} + } + return s.queryUserIDs( + `SELECT owner_id FROM workspace_shared_folders WHERE id = ? + UNION SELECT target_id FROM workspace_folder_shares WHERE folder_id = ? AND target_type = 'user' + UNION SELECT tm.user_id FROM team_members tm + WHERE tm.team_id IN (SELECT target_id FROM workspace_folder_shares WHERE folder_id = ? AND target_type = 'team')`, + folderID, folderID, folderID, + ) +} + func (s *Store) getMembersForTaskList(tl *domain.TaskList) []domain.TaskListMember { memberMap := make(map[string]domain.TaskListMember) diff --git a/internal/ws/hub.go b/internal/ws/hub.go index 533815d..5f7ac91 100644 --- a/internal/ws/hub.go +++ b/internal/ws/hub.go @@ -93,6 +93,27 @@ type Hub struct { sseUserRooms map[string]map[*SSEClient]bool sseRooms map[string]map[*SSEClient]bool closed bool + + // authorizeRoom decides whether a user may receive events of a room; without it every join is denied + authorizeRoom func(userID, roomID string) bool +} + +// SetRoomAuthorizer installs the permission check used for every room join (URL room, JOIN_ROOM, SSE rooms). +func (h *Hub) SetRoomAuthorizer(f func(userID, roomID string) bool) { + h.mu.Lock() + h.authorizeRoom = f + h.mu.Unlock() +} + +// canJoin must be called without holding h.mu, since the authorizer queries the database. +func (h *Hub) canJoin(userID, roomID string) bool { + if roomID == "" || userID == "" || userID == "anonymous" { + return false + } + h.mu.RLock() + f := h.authorizeRoom + h.mu.RUnlock() + return f != nil && f(userID, roomID) } func NewHub() *Hub { @@ -107,6 +128,9 @@ func NewHub() *Hub { } func (h *Hub) Register(c *Client) { + if c.roomID != "" && !h.canJoin(c.userID, c.roomID) { + c.roomID = "" + } h.mu.Lock() h.allClients[c] = true if c.roomID != "" { @@ -167,6 +191,14 @@ func (h *Hub) Unregister(c *Client) { } func (h *Hub) RegisterSSE(c *SSEClient) { + if c.roomID != "" && !h.canJoin(c.userID, c.roomID) { + c.roomID = "" + } + for rID := range c.subscribedRooms { + if !h.canJoin(c.userID, rID) { + delete(c.subscribedRooms, rID) + } + } h.mu.Lock() h.sseClients[c] = true if c.roomID != "" { @@ -232,11 +264,15 @@ func (h *Hub) UnregisterSSE(c *SSEClient) { h.mu.Unlock() } -func (h *Hub) JoinSSERoomByID(sseID, roomID string) bool { +// JoinSSERoomByID adds the SSE connection sseID to a room; it only succeeds for the user owning that connection. +func (h *Hub) JoinSSERoomByID(sseID, userID, roomID string) bool { + if !h.canJoin(userID, roomID) { + return false + } h.mu.Lock() defer h.mu.Unlock() for client := range h.sseClients { - if client.id == sseID { + if client.id == sseID && client.userID == userID { if h.sseRooms[roomID] == nil { h.sseRooms[roomID] = make(map[*SSEClient]bool) } @@ -251,11 +287,11 @@ func (h *Hub) JoinSSERoomByID(sseID, roomID string) bool { return false } -func (h *Hub) LeaveSSERoomByID(sseID, roomID string) bool { +func (h *Hub) LeaveSSERoomByID(sseID, userID, roomID string) bool { h.mu.Lock() defer h.mu.Unlock() for client := range h.sseClients { - if client.id == sseID { + if client.id == sseID && client.userID == userID { if room, ok := h.sseRooms[roomID]; ok { delete(room, client) if len(room) == 0 { @@ -305,32 +341,6 @@ func (h *Hub) Close() { // Broadcasting Methods -func (h *Hub) BroadcastToAll(event string, payload interface{}) { - msg := domain.WSMessage{ - Event: event, - Payload: payload, - Timestamp: domain.NowISO(), - } - data, err := json.Marshal(msg) - if err != nil { - return - } - - h.mu.RLock() - if h.closed { - h.mu.RUnlock() - return - } - defer h.mu.RUnlock() - - for client := range h.allClients { - client.trySend(data) - } - for sseClient := range h.sseClients { - sseClient.trySend(data) - } -} - func (h *Hub) BroadcastToUser(userID, event string, payload interface{}) { msg := domain.WSMessage{ Event: event, @@ -361,15 +371,21 @@ func (h *Hub) BroadcastToUser(userID, event string, payload interface{}) { } } -func (h *Hub) BroadcastTaskEvent(userID, event string, payload interface{}) { - h.BroadcastToUser(userID, event, payload) +// BroadcastToUsers sends one event to every connection of the given users (duplicates are ignored). +func (h *Hub) BroadcastToUsers(userIDs []string, event string, payload interface{}) { + seen := make(map[string]bool, len(userIDs)) + for _, uid := range userIDs { + if uid == "" || seen[uid] { + continue + } + seen[uid] = true + h.BroadcastToUser(uid, event, payload) + } } func (h *Hub) BroadcastDocEvent(docID, event string, payload interface{}) { if docID != "" { h.BroadcastRoom("doc:"+docID, event, payload, nil) - } else { - h.BroadcastToAll(event, payload) } } @@ -413,7 +429,10 @@ func (h *Hub) BroadcastListEvent(listID, event string, payload interface{}) { h.Broadcast("", listID, event, payload) } -func (h *Hub) JoinRoom(c *Client, roomID string) { +func (h *Hub) JoinRoom(c *Client, roomID string) bool { + if !h.canJoin(c.userID, roomID) { + return false + } h.mu.Lock() defer h.mu.Unlock() if h.rooms[roomID] == nil { @@ -424,6 +443,7 @@ func (h *Hub) JoinRoom(c *Client, roomID string) { c.subscribedRooms = make(map[string]bool) } c.subscribedRooms[roomID] = true + return true } func (h *Hub) LeaveRoom(c *Client, roomID string) { @@ -512,11 +532,13 @@ func (h *Hub) ServeSSE(w http.ResponseWriter, r *http.Request, roomID, userID st return } + // The server's WriteTimeout would cut long-lived streams; lift it for this response only + _ = http.NewResponseController(w).SetWriteDeadline(time.Time{}) + w.Header().Set("Content-Type", "text/event-stream") w.Header().Set("Cache-Control", "no-cache, no-transform") w.Header().Set("Connection", "keep-alive") w.Header().Set("X-Accel-Buffering", "no") - w.Header().Set("Access-Control-Allow-Origin", "*") client := &SSEClient{ id: domain.NewID("sse"), @@ -591,7 +613,7 @@ func (c *Client) readPump() { break } - // Handle client-initiated lock, CRDT, room subscription, and presence messages + // Handle client-initiated room subscription messages var incoming struct { Event string `json:"event"` DocID string `json:"docId"` diff --git a/internal/ws/hub_test.go b/internal/ws/hub_test.go index eb47855..e107485 100644 --- a/internal/ws/hub_test.go +++ b/internal/ws/hub_test.go @@ -22,7 +22,7 @@ func TestHubCloseAndGracefulShutdown(t *testing.T) { } // Verify broadcast works - hub.BroadcastToAll("TEST_EVENT", map[string]string{"msg": "hello"}) + hub.BroadcastToUser("user-1", "TEST_EVENT", map[string]string{"msg": "hello"}) // Concurrently invoke Close() and Broadcast() to verify no panics or deadlocks var wg sync.WaitGroup @@ -31,7 +31,7 @@ func TestHubCloseAndGracefulShutdown(t *testing.T) { go func() { defer wg.Done() for i := 0; i < 50; i++ { - hub.BroadcastToAll("SPAM", map[string]int{"i": i}) + hub.BroadcastToUser("user-1", "SPAM", map[string]int{"i": i}) time.Sleep(1 * time.Millisecond) } }() @@ -56,3 +56,37 @@ func TestHubCloseAndGracefulShutdown(t *testing.T) { } } } + +func TestRoomJoinsRequireAuthorization(t *testing.T) { + hub := NewHub() + c := &Client{hub: hub, userID: "user-1", send: make(chan []byte, 10)} + + if hub.JoinRoom(c, "lst_1") { + t.Fatal("join must be denied without an authorizer") + } + + hub.SetRoomAuthorizer(func(userID, roomID string) bool { + return userID == "user-1" && roomID == "lst_allowed" + }) + if hub.JoinRoom(c, "lst_other") { + t.Fatal("join to an unauthorized room must be denied") + } + if !hub.JoinRoom(c, "lst_allowed") { + t.Fatal("join to an authorized room must succeed") + } + + anon := &Client{hub: hub, userID: "anonymous", send: make(chan []byte, 10)} + if hub.JoinRoom(anon, "lst_allowed") { + t.Fatal("anonymous clients must never join rooms") + } + + sse := &SSEClient{id: "sse_1", userID: "user-1", send: make(chan []byte, 10), + subscribedRooms: map[string]bool{"lst_allowed": true, "doc:secret": true}} + hub.RegisterSSE(sse) + if sse.subscribedRooms["doc:secret"] { + t.Fatal("unauthorized initial SSE rooms must be dropped") + } + if hub.JoinSSERoomByID("sse_1", "user-2", "lst_allowed") { + t.Fatal("joining with another user's sseId must fail") + } +} From 1ea0c78b4108ef2dfcd4b775841f7600b2c4812b Mon Sep 17 00:00:00 2001 From: Sandro Date: Wed, 28 Oct 2026 11:47:56 +0100 Subject: [PATCH 04/11] feat(notes): operation log, block leases and sync API for collaborative editing Phase 1 of the collaboration plan (server side): - internal/collab: block tree with parent IDs and fractional order keys, operations with conflict rules (version check, block leases, conflict copies instead of lost text, restore on late edits, cycle checks), in-memory leases/sessions and long-poll notifier. - Storage: per-note lock + transaction, append-only effect log (doc_ops), idempotent results per opId (doc_op_results), snapshot, log paging, central DocRole. Old flat block data is converted to the tree. - The existing whole-document save (PUT/PATCH) runs as an adapter through the same engine, so the current editor keeps working and writes to the log. - API: GET /docs/{id}/snapshot, POST /docs/{id}/sync, GET /docs/{id}/changes (long poll), POST /docs/{id}/leave; DOC_SEQ poke for WebSocket/SSE rooms. - Backend accepts editor (write) shares for notes and folders; the forced read-only migration is gone. The share dialog still offers read only. - Tests: conflict matrix, order keys, leases, storage adapter round trip, idempotency, API protocol incl. long poll, and a randomized simulation (4 clients, lost/duplicated requests and responses, offline phases, lease expiry) checking convergence, no lost text and exactly-once results. Co-Authored-By: Claude Opus 5.5 --- Doku/api/rest.md | 41 +- Doku/api/websocket.md | 1 + Doku/architecture.md | 1 + Doku/database.md | 8 +- Doku/flaws.md | 6 +- Doku/modules/workspace.md | 2 +- Doku/plans/kollaboration-notizen.md | 2 +- Doku/sharing-and-permissions.md | 6 +- internal/api/docsync.go | 228 +++++++++ internal/api/docsync_test.go | 198 ++++++++ internal/api/server.go | 15 +- internal/collab/doc.go | 201 ++++++++ internal/collab/lease.go | 197 ++++++++ internal/collab/notify.go | 34 ++ internal/collab/ops.go | 396 ++++++++++++++++ internal/collab/ops_test.go | 263 +++++++++++ internal/collab/orderkey.go | 267 +++++++++++ internal/collab/orderkey_test.go | 90 ++++ internal/storage/docsync.go | 663 +++++++++++++++++++++++++++ internal/storage/docsync_sim_test.go | 402 ++++++++++++++++ internal/storage/docsync_test.go | 186 ++++++++ internal/storage/sqlite.go | 344 +++----------- internal/storage/sqlite_test.go | 2 +- 23 files changed, 3266 insertions(+), 287 deletions(-) create mode 100644 internal/api/docsync.go create mode 100644 internal/api/docsync_test.go create mode 100644 internal/collab/doc.go create mode 100644 internal/collab/lease.go create mode 100644 internal/collab/notify.go create mode 100644 internal/collab/ops.go create mode 100644 internal/collab/ops_test.go create mode 100644 internal/collab/orderkey.go create mode 100644 internal/collab/orderkey_test.go create mode 100644 internal/storage/docsync.go create mode 100644 internal/storage/docsync_sim_test.go create mode 100644 internal/storage/docsync_test.go diff --git a/Doku/api/rest.md b/Doku/api/rest.md index db53683..e1bcb8a 100644 --- a/Doku/api/rest.md +++ b/Doku/api/rest.md @@ -67,17 +67,50 @@ Schreibzugriffe von Lesern auf Listen oder deren Aufgaben liefern `403` („Kein | GET/POST | `/workspace/folders/shared` | Geteilte Ordner; anlegen (`name`, `mountPath`) | | GET/DELETE | `/workspace/folders/shared/{id}` | Abruf; Löschen | | PUT | `/workspace/folders/shared/{id}/mount` | Eigenen Einhängepfad ändern (`mountPath`) | -| GET/POST/DELETE | `/workspace/folders/shared/{id}/shares[/{shareId}]` | Freigaben (nur Lesen wirksam) | +| GET/POST/DELETE | `/workspace/folders/shared/{id}/shares[/{shareId}]` | Freigaben (`permission`: `read`/`write`) | | GET | `/workspace/tags`, `/workspace/tags/{tag}` | Tag-Übersicht und -Details | | PUT | `/workspace/tags/{tag}/note` | Tag-Seite speichern (`content`) | | GET/POST | `/documents` | Notizen auflisten; anlegen (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`) | | GET/PUT/DELETE | `/documents/{id}` | Abruf; Ändern (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`); Löschen | -| PATCH | `/documents/{id}/blocks` | Teil-Update der Blöcke (`upserted`, `deletedIds`, Reihenfolge); löst `DOC_BLOCKS_PATCHED` aus | +| PATCH | `/documents/{id}/blocks` | Teil-Update der Blöcke (`upserted`, `deletedIds`, `blockOrder`); läuft intern über das Änderungs-Log; löst `DOC_BLOCKS_PATCHED` aus | | PUT | `/documents/{id}/move` | Ordner wechseln (`folderPath`) | | POST | `/documents/{id}/blocks/{blockId}/enhanced-task` | Aus einem Block eine verknüpfte Aufgabe machen (`content`, `checked`) | -| GET/POST/DELETE | `/documents/{id}/shares[/{userId}]` | Notiz-Freigaben (`userId`/`email`/`teamId`; wirksam nur Lesen) | +| GET/POST/DELETE | `/documents/{id}/shares[/{userId}]` | Notiz-Freigaben (`userId`/`email`/`teamId`, `permission`: `read`/`write`) | -Nur der Eigentümer schreibt in eine Notiz; Empfänger erhalten `403`/„nur Lesezugriff“ ([FLAW-17](../flaws.md)). +Die Endpunkte `PUT /documents/{id}` und `PATCH /documents/{id}/blocks` sind der bisherige Speicherweg (ganzes Dokument). Sie werden intern in Operationen umgerechnet und landen im selben Log wie `/sync`; der letzte Schreiber gewinnt. Sie bleiben, bis der Editor auf `/sync` umgestellt ist. + +## Gemeinsames Bearbeiten von Notizen + +Grundlage: [plans/kollaboration-notizen.md](../plans/kollaboration-notizen.md). Zugriff über die Rolle `owner`/`editor`/`viewer` (`Store.DocRole`); ohne Rolle `404`. + +| Methode | Pfad | Zweck | +|---|---|---| +| GET | `/docs/{id}/snapshot` | Kompletter Blockbaum inkl. Grabsteinen, `title`, `titleVersion`, `headSeq`, `role`, `shared` | +| POST | `/docs/{id}/sync` | Ein Rundlauf: eigene Operationen senden, Leases halten/abgeben, Log ab `lastSeq` abholen | +| GET | `/docs/{id}/changes?since=N&wait=S` | Long-Polling: antwortet sofort, wenn `headSeq > N`, sonst bei der nächsten Änderung (auch Leases/Präsenz) oder nach höchstens 25 s. Antwort `{headSeq, changed}`; Daten holt der Client danach per `/sync` | +| POST | `/docs/{id}/leave` | Sitzung beenden, alle Leases des Clients freigeben (`{clientId}`) | + +`/sync`-Anfrage: + +```json +{ + "clientId": "tab-7f3…", + "lastSeq": 412, + "ops": [ + { "opId": "c1a…", "kind": "set_content", "blockId": "b9…", "baseVersion": 7, "content": "Milch und Brot" }, + { "opId": "c1b…", "kind": "insert", "blockId": "bA…", "parentId": "b2…", "orderKey": "a0V", "type": "bullet", "content": "" } + ], + "leases": { "hold": ["b9…", "bA…"], "release": [] } +} +``` + +- Operationsarten: `insert`, `set_content`, `set_props` (`patch`: `type`, `checked`, `collapsed`, `lang`), `move`, `delete`, `restore` (`ids`), `set_title` (`baseVersion`, `title`). +- `leases.hold` ist die vollständige Liste der Blöcke, die der Client halten will; alle anderen Leases des Clients werden freigegeben. Neu eingefügte Blöcke und Konfliktkopien bekommt der Autor automatisch. +- Höchstens 500 Operationen pro Anfrage. Leser dürfen keine Operationen senden (`403`), aber abfragen. + +Antwort: `results` (je Operation `status` `applied`/`conflict`/`rejected`, ggf. `conflictBlockId`, `version`, `seq`), `ops` (Log-Einträge `{seq, opId, userId, clientId, effect}` ab `lastSeq`, höchstens 1000; `hasMore` bei mehr), `headSeq`, `resync` (Client ist dem Server voraus, Snapshot neu laden), `role`, `shared`, `sessions`, `presence`, `leases` (`granted`, `denied`, `others`). + +Wiederholte Operationen (gleiche `opId`) werden nicht erneut angewendet; die Antwort enthält das gespeicherte Ergebnis. ## Geteilte Listen diff --git a/Doku/api/websocket.md b/Doku/api/websocket.md index 16a88ea..e905102 100644 --- a/Doku/api/websocket.md +++ b/Doku/api/websocket.md @@ -43,6 +43,7 @@ Client → Server versteht der Server nur: `{"event":"JOIN_ROOM","room":""}` | `TASK_REORDERED` | Aufgaben umsortieren | Benutzer aller betroffenen Listen | | `TASK_LIST_UPDATED` / `TASK_LIST_DELETED` | Liste ändern, (un)teilen, löschen; Payload nur `{id}`, Clients laden ihre Listen neu | Benutzer vor und nach der Änderung | | `DOC_CREATED`, `DOC_UPDATED`, `DOC_DELETED` | Notiz anlegen/ändern/löschen, Freigaben | berechtigte Benutzer | +| `DOC_SEQ` | Änderung an Notiz, Leases oder Präsenz (`docId`, `headSeq`); Clients holen die Daten per `/docs/{id}/sync` | Raum `doc:` | | `DOC_BLOCKS_PATCHED` | `PATCH /documents/{id}/blocks` (`docId`, `updatedBy`, `upserted`, `deletedIds`, `blockOrder`) | Raum `doc:` und berechtigte Benutzer | | `WORKSPACE_FOLDERS_UPDATED` | Ordner/Mounts ändern | Benutzer selbst; bei geteilten Ordnern Eigentümer und alle Beteiligten | | `LIST_CREATED`, `LIST_UPDATED`, `LIST_DELETED`, `LIST_SHARED`, `LIST_UNSHARED`, `LIST_CLEARED` | Listenverwaltung | Benutzer bzw. Raum der Liste | diff --git a/Doku/architecture.md b/Doku/architecture.md index e084702..4f5a235 100644 --- a/Doku/architecture.md +++ b/Doku/architecture.md @@ -34,6 +34,7 @@ Das Frontend liegt per `go:embed` im Binary ([cmd/server/main.go](../cmd/server/ | `internal/storage/` | `sqlite.go` (Schema, Migrationen, gesamte Datenlogik) | | `internal/domain/` | Strukturen (`models.go`), ID-Erzeugung, Tag-Extraktion | | `internal/ws/` | `hub.go` | +| `internal/collab/` | Kern der Notiz-Synchronisation ohne Datenbank: Blockbaum, Operationen und Konfliktregeln, Sortierschlüssel, Leases, Long-Poll-Benachrichtigung | | `internal/auth/` | Tokens und Passwörter | | `internal/config/` | Umgebungsvariablen | | `frontend/src/features/` | `triage`, `workspace`, `lists`, `teams`, `settings`, `capture`, `auth`, `help` | diff --git a/Doku/database.md b/Doku/database.md index eaccf8c..5c6f6cc 100644 --- a/Doku/database.md +++ b/Doku/database.md @@ -28,11 +28,13 @@ Eine Aufgabe ist ein `thought` mit `status='kept'` oder `status='converted' AND | Tabelle | Zweck | |---|---| | `documents` | Notizen: `title`, `icon`, `folder_path`, `tags` (JSON), `is_archived`, `markdown_cache`, `shared_folder_id`, `sub_folder_path`, Altspalten `content`, `blocks_json` | -| `blocks` | Outliner-Blöcke: `document_id`, `type`, `content`, `level`, `position` (REAL), `checked`, `collapsed`, `lang` | +| `blocks` | Outliner-Blöcke als Baum: `document_id`, `parent_id` (`''` = oberste Ebene), `order_key` (Sortierschlüssel unter Geschwistern), `type`, `content`, `checked`, `collapsed`, `lang`, `version` (Textversion), `conflict_of`/`conflict_author` (Konfliktkopien), `updated_by`, `deleted` (Grabstein). `level`/`position` sind Altspalten und werden nicht mehr gepflegt; Ebene und Reihenfolge berechnet der Server beim Lesen aus dem Baum. | +| `doc_ops` | Änderungs-Log je Notiz: `seq` (fortlaufend pro Notiz), `op_id`, `user_id`, `client_id`, `kind`, `block_id`, `payload` (Effekt als JSON). Abspielen aller Einträge ergibt den Stand. | +| `doc_op_results` | Ergebnis je Client-Operation (`op_id` eindeutig) für idempotente Wiederholungen | | `block_tags` | `#Hashtags` je Block (`tag`, `document_id`, `block_id`) | | `tag_pages` | Freitext-Notiz je Benutzer und Tag | | `doc_pins` | Favoriten je Benutzer | -| `doc_shares`, `doc_team_shares` | Notiz-Freigaben an Benutzer bzw. Gruppen (`permission` effektiv immer `read`) | +| `doc_shares`, `doc_team_shares` | Notiz-Freigaben an Benutzer bzw. Gruppen (`permission`: `read` = Leser, `write` = Bearbeiter) | | `workspace_folders` | Explizit angelegte Ordnerpfade | | `workspace_shared_folders` | Geteilte Ordner (Entität mit Eigentümer) | | `workspace_folder_shares` | Freigaben geteilter Ordner | @@ -74,7 +76,7 @@ teams 1─n team_members n─1 users - Spalten/Tabellen/Indizes werden nachgezogen (`ALTER TABLE … ADD COLUMN`, Fehler werden ignoriert). - Veraltete Tabellen werden gelöscht: `spaces`, `space_members`, `space_team_shares`, `document_crdt_state`, `document_crdt_updates`, `document_blocks`, `doc_block_tags`. - `users.username` wird aus dem E-Mail-Präfix aufgefüllt. -- Alle Dokument-, Team- und Ordnerfreigaben werden auf `read` gesetzt (bei **jedem** Start, siehe [FLAW-17](flaws.md)). +- Notiz-Synchronisation (`migrateDocSync`): neue Spalten an `blocks` und `documents` (`head_seq`, `title_version`), Tabellen `doc_ops` und `doc_op_results`. Blöcke im alten flachen Format (nur `level`/`position`) werden einmalig in den Baum umgewandelt. - Standardwerte in `system_settings` (`INSERT OR IGNORE`): `sharing_mode=teams_only`, `allow_registration=false`, `instance_name=MindSift`, SMTP leer. - Die Tabelle `teams` wird einmalig ohne Fremdschlüssel auf `organizations` neu aufgebaut. diff --git a/Doku/flaws.md b/Doku/flaws.md index 7a6043d..f826cef 100644 --- a/Doku/flaws.md +++ b/Doku/flaws.md @@ -20,13 +20,13 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( | FLAW-08 | Auth | Webhook-Ingest mit JWT prüft weder Sperrliste noch `token_version` noch `is_active` | Mittel | | FLAW-09 | Auth | Kein Rate-Limit, keine Sperre bei Fehlversuchen, Mindestpasswort 4 Zeichen | Mittel | | FLAW-10 | Freigaben | `sharing_mode` wird bei Freigaben nicht durchgesetzt; Benutzerverzeichnis leakt alle Konten | Mittel | -| FLAW-11 | Workspace | Block-Upsert (`ON CONFLICT(id)`) prüft das Dokument nicht | Mittel | +| FLAW-11 | Workspace | ~~Block-Upsert (`ON CONFLICT(id)`) prüft das Dokument nicht~~ | **Behoben** | | FLAW-12 | Benachrichtigungen | `notify_before_minutes` wirkungslos; Alarm feuert nach Umplanung nie wieder | Mittel | | FLAW-13 | Datenbank | Keine Schema-Versionierung; Migrationsfehler werden verschluckt; `migrations/001_init.sql` ist toter Code | Mittel | | FLAW-14 | Backend | Fehler von Schreiboperationen werden häufig ignoriert (`_, _ =`) | Mittel | | FLAW-15 | Realtime | ~~`WriteTimeout` von 30 s beendet SSE-Streams~~ | **Behoben** | | FLAW-16 | Setup | `/auth/setup` ist bis zum ersten Konto offen; Demo-Modus legt `admin/admin` an | Mittel | -| FLAW-17 | Workspace | Dokument- und Ordnerfreigaben sind per Migration erzwungen schreibgeschützt; tote Schreibpfade | Niedrig | +| FLAW-17 | Workspace | Bearbeiter-Freigaben für Notizen erst im Backend aktiv, nicht im UI | Niedrig | | FLAW-18 | Suche | Nur eigene Gedanken und Dokumente, `LIKE` ohne Escaping, keine FTS | Niedrig | | FLAW-19 | Qualität | Frontend-Tests nur für den Outliner, `lint` ist ein Platzhalter | Niedrig | | FLAW-20 | Qualität | Sehr große Dateien, duplizierte Zugriffslogik | Niedrig | @@ -96,6 +96,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( - **Maßnahme:** Eine zentrale Funktion `canShareWith(owner, target)` und in allen Share-Handlern verwenden; Verzeichnisse darauf einschränken. ### FLAW-11: Block-Upsert prüft das Dokument nicht +- **Status: Behoben.** Alle Blockänderungen laufen über `Store.applyDocChange`: Einfügen mit einer Block-ID aus einer anderen Notiz wird abgelehnt (`exists`), und der Upsert ändert nur Zeilen derselben Notiz (`WHERE blocks.document_id = excluded.document_id`). Test: `TestInsertCannotReuseBlockIDOfAnotherNote`. - **Fundstelle:** `Store.PatchDocumentBlocks`, `INSERT INTO blocks … ON CONFLICT(id) DO UPDATE`. - **Problem:** `blocks.id` ist global eindeutig. Existiert die ID bereits in einem **anderen** Dokument, überschreibt der Konflikt-Zweig dessen Inhalt, ohne `document_id` zu prüfen. Voraussetzung: Schreibrecht im eigenen Dokument und Kenntnis einer fremden Block-ID. - **Maßnahme:** `DO UPDATE … WHERE blocks.document_id = excluded.document_id`; fremde IDs ablehnen. @@ -134,6 +135,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( - **Maßnahme:** Setup per einmaligem Token aus dem Server-Log absichern; Demo-Modus nur zusammen mit expliziter Warnung/Bestätigung. ### FLAW-17: Schreibgeschützte Dokumentfreigaben +- **Stand:** Die Zwangsmigration auf `read` ist entfernt, das Backend speichert und prüft Bearbeiter-Freigaben (`DocRole`, `/docs/{id}/sync`). Offen bis Phase 5 des [Kollaborationsplans](plans/kollaboration-notizen.md): Freigabe-Dialog bietet nur „Nur Lesen“, der alte Editor (`PUT /documents/{id}`) schreibt nur für den Eigentümer. - **Fundstellen:** Migration in `Store.migrate` (`UPDATE doc_shares SET permission = 'read' …`, ebenso `doc_team_shares`, `workspace_folder_shares`), `PatchDocumentBlocks`/`UpdateDocument` (nur Eigentümer). - **Hinweis:** Das ist offenbar gewollt: Nur der Eigentümer schreibt in ein Dokument, Empfänger lesen. Es gibt daher **kein** kollaboratives Bearbeiten. Das Schema (`CHECK(permission IN ('read','write'))`), die Zweige für `write` im Code und frühere Dokumentation suggerieren etwas anderes. Es gibt weder Sperren noch Konfliktauflösung, noch Cursor-Presence. - **Maßnahme:** Entweder Schreibfreigaben mit Konfliktbehandlung ergänzen oder den toten Code und das Schema bereinigen. diff --git a/Doku/modules/workspace.md b/Doku/modules/workspace.md index 857c1ee..b384211 100644 --- a/Doku/modules/workspace.md +++ b/Doku/modules/workspace.md @@ -4,7 +4,7 @@ Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.ts ## Datenhaltung -- Eine Notiz ist ein Datensatz in `documents` plus geordnete Zeilen in `blocks` (`position` als Kommazahl, `level` für die Einrückung). Alles liegt in SQLite; es gibt keine Dateien auf der Platte. +- Eine Notiz ist ein Datensatz in `documents` plus ein Baum von Blöcken in `blocks` (`parent_id`, Sortierschlüssel `order_key`). Jede Änderung läuft über `internal/collab` und wird im Log `doc_ops` festgehalten. Ebene und Position für die Oberfläche berechnet der Server beim Lesen. Alles liegt in SQLite; es gibt keine Dateien auf der Platte. - Bei jeder Änderung wird `documents.markdown_cache` neu erzeugt; `block_tags` wird aus `#Hashtags` der Blöcke neu aufgebaut. - Der Export als `.md` erfolgt im Browser (Button im Editor, Blob-Download). - Blocktypen: Überschriften `h1`–`h3`, `paragraph`/`text`, `bullet`, `todo`, `code`, `callout`, `divider`, `table`, `progress`. diff --git a/Doku/plans/kollaboration-notizen.md b/Doku/plans/kollaboration-notizen.md index a1ee83e..4418fa0 100644 --- a/Doku/plans/kollaboration-notizen.md +++ b/Doku/plans/kollaboration-notizen.md @@ -278,7 +278,7 @@ Damit private Notizen keinen unnötigen Aufwand erzeugen, liefert der Server im | Phase | Inhalt | Ergebnis | |---|---|---| | 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Umsortieren (FLAW-03), Stream-Timeout (FLAW-15). **Erledigt.** Die zentrale `docRole` wandert in Phase 1, weil sie auf dem neuen Freigabemodell (`workspace_access`) aufbaut. | Grundlage dicht | -| 1 | Neues Schema inkl. `workspace_access` und zentraler `docRole`, Operations-Log, `/snapshot`, `/sync`, Konfliktregeln, Simulationstests | Server fertig und abgesichert | +| 1 | Operations-Log, `/snapshot`, `/sync`, `/changes`, `/leave`, Konfliktregeln, Leases, zentrale `DocRole`, Simulationstests. **Erledigt.** Abweichungen: Die bestehenden Tabellen `blocks`/`documents` wurden erweitert statt ersetzt (Baum per `parent_id`/`order_key`, alte Daten werden umgewandelt); der bisherige Speicherweg (`PUT`/`PATCH`) läuft als Adapter durch dieselbe Engine. `DocRole` arbeitet auf den vorhandenen Freigabetabellen; deren Zusammenlegung zu `workspace_access` erfolgt mit dem Freigabe-Dialog in Phase 5. `/changes` liefert nur `{headSeq, changed}`, die Daten kommen immer über `/sync`. | Server fertig und abgesichert | | 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt (Baum statt `level`) | Bearbeiten funktioniert überall, auch offline | | 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | | 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt | diff --git a/Doku/sharing-and-permissions.md b/Doku/sharing-and-permissions.md index 426c175..7e5daa8 100644 --- a/Doku/sharing-and-permissions.md +++ b/Doku/sharing-and-permissions.md @@ -14,10 +14,10 @@ |---|---|---|---|---| | **Aufgabenliste** (`task_list_shares.permission`) | alles, löschen, (un)teilen | Aufgaben anlegen/ändern/abhaken/verschieben/löschen, Liste umbenennen, Spalten ändern | nur ansehen | Benutzer, Gruppen | | **Geteilte Liste** (`list_shares.role`) | alles, löschen, (un)teilen | Einträge anlegen/ändern/abhaken/löschen | nur ansehen | Benutzer, Gruppen | -| **Notiz** (`doc_shares`, `doc_team_shares`) | schreiben | – | nur lesen | Benutzer, Gruppen | -| **Geteilter Ordner** (`workspace_folder_shares`) | schreiben | – | nur lesen | Benutzer, Gruppen | +| **Notiz** (`doc_shares`, `doc_team_shares`) | alles, Freigaben, löschen | Inhalt, Struktur, Titel | nur lesen | Benutzer, Gruppen | +| **Geteilter Ordner** (`workspace_folder_shares`) | alles | Notizen darin bearbeiten | nur lesen | Benutzer, Gruppen | -- **Notizen und Ordner sind absichtlich schreibgeschützt für Empfänger.** Das erzwingt eine Migration beim Start (`permission = 'read'`); Schreibpfade im Code sind tot ([FLAW-17](flaws.md)). +- **Notizen und Ordner:** Die effektive Rolle berechnet `Store.DocRole` (Eigentümer, Eigentümer des geteilten Ordners = Bearbeiter, direkte Freigabe, Gruppenfreigabe, Ordnerfreigabe; höchste gewinnt). Das Backend akzeptiert Bearbeiter-Freigaben (`write`). Der Freigabe-Dialog bietet bis zur Umstellung des Editors (Phase 5 im [Plan](plans/kollaboration-notizen.md)) noch nur „Nur Lesen“ an, und der alte Editor schreibt weiterhin nur für den Eigentümer. - Bei Aufgabenlisten und Listen gilt: Hat jemand mehrere Zugriffswege (direkt und über Gruppen), zählt die **höchste** Stufe. - Für Gruppen gilt die Freigabe für alle aktuellen und künftigen Mitglieder. - Aufgabenlisten besitzen zusätzlich das Altfeld `team_id`: Mitglieder dieser Gruppe haben ohne Freigabezeile Bearbeiterrechte. Eine Gruppenfreigabe über den Dialog setzt `team_id`, falls es noch leer ist. diff --git a/internal/api/docsync.go b/internal/api/docsync.go new file mode 100644 index 0000000..91b4fca --- /dev/null +++ b/internal/api/docsync.go @@ -0,0 +1,228 @@ +package api + +import ( + "encoding/json" + "net/http" + "strconv" + "time" + + "github.com/go-chi/chi/v5" + "github.com/sandrodach/mindsift/internal/auth" + "github.com/sandrodach/mindsift/internal/collab" + "github.com/sandrodach/mindsift/internal/storage" +) + +const ( + syncMaxOps = 500 + syncMaxLog = 1000 + syncMaxBody = 8 << 20 + longPollMax = 25 * time.Second + maxClientIDLen = 64 +) + +type syncLeases struct { + Granted []string `json:"granted"` + Denied []string `json:"denied"` + Others []collab.LeaseInfo `json:"others"` +} + +type syncResponse struct { + Results []collab.Result `json:"results"` + Ops []storage.DocLogEntry `json:"ops"` + HeadSeq int64 `json:"headSeq"` + HasMore bool `json:"hasMore"` + Resync bool `json:"resync,omitempty"` // client state is ahead of the server: reload the snapshot + Role string `json:"role"` + Shared bool `json:"shared"` + Sessions int `json:"sessions"` + Presence []collab.Presence `json:"presence"` + Leases syncLeases `json:"leases"` +} + +// docAccess resolves the caller's role; it writes 404 and returns "" without access. +func (s *Server) docAccess(w http.ResponseWriter, r *http.Request) (userID, docID, role string) { + userID = auth.GetUserID(r.Context()) + docID = chi.URLParam(r, "id") + role = s.store.DocRole(userID, docID) + if role == "" { + writeError(w, http.StatusNotFound, storage.ErrDocNotFound.Error()) + } + return userID, docID, role +} + +func (s *Server) handleDocSnapshot(w http.ResponseWriter, r *http.Request) { + _, docID, role := s.docAccess(w, r) + if role == "" { + return + } + snap, err := s.store.GetDocSnapshot(docID) + if err != nil { + writeError(w, http.StatusNotFound, err.Error()) + return + } + writeJSON(w, http.StatusOK, map[string]interface{}{ + "docId": snap.DocID, + "title": snap.Title, + "titleVersion": snap.TitleVersion, + "headSeq": snap.HeadSeq, + "blocks": snap.Blocks, + "role": role, + "shared": s.store.IsDocShared(docID), + }) +} + +// handleDocSync is the single round trip of the sync protocol: send own operations, hold or release +// block leases, and receive every log entry after lastSeq. +func (s *Server) handleDocSync(w http.ResponseWriter, r *http.Request) { + userID, docID, role := s.docAccess(w, r) + if role == "" { + return + } + var req struct { + ClientID string `json:"clientId"` + LastSeq int64 `json:"lastSeq"` + Ops []collab.Op `json:"ops"` + Leases *struct { + Hold []string `json:"hold"` + Release []string `json:"release"` + } `json:"leases"` + } + if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, syncMaxBody)).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "Ungültiges JSON") + return + } + if req.ClientID == "" || len(req.ClientID) > maxClientIDLen { + writeError(w, http.StatusBadRequest, "clientId fehlt oder ist zu lang") + return + } + if len(req.Ops) > syncMaxOps { + writeError(w, http.StatusRequestEntityTooLarge, "Zu viele Änderungen in einer Anfrage") + return + } + canEdit := storage.CanEditDoc(role) + if len(req.Ops) > 0 && !canEdit { + writeError(w, http.StatusForbidden, "Keine Schreibrechte für diese Notiz (nur Lesen)") + return + } + + actor := collab.Actor{UserID: userID, ClientID: req.ClientID} + if u, err := s.store.GetUserByID(userID); err == nil { + actor.Name = u.DisplayName + } + s.leases.Touch(docID, actor) + + resp := syncResponse{Results: []collab.Result{}, Role: role, Leases: syncLeases{Granted: []string{}, Denied: []string{}}} + changed := false + if req.Leases != nil && canEdit { + s.leases.ReleaseAllExcept(docID, req.ClientID, req.Leases.Hold) + s.leases.Release(docID, req.ClientID, req.Leases.Release) + resp.Leases.Granted, resp.Leases.Denied = s.leases.Hold(docID, actor, req.Leases.Hold) + changed = true + } + + if len(req.Ops) > 0 { + results, _, err := s.store.ApplyDocOps(docID, actor, req.Ops, s.leases.View(docID)) + if err != nil { + writeError(w, http.StatusInternalServerError, err.Error()) + return + } + resp.Results = results + // New blocks (and conflict copies) belong to their author right away + var own []string + for i, res := range results { + if res.Status == collab.StatusApplied && req.Ops[i].Kind == collab.OpInsert { + own = append(own, req.Ops[i].BlockID) + } + if res.ConflictBlockID != "" { + own = append(own, res.ConflictBlockID) + } + if res.Seq > 0 { + changed = true + } + } + if len(own) > 0 { + granted, _ := s.leases.Hold(docID, actor, own) + resp.Leases.Granted = append(resp.Leases.Granted, granted...) + } + } + + entries, head, err := s.store.GetDocOpsSince(docID, req.LastSeq, syncMaxLog) + if err != nil { + writeError(w, http.StatusInternalServerError, err.Error()) + return + } + if req.LastSeq > head { + resp.Resync = true + entries = []storage.DocLogEntry{} + } + resp.Ops = entries + resp.HeadSeq = head + resp.HasMore = len(entries) == syncMaxLog && entries[len(entries)-1].Seq < head + resp.Shared = s.store.IsDocShared(docID) + resp.Sessions, resp.Presence = s.leases.Sessions(docID) + resp.Leases.Others = s.leases.Others(docID, req.ClientID) + + if changed { + s.notifyDocChanged(docID, head) + } + writeJSON(w, http.StatusOK, resp) +} + +// notifyDocChanged wakes long-poll requests and pokes WebSocket/SSE clients in the note's room. +func (s *Server) notifyDocChanged(docID string, head int64) { + s.notifier.Notify(docID) + s.wsHub.BroadcastDocEvent(docID, "DOC_SEQ", map[string]interface{}{"docId": docID, "headSeq": head}) +} + +// handleDocChanges is the long-poll fallback: it returns as soon as the note changes (or leases/presence +// change), at the latest after `wait` seconds. Clients then call /sync to fetch the data. +func (s *Server) handleDocChanges(w http.ResponseWriter, r *http.Request) { + _, docID, role := s.docAccess(w, r) + if role == "" { + return + } + since, _ := strconv.ParseInt(r.URL.Query().Get("since"), 10, 64) + wait := longPollMax + if sec, err := strconv.Atoi(r.URL.Query().Get("wait")); err == nil && sec >= 0 && time.Duration(sec)*time.Second < wait { + wait = time.Duration(sec) * time.Second + } + _ = http.NewResponseController(w).SetWriteDeadline(time.Now().Add(wait + 10*time.Second)) + + woken := s.notifier.Changed(docID) // subscribe before reading, so no change can slip through + _, head, err := s.store.GetDocOpsSince(docID, since, 0) + if err != nil { + writeError(w, http.StatusInternalServerError, err.Error()) + return + } + changed := head > since + if !changed { + select { + case <-woken: + changed = true + case <-time.After(wait): + case <-r.Context().Done(): + return + } + _, head, _ = s.store.GetDocOpsSince(docID, since, 0) + } + writeJSON(w, http.StatusOK, map[string]interface{}{"headSeq": head, "changed": changed}) +} + +// handleDocLeave ends a client's session on a note and frees its leases (tab closed, app in background). +func (s *Server) handleDocLeave(w http.ResponseWriter, r *http.Request) { + _, docID, role := s.docAccess(w, r) + if role == "" { + return + } + var req struct { + ClientID string `json:"clientId"` + } + if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&req); err != nil || req.ClientID == "" { + writeError(w, http.StatusBadRequest, "clientId fehlt") + return + } + s.leases.Leave(docID, req.ClientID) + _, head, _ := s.store.GetDocOpsSince(docID, 0, 0) + s.notifyDocChanged(docID, head) + writeJSON(w, http.StatusOK, map[string]bool{"success": true}) +} diff --git a/internal/api/docsync_test.go b/internal/api/docsync_test.go new file mode 100644 index 0000000..1319d59 --- /dev/null +++ b/internal/api/docsync_test.go @@ -0,0 +1,198 @@ +package api + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/sandrodach/mindsift/internal/collab" + "github.com/sandrodach/mindsift/internal/domain" +) + +func syncDoc(t *testing.T, ts *httptest.Server, token, docID string, body map[string]interface{}) (int, syncResponse) { + t.Helper() + resp := doJSON(t, ts, http.MethodPost, "/api/v1/docs/"+docID+"/sync", token, body) + var out syncResponse + if resp.StatusCode == http.StatusOK { + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil { + t.Fatalf("decode sync response: %v", err) + } + } + return resp.StatusCode, out +} + +func strp(s string) *string { return &s } + +type collabFixture struct { + ts *httptest.Server + docID string + ownerTok, editorTok, viewerTok string + outsiderTok string +} + +func newCollabFixture(t *testing.T) collabFixture { + t.Helper() + _, store, ts, owner, ownerTok := startRealtimeServer(t) + editor, editorTok := newTestUser(t, store, "editor@example.com") + viewer, viewerTok := newTestUser(t, store, "viewer@example.com") + _, outsiderTok := newTestUser(t, store, "outsider2@example.com") + doc, err := store.CreateDocument(owner.ID, "", "Gemeinsam", "📄", "", []domain.DocumentBlock{{ID: "b1", Type: "bullet", Content: "Erster Punkt"}}, nil, false) + if err != nil { + t.Fatalf("create doc: %v", err) + } + if _, err := store.CreateDocShare(owner.ID, doc.ID, editor.ID, "write"); err != nil { + t.Fatalf("share editor: %v", err) + } + if _, err := store.CreateDocShare(owner.ID, doc.ID, viewer.ID, "read"); err != nil { + t.Fatalf("share viewer: %v", err) + } + return collabFixture{ts: ts, docID: doc.ID, ownerTok: ownerTok, editorTok: editorTok, viewerTok: viewerTok, outsiderTok: outsiderTok} +} + +func TestDocSyncBetweenTwoEditors(t *testing.T) { + f := newCollabFixture(t) + + // Owner inserts a block and edits the existing one in one round trip + code, a := syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{ + "clientId": "owner-tab", "lastSeq": 0, + "ops": []collab.Op{ + {OpID: "o1", Kind: collab.OpInsert, BlockID: "b2", ParentID: strp(""), OrderKey: "a5", Content: strp("Zweiter Punkt")}, + {OpID: "o2", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Erster Punkt (bearbeitet)")}, + }, + }) + if code != http.StatusOK || a.Results[0].Status != collab.StatusApplied || a.Results[1].Status != collab.StatusApplied { + t.Fatalf("owner sync: %d %+v", code, a.Results) + } + if len(a.Leases.Granted) == 0 || a.Leases.Granted[0] != "b2" { + t.Fatalf("inserted block must be leased to its author: %+v", a.Leases) + } + + // Editor pulls everything since 0 and replays it + code, b := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{"clientId": "editor-tab", "lastSeq": 0}) + if code != http.StatusOK || b.HeadSeq != a.HeadSeq || b.Role != "editor" || !b.Shared { + t.Fatalf("editor pull: %d head=%d role=%s shared=%v", code, b.HeadSeq, b.Role, b.Shared) + } + replay := collab.NewDoc() + for _, e := range b.Ops { + replay.ApplyEffect(e.Effect) + } + flat := replay.Flatten() + if len(flat) != 2 || flat[0].Content != "Erster Punkt (bearbeitet)" || flat[1].Content != "Zweiter Punkt" { + t.Fatalf("replayed state wrong: %+v", flat) + } + if len(b.Leases.Others) == 0 { + t.Fatal("editor must see the owner's lease on b2") + } + + // Editor writes into the leased block: owner keeps it, editor's text becomes a conflict copy + _, c := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{ + "clientId": "editor-tab", "lastSeq": b.HeadSeq, + "ops": []collab.Op{{OpID: "e1", Kind: collab.OpSetContent, BlockID: "b2", BaseVersion: 1, Content: strp("Editor war hier")}}, + }) + if c.Results[0].Status != collab.StatusConflict || c.Results[0].ConflictBlockID == "" { + t.Fatalf("write into foreign lease must conflict: %+v", c.Results) + } +} + +func TestDocSyncRetryIsIdempotent(t *testing.T) { + f := newCollabFixture(t) + body := map[string]interface{}{ + "clientId": "tab", "lastSeq": 0, + "ops": []collab.Op{{OpID: "same-op", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Einmal")}}, + } + _, first := syncDoc(t, f.ts, f.ownerTok, f.docID, body) + _, second := syncDoc(t, f.ts, f.ownerTok, f.docID, body) // response of the first was "lost" + if first.Results[0] != second.Results[0] || first.HeadSeq != second.HeadSeq { + t.Fatalf("retry applied twice: %+v / %+v", first, second) + } +} + +func TestDocSyncPermissions(t *testing.T) { + f := newCollabFixture(t) + op := map[string]interface{}{ + "clientId": "tab", "lastSeq": 0, + "ops": []collab.Op{{OpID: "v1", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Leser schreibt")}}, + } + if code, _ := syncDoc(t, f.ts, f.viewerTok, f.docID, op); code != http.StatusForbidden { + t.Errorf("viewer sending ops: want 403, got %d", code) + } + code, pull := syncDoc(t, f.ts, f.viewerTok, f.docID, map[string]interface{}{"clientId": "tab", "lastSeq": 0, "leases": map[string]interface{}{"hold": []string{"b1"}}}) + if code != http.StatusOK || pull.Role != "viewer" || len(pull.Leases.Granted) != 0 { + t.Errorf("viewer pull: %d role=%s granted=%v", code, pull.Role, pull.Leases.Granted) + } + if code, _ := syncDoc(t, f.ts, f.outsiderTok, f.docID, map[string]interface{}{"clientId": "tab"}); code != http.StatusNotFound { + t.Errorf("outsider: want 404, got %d", code) + } + for _, path := range []string{"/snapshot", "/changes?wait=0"} { + if resp := doJSON(t, f.ts, http.MethodGet, "/api/v1/docs/"+f.docID+path, f.outsiderTok, nil); resp.StatusCode != http.StatusNotFound { + t.Errorf("outsider %s: want 404, got %d", path, resp.StatusCode) + } + } +} + +func TestDocSnapshotMatchesLog(t *testing.T) { + f := newCollabFixture(t) + syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{ + "clientId": "tab", "lastSeq": 0, + "ops": []collab.Op{{OpID: "t1", Kind: collab.OpSetTitle, BaseVersion: 0, Title: strp("Neuer Titel")}}, + }) + resp := doJSON(t, f.ts, http.MethodGet, "/api/v1/docs/"+f.docID+"/snapshot", f.editorTok, nil) + var snap struct { + Title string `json:"title"` + HeadSeq int64 `json:"headSeq"` + Blocks []collab.Block `json:"blocks"` + Role string `json:"role"` + } + if err := json.NewDecoder(resp.Body).Decode(&snap); err != nil { + t.Fatal(err) + } + if snap.Title != "Neuer Titel" || snap.HeadSeq == 0 || len(snap.Blocks) != 1 || snap.Role != "editor" { + t.Fatalf("snapshot: %+v", snap) + } +} + +func TestDocChangesLongPollWakesOnChange(t *testing.T) { + f := newCollabFixture(t) + _, base := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{"clientId": "editor-tab", "lastSeq": 0}) + + type pollResult struct { + Changed bool `json:"changed"` + HeadSeq int64 `json:"headSeq"` + } + done := make(chan pollResult, 1) + start := time.Now() + go func() { + req, _ := http.NewRequest(http.MethodGet, f.ts.URL+"/api/v1/docs/"+f.docID+"/changes?wait=20&since="+itoa(base.HeadSeq), nil) + req.Header.Set("Authorization", "Bearer "+f.editorTok) + resp, err := http.DefaultClient.Do(req) + if err != nil { + done <- pollResult{} + return + } + defer resp.Body.Close() + var r pollResult + _ = json.NewDecoder(resp.Body).Decode(&r) + done <- r + }() + + time.Sleep(300 * time.Millisecond) + syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{ + "clientId": "owner-tab", "lastSeq": 0, + "ops": []collab.Op{{OpID: "lp1", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Weckruf")}}, + }) + select { + case r := <-done: + if !r.Changed || r.HeadSeq <= base.HeadSeq || time.Since(start) > 5*time.Second { + t.Fatalf("long poll result: %+v after %v", r, time.Since(start)) + } + case <-time.After(10 * time.Second): + t.Fatal("long poll did not wake up") + } +} + +func itoa(n int64) string { + b, _ := json.Marshal(n) + return string(b) +} diff --git a/internal/api/server.go b/internal/api/server.go index 0b9c0dc..9c49d8d 100644 --- a/internal/api/server.go +++ b/internal/api/server.go @@ -18,6 +18,7 @@ import ( "github.com/go-chi/chi/v5/middleware" "github.com/go-chi/cors" "github.com/sandrodach/mindsift/internal/auth" + "github.com/sandrodach/mindsift/internal/collab" "github.com/sandrodach/mindsift/internal/config" "github.com/sandrodach/mindsift/internal/domain" "github.com/sandrodach/mindsift/internal/storage" @@ -29,6 +30,9 @@ type Server struct { store *storage.Store wsHub *ws.Hub staticFS fs.FS + + leases *collab.LeaseManager // block leases and open sessions of notes + notifier *collab.Notifier // wakes long-poll requests on note changes } func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS fs.FS) *Server { @@ -37,6 +41,8 @@ func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS store: store, wsHub: wsHub, staticFS: staticFS, + leases: collab.NewLeaseManager(nil), + notifier: collab.NewNotifier(), } wsHub.SetRoomAuthorizer(srv.canJoinRoom) go srv.startAlarmTicker() @@ -46,8 +52,7 @@ func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS // canJoinRoom allows realtime rooms only to users with read access: "doc:" for notes, otherwise a shared list ID. func (s *Server) canJoinRoom(userID, room string) bool { if docID, ok := strings.CutPrefix(room, "doc:"); ok { - _, err := s.store.GetDocumentByID(userID, docID) - return err == nil + return s.store.DocRole(userID, docID) != "" } _, err := s.store.GetListByID(userID, room) return err == nil @@ -159,6 +164,12 @@ func (s *Server) Router() http.Handler { r.Delete("/api/v1/documents/{id}", s.handleDeleteDocument) r.Post("/api/v1/documents/{id}/blocks/{blockId}/enhanced-task", s.handleCreateEnhancedTask) + // Collaborative note editing (operation log) + r.Get("/api/v1/docs/{id}/snapshot", s.handleDocSnapshot) + r.Post("/api/v1/docs/{id}/sync", s.handleDocSync) + r.Get("/api/v1/docs/{id}/changes", s.handleDocChanges) + r.Post("/api/v1/docs/{id}/leave", s.handleDocLeave) + // Document Sharing & Collaboration r.Get("/api/v1/documents/{id}/shares", s.handleGetDocShares) r.Post("/api/v1/documents/{id}/shares", s.handleCreateDocShare) diff --git a/internal/collab/doc.go b/internal/collab/doc.go new file mode 100644 index 0000000..00d7673 --- /dev/null +++ b/internal/collab/doc.go @@ -0,0 +1,201 @@ +// Package collab contains the database-independent core of collaborative note editing: the block tree, +// operations with their conflict rules, block leases and change notification. The same rules run on the +// server for every incoming operation; clients replay the resulting effects to stay in sync. +package collab + +import ( + "fmt" + "sort" +) + +// Props are per-block settings that change independently of the text. +type Props struct { + Checked bool `json:"checked,omitempty"` + Collapsed bool `json:"collapsed,omitempty"` + Lang string `json:"lang,omitempty"` + ConflictOf string `json:"conflictOf,omitempty"` // ID of the block (or "title") this conflict copy belongs to + ConflictAuthor string `json:"conflictAuthor,omitempty"` // display name of the author whose text lost +} + +type Block struct { + ID string `json:"id"` + ParentID string `json:"parentId"` // "" = top level + OrderKey string `json:"orderKey"` + Type string `json:"type"` + Content string `json:"content"` + Props Props `json:"props"` + Version int `json:"version"` // content version; props changes do not bump it + Deleted bool `json:"deleted,omitempty"` + UpdatedBy string `json:"updatedBy,omitempty"` +} + +// Doc is the complete state of one note including deleted blocks (tombstones). +type Doc struct { + Title string + TitleVersion int + Blocks map[string]*Block +} + +func NewDoc() *Doc { + return &Doc{Blocks: make(map[string]*Block)} +} + +func (d *Doc) Clone() *Doc { + c := &Doc{Title: d.Title, TitleVersion: d.TitleVersion, Blocks: make(map[string]*Block, len(d.Blocks))} + for id, b := range d.Blocks { + cp := *b + c.Blocks[id] = &cp + } + return c +} + +func siblingLess(a, b *Block) bool { + if a.OrderKey != b.OrderKey { + return a.OrderKey < b.OrderKey + } + return a.ID < b.ID +} + +// Children returns the live children of parentID in display order. +func (d *Doc) Children(parentID string) []*Block { + out := make([]*Block, 0) + for _, b := range d.Blocks { + if !b.Deleted && b.ParentID == parentID { + out = append(out, b) + } + } + sort.Slice(out, func(i, j int) bool { return siblingLess(out[i], out[j]) }) + return out +} + +// FlatBlock is a live block with its depth, in document (depth-first) order. +type FlatBlock struct { + *Block + Level int +} + +// Flatten returns all live blocks in display order with their nesting level. +func (d *Doc) Flatten() []FlatBlock { + children := make(map[string][]*Block) + for _, b := range d.Blocks { + if !b.Deleted { + children[b.ParentID] = append(children[b.ParentID], b) + } + } + for _, list := range children { + sort.Slice(list, func(i, j int) bool { return siblingLess(list[i], list[j]) }) + } + out := make([]FlatBlock, 0, len(d.Blocks)) + var walk func(parent string, level int) + walk = func(parent string, level int) { + for _, b := range children[parent] { + out = append(out, FlatBlock{Block: b, Level: level}) + walk(b.ID, level+1) + } + } + walk("", 0) + return out +} + +// subtree returns id and all descendants; with liveOnly, deleted blocks are skipped. +func (d *Doc) subtree(id string, liveOnly bool) []string { + children := make(map[string][]string) + for _, b := range d.Blocks { + if liveOnly && b.Deleted { + continue + } + children[b.ParentID] = append(children[b.ParentID], b.ID) + } + out := []string{id} + for i := 0; i < len(out); i++ { + out = append(out, children[out[i]]...) + } + sort.Strings(out[1:]) + return out +} + +// liveAncestor walks up from id until it finds a live block; "" means top level. +func (d *Doc) liveAncestor(id string) string { + seen := map[string]bool{} + for id != "" && !seen[id] { + seen[id] = true + b, ok := d.Blocks[id] + if !ok { + return "" + } + if !b.Deleted { + return id + } + id = b.ParentID + } + return "" +} + +// isAncestor reports whether ancestorID is ancestorID itself or above id in the tree. +func (d *Doc) isAncestor(ancestorID, id string) bool { + seen := map[string]bool{} + for id != "" && !seen[id] { + if id == ancestorID { + return true + } + seen[id] = true + b, ok := d.Blocks[id] + if !ok { + return false + } + id = b.ParentID + } + return false +} + +// keyAfter returns a key directly after block b among its live siblings. +func (d *Doc) keyAfter(b *Block) string { + next := "" + for _, s := range d.Children(b.ParentID) { + if siblingLess(b, s) { + next = s.OrderKey + break + } + } + if next == b.OrderKey { + // Equal keys are only ordered by ID; append after the whole run instead + next = "" + for _, s := range d.Children(b.ParentID) { + if s.OrderKey > b.OrderKey { + next = s.OrderKey + break + } + } + } + k, err := KeyBetween(b.OrderKey, next) + if err != nil { + k, _ = KeyBetween(b.OrderKey, "") + } + return k +} + +// Validate checks the tree invariants: parents exist, live blocks have live parents, no cycles, valid keys. +func (d *Doc) Validate() error { + for id, b := range d.Blocks { + if b.ID != id { + return fmt.Errorf("block %s stored under key %s", b.ID, id) + } + if !ValidOrderKey(b.OrderKey) { + return fmt.Errorf("block %s has invalid order key %q", id, b.OrderKey) + } + if b.ParentID == "" { + continue + } + p, ok := d.Blocks[b.ParentID] + if !ok { + return fmt.Errorf("block %s has missing parent %s", id, b.ParentID) + } + if !b.Deleted && p.Deleted { + return fmt.Errorf("live block %s has deleted parent %s", id, p.ID) + } + if d.isAncestor(id, b.ParentID) { + return fmt.Errorf("block %s is part of a cycle", id) + } + } + return nil +} diff --git a/internal/collab/lease.go b/internal/collab/lease.go new file mode 100644 index 0000000..c3525b3 --- /dev/null +++ b/internal/collab/lease.go @@ -0,0 +1,197 @@ +package collab + +import ( + "sort" + "sync" + "time" +) + +const ( + LeaseTTL = 20 * time.Second + SessionTimeout = 30 * time.Second +) + +type lease struct { + clientID string + userID string + name string + expires time.Time +} + +type session struct { + userID string + name string + lastSeen time.Time +} + +// LeaseInfo describes a block held by another client, for display ("Anna bearbeitet …"). +type LeaseInfo struct { + BlockID string `json:"blockId"` + UserID string `json:"userId"` + Name string `json:"name"` +} + +// Presence is one client that synced a document recently. +type Presence struct { + UserID string `json:"userId"` + Name string `json:"name"` +} + +// LeaseManager keeps block leases and active sessions in memory. Leases are short-lived; after a server +// restart clients simply request them again with their next sync. +type LeaseManager struct { + mu sync.Mutex + now func() time.Time + leases map[string]map[string]lease // docID -> blockID -> lease + sessions map[string]map[string]session // docID -> clientID -> session +} + +func NewLeaseManager(now func() time.Time) *LeaseManager { + if now == nil { + now = time.Now + } + return &LeaseManager{now: now, leases: map[string]map[string]lease{}, sessions: map[string]map[string]session{}} +} + +func (m *LeaseManager) docLeases(docID string) map[string]lease { + l := m.leases[docID] + if l == nil { + l = map[string]lease{} + m.leases[docID] = l + } + return l +} + +func (m *LeaseManager) prune(docID string, now time.Time) { + for id, l := range m.leases[docID] { + if !now.Before(l.expires) { + delete(m.leases[docID], id) + } + } + for cid, s := range m.sessions[docID] { + if now.Sub(s.lastSeen) > SessionTimeout { + delete(m.sessions[docID], cid) + } + } +} + +// Hold acquires or renews leases for a client. A block held by another live client is denied. +func (m *LeaseManager) Hold(docID string, a Actor, blockIDs []string) (granted, denied []string) { + m.mu.Lock() + defer m.mu.Unlock() + now := m.now() + m.prune(docID, now) + l := m.docLeases(docID) + for _, id := range blockIDs { + if cur, ok := l[id]; ok && cur.clientID != a.ClientID { + denied = append(denied, id) + continue + } + l[id] = lease{clientID: a.ClientID, userID: a.UserID, name: a.Name, expires: now.Add(LeaseTTL)} + granted = append(granted, id) + } + return granted, denied +} + +// Release gives up leases held by this client. +func (m *LeaseManager) Release(docID, clientID string, blockIDs []string) { + m.mu.Lock() + defer m.mu.Unlock() + l := m.leases[docID] + for _, id := range blockIDs { + if cur, ok := l[id]; ok && cur.clientID == clientID { + delete(l, id) + } + } +} + +// ReleaseAllExcept drops every lease of the client in this document that is not in keep. +func (m *LeaseManager) ReleaseAllExcept(docID, clientID string, keep []string) { + m.mu.Lock() + defer m.mu.Unlock() + keepSet := make(map[string]bool, len(keep)) + for _, id := range keep { + keepSet[id] = true + } + for id, cur := range m.leases[docID] { + if cur.clientID == clientID && !keepSet[id] { + delete(m.leases[docID], id) + } + } +} + +// View returns a snapshot of the current leases of a document for operation validation. +func (m *LeaseManager) View(docID string) LeaseView { + m.mu.Lock() + defer m.mu.Unlock() + now := m.now() + m.prune(docID, now) + snap := make(leaseSnapshot, len(m.leases[docID])) + for id, l := range m.leases[docID] { + snap[id] = l.clientID + } + return snap +} + +// Others lists blocks held by clients other than clientID. +func (m *LeaseManager) Others(docID, clientID string) []LeaseInfo { + m.mu.Lock() + defer m.mu.Unlock() + m.prune(docID, m.now()) + out := make([]LeaseInfo, 0) + for id, l := range m.leases[docID] { + if l.clientID != clientID { + out = append(out, LeaseInfo{BlockID: id, UserID: l.userID, Name: l.name}) + } + } + sort.Slice(out, func(i, j int) bool { return out[i].BlockID < out[j].BlockID }) + return out +} + +// Touch records that a client synced the document; it counts as an open session for SessionTimeout. +func (m *LeaseManager) Touch(docID string, a Actor) { + m.mu.Lock() + defer m.mu.Unlock() + s := m.sessions[docID] + if s == nil { + s = map[string]session{} + m.sessions[docID] = s + } + s[a.ClientID] = session{userID: a.UserID, name: a.Name, lastSeen: m.now()} +} + +// Leave removes a client's session and all its leases (tab closed, app sent to background). +func (m *LeaseManager) Leave(docID, clientID string) { + m.mu.Lock() + defer m.mu.Unlock() + delete(m.sessions[docID], clientID) + for id, l := range m.leases[docID] { + if l.clientID == clientID { + delete(m.leases[docID], id) + } + } +} + +// Sessions returns how many clients have the document open and who they are. +func (m *LeaseManager) Sessions(docID string) (int, []Presence) { + m.mu.Lock() + defer m.mu.Unlock() + m.prune(docID, m.now()) + seen := map[string]bool{} + people := make([]Presence, 0) + for _, s := range m.sessions[docID] { + if !seen[s.userID] { + seen[s.userID] = true + people = append(people, Presence{UserID: s.userID, Name: s.name}) + } + } + sort.Slice(people, func(i, j int) bool { return people[i].UserID < people[j].UserID }) + return len(m.sessions[docID]), people +} + +type leaseSnapshot map[string]string + +func (s leaseSnapshot) HolderOf(blockID string) (string, bool) { + c, ok := s[blockID] + return c, ok +} diff --git a/internal/collab/notify.go b/internal/collab/notify.go new file mode 100644 index 0000000..8770e1c --- /dev/null +++ b/internal/collab/notify.go @@ -0,0 +1,34 @@ +package collab + +import "sync" + +// Notifier wakes up long-poll requests waiting for changes of a document. +type Notifier struct { + mu sync.Mutex + wait map[string]chan struct{} +} + +func NewNotifier() *Notifier { + return &Notifier{wait: map[string]chan struct{}{}} +} + +// Changed returns a channel that is closed on the next Notify for docID. +func (n *Notifier) Changed(docID string) <-chan struct{} { + n.mu.Lock() + defer n.mu.Unlock() + ch, ok := n.wait[docID] + if !ok { + ch = make(chan struct{}) + n.wait[docID] = ch + } + return ch +} + +func (n *Notifier) Notify(docID string) { + n.mu.Lock() + defer n.mu.Unlock() + if ch, ok := n.wait[docID]; ok { + close(ch) + delete(n.wait, docID) + } +} diff --git a/internal/collab/ops.go b/internal/collab/ops.go new file mode 100644 index 0000000..14dfad5 --- /dev/null +++ b/internal/collab/ops.go @@ -0,0 +1,396 @@ +package collab + +import ( + "crypto/sha256" + "encoding/hex" + "sort" +) + +const ( + OpInsert = "insert" + OpSetContent = "set_content" + OpSetProps = "set_props" + OpMove = "move" + OpDelete = "delete" + OpRestore = "restore" + OpSetTitle = "set_title" + + StatusApplied = "applied" + StatusConflict = "conflict" + StatusRejected = "rejected" + + maxContentLength = 100_000 + maxTypeLength = 20 +) + +// PropsPatch changes selected block settings; nil fields stay as they are. +type PropsPatch struct { + Type *string `json:"type,omitempty"` + Checked *bool `json:"checked,omitempty"` + Collapsed *bool `json:"collapsed,omitempty"` + Lang *string `json:"lang,omitempty"` +} + +// Op is a change requested by a client. OpID is chosen by the client and makes retries idempotent. +type Op struct { + OpID string `json:"opId"` + Kind string `json:"kind"` + BlockID string `json:"blockId,omitempty"` + BaseVersion int `json:"baseVersion,omitempty"` // set_content, set_title + ParentID *string `json:"parentId,omitempty"` // insert, move ("" = top level) + OrderKey string `json:"orderKey,omitempty"` // insert, move + Type string `json:"type,omitempty"` // insert + Content *string `json:"content,omitempty"` // insert, set_content + Props *Props `json:"props,omitempty"` // insert + Patch *PropsPatch `json:"patch,omitempty"` // set_props + IDs []string `json:"ids,omitempty"` // restore + Title *string `json:"title,omitempty"` // set_title +} + +// Effect is a normalized, already validated change. The server log stores effects; replaying them in +// order on an empty Doc reproduces the server state exactly. +type Effect struct { + Kind string `json:"kind"` + BlockID string `json:"blockId,omitempty"` + ParentID string `json:"parentId,omitempty"` + OrderKey string `json:"orderKey,omitempty"` + Type string `json:"type,omitempty"` + Content string `json:"content,omitempty"` + Props *Props `json:"props,omitempty"` + Version int `json:"version,omitempty"` + IDs []string `json:"ids,omitempty"` + Title string `json:"title,omitempty"` + TitleVersion int `json:"titleVersion,omitempty"` + By string `json:"by,omitempty"` +} + +type Result struct { + OpID string `json:"opId"` + Status string `json:"status"` + Reason string `json:"reason,omitempty"` + Version int `json:"version,omitempty"` + ConflictBlockID string `json:"conflictBlockId,omitempty"` + Seq int64 `json:"seq,omitempty"` +} + +// Actor identifies who sends an operation. ClientID is one browser tab / sync engine instance. +type Actor struct { + UserID string + ClientID string + Name string +} + +// LeaseView answers which client currently holds the lease on a block. +type LeaseView interface { + HolderOf(blockID string) (clientID string, held bool) +} + +// Env provides outside knowledge needed to validate operations. +type Env struct { + Leases LeaseView // nil = no leases (e.g. server-internal writes) + IDTaken func(id string) bool // true if the block ID exists in another document +} + +// ApplyEffect mutates the document with one effect. It never fails; effects are validated by Apply. +func (d *Doc) ApplyEffect(e Effect) { + switch e.Kind { + case OpInsert: + b := &Block{ID: e.BlockID, ParentID: e.ParentID, OrderKey: e.OrderKey, Type: e.Type, Content: e.Content, Version: e.Version, UpdatedBy: e.By} + if e.Props != nil { + b.Props = *e.Props + } + d.Blocks[e.BlockID] = b + case OpSetContent: + if b, ok := d.Blocks[e.BlockID]; ok { + b.Content, b.Version, b.UpdatedBy = e.Content, e.Version, e.By + } + case OpSetProps: + if b, ok := d.Blocks[e.BlockID]; ok { + b.Type = e.Type + if e.Props != nil { + b.Props = *e.Props + } + } + case OpMove: + if b, ok := d.Blocks[e.BlockID]; ok { + b.ParentID, b.OrderKey = e.ParentID, e.OrderKey + } + case OpDelete: + for _, id := range e.IDs { + if b, ok := d.Blocks[id]; ok { + b.Deleted = true + } + } + case OpRestore: + for _, id := range e.IDs { + if b, ok := d.Blocks[id]; ok { + b.Deleted = false + } + } + case OpSetTitle: + d.Title, d.TitleVersion = e.Title, e.TitleVersion + } +} + +// ConflictBlockID derives the ID of a conflict copy from the losing operation, so retries never duplicate it. +func ConflictBlockID(opID string) string { + sum := sha256.Sum256([]byte("conflict:" + opID)) + return "blk_cf_" + hex.EncodeToString(sum[:8]) +} + +type applier struct { + d *Doc + a Actor + env Env + effects []Effect +} + +func (ap *applier) emit(e Effect) { + e.By = ap.a.UserID + ap.d.ApplyEffect(e) + ap.effects = append(ap.effects, e) +} + +func (ap *applier) foreignLease(blockID string) bool { + if ap.env.Leases == nil { + return false + } + holder, held := ap.env.Leases.HolderOf(blockID) + return held && holder != ap.a.ClientID +} + +func rejected(op Op, reason string) Result { + return Result{OpID: op.OpID, Status: StatusRejected, Reason: reason} +} + +// Apply validates op against d, mutates d and returns the produced effects. Rejected operations +// produce no effects; conflicting text is kept as a separate conflict block. +func Apply(d *Doc, op Op, a Actor, env Env) ([]Effect, Result) { + ap := &applier{d: d, a: a, env: env} + res := ap.apply(op) + res.OpID = op.OpID + return ap.effects, res +} + +func (ap *applier) apply(op Op) Result { + d := ap.d + switch op.Kind { + case OpInsert: + if op.BlockID == "" || len(op.BlockID) > 64 { + return rejected(op, "invalid_id") + } + if _, exists := d.Blocks[op.BlockID]; exists || (ap.env.IDTaken != nil && ap.env.IDTaken(op.BlockID)) { + return rejected(op, "exists") + } + parent := "" + if op.ParentID != nil { + parent = *op.ParentID + } + if parent != "" { + if _, ok := d.Blocks[parent]; !ok { + return rejected(op, "parent_missing") + } + parent = d.liveAncestor(parent) + } + if !ValidOrderKey(op.OrderKey) { + return rejected(op, "invalid_order_key") + } + typ := op.Type + if typ == "" { + typ = "bullet" + } + if len(typ) > maxTypeLength { + return rejected(op, "invalid_type") + } + content := "" + if op.Content != nil { + content = *op.Content + } + if len(content) > maxContentLength { + return rejected(op, "too_long") + } + props := Props{} + if op.Props != nil { + props = *op.Props + } + ap.emit(Effect{Kind: OpInsert, BlockID: op.BlockID, ParentID: parent, OrderKey: op.OrderKey, Type: typ, Content: content, Props: &props, Version: 1}) + return Result{Status: StatusApplied, Version: 1} + + case OpSetContent: + b, ok := d.Blocks[op.BlockID] + if !ok { + return rejected(op, "missing") + } + if op.Content == nil { + return rejected(op, "no_content") + } + if len(*op.Content) > maxContentLength { + return rejected(op, "too_long") + } + if b.Deleted { + // A late edit on a deleted block brings it back instead of losing the text + ap.restore([]string{b.ID}) + } + if *op.Content == b.Content { + return Result{Status: StatusApplied, Version: b.Version} + } + if ap.foreignLease(b.ID) || op.BaseVersion != b.Version { + return ap.conflictCopy(op, b, *op.Content) + } + ap.emit(Effect{Kind: OpSetContent, BlockID: b.ID, Content: *op.Content, Version: b.Version + 1}) + return Result{Status: StatusApplied, Version: b.Version} + + case OpSetProps: + b, ok := d.Blocks[op.BlockID] + if !ok || b.Deleted { + return rejected(op, "missing") + } + if op.Patch == nil { + return rejected(op, "no_patch") + } + typ, props := b.Type, b.Props + if p := op.Patch; p != nil { + if p.Type != nil && *p.Type != "" && len(*p.Type) <= maxTypeLength { + typ = *p.Type + } + if p.Checked != nil { + props.Checked = *p.Checked + } + if p.Collapsed != nil { + props.Collapsed = *p.Collapsed + } + if p.Lang != nil { + props.Lang = *p.Lang + } + } + if typ != b.Type || props != b.Props { + ap.emit(Effect{Kind: OpSetProps, BlockID: b.ID, Type: typ, Props: &props}) + } + return Result{Status: StatusApplied} + + case OpMove: + b, ok := d.Blocks[op.BlockID] + if !ok || b.Deleted { + return rejected(op, "missing") + } + if ap.foreignLease(b.ID) { + return rejected(op, "locked") + } + parent := "" + if op.ParentID != nil { + parent = *op.ParentID + } + if parent != "" { + p, ok := d.Blocks[parent] + if !ok || p.Deleted { + return rejected(op, "parent_missing") + } + if d.isAncestor(b.ID, parent) { + return rejected(op, "cycle") + } + } + if !ValidOrderKey(op.OrderKey) { + return rejected(op, "invalid_order_key") + } + if parent != b.ParentID || op.OrderKey != b.OrderKey { + ap.emit(Effect{Kind: OpMove, BlockID: b.ID, ParentID: parent, OrderKey: op.OrderKey}) + } + return Result{Status: StatusApplied} + + case OpDelete: + b, ok := d.Blocks[op.BlockID] + if !ok { + return rejected(op, "missing") + } + if b.Deleted { + return Result{Status: StatusApplied} + } + ids := d.subtree(b.ID, true) + for _, id := range ids { + if ap.foreignLease(id) { + return rejected(op, "locked") + } + } + ap.emit(Effect{Kind: OpDelete, IDs: ids}) + return Result{Status: StatusApplied} + + case OpRestore: + ids := make([]string, 0, len(op.IDs)) + for _, id := range op.IDs { + if b, ok := d.Blocks[id]; ok && b.Deleted { + ids = append(ids, id) + } + } + if len(ids) > 0 { + ap.restore(ids) + } + return Result{Status: StatusApplied} + + case OpSetTitle: + if op.Title == nil { + return rejected(op, "no_title") + } + if len(*op.Title) > 1000 { + return rejected(op, "too_long") + } + if *op.Title == d.Title { + return Result{Status: StatusApplied, Version: d.TitleVersion} + } + if op.BaseVersion != d.TitleVersion { + return ap.titleConflict(op, *op.Title) + } + ap.emit(Effect{Kind: OpSetTitle, Title: *op.Title, TitleVersion: d.TitleVersion + 1}) + return Result{Status: StatusApplied, Version: d.TitleVersion} + } + return rejected(op, "unknown_kind") +} + +// restore brings back deleted blocks; blocks whose parent stays deleted move to the nearest live ancestor. +func (ap *applier) restore(ids []string) { + d := ap.d + set := make(map[string]bool, len(ids)) + for _, id := range ids { + set[id] = true + } + sorted := append([]string(nil), ids...) + sort.Strings(sorted) + ap.emit(Effect{Kind: OpRestore, IDs: sorted}) + for _, id := range sorted { + b := d.Blocks[id] + if b.ParentID == "" || set[b.ParentID] { + continue + } + if p, ok := d.Blocks[b.ParentID]; ok && !p.Deleted { + continue + } + ap.emit(Effect{Kind: OpMove, BlockID: b.ID, ParentID: d.liveAncestor(b.ParentID), OrderKey: b.OrderKey}) + } +} + +// conflictCopy keeps the losing text as a new block directly below the contested one. +func (ap *applier) conflictCopy(op Op, b *Block, content string) Result { + id := ConflictBlockID(op.OpID) + if _, exists := ap.d.Blocks[id]; !exists { + props := Props{ConflictOf: b.ID, ConflictAuthor: ap.a.Name} + ap.emit(Effect{Kind: OpInsert, BlockID: id, ParentID: b.ParentID, OrderKey: ap.d.keyAfter(b), Type: b.Type, Content: content, Props: &props, Version: 1}) + } + return Result{Status: StatusConflict, Version: b.Version, ConflictBlockID: id} +} + +// titleConflict keeps a losing title as a conflict block at the very top of the note. +func (ap *applier) titleConflict(op Op, title string) Result { + id := ConflictBlockID(op.OpID) + if _, exists := ap.d.Blocks[id]; !exists { + first := "" + if roots := ap.d.Children(""); len(roots) > 0 { + first = roots[0].OrderKey + } + key, err := KeyBetween("", first) + if err != nil { + key, _ = KeyBetween("", "") + } + props := Props{ConflictOf: "title", ConflictAuthor: ap.a.Name} + ap.emit(Effect{Kind: OpInsert, BlockID: id, OrderKey: key, Type: "text", Content: title, Props: &props, Version: 1}) + } + return Result{Status: StatusConflict, Version: ap.d.TitleVersion, ConflictBlockID: id} +} diff --git a/internal/collab/ops_test.go b/internal/collab/ops_test.go new file mode 100644 index 0000000..adce525 --- /dev/null +++ b/internal/collab/ops_test.go @@ -0,0 +1,263 @@ +package collab + +import ( + "testing" + "time" +) + +var ( + anna = Actor{UserID: "u_anna", ClientID: "c_anna", Name: "Anna"} + ben = Actor{UserID: "u_ben", ClientID: "c_ben", Name: "Ben"} +) + +func str(s string) *string { return &s } + +type fixedLeases map[string]string + +func (f fixedLeases) HolderOf(id string) (string, bool) { c, ok := f[id]; return c, ok } + +// docWith builds a doc via real operations so versions and keys are realistic. +func docWith(t *testing.T, blocks ...[3]string) *Doc { // {id, parent, content} + t.Helper() + d := NewDoc() + keys := map[string]string{} + for i, b := range blocks { + k, _ := KeyBetween(keys[b[1]], "") + keys[b[1]] = k + _, res := Apply(d, Op{OpID: "seed" + string(rune('a'+i)), Kind: OpInsert, BlockID: b[0], ParentID: str(b[1]), OrderKey: k, Content: str(b[2])}, anna, Env{}) + if res.Status != StatusApplied { + t.Fatalf("seed %v: %+v", b, res) + } + } + return d +} + +func mustValid(t *testing.T, d *Doc) { + t.Helper() + if err := d.Validate(); err != nil { + t.Fatal(err) + } +} + +func TestSetContentAppliesOnMatchingVersion(t *testing.T) { + d := docWith(t, [3]string{"a", "", "Hallo"}) + _, res := Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Hallo Welt")}, anna, Env{}) + if res.Status != StatusApplied || d.Blocks["a"].Content != "Hallo Welt" || d.Blocks["a"].Version != 2 { + t.Fatalf("got %+v, block %+v", res, d.Blocks["a"]) + } +} + +func TestStaleVersionCreatesConflictCopy(t *testing.T) { + d := docWith(t, [3]string{"a", "", "Start"}, [3]string{"b", "", "Danach"}) + Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Von Anna")}, anna, Env{}) + _, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Von Ben")}, ben, Env{}) + + if res.Status != StatusConflict || res.ConflictBlockID == "" { + t.Fatalf("want conflict, got %+v", res) + } + if d.Blocks["a"].Content != "Von Anna" { + t.Fatal("first writer must keep the block") + } + flat := d.Flatten() + if len(flat) != 3 || flat[1].ID != res.ConflictBlockID || flat[1].Content != "Von Ben" || flat[2].ID != "b" { + t.Fatalf("conflict copy must sit directly below the block: %+v", flat) + } + if c := d.Blocks[res.ConflictBlockID].Props; c.ConflictOf != "a" || c.ConflictAuthor != "Ben" { + t.Fatalf("conflict props wrong: %+v", c) + } + mustValid(t, d) +} + +func TestForeignLeaseCreatesConflictCopy(t *testing.T) { + d := docWith(t, [3]string{"a", "", "Start"}) + _, res := Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Ben tippt")}, ben, Env{Leases: fixedLeases{"a": "c_anna"}}) + if res.Status != StatusConflict || d.Blocks["a"].Content != "Start" { + t.Fatalf("lease holder must win: %+v", res) + } + // The lease holder itself may write + _, res = Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Anna tippt")}, anna, Env{Leases: fixedLeases{"a": "c_anna"}}) + if res.Status != StatusApplied { + t.Fatalf("lease holder rejected: %+v", res) + } +} + +func TestIdenticalContentIsNoConflict(t *testing.T) { + d := docWith(t, [3]string{"a", "", "Gleich"}) + Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Neu")}, anna, Env{}) + _, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Neu")}, ben, Env{}) + if res.Status != StatusApplied || len(d.Blocks) != 1 { + t.Fatalf("same text must not produce a conflict copy: %+v", res) + } +} + +func TestEditOnDeletedBlockRestoresIt(t *testing.T) { + d := docWith(t, [3]string{"p", "", "Eltern"}, [3]string{"c", "p", "Kind"}) + Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "p"}, anna, Env{}) + _, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "c", BaseVersion: 1, Content: str("Spät getippt")}, ben, Env{}) + if res.Status != StatusApplied { + t.Fatalf("got %+v", res) + } + c := d.Blocks["c"] + if c.Deleted || c.Content != "Spät getippt" || c.ParentID != "" { + t.Fatalf("block must be restored at the nearest live ancestor: %+v", c) + } + mustValid(t, d) +} + +func TestDeleteIsBlockedByForeignLeaseInSubtree(t *testing.T) { + d := docWith(t, [3]string{"p", "", "Ast"}, [3]string{"c", "p", "Unterpunkt"}) + _, res := Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "p"}, anna, Env{Leases: fixedLeases{"c": "c_ben"}}) + if res.Status != StatusRejected || res.Reason != "locked" || d.Blocks["p"].Deleted { + t.Fatalf("got %+v", res) + } + _, res = Apply(d, Op{OpID: "2", Kind: OpDelete, BlockID: "p"}, anna, Env{}) + if res.Status != StatusApplied || !d.Blocks["c"].Deleted { + t.Fatal("delete must tombstone the whole subtree") + } +} + +func TestMoveRejectsCyclesAndLockedBlocks(t *testing.T) { + d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"}) + if _, res := Apply(d, Op{OpID: "1", Kind: OpMove, BlockID: "a", ParentID: str("b"), OrderKey: "a0"}, anna, Env{}); res.Reason != "cycle" { + t.Fatalf("want cycle, got %+v", res) + } + if _, res := Apply(d, Op{OpID: "2", Kind: OpMove, BlockID: "b", ParentID: str(""), OrderKey: "a5"}, anna, Env{Leases: fixedLeases{"b": "c_ben"}}); res.Reason != "locked" { + t.Fatalf("want locked, got %+v", res) + } + if _, res := Apply(d, Op{OpID: "3", Kind: OpMove, BlockID: "b", ParentID: str(""), OrderKey: "a5"}, anna, Env{}); res.Status != StatusApplied { + t.Fatalf("got %+v", res) + } + mustValid(t, d) +} + +func TestInsertIntoDeletedParentGoesToLiveAncestor(t *testing.T) { + d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"}) + Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "b"}, anna, Env{}) + _, res := Apply(d, Op{OpID: "2", Kind: OpInsert, BlockID: "n", ParentID: str("b"), OrderKey: "a0", Content: str("neu")}, ben, Env{}) + if res.Status != StatusApplied || d.Blocks["n"].ParentID != "a" { + t.Fatalf("got %+v parent=%q", res, d.Blocks["n"].ParentID) + } +} + +func TestInsertRejectsTakenIDs(t *testing.T) { + d := docWith(t, [3]string{"a", "", "A"}) + if _, res := Apply(d, Op{OpID: "1", Kind: OpInsert, BlockID: "a", OrderKey: "a5"}, anna, Env{}); res.Reason != "exists" { + t.Fatalf("got %+v", res) + } + env := Env{IDTaken: func(id string) bool { return id == "other_doc_block" }} + if _, res := Apply(d, Op{OpID: "2", Kind: OpInsert, BlockID: "other_doc_block", OrderKey: "a5"}, anna, env); res.Reason != "exists" { + t.Fatalf("block IDs of other documents must be rejected: %+v", res) + } +} + +func TestTitleConflictBecomesTopBlock(t *testing.T) { + d := docWith(t, [3]string{"a", "", "Erster"}) + Apply(d, Op{OpID: "1", Kind: OpSetTitle, BaseVersion: 0, Title: str("Titel Anna")}, anna, Env{}) + _, res := Apply(d, Op{OpID: "2", Kind: OpSetTitle, BaseVersion: 0, Title: str("Titel Ben")}, ben, Env{}) + if res.Status != StatusConflict || d.Title != "Titel Anna" { + t.Fatalf("got %+v title=%q", res, d.Title) + } + if flat := d.Flatten(); flat[0].ID != res.ConflictBlockID || flat[0].Props.ConflictOf != "title" { + t.Fatalf("title conflict must be the first block: %+v", flat[0]) + } +} + +func TestSetPropsDoesNotBumpContentVersion(t *testing.T) { + d := docWith(t, [3]string{"a", "", "Todo"}) + Apply(d, Op{OpID: "1", Kind: OpSetProps, BlockID: "a", Patch: &PropsPatch{Checked: new(bool)}}, anna, Env{}) + yes := true + Apply(d, Op{OpID: "2", Kind: OpSetProps, BlockID: "a", Patch: &PropsPatch{Checked: &yes}}, ben, Env{Leases: fixedLeases{"a": "c_anna"}}) + if !d.Blocks["a"].Props.Checked || d.Blocks["a"].Version != 1 { + t.Fatalf("props must apply independently of text and leases: %+v", d.Blocks["a"]) + } +} + +// Replaying the effects of every operation on an empty doc must give exactly the same state. +func TestEffectsReplayToSameState(t *testing.T) { + d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"}) + replay := d.Clone() + ops := []struct { + op Op + a Actor + }{ + {Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("A2")}, anna}, + {Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("A3")}, ben}, + {Op{OpID: "3", Kind: OpDelete, BlockID: "a"}, anna}, + {Op{OpID: "4", Kind: OpSetContent, BlockID: "b", BaseVersion: 1, Content: str("B2")}, ben}, + {Op{OpID: "5", Kind: OpSetTitle, Title: str("T")}, anna}, + } + for _, o := range ops { + effects, _ := Apply(d, o.op, o.a, Env{}) + for _, e := range effects { + replay.ApplyEffect(e) + } + } + if !docsEqual(d, replay) { + t.Fatal("replayed effects differ from server state") + } +} + +func docsEqual(a, b *Doc) bool { + if a.Title != b.Title || a.TitleVersion != b.TitleVersion || len(a.Blocks) != len(b.Blocks) { + return false + } + for id, x := range a.Blocks { + y, ok := b.Blocks[id] + if !ok || *x != *y { + return false + } + } + return true +} + +func TestLeaseManager(t *testing.T) { + now := time.Unix(1000, 0) + m := NewLeaseManager(func() time.Time { return now }) + + g, den := m.Hold("d", anna, []string{"a", "b"}) + if len(g) != 2 || len(den) != 0 { + t.Fatalf("anna: granted=%v denied=%v", g, den) + } + if g, den = m.Hold("d", ben, []string{"a", "c"}); len(g) != 1 || den[0] != "a" { + t.Fatalf("ben: granted=%v denied=%v", g, den) + } + if others := m.Others("d", "c_ben"); len(others) != 2 || others[0].Name != "Anna" { + t.Fatalf("others: %+v", others) + } + + now = now.Add(LeaseTTL + time.Second) + if g, _ = m.Hold("d", ben, []string{"a"}); len(g) != 1 { + t.Fatal("expired lease must be available") + } + + m.ReleaseAllExcept("d", "c_ben", []string{"a"}) + if _, held := m.View("d").HolderOf("c"); held { + t.Fatal("lease not in keep set must be released") + } + + m.Touch("d", anna) + m.Touch("d", ben) + if n, people := m.Sessions("d"); n != 2 || len(people) != 2 { + t.Fatalf("sessions: %d %+v", n, people) + } + m.Leave("d", "c_ben") + if _, held := m.View("d").HolderOf("a"); held { + t.Fatal("leaving must drop the client's leases") + } +} + +func TestNotifier(t *testing.T) { + n := NewNotifier() + ch := n.Changed("d") + select { + case <-ch: + t.Fatal("must not fire before Notify") + default: + } + n.Notify("d") + select { + case <-ch: + case <-time.After(time.Second): + t.Fatal("Notify must wake waiters") + } +} diff --git a/internal/collab/orderkey.go b/internal/collab/orderkey.go new file mode 100644 index 0000000..f91af62 --- /dev/null +++ b/internal/collab/orderkey.go @@ -0,0 +1,267 @@ +package collab + +import ( + "errors" + "strings" +) + +// Order keys sort siblings lexicographically (ties broken by block ID) and allow inserting between any two +// keys without renumbering. Format as in the "fractional-indexing" scheme: an integer part whose first +// character encodes its length ('a'..'z' positive, 'A'..'Z' negative) followed by base-62 digits, then an +// optional fraction without trailing '0'. Appending stays short ("a0", "a1", …, "az", "b00", …). +const keyDigits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" + +const smallestInteger = "A00000000000000000000000000" + +var errInvalidOrderKey = errors.New("ungültiger Sortierschlüssel") + +func digitIndex(c byte) int { + return strings.IndexByte(keyDigits, c) +} + +func integerLength(head byte) int { + switch { + case head >= 'a' && head <= 'z': + return int(head-'a') + 2 + case head >= 'A' && head <= 'Z': + return int('Z'-head) + 2 + } + return 0 +} + +func integerPart(key string) (string, bool) { + if key == "" { + return "", false + } + n := integerLength(key[0]) + if n == 0 || n > len(key) { + return "", false + } + return key[:n], true +} + +// ValidOrderKey reports whether k is a well-formed key. +func ValidOrderKey(k string) bool { + if k == "" || len(k) > 256 || k == smallestInteger { + return false + } + ip, ok := integerPart(k) + if !ok { + return false + } + for i := 1; i < len(k); i++ { + if digitIndex(k[i]) < 0 { + return false + } + } + frac := k[len(ip):] + return frac == "" || frac[len(frac)-1] != '0' +} + +func incrementInteger(x string) (string, bool) { + head, digs := x[0], []byte(x[1:]) + carry := true + for i := len(digs) - 1; carry && i >= 0; i-- { + d := digitIndex(digs[i]) + 1 + if d == len(keyDigits) { + digs[i] = '0' + } else { + digs[i] = keyDigits[d] + carry = false + } + } + if !carry { + return string(head) + string(digs), true + } + if head == 'Z' { + return "a0", true + } + if head == 'z' { + return "", false + } + h := head + 1 + if h > 'a' { + digs = append(digs, '0') + } else { + digs = digs[:len(digs)-1] + } + return string(h) + string(digs), true +} + +func decrementInteger(x string) (string, bool) { + head, digs := x[0], []byte(x[1:]) + borrow := true + for i := len(digs) - 1; borrow && i >= 0; i-- { + d := digitIndex(digs[i]) - 1 + if d == -1 { + digs[i] = 'z' + } else { + digs[i] = keyDigits[d] + borrow = false + } + } + if !borrow { + return string(head) + string(digs), true + } + if head == 'a' { + return "Z" + string(keyDigits[len(keyDigits)-1]), true + } + if head == 'A' { + return "", false + } + h := head - 1 + if h < 'Z' { + digs = append(digs, 'z') + } else { + digs = digs[:len(digs)-1] + } + return string(h) + string(digs), true +} + +// midpoint returns a fraction between a and b (a may be "" = 0, b may be "" = 1). +func midpoint(a, b string) string { + if b != "" { + n := 0 + for n < len(b) { + ca := byte('0') + if n < len(a) { + ca = a[n] + } + if ca != b[n] { + break + } + n++ + } + if n > 0 { + rest := "" + if n < len(a) { + rest = a[n:] + } + return b[:n] + midpoint(rest, b[n:]) + } + } + da := 0 + if a != "" { + da = digitIndex(a[0]) + } + db := len(keyDigits) + if b != "" { + db = digitIndex(b[0]) + } + if db-da > 1 { + return string(keyDigits[(da+db)/2]) + } + if b != "" && len(b) > 1 { + return b[:1] + } + rest := "" + if len(a) > 1 { + rest = a[1:] + } + return string(keyDigits[da]) + midpoint(rest, "") +} + +// KeyBetween returns a key strictly between a and b. An empty a means "before everything", +// an empty b means "after everything". +func KeyBetween(a, b string) (string, error) { + if (a != "" && !ValidOrderKey(a)) || (b != "" && !ValidOrderKey(b)) { + return "", errInvalidOrderKey + } + if a != "" && b != "" && a >= b { + return "", errInvalidOrderKey + } + switch { + case a == "" && b == "": + return "a0", nil + case a == "": + ib, _ := integerPart(b) + fb := b[len(ib):] + if ib == smallestInteger { + return ib + midpoint("", fb), nil + } + if ib < b { + return ib, nil + } + res, ok := decrementInteger(ib) + if !ok { + return "", errInvalidOrderKey + } + return res, nil + case b == "": + ia, _ := integerPart(a) + fa := a[len(ia):] + if i, ok := incrementInteger(ia); ok { + return i, nil + } + return ia + midpoint(fa, ""), nil + } + ia, _ := integerPart(a) + fa := a[len(ia):] + ib, _ := integerPart(b) + fb := b[len(ib):] + if ia == ib { + return ia + midpoint(fa, fb), nil + } + i, ok := incrementInteger(ia) + if !ok { + return "", errInvalidOrderKey + } + if i < b { + return i, nil + } + return ia + midpoint(fa, ""), nil +} + +// KeysBetween returns n increasing keys between a and b, spread so that bulk inserts stay short. +func KeysBetween(a, b string, n int) ([]string, error) { + if n <= 0 { + return nil, nil + } + if n == 1 { + k, err := KeyBetween(a, b) + if err != nil { + return nil, err + } + return []string{k}, nil + } + if b == "" { + keys := make([]string, 0, n) + prev := a + for i := 0; i < n; i++ { + k, err := KeyBetween(prev, "") + if err != nil { + return nil, err + } + keys = append(keys, k) + prev = k + } + return keys, nil + } + if a == "" { + keys := make([]string, n) + next := b + for i := n - 1; i >= 0; i-- { + k, err := KeyBetween("", next) + if err != nil { + return nil, err + } + keys[i] = k + next = k + } + return keys, nil + } + mid := n / 2 + c, err := KeyBetween(a, b) + if err != nil { + return nil, err + } + left, err := KeysBetween(a, c, mid) + if err != nil { + return nil, err + } + right, err := KeysBetween(c, b, n-mid-1) + if err != nil { + return nil, err + } + return append(append(left, c), right...), nil +} diff --git a/internal/collab/orderkey_test.go b/internal/collab/orderkey_test.go new file mode 100644 index 0000000..895d1e4 --- /dev/null +++ b/internal/collab/orderkey_test.go @@ -0,0 +1,90 @@ +package collab + +import ( + "math/rand" + "sort" + "testing" +) + +func TestKeyBetweenBasics(t *testing.T) { + cases := [][2]string{{"", ""}, {"", "a0"}, {"a0", ""}, {"a0", "a1"}, {"a0", "a0V"}, {"a1", "a2"}, {"a0V", "a1"}, {"Zz", "a0"}, {"az", ""}, {"", "Zz"}} + for _, c := range cases { + k, err := KeyBetween(c[0], c[1]) + if err != nil { + t.Fatalf("KeyBetween(%q,%q): %v", c[0], c[1], err) + } + if !ValidOrderKey(k) { + t.Fatalf("KeyBetween(%q,%q) = %q is not a valid key", c[0], c[1], k) + } + if (c[0] != "" && k <= c[0]) || (c[1] != "" && k >= c[1]) { + t.Fatalf("KeyBetween(%q,%q) = %q is not between", c[0], c[1], k) + } + } + if _, err := KeyBetween("a1", "a0"); err == nil { + t.Fatal("reversed range must fail") + } + if _, err := KeyBetween("a0V0", ""); err == nil { + t.Fatal("trailing zero key must be rejected") + } +} + +// Random inserts between neighbours must always produce a strictly increasing sequence. +func TestKeyBetweenRandomInserts(t *testing.T) { + r := rand.New(rand.NewSource(42)) + keys := []string{} + for i := 0; i < 3000; i++ { + pos := r.Intn(len(keys) + 1) + a, b := "", "" + if pos > 0 { + a = keys[pos-1] + } + if pos < len(keys) { + b = keys[pos] + } + k, err := KeyBetween(a, b) + if err != nil { + t.Fatalf("step %d: %v", i, err) + } + keys = append(keys[:pos], append([]string{k}, keys[pos:]...)...) + } + if !sort.StringsAreSorted(keys) { + t.Fatal("keys are not sorted") + } + for i := 1; i < len(keys); i++ { + if keys[i] == keys[i-1] { + t.Fatalf("duplicate key %q", keys[i]) + } + } +} + +// Appending at the end, the most common case, must keep keys short. +func TestKeyBetweenAppendStaysShort(t *testing.T) { + last := "" + for i := 0; i < 1000; i++ { + k, _ := KeyBetween(last, "") + last = k + } + if len(last) > 4 { + t.Fatalf("key after 1000 appends too long: %d chars", len(last)) + } +} + +func TestKeysBetweenBulk(t *testing.T) { + for _, r := range [][2]string{{"", ""}, {"a0", ""}, {"", "a0"}, {"a0", "a1"}} { + keys, err := KeysBetween(r[0], r[1], 500) + if err != nil { + t.Fatalf("%v: %v", r, err) + } + if len(keys) != 500 || !sort.StringsAreSorted(keys) { + t.Fatalf("%v: wrong or unsorted result", r) + } + for i, k := range keys { + if !ValidOrderKey(k) || (i > 0 && keys[i-1] == k) { + t.Fatalf("%v: bad key %q", r, k) + } + if (r[0] != "" && k <= r[0]) || (r[1] != "" && k >= r[1]) { + t.Fatalf("%v: key %q out of range", r, k) + } + } + } +} diff --git a/internal/storage/docsync.go b/internal/storage/docsync.go new file mode 100644 index 0000000..af01f8a --- /dev/null +++ b/internal/storage/docsync.go @@ -0,0 +1,663 @@ +package storage + +import ( + "database/sql" + "encoding/json" + "errors" + "fmt" + "log/slog" + "sort" + "strings" + "sync" + + "github.com/sandrodach/mindsift/internal/collab" + "github.com/sandrodach/mindsift/internal/domain" +) + +// Collaborative note editing: every change to a note's blocks or title runs through collab.Apply under a +// per-document lock, is appended to doc_ops (the log) and recorded in doc_op_results (idempotency). + +// ErrDocNotFound is returned when a note does not exist or the user has no access to it. +var ErrDocNotFound = errors.New("Notiz nicht gefunden oder kein Zugriff") + +// Roles a user can have on a note. +const ( + DocRoleOwner = "owner" + DocRoleEditor = "editor" + DocRoleViewer = "viewer" +) + +// DocLogEntry is one effect from the log, as sent to clients. +type DocLogEntry struct { + Seq int64 `json:"seq"` + OpID string `json:"opId,omitempty"` + UserID string `json:"userId"` + ClientID string `json:"clientId"` + Effect collab.Effect `json:"effect"` +} + +// DocSnapshot is the complete state of a note including tombstones, needed to replay later effects. +type DocSnapshot struct { + DocID string `json:"docId"` + Title string `json:"title"` + TitleVersion int `json:"titleVersion"` + HeadSeq int64 `json:"headSeq"` + Blocks []collab.Block `json:"blocks"` +} + +var docLocks sync.Map // docID -> *sync.Mutex + +func lockDoc(docID string) func() { + m, _ := docLocks.LoadOrStore(docID, &sync.Mutex{}) + mu := m.(*sync.Mutex) + mu.Lock() + return mu.Unlock +} + +type docQueryer interface { + QueryRow(query string, args ...interface{}) *sql.Row + Query(query string, args ...interface{}) (*sql.Rows, error) +} + +// migrateDocSync adds the columns and tables of the collaboration model and converts old flat block lists. +func (s *Store) migrateDocSync() { + stmts := []string{ + "ALTER TABLE blocks ADD COLUMN parent_id TEXT NOT NULL DEFAULT '';", + "ALTER TABLE blocks ADD COLUMN order_key TEXT NOT NULL DEFAULT '';", + "ALTER TABLE blocks ADD COLUMN version INTEGER NOT NULL DEFAULT 1;", + "ALTER TABLE blocks ADD COLUMN conflict_of TEXT NOT NULL DEFAULT '';", + "ALTER TABLE blocks ADD COLUMN conflict_author TEXT NOT NULL DEFAULT '';", + "ALTER TABLE blocks ADD COLUMN updated_by TEXT NOT NULL DEFAULT '';", + "ALTER TABLE blocks ADD COLUMN deleted INTEGER NOT NULL DEFAULT 0;", + "CREATE INDEX IF NOT EXISTS idx_blocks_doc_parent ON blocks(document_id, parent_id);", + "ALTER TABLE documents ADD COLUMN head_seq INTEGER NOT NULL DEFAULT 0;", + "ALTER TABLE documents ADD COLUMN title_version INTEGER NOT NULL DEFAULT 0;", + `CREATE TABLE IF NOT EXISTS doc_ops ( + doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE, + seq INTEGER NOT NULL, + op_id TEXT NOT NULL DEFAULT '', + user_id TEXT NOT NULL DEFAULT '', + client_id TEXT NOT NULL DEFAULT '', + kind TEXT NOT NULL, + block_id TEXT NOT NULL DEFAULT '', + payload TEXT NOT NULL, + created_at TEXT NOT NULL, + PRIMARY KEY (doc_id, seq) + );`, + `CREATE TABLE IF NOT EXISTS doc_op_results ( + op_id TEXT PRIMARY KEY, + doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE, + user_id TEXT NOT NULL DEFAULT '', + result TEXT NOT NULL, + created_at TEXT NOT NULL + );`, + "CREATE INDEX IF NOT EXISTS idx_doc_op_results_doc ON doc_op_results(doc_id);", + } + for _, q := range stmts { + if _, err := s.db.Exec(q); err != nil && !strings.Contains(err.Error(), "duplicate column") { + slog.Error("Migration der Notiz-Synchronisation fehlgeschlagen", "error", err, "statement", q) + } + } + s.backfillBlockTree() +} + +// backfillBlockTree converts blocks stored as a flat list (level + position) into parent_id + order_key. +func (s *Store) backfillBlockTree() { + rows, err := s.db.Query(`SELECT DISTINCT document_id FROM blocks WHERE order_key = ''`) + if err != nil { + return + } + var docIDs []string + for rows.Next() { + var id string + if rows.Scan(&id) == nil { + docIDs = append(docIDs, id) + } + } + _ = rows.Close() + + for _, docID := range docIDs { + bRows, err := s.db.Query(`SELECT id, level FROM blocks WHERE document_id = ? ORDER BY position ASC, rowid ASC`, docID) + if err != nil { + continue + } + type flat struct { + id string + level int + } + var list []flat + for bRows.Next() { + var f flat + if bRows.Scan(&f.id, &f.level) == nil { + list = append(list, f) + } + } + _ = bRows.Close() + + parents, keys := treeFromLevels(len(list), func(i int) int { return list[i].level }) + tx, err := s.db.Begin() + if err != nil { + continue + } + for i, f := range list { + parent := "" + if parents[i] >= 0 { + parent = list[parents[i]].id + } + _, _ = tx.Exec(`UPDATE blocks SET parent_id = ?, order_key = ? WHERE id = ?`, parent, keys[i], f.id) + } + if err := tx.Commit(); err != nil { + slog.Error("Umwandlung der Blockstruktur fehlgeschlagen", "doc_id", docID, "error", err) + } + } +} + +// treeFromLevels turns a flat list with nesting levels into parent indices (-1 = top level) and +// sequential order keys per parent. Levels deeper than possible are clamped. +func treeFromLevels(n int, level func(int) int) ([]int, []string) { + parents := make([]int, n) + keys := make([]string, n) + lastKey := map[int]string{} + var stack []int // stack[l] = index of the last block at level l + for i := 0; i < n; i++ { + lvl := level(i) + if lvl < 0 { + lvl = 0 + } + if lvl > len(stack) { + lvl = len(stack) + } + stack = stack[:lvl] + parent := -1 + if lvl > 0 { + parent = stack[lvl-1] + } + stack = append(stack, i) + parents[i] = parent + k, _ := collab.KeyBetween(lastKey[parent], "") + lastKey[parent] = k + keys[i] = k + } + return parents, keys +} + +func loadCollabDoc(q docQueryer, docID string) (*collab.Doc, int64, error) { + d := collab.NewDoc() + var head int64 + err := q.QueryRow(`SELECT title, title_version, head_seq FROM documents WHERE id = ?`, docID).Scan(&d.Title, &d.TitleVersion, &head) + if err != nil { + return nil, 0, err + } + rows, err := q.Query( + `SELECT id, parent_id, order_key, type, content, checked, collapsed, lang, conflict_of, conflict_author, version, deleted, updated_by + FROM blocks WHERE document_id = ?`, docID) + if err != nil { + return nil, 0, err + } + defer rows.Close() + for rows.Next() { + b := &collab.Block{} + var chk, col, del int + if err := rows.Scan(&b.ID, &b.ParentID, &b.OrderKey, &b.Type, &b.Content, &chk, &col, &b.Props.Lang, + &b.Props.ConflictOf, &b.Props.ConflictAuthor, &b.Version, &del, &b.UpdatedBy); err != nil { + return nil, 0, err + } + b.Props.Checked, b.Props.Collapsed, b.Deleted = chk != 0, col != 0, del != 0 + d.Blocks[b.ID] = b + } + return d, head, rows.Err() +} + +// flatDocumentBlocks converts the live tree into the flat list (level + position) the existing API returns. +func flatDocumentBlocks(docID string, d *collab.Doc) []domain.DocumentBlock { + flat := d.Flatten() + out := make([]domain.DocumentBlock, len(flat)) + for i, f := range flat { + out[i] = domain.DocumentBlock{ + ID: f.ID, DocumentID: docID, Type: f.Type, Content: f.Content, Checked: f.Props.Checked, + Lang: f.Props.Lang, Level: f.Level, Position: float64((i + 1) * 1000), Collapsed: f.Props.Collapsed, + } + } + return out +} + +func boolInt(b bool) int { + if b { + return 1 + } + return 0 +} + +// ApplyDocOps applies client operations to a note. Operations whose opId was already processed return +// the stored result without being applied again, so clients can safely resend after lost responses. +func (s *Store) ApplyDocOps(docID string, a collab.Actor, ops []collab.Op, leases collab.LeaseView) ([]collab.Result, int64, error) { + return s.applyDocChange(docID, a, leases, func(*collab.Doc) []collab.Op { return ops }) +} + +// applyDocChange builds operations from the current state (under the document lock) and applies them. +func (s *Store) applyDocChange(docID string, a collab.Actor, leases collab.LeaseView, build func(*collab.Doc) []collab.Op) ([]collab.Result, int64, error) { + unlock := lockDoc(docID) + defer unlock() + + tx, err := s.db.Begin() + if err != nil { + return nil, 0, err + } + defer tx.Rollback() + + doc, head, err := loadCollabDoc(tx, docID) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil, 0, ErrDocNotFound + } + return nil, 0, err + } + before := doc.Clone() + startHead := head + env := collab.Env{ + Leases: leases, + IDTaken: func(id string) bool { + var owner string + return tx.QueryRow(`SELECT document_id FROM blocks WHERE id = ?`, id).Scan(&owner) == nil && owner != docID + }, + } + now := domain.NowISO() + ops := build(doc) + results := make([]collab.Result, 0, len(ops)) + + for _, op := range ops { + if op.OpID == "" || len(op.OpID) > 100 { + results = append(results, collab.Result{OpID: op.OpID, Status: collab.StatusRejected, Reason: "invalid_op_id"}) + continue + } + var stored, storedDoc string + err := tx.QueryRow(`SELECT result, doc_id FROM doc_op_results WHERE op_id = ?`, op.OpID).Scan(&stored, &storedDoc) + if err == nil { + var res collab.Result + if storedDoc != docID || json.Unmarshal([]byte(stored), &res) != nil { + res = collab.Result{OpID: op.OpID, Status: collab.StatusRejected, Reason: "op_id_reused"} + } + results = append(results, res) + continue + } + if !errors.Is(err, sql.ErrNoRows) { + return nil, 0, err + } + + effects, res := collab.Apply(doc, op, a, env) + for _, e := range effects { + head++ + payload, _ := json.Marshal(e) + if _, err := tx.Exec( + `INSERT INTO doc_ops (doc_id, seq, op_id, user_id, client_id, kind, block_id, payload, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`, + docID, head, op.OpID, a.UserID, a.ClientID, e.Kind, e.BlockID, string(payload), now, + ); err != nil { + return nil, 0, err + } + } + if len(effects) > 0 { + res.Seq = head + } + resJSON, _ := json.Marshal(res) + if _, err := tx.Exec(`INSERT INTO doc_op_results (op_id, doc_id, user_id, result, created_at) VALUES (?, ?, ?, ?, ?)`, + op.OpID, docID, a.UserID, string(resJSON), now); err != nil { + return nil, 0, err + } + results = append(results, res) + } + + if head != startHead { + if err := persistDocState(tx, docID, before, doc, head, now); err != nil { + return nil, 0, err + } + } + if err := tx.Commit(); err != nil { + return nil, 0, err + } + return results, head, nil +} + +// persistDocState writes changed blocks and refreshes data derived from the note (markdown, search text, tags). +func persistDocState(tx *sql.Tx, docID string, before, after *collab.Doc, head int64, now string) error { + stmt, err := tx.Prepare(` + INSERT INTO blocks (id, document_id, parent_id, order_key, type, content, checked, collapsed, lang, + conflict_of, conflict_author, version, deleted, updated_by, level, position) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, 0) + ON CONFLICT(id) DO UPDATE SET + parent_id = excluded.parent_id, order_key = excluded.order_key, type = excluded.type, + content = excluded.content, checked = excluded.checked, collapsed = excluded.collapsed, + lang = excluded.lang, conflict_of = excluded.conflict_of, conflict_author = excluded.conflict_author, + version = excluded.version, deleted = excluded.deleted, updated_by = excluded.updated_by + WHERE blocks.document_id = excluded.document_id`) + if err != nil { + return err + } + defer stmt.Close() + tagsChanged := false + for id, b := range after.Blocks { + if old, ok := before.Blocks[id]; ok && *old == *b { + continue + } + tagsChanged = true + if _, err := stmt.Exec(b.ID, docID, b.ParentID, b.OrderKey, b.Type, b.Content, boolInt(b.Props.Checked), + boolInt(b.Props.Collapsed), b.Props.Lang, b.Props.ConflictOf, b.Props.ConflictAuthor, b.Version, + boolInt(b.Deleted), b.UpdatedBy); err != nil { + return err + } + } + + flat := flatDocumentBlocks(docID, after) + var plain strings.Builder + for _, b := range flat { + if b.Content != "" { + plain.WriteString(b.Content) + plain.WriteString("\n") + } + } + blocksJSON, _ := json.Marshal(flat) + if _, err := tx.Exec( + `UPDATE documents SET title = ?, title_version = ?, head_seq = ?, markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? WHERE id = ?`, + after.Title, after.TitleVersion, head, domain.CompileBlocksToMarkdown(flat), plain.String(), string(blocksJSON), now, docID, + ); err != nil { + return err + } + + if tagsChanged { + if _, err := tx.Exec(`DELETE FROM block_tags WHERE document_id = ?`, docID); err != nil { + return err + } + for _, b := range flat { + for _, tag := range domain.ExtractTags(b.Content) { + if _, err := tx.Exec(`INSERT OR IGNORE INTO block_tags (tag, document_id, block_id) VALUES (?, ?, ?)`, tag, docID, b.ID); err != nil { + return err + } + } + } + } + return nil +} + +// GetDocOpsSince returns up to limit log entries after seq, plus the current head. +func (s *Store) GetDocOpsSince(docID string, since int64, limit int) ([]DocLogEntry, int64, error) { + var head int64 + if err := s.db.QueryRow(`SELECT head_seq FROM documents WHERE id = ?`, docID).Scan(&head); err != nil { + return nil, 0, err + } + rows, err := s.db.Query( + `SELECT seq, op_id, user_id, client_id, payload FROM doc_ops WHERE doc_id = ? AND seq > ? ORDER BY seq ASC LIMIT ?`, + docID, since, limit) + if err != nil { + return nil, 0, err + } + defer rows.Close() + out := make([]DocLogEntry, 0) + for rows.Next() { + var e DocLogEntry + var payload string + if err := rows.Scan(&e.Seq, &e.OpID, &e.UserID, &e.ClientID, &payload); err != nil { + return nil, 0, err + } + if err := json.Unmarshal([]byte(payload), &e.Effect); err != nil { + return nil, 0, fmt.Errorf("log entry %d unreadable: %w", e.Seq, err) + } + out = append(out, e) + } + return out, head, rows.Err() +} + +// GetDocSnapshot returns the full block tree (including tombstones) and the log position it corresponds to. +func (s *Store) GetDocSnapshot(docID string) (*DocSnapshot, error) { + unlock := lockDoc(docID) + defer unlock() + d, head, err := loadCollabDoc(s.db, docID) + if err != nil { + if errors.Is(err, sql.ErrNoRows) { + return nil, ErrDocNotFound + } + return nil, err + } + snap := &DocSnapshot{DocID: docID, Title: d.Title, TitleVersion: d.TitleVersion, HeadSeq: head, Blocks: make([]collab.Block, 0, len(d.Blocks))} + for _, b := range d.Blocks { + snap.Blocks = append(snap.Blocks, *b) + } + sort.Slice(snap.Blocks, func(i, j int) bool { return snap.Blocks[i].ID < snap.Blocks[j].ID }) + return snap, nil +} + +// DocRole returns the user's role on a note ("owner", "editor", "viewer") or "" without access. +// Sources: ownership, direct shares, team shares and shares of the shared folder the note lives in. +func (s *Store) DocRole(userID, docID string) string { + if userID == "" { + return "" + } + var ownerID, sharedFolderID, folderOwner string + err := s.db.QueryRow( + `SELECT d.user_id, d.shared_folder_id, COALESCE(wsf.owner_id, '') + FROM documents d LEFT JOIN workspace_shared_folders wsf ON wsf.id = d.shared_folder_id + WHERE d.id = ?`, docID).Scan(&ownerID, &sharedFolderID, &folderOwner) + if err != nil { + return "" + } + if ownerID == userID { + return DocRoleOwner + } + if folderOwner == userID { + return DocRoleEditor + } + rows, err := s.db.Query( + `SELECT permission FROM doc_shares WHERE doc_id = ? AND user_id = ? + UNION ALL SELECT dts.permission FROM doc_team_shares dts JOIN team_members tm ON tm.team_id = dts.team_id + WHERE dts.doc_id = ? AND tm.user_id = ? + UNION ALL SELECT wfs.permission FROM workspace_folder_shares wfs + WHERE wfs.folder_id = ? AND wfs.folder_id != '' AND ( + (wfs.target_type = 'user' AND wfs.target_id = ?) + OR (wfs.target_type = 'team' AND wfs.target_id IN (SELECT team_id FROM team_members WHERE user_id = ?)))`, + docID, userID, docID, userID, sharedFolderID, userID, userID) + if err != nil { + return "" + } + defer rows.Close() + role := "" + for rows.Next() { + var p string + if rows.Scan(&p) != nil { + continue + } + if p == "write" { + return DocRoleEditor + } + role = DocRoleViewer + } + return role +} + +// CanEditDoc reports whether the role allows changing a note's content. +func CanEditDoc(role string) bool { + return role == DocRoleOwner || role == DocRoleEditor +} + +// IsDocShared reports whether anyone besides the owner can access the note. +func (s *Store) IsDocShared(docID string) bool { + var n int + _ = s.db.QueryRow( + `SELECT (SELECT COUNT(*) FROM doc_shares WHERE doc_id = ?) + + (SELECT COUNT(*) FROM doc_team_shares WHERE doc_id = ?) + + (SELECT COUNT(*) FROM documents d JOIN workspace_folder_shares wfs ON wfs.folder_id = d.shared_folder_id + WHERE d.id = ? AND d.shared_folder_id != '')`, + docID, docID, docID).Scan(&n) + return n > 0 +} + +// --- Adapter for the existing whole-document API (PUT /documents/{id}) --- + +const legacyClientID = "legacy" + +// replaceDocBlocksInDB stores a complete flat block list by diffing it against the current tree and +// applying the difference as regular operations, so it shows up in the log like any other change. +func (s *Store) replaceDocBlocksInDB(docID, userID string, blocks []domain.DocumentBlock, _ string) { + a := collab.Actor{UserID: userID, ClientID: legacyClientID} + if _, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op { return legacyOps(d, blocks) }); err != nil { + slog.Error("Speichern der Notiz fehlgeschlagen", "doc_id", docID, "error", err) + } +} + +// applyLegacyTitle records a title change from the whole-document API in the log. +func (s *Store) applyLegacyTitle(docID, userID, title string) { + a := collab.Actor{UserID: userID, ClientID: legacyClientID} + if _, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op { + if d.Title == title { + return nil + } + return []collab.Op{{OpID: domain.NewID("op"), Kind: collab.OpSetTitle, BaseVersion: d.TitleVersion, Title: &title}} + }); err != nil { + slog.Error("Speichern des Notiztitels fehlgeschlagen", "doc_id", docID, "error", err) + } +} + +// legacyOps computes the operations that turn the current tree into the given flat list. +// Last writer wins, exactly like the old whole-document save. +func legacyOps(d *collab.Doc, blocks []domain.DocumentBlock) []collab.Op { + // Normalize IDs (missing or duplicated IDs get new ones) + seen := make(map[string]bool, len(blocks)) + list := make([]domain.DocumentBlock, len(blocks)) + for i, b := range blocks { + id := strings.TrimSpace(b.ID) + if id == "" || seen[id] { + id = domain.NewID("blk") + } + seen[id] = true + b.ID = id + if b.Type == "" { + b.Type = "bullet" + } + list[i] = b + } + + parentIdx, _ := treeFromLevels(len(list), func(i int) int { return list[i].Level }) + parentOf := make([]string, len(list)) + children := map[string][]int{} + for i := range list { + if parentIdx[i] >= 0 { + parentOf[i] = list[parentIdx[i]].ID + } + children[parentOf[i]] = append(children[parentOf[i]], i) + } + keys := make([]string, len(list)) + for parent, idxs := range children { + assignKeys(d, parent, idxs, list, keys) + } + + var ops []collab.Op + newOp := func(op collab.Op) { + op.OpID = domain.NewID("op") + ops = append(ops, op) + } + wanted := make(map[string]bool, len(list)) + for i, b := range list { + wanted[b.ID] = true + parent := parentOf[i] + patch := &collab.PropsPatch{Type: &list[i].Type, Checked: &list[i].Checked, Collapsed: &list[i].Collapsed, Lang: &list[i].Lang} + cur, exists := d.Blocks[b.ID] + if !exists { + content := b.Content + props := collab.Props{Checked: b.Checked, Collapsed: b.Collapsed, Lang: b.Lang} + newOp(collab.Op{Kind: collab.OpInsert, BlockID: b.ID, ParentID: &parent, OrderKey: keys[i], Type: b.Type, Content: &content, Props: &props}) + continue + } + if cur.Deleted { + newOp(collab.Op{Kind: collab.OpRestore, IDs: []string{b.ID}}) + } + if cur.ParentID != parent || cur.OrderKey != keys[i] || cur.Deleted { + newOp(collab.Op{Kind: collab.OpMove, BlockID: b.ID, ParentID: &parent, OrderKey: keys[i]}) + } + if cur.Content != b.Content { + content := b.Content + newOp(collab.Op{Kind: collab.OpSetContent, BlockID: b.ID, BaseVersion: cur.Version, Content: &content}) + } + if cur.Type != b.Type || cur.Props.Checked != b.Checked || cur.Props.Collapsed != b.Collapsed || cur.Props.Lang != b.Lang { + newOp(collab.Op{Kind: collab.OpSetProps, BlockID: b.ID, Patch: patch}) + } + } + // Deletes last, after moves have rescued wanted children out of deleted branches + for _, fb := range d.Flatten() { + if !wanted[fb.ID] { + newOp(collab.Op{Kind: collab.OpDelete, BlockID: fb.ID}) + } + } + return ops +} + +// assignKeys picks order keys for the children idxs of parent. Existing keys that are already in the right +// relative order (longest increasing subsequence) are kept, so unchanged blocks do not show up as moves. +func assignKeys(d *collab.Doc, parent string, idxs []int, list []domain.DocumentBlock, keys []string) { + n := len(idxs) + existing := make([]string, n) + for j, i := range idxs { + if b, ok := d.Blocks[list[i].ID]; ok && !b.Deleted && b.ParentID == parent { + existing[j] = b.OrderKey + } + } + keep := longestIncreasing(existing) + for j := 0; j < n; { + if keep[j] { + keys[idxs[j]] = existing[j] + j++ + continue + } + start := j + for j < n && !keep[j] { + j++ + } + left, right := "", "" + if start > 0 { + left = keys[idxs[start-1]] + } + if j < n { + right = existing[j] + } + fresh, err := collab.KeysBetween(left, right, j-start) + if err != nil { + fresh, _ = collab.KeysBetween(left, "", j-start) + } + for k := start; k < j; k++ { + keys[idxs[k]] = fresh[k-start] + } + } +} + +// longestIncreasing marks a longest strictly increasing subsequence of the non-empty keys. +func longestIncreasing(keys []string) []bool { + n := len(keys) + keep := make([]bool, n) + tails := []int{} // indices of the smallest tail for each length + prev := make([]int, n) // predecessor in the subsequence + for i, k := range keys { + prev[i] = -1 + if k == "" { + continue + } + lo, hi := 0, len(tails) + for lo < hi { + mid := (lo + hi) / 2 + if keys[tails[mid]] < k { + lo = mid + 1 + } else { + hi = mid + } + } + if lo > 0 { + prev[i] = tails[lo-1] + } + if lo == len(tails) { + tails = append(tails, i) + } else { + tails[lo] = i + } + } + if len(tails) > 0 { + for i := tails[len(tails)-1]; i >= 0; i = prev[i] { + keep[i] = true + } + } + return keep +} diff --git a/internal/storage/docsync_sim_test.go b/internal/storage/docsync_sim_test.go new file mode 100644 index 0000000..e4dc7e5 --- /dev/null +++ b/internal/storage/docsync_sim_test.go @@ -0,0 +1,402 @@ +package storage + +import ( + "fmt" + "math/rand" + "sort" + "testing" + "time" + + "github.com/sandrodach/mindsift/internal/collab" +) + +// The simulation drives several clients against the real store over an unreliable network and checks +// that everyone converges to the server state without losing text or applying anything twice. + +type simClient struct { + actor collab.Actor + confirmed *collab.Doc + lastSeq int64 + pending []collab.Op // not sent yet; base versions are filled in when a batch is formed + inflight []collab.Op // sent, response not received; resent unchanged (same op IDs) + holds map[string]bool + offline bool + counter int +} + +type simWorld struct { + t *testing.T + r *rand.Rand + store *Store + docID string + leases *collab.LeaseManager + now time.Time + clients []*simClient + sent map[string]collab.Op // every op ever sent, by ID + results map[string]collab.Result // first result received per op ID +} + +func (w *simWorld) newOpID(c *simClient) string { + c.counter++ + return fmt.Sprintf("%s-%d", c.actor.ClientID, c.counter) +} + +// localView is what the client's editor shows: confirmed state plus its own unconfirmed operations. +func (c *simClient) localView() *collab.Doc { + v := c.confirmed.Clone() + for _, op := range append(append([]collab.Op{}, c.inflight...), c.pending...) { + if op.Kind == collab.OpSetContent { + if b, ok := v.Blocks[op.BlockID]; ok { + op.BaseVersion = b.Version + } + } + if op.Kind == collab.OpSetTitle { + op.BaseVersion = v.TitleVersion + } + collab.Apply(v, op, c.actor, collab.Env{}) + } + return v +} + +func pick[T any](r *rand.Rand, list []T) (T, bool) { + var zero T + if len(list) == 0 { + return zero, false + } + return list[r.Intn(len(list))], true +} + +// edit lets a client perform one random editor action on its local view. +func (w *simWorld) edit(c *simClient) { + v := c.localView() + flat := v.Flatten() + var target *collab.Block + if fb, ok := pick(w.r, flat); ok { + target = fb.Block + } + switch n := w.r.Intn(100); { + case n < 45 && target != nil: // typing + content := target.Content + fmt.Sprintf(" %s%d", c.actor.ClientID, c.counter) + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetContent, BlockID: target.ID, Content: &content}) + c.holds[target.ID] = true + case n < 65: // Enter: new block after target (or first block) + parent, after := "", "" + if target != nil { + parent, after = target.ParentID, target.OrderKey + } + next := "" + for _, s := range v.Children(parent) { + if s.OrderKey > after { + next = s.OrderKey + break + } + } + key, err := collab.KeyBetween(after, next) + if err != nil { + return + } + id := fmt.Sprintf("blk_%s_%d", c.actor.ClientID, c.counter) + content := "neu von " + c.actor.ClientID + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpInsert, BlockID: id, ParentID: &parent, OrderKey: key, Content: &content}) + c.holds[id] = true + case n < 73 && target != nil: // indent under previous sibling + siblings := v.Children(target.ParentID) + for i, s := range siblings { + if s.ID == target.ID && i > 0 { + prev := siblings[i-1] + last := "" + if kids := v.Children(prev.ID); len(kids) > 0 { + last = kids[len(kids)-1].OrderKey + } + key, _ := collab.KeyBetween(last, "") + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpMove, BlockID: target.ID, ParentID: &prev.ID, OrderKey: key}) + } + } + case n < 80 && target != nil: // outdent + if p, ok := v.Blocks[target.ParentID]; ok { + key, err := collab.KeyBetween(p.OrderKey, "") + if err == nil { + for _, s := range v.Children(p.ParentID) { + if s.OrderKey > p.OrderKey { + key, _ = collab.KeyBetween(p.OrderKey, s.OrderKey) + break + } + } + grand := p.ParentID + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpMove, BlockID: target.ID, ParentID: &grand, OrderKey: key}) + } + } + case n < 87 && target != nil: // delete branch + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpDelete, BlockID: target.ID}) + case n < 93 && target != nil: // tick checkbox + checked := !target.Props.Checked + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetProps, BlockID: target.ID, Patch: &collab.PropsPatch{Checked: &checked}}) + case n < 96: // rename note + title := fmt.Sprintf("Titel %s%d", c.actor.ClientID, c.counter) + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetTitle, Title: &title}) + default: // stop editing some block + for id := range c.holds { + delete(c.holds, id) + break + } + } +} + +// formBatch takes pending operations, keeps only the last text change per block and fills in base versions. +func (c *simClient) formBatch() []collab.Op { + lastText := map[string]int{} + lastTitle := -1 + for i, op := range c.pending { + switch op.Kind { + case collab.OpSetContent: + lastText[op.BlockID] = i + case collab.OpSetTitle: + lastTitle = i + } + } + var batch []collab.Op + for i, op := range c.pending { + switch op.Kind { + case collab.OpSetContent: + if lastText[op.BlockID] != i { + continue + } + op.BaseVersion = 1 + if b, ok := c.confirmed.Blocks[op.BlockID]; ok { + op.BaseVersion = b.Version + } + case collab.OpSetTitle: + if i != lastTitle { + continue + } + op.BaseVersion = c.confirmed.TitleVersion + } + batch = append(batch, op) + } + c.pending = nil + return batch +} + +// serverSync is what POST /sync does, minus HTTP. +func (w *simWorld) serverSync(c *simClient, batch []collab.Op, holds []string) []collab.Result { + w.leases.Touch(w.docID, c.actor) + w.leases.ReleaseAllExcept(w.docID, c.actor.ClientID, holds) + w.leases.Hold(w.docID, c.actor, holds) + results, _, err := w.store.ApplyDocOps(w.docID, c.actor, batch, w.leases.View(w.docID)) + if err != nil { + w.t.Fatalf("ApplyDocOps: %v", err) + } + var own []string + for i, res := range results { + if res.Status == collab.StatusApplied && batch[i].Kind == collab.OpInsert { + own = append(own, batch[i].BlockID) + } + if res.ConflictBlockID != "" { + own = append(own, res.ConflictBlockID) + } + } + w.leases.Hold(w.docID, c.actor, own) + return results +} + +// pull fetches the log after lastSeq in small pages (to exercise paging) and replays it. +func (w *simWorld) pull(c *simClient) { + for { + entries, head, err := w.store.GetDocOpsSince(w.docID, c.lastSeq, 7) + if err != nil { + w.t.Fatalf("GetDocOpsSince: %v", err) + } + for _, e := range entries { + if e.Seq != c.lastSeq+1 { + w.t.Fatalf("%s: log gap, expected seq %d got %d", c.actor.ClientID, c.lastSeq+1, e.Seq) + } + c.confirmed.ApplyEffect(e.Effect) + c.lastSeq = e.Seq + } + if c.lastSeq >= head { + return + } + } +} + +// sync runs one round trip over the unreliable (or, with reliable=true, a perfect) network. +func (w *simWorld) sync(c *simClient, reliable bool) { + if c.inflight == nil { + c.inflight = c.formBatch() + for _, op := range c.inflight { + w.sent[op.OpID] = op + } + } + holds := make([]string, 0, len(c.holds)) + for id := range c.holds { + holds = append(holds, id) + } + sort.Strings(holds) + + fate := 100 + if !reliable { + fate = w.r.Intn(100) + } + switch { + case fate < 10: // request lost + return + case fate < 20: // server applied it, response lost + w.serverSync(c, c.inflight, holds) + return + case fate < 25: // request duplicated by a retrying proxy + w.serverSync(c, c.inflight, holds) + } + results := w.serverSync(c, c.inflight, holds) + for _, res := range results { + if prev, ok := w.results[res.OpID]; ok && prev != res { + w.t.Fatalf("op %s answered differently on retry: %+v vs %+v", res.OpID, prev, res) + } + w.results[res.OpID] = res + } + c.inflight = nil + w.pull(c) +} + +func runSimulation(t *testing.T, seed int64, steps int) { + store, u := newSyncTestStore(t) + doc, err := store.CreateDocument(u.ID, "", "Sim", "📄", "", flatBlocks([3]interface{}{"root", 0, "Start"}), nil, false) + if err != nil { + t.Fatal(err) + } + w := &simWorld{t: t, r: rand.New(rand.NewSource(seed)), store: store, docID: doc.ID, + now: time.Unix(1_700_000_000, 0), sent: map[string]collab.Op{}, results: map[string]collab.Result{}} + w.leases = collab.NewLeaseManager(func() time.Time { return w.now }) + + for i := 0; i < 4; i++ { + snap, err := store.GetDocSnapshot(doc.ID) + if err != nil { + t.Fatal(err) + } + c := &simClient{actor: collab.Actor{UserID: fmt.Sprintf("u%d", i%2), ClientID: fmt.Sprintf("c%d", i), Name: fmt.Sprintf("Client %d", i)}, + confirmed: collab.NewDoc(), lastSeq: snap.HeadSeq, holds: map[string]bool{}} + c.confirmed.Title, c.confirmed.TitleVersion = snap.Title, snap.TitleVersion + for _, b := range snap.Blocks { + b := b + c.confirmed.Blocks[b.ID] = &b + } + w.clients = append(w.clients, c) + } + + for step := 0; step < steps; step++ { + c := w.clients[w.r.Intn(len(w.clients))] + switch n := w.r.Intn(100); { + case n < 55: + w.edit(c) + case n < 90: + if !c.offline { + w.sync(c, false) + } + case n < 95: + c.offline = !c.offline + default: + w.now = w.now.Add(time.Duration(w.r.Intn(25)) * time.Second) // leases may expire + } + } + + // Everyone comes back online and syncs until nothing is left + for round := 0; round < 10; round++ { + for _, c := range w.clients { + c.offline = false + w.sync(c, true) + } + } + for _, c := range w.clients { + if len(c.pending) > 0 || len(c.inflight) > 0 { + t.Fatalf("%s still has unsent operations", c.actor.ClientID) + } + w.pull(c) + } + + server, head, err := loadCollabDoc(store.db, doc.ID) + if err != nil { + t.Fatal(err) + } + if err := server.Validate(); err != nil { + t.Fatalf("seed %d: invalid tree: %v", seed, err) + } + for _, c := range w.clients { + if c.lastSeq != head || !docsEqual(c.confirmed, server) { + t.Fatalf("seed %d: %s diverged from the server (seq %d vs %d)", seed, c.actor.ClientID, c.lastSeq, head) + } + } + + // Every operation got exactly one stored result, and no text was lost + var stored int + if err := store.db.QueryRow(`SELECT COUNT(*) FROM doc_op_results WHERE doc_id = ? AND user_id IN ('u0', 'u1')`, doc.ID).Scan(&stored); err != nil { + t.Fatal(err) + } + if stored != len(w.sent) { + t.Fatalf("seed %d: %d operations sent but %d results stored", seed, len(w.sent), stored) + } + log, _, _ := store.GetDocOpsSince(doc.ID, 0, 1_000_000) + bySeq := map[int64]collab.Effect{} + inserted := map[string]string{} + for _, e := range log { + bySeq[e.Seq] = e.Effect + if e.Effect.Kind == collab.OpInsert { + inserted[e.Effect.BlockID] = e.Effect.Content + } + } + conflicts := 0 + for id, op := range w.sent { + res, ok := w.results[id] + if !ok { + continue // result only seen by the server (response lost, never retried after success) is covered by the count above + } + switch op.Kind { + case collab.OpInsert, collab.OpSetContent, collab.OpSetTitle: + if res.Status == collab.StatusRejected { + t.Fatalf("seed %d: text-carrying op %s was rejected: %+v", seed, id, res) + } + } + if op.Kind == collab.OpSetContent && res.Status == collab.StatusApplied && res.Seq > 0 { + if e := bySeq[res.Seq]; e.Kind != collab.OpSetContent || e.Content != *op.Content { + t.Fatalf("seed %d: applied text of %s not in the log", seed, id) + } + } + if res.Status == collab.StatusConflict { + conflicts++ + text := "" + if op.Content != nil { + text = *op.Content + } else if op.Title != nil { + text = *op.Title + } + if got, ok := inserted[res.ConflictBlockID]; !ok || got != text { + t.Fatalf("seed %d: conflict copy of %s missing or wrong", seed, id) + } + } + } + t.Logf("seed %d: %d ops, %d log entries, %d conflict copies, %d blocks", seed, len(w.sent), len(log), conflicts, len(server.Blocks)) +} + +func docsEqual(a, b *collab.Doc) bool { + if a.Title != b.Title || a.TitleVersion != b.TitleVersion || len(a.Blocks) != len(b.Blocks) { + return false + } + for id, x := range a.Blocks { + y, ok := b.Blocks[id] + if !ok || *x != *y { + return false + } + } + return true +} + +func TestSyncSimulationUnreliableNetwork(t *testing.T) { + seeds := 25 + if testing.Short() { + seeds = 3 + } + for seed := int64(1); seed <= int64(seeds); seed++ { + seed := seed + t.Run(fmt.Sprintf("seed-%d", seed), func(t *testing.T) { + runSimulation(t, seed, 600) + }) + } +} diff --git a/internal/storage/docsync_test.go b/internal/storage/docsync_test.go new file mode 100644 index 0000000..ad438a6 --- /dev/null +++ b/internal/storage/docsync_test.go @@ -0,0 +1,186 @@ +package storage + +import ( + "path/filepath" + "testing" + + "github.com/sandrodach/mindsift/internal/collab" + "github.com/sandrodach/mindsift/internal/domain" +) + +func newSyncTestStore(t *testing.T) (*Store, *domain.User) { + t.Helper() + store, err := NewStore(filepath.Join(t.TempDir(), "sync.db")) + if err != nil { + t.Fatalf("store: %v", err) + } + t.Cleanup(func() { _ = store.Close() }) + u, err := store.CreateUser("owner@example.com", "hash", "Owner", "member") + if err != nil { + t.Fatalf("user: %v", err) + } + return store, u +} + +func flatBlocks(spec ...[3]interface{}) []domain.DocumentBlock { // {id, level, content} + out := make([]domain.DocumentBlock, len(spec)) + for i, s := range spec { + out[i] = domain.DocumentBlock{ID: s[0].(string), Level: s[1].(int), Content: s[2].(string), Type: "bullet"} + } + return out +} + +func outline(blocks []domain.DocumentBlock) [][2]interface{} { + out := make([][2]interface{}, len(blocks)) + for i, b := range blocks { + out[i] = [2]interface{}{b.ID, b.Level} + } + return out +} + +func sameOutline(t *testing.T, got []domain.DocumentBlock, want []domain.DocumentBlock) { + t.Helper() + if len(got) != len(want) { + t.Fatalf("got %v, want %v", outline(got), outline(want)) + } + for i := range got { + if got[i].ID != want[i].ID || got[i].Level != want[i].Level || got[i].Content != want[i].Content { + t.Fatalf("block %d: got %v, want %v", i, outline(got), outline(want)) + } + } +} + +func headSeq(t *testing.T, s *Store, docID string) int64 { + t.Helper() + _, head, err := s.GetDocOpsSince(docID, 0, 1) + if err != nil { + t.Fatal(err) + } + return head +} + +func TestLegacySaveRoundTripsThroughTheLog(t *testing.T) { + s, u := newSyncTestStore(t) + initial := flatBlocks( + [3]interface{}{"a", 0, "Projekt"}, + [3]interface{}{"b", 1, "Aufgabe 1"}, + [3]interface{}{"c", 2, "Detail"}, + [3]interface{}{"d", 1, "Aufgabe 2"}, + [3]interface{}{"e", 0, "Anderes"}, + ) + doc, err := s.CreateDocument(u.ID, "", "Notiz", "📄", "", initial, nil, false) + if err != nil { + t.Fatal(err) + } + got, _ := s.GetDocumentByID(u.ID, doc.ID) + sameOutline(t, got.Blocks, initial) + + // Saving the same content again must not produce any log entries + head := headSeq(t, s, doc.ID) + if _, err := s.UpdateDocument(u.ID, doc.ID, nil, nil, nil, &initial, nil, nil); err != nil { + t.Fatal(err) + } + if h := headSeq(t, s, doc.ID); h != head { + t.Fatalf("unchanged save wrote %d log entries", h-head) + } + + // Reorder, outdent, edit and delete in one save + changed := flatBlocks( + [3]interface{}{"e", 0, "Anderes"}, + [3]interface{}{"a", 0, "Projekt (neu)"}, + [3]interface{}{"d", 1, "Aufgabe 2"}, + [3]interface{}{"c", 1, "Detail"}, + [3]interface{}{"n", 2, "Neu"}, + ) + if _, err := s.UpdateDocument(u.ID, doc.ID, nil, nil, nil, &changed, nil, nil); err != nil { + t.Fatal(err) + } + got, _ = s.GetDocumentByID(u.ID, doc.ID) + sameOutline(t, got.Blocks, changed) + + // The log alone must rebuild exactly this state + entries, _, err := s.GetDocOpsSince(doc.ID, 0, 10_000) + if err != nil { + t.Fatal(err) + } + replay := collab.NewDoc() + for _, e := range entries { + replay.ApplyEffect(e.Effect) + } + sameOutline(t, flatDocumentBlocks(doc.ID, replay), changed) +} + +func TestApplyDocOpsIsIdempotent(t *testing.T) { + s, u := newSyncTestStore(t) + doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", flatBlocks([3]interface{}{"a", 0, "Start"}), nil, false) + a := collab.Actor{UserID: u.ID, ClientID: "tab1", Name: "Owner"} + text := "Geändert" + op := collab.Op{OpID: "op-1", Kind: collab.OpSetContent, BlockID: "a", BaseVersion: 1, Content: &text} + + first, head1, err := s.ApplyDocOps(doc.ID, a, []collab.Op{op}, nil) + if err != nil || first[0].Status != collab.StatusApplied { + t.Fatalf("first: %+v %v", first, err) + } + // Resend after a lost response: same result, nothing applied twice + again, head2, err := s.ApplyDocOps(doc.ID, a, []collab.Op{op}, nil) + if err != nil || again[0] != first[0] || head2 != head1 { + t.Fatalf("retry changed something: %+v head %d -> %d", again, head1, head2) + } +} + +func TestInsertCannotReuseBlockIDOfAnotherNote(t *testing.T) { + s, u := newSyncTestStore(t) + other, _ := s.CreateDocument(u.ID, "", "Andere", "📄", "", flatBlocks([3]interface{}{"shared_id", 0, "Fremd"}), nil, false) + doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", nil, nil, false) + a := collab.Actor{UserID: u.ID, ClientID: "tab1"} + content, root := "Übernahme", "" + res, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "x", Kind: collab.OpInsert, BlockID: "shared_id", ParentID: &root, OrderKey: "a0", Content: &content}}, nil) + if err != nil || res[0].Reason != "exists" { + t.Fatalf("got %+v %v", res, err) + } + got, _ := s.GetDocumentByID(u.ID, other.ID) + if got.Blocks[0].Content != "Fremd" { + t.Fatal("block of another note was modified") + } +} + +func TestDocRole(t *testing.T) { + s, owner := newSyncTestStore(t) + reader, _ := s.CreateUser("reader@example.com", "hash", "Reader", "member") + writer, _ := s.CreateUser("writer@example.com", "hash", "Writer", "member") + stranger, _ := s.CreateUser("stranger@example.com", "hash", "Stranger", "member") + doc, _ := s.CreateDocument(owner.ID, "", "Notiz", "📄", "", nil, nil, false) + + now := domain.NowISO() + if _, err := s.db.Exec(`INSERT INTO doc_shares (id, doc_id, user_id, permission, created_at, created_by) VALUES ('s1', ?, ?, 'read', ?, ?), ('s2', ?, ?, 'write', ?, ?)`, + doc.ID, reader.ID, now, owner.ID, doc.ID, writer.ID, now, owner.ID); err != nil { + t.Fatal(err) + } + for user, want := range map[string]string{owner.ID: DocRoleOwner, writer.ID: DocRoleEditor, reader.ID: DocRoleViewer, stranger.ID: ""} { + if got := s.DocRole(user, doc.ID); got != want { + t.Errorf("role of %s: got %q, want %q", user, got, want) + } + } + if !s.IsDocShared(doc.ID) { + t.Error("note with shares must count as shared") + } +} + +func TestBackfillConvertsFlatBlocksToTree(t *testing.T) { + s, u := newSyncTestStore(t) + doc, _ := s.CreateDocument(u.ID, "", "Alt", "📄", "", nil, nil, false) + if _, err := s.db.Exec(`DELETE FROM blocks WHERE document_id = ?`, doc.ID); err != nil { + t.Fatal(err) + } + // Rows as written by the old schema: only level and position + for i, row := range [][2]interface{}{{"x", 0}, {"y", 1}, {"z", 1}, {"w", 0}} { + if _, err := s.db.Exec(`INSERT INTO blocks (id, document_id, content, level, position, type) VALUES (?, ?, ?, ?, ?, 'bullet')`, + row[0], doc.ID, row[0], row[1], (i+1)*1000); err != nil { + t.Fatal(err) + } + } + s.backfillBlockTree() + got, _ := s.GetDocumentByID(u.ID, doc.ID) + want := flatBlocks([3]interface{}{"x", 0, "x"}, [3]interface{}{"y", 1, "y"}, [3]interface{}{"z", 1, "z"}, [3]interface{}{"w", 0, "w"}) + sameOutline(t, got.Blocks, want) +} diff --git a/internal/storage/sqlite.go b/internal/storage/sqlite.go index 12cf1c6..6bab8a5 100644 --- a/internal/storage/sqlite.go +++ b/internal/storage/sqlite.go @@ -415,11 +415,6 @@ func (s *Store) migrate() error { // Backfill username for existing accounts where username is empty _, _ = s.db.Exec(`UPDATE users SET username = LOWER(SUBSTR(email, 1, INSTR(email, '@') - 1)) WHERE (username IS NULL OR username = '') AND INSTR(email, '@') > 0`) - // Ensure all existing document & folder shares are strictly read-only - _, _ = s.db.Exec(`UPDATE doc_shares SET permission = 'read' WHERE permission != 'read'`) - _, _ = s.db.Exec(`UPDATE doc_team_shares SET permission = 'read' WHERE permission != 'read'`) - _, _ = s.db.Exec(`UPDATE workspace_folder_shares SET permission = 'read' WHERE permission != 'read'`) - // Default system settings (public registration disabled by default; invite or admin creation required) _, _ = s.db.Exec(`INSERT OR IGNORE INTO system_settings (key, value) VALUES ('sharing_mode', 'teams_only'), @@ -463,6 +458,9 @@ func (s *Store) migrate() error { _, _ = s.db.Exec(`UPDATE team_members SET role = 'member' WHERE role NOT IN ('admin', 'member')`) _, _ = s.db.Exec(`UPDATE users SET role = 'member' WHERE role = 'team_admin'`) + // Block tree, operation log and idempotency tables for collaborative editing + s.migrateDocSync() + // Migrate workspace documents, blocks, folders, and strip legacy folder:* tags into pure DB tables s.migrateWorkspaceToPureDB() @@ -1449,86 +1447,22 @@ func (s *Store) migrateWorkspaceToPureDB() { } func (s *Store) loadDocBlocks(docID string, fallbackJSON string) []domain.DocumentBlock { - rows, err := s.db.Query(` - SELECT id, document_id, type, content, level, position, checked, collapsed, lang - FROM blocks - WHERE document_id = ? - ORDER BY position ASC, rowid ASC - `, docID) - if err == nil { - blocks := make([]domain.DocumentBlock, 0) - for rows.Next() { - var b domain.DocumentBlock - var chk, col int - if scanErr := rows.Scan(&b.ID, &b.DocumentID, &b.Type, &b.Content, &b.Level, &b.Position, &chk, &col, &b.Lang); scanErr == nil { - b.Checked = chk != 0 - b.Collapsed = col != 0 - blocks = append(blocks, b) - } - } - _ = rows.Close() - if len(blocks) > 0 { - return blocks - } + if d, _, err := loadCollabDoc(s.db, docID); err == nil && len(d.Blocks) > 0 { + return flatDocumentBlocks(docID, d) } + // Very old notes only stored blocks_json; convert them once blocks := make([]domain.DocumentBlock, 0) if fallbackJSON != "" && fallbackJSON != "[]" { _ = json.Unmarshal([]byte(fallbackJSON), &blocks) if len(blocks) > 0 { s.replaceDocBlocksInDB(docID, "", blocks, domain.NowISO()) - } - } - return blocks -} - -func (s *Store) replaceDocBlocksInDB(docID, _ string, blocks []domain.DocumentBlock, now string) { - _, _ = s.db.Exec(`DELETE FROM blocks WHERE document_id = ?`, docID) - if len(blocks) > 0 { - stmt, err := s.db.Prepare(` - INSERT INTO blocks (id, document_id, content, level, position, type, checked, collapsed, lang) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) - ON CONFLICT(id) DO UPDATE SET - document_id = excluded.document_id, - content = excluded.content, - level = excluded.level, - position = excluded.position, - type = excluded.type, - checked = excluded.checked, - collapsed = excluded.collapsed, - lang = excluded.lang - `) - if err == nil { - defer stmt.Close() - for i, b := range blocks { - bID := strings.TrimSpace(b.ID) - if bID == "" { - bID = domain.NewID("blk") - } - bType := b.Type - if bType == "" { - bType = "bullet" - } - chk := 0 - if b.Checked { - chk = 1 - } - col := 0 - if b.Collapsed { - col = 1 - } - pos := b.Position - if pos <= 0 { - pos = float64((i + 1) * 1000) - } - _, _ = stmt.Exec(bID, docID, b.Content, b.Level, pos, bType, chk, col, b.Lang) + if d, _, err := loadCollabDoc(s.db, docID); err == nil { + return flatDocumentBlocks(docID, d) } } } - - // Update markdown_cache on documents - mdCache := domain.CompileBlocksToMarkdown(blocks) - _, _ = s.db.Exec(`UPDATE documents SET markdown_cache = ?, updated_at = ? WHERE id = ?`, mdCache, now, docID) + return blocks } func (s *Store) LoadDocumentState(docID string) (string, []domain.DocumentBlock, error) { @@ -1541,222 +1475,85 @@ func (s *Store) LoadDocumentState(docID string) (string, []domain.DocumentBlock, return title, blocks, nil } -// FlushDocumentState persists the in-memory blocks into `blocks`, extracts all #tags into `block_tags`, -// and compiles the hierarchical Markdown into `documents.markdown_cache`. +// FlushDocumentState stores a complete block list (and optionally the title) through the operation log. func (s *Store) FlushDocumentState(docID string, title string, blocks []domain.DocumentBlock) error { - now := domain.NowISO() - - // Normalize positions sequentially - normalized := make([]domain.DocumentBlock, len(blocks)) - var plainBuilder strings.Builder - for i, b := range blocks { - if strings.TrimSpace(b.ID) == "" { - b.ID = domain.NewID("blk") - } - if b.Type == "" { - b.Type = "bullet" - } - b.DocumentID = docID - b.Position = float64((i + 1) * 1000) - normalized[i] = b - if b.Content != "" { - plainBuilder.WriteString(b.Content) - plainBuilder.WriteString("\n") - } - } - - s.replaceDocBlocksInDB(docID, "", normalized, now) - _ = s.indexDocBlockTags(docID, "", normalized) - - mdCache := domain.CompileBlocksToMarkdown(normalized) - plainContent := plainBuilder.String() - blocksJSON, _ := json.Marshal(normalized) - + s.replaceDocBlocksInDB(docID, "", blocks, domain.NowISO()) if strings.TrimSpace(title) != "" { - _, err := s.db.Exec(` - UPDATE documents - SET title = ?, markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? - WHERE id = ? - `, title, mdCache, plainContent, string(blocksJSON), now, docID) - return err + s.applyLegacyTitle(docID, "", title) } - - _, err := s.db.Exec(` - UPDATE documents - SET markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? - WHERE id = ? - `, mdCache, plainContent, string(blocksJSON), now, docID) - return err + return nil } -// PatchDocumentBlocks applies atomic block-level upserts, deletions, and order reconciliation without overwriting concurrent edits from other sessions. +// PatchDocumentBlocks applies block upserts, deletions and an optional order to a note through the +// operation log. Blocks not mentioned keep their position relative to each other. func (s *Store) PatchDocumentBlocks(userID, docID string, req domain.DocBlocksPatchRequest) (*domain.Document, []string, error) { + if !CanEditDoc(s.DocRole(userID, docID)) { + return nil, nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)") + } doc, err := s.GetDocumentByID(userID, docID) if err != nil { return nil, nil, err } - if !doc.IsOwner { - return nil, nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)") + + deleted := make(map[string]bool, len(req.DeletedIDs)) + for _, id := range req.DeletedIDs { + deleted[strings.TrimSpace(id)] = true } - - now := domain.NowISO() - - // Ensure blocks table is seeded if previously only in blocks_json - var existingCount int - _ = s.db.QueryRow(`SELECT COUNT(*) FROM blocks WHERE document_id = ?`, docID).Scan(&existingCount) - if existingCount == 0 && len(doc.Blocks) > 0 { - s.replaceDocBlocksInDB(docID, doc.UserID, doc.Blocks, doc.UpdatedAt) - } - - // Read current ordered block IDs from DB before patch - var existingOrder []string - ordRows, err := s.db.Query(`SELECT id FROM blocks WHERE document_id = ? ORDER BY position ASC, rowid ASC`, docID) - if err == nil { - for ordRows.Next() { - var bid string - if err := ordRows.Scan(&bid); err == nil { - existingOrder = append(existingOrder, bid) - } + upserts := make(map[string]domain.DocumentBlock, len(req.Upserted)) + for _, b := range req.Upserted { + if id := strings.TrimSpace(b.ID); id != "" && !deleted[id] { + upserts[id] = b } - _ = ordRows.Close() } - // 1. Delete explicitly deleted block IDs - deletedSet := make(map[string]bool, len(req.DeletedIDs)) - for _, delID := range req.DeletedIDs { - delID = strings.TrimSpace(delID) - if delID == "" { + merged := make([]domain.DocumentBlock, 0, len(doc.Blocks)+len(upserts)) + used := make(map[string]bool, len(upserts)) + for _, b := range doc.Blocks { + if deleted[b.ID] { continue } - deletedSet[delID] = true - _, _ = s.db.Exec(`DELETE FROM blocks WHERE document_id = ? AND id = ?`, docID, delID) - } - - // Filter existingOrder to exclude deleted blocks - survivingExisting := make([]string, 0, len(existingOrder)) - survivingSet := make(map[string]bool, len(existingOrder)) - for _, bid := range existingOrder { - if !deletedSet[bid] { - survivingExisting = append(survivingExisting, bid) - survivingSet[bid] = true - } - } - - // 2. Upsert modified or newly created blocks - upsertedIDs := make([]string, 0, len(req.Upserted)) - for idx, b := range req.Upserted { - bID := strings.TrimSpace(b.ID) - if bID == "" || deletedSet[bID] { + if up, ok := upserts[b.ID]; ok { + merged = append(merged, up) + used[b.ID] = true continue } - bType := b.Type - if bType == "" { - bType = "bullet" + merged = append(merged, b) + } + for _, b := range req.Upserted { + if id := strings.TrimSpace(b.ID); id != "" && !used[id] && !deleted[id] { + merged = append(merged, b) + used[id] = true } - chk := 0 - if b.Checked { - chk = 1 - } - col := 0 - if b.Collapsed { - col = 1 - } - defaultSort := float64((len(survivingExisting) + idx + 1) * 1000) - _, _ = s.db.Exec(` - INSERT INTO blocks (id, document_id, content, level, position, type, checked, collapsed, lang) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) - ON CONFLICT(id) DO UPDATE SET - type = excluded.type, - content = excluded.content, - level = excluded.level, - checked = excluded.checked, - collapsed = excluded.collapsed, - lang = excluded.lang - `, bID, docID, b.Content, b.Level, defaultSort, bType, chk, col, b.Lang) - upsertedIDs = append(upsertedIDs, bID) - survivingSet[bID] = true } - - // 3. Reconcile block ordering without dropping blocks concurrently created by other users - var mergedOrder []string - inMerged := make(map[string]bool) - if len(req.BlockOrder) > 0 { - for _, bid := range req.BlockOrder { - if !deletedSet[bid] && survivingSet[bid] && !inMerged[bid] { - mergedOrder = append(mergedOrder, bid) - inMerged[bid] = true - } + rank := make(map[string]int, len(req.BlockOrder)) + for i, id := range req.BlockOrder { + rank[id] = i } - for i, bid := range survivingExisting { - if inMerged[bid] { - continue + sort.SliceStable(merged, func(i, j int) bool { + ri, okI := rank[merged[i].ID] + rj, okJ := rank[merged[j].ID] + if okI && okJ { + return ri < rj } - insertPos := 0 - for j := i - 1; j >= 0; j-- { - predID := survivingExisting[j] - if inMerged[predID] { - for k, mID := range mergedOrder { - if mID == predID { - insertPos = k + 1 - break - } - } - break - } - } - mergedOrder = append(mergedOrder[:insertPos], append([]string{bid}, mergedOrder[insertPos:]...)...) - inMerged[bid] = true - } - for _, bid := range upsertedIDs { - if !inMerged[bid] { - mergedOrder = append(mergedOrder, bid) - inMerged[bid] = true - } - } - } else { - for _, bid := range survivingExisting { - if !inMerged[bid] { - mergedOrder = append(mergedOrder, bid) - inMerged[bid] = true - } - } - for _, bid := range upsertedIDs { - if !inMerged[bid] { - mergedOrder = append(mergedOrder, bid) - inMerged[bid] = true - } - } + return okI && !okJ + }) } - // Update position in blocks - for idx, bid := range mergedOrder { - _, _ = s.db.Exec(`UPDATE blocks SET position = ? WHERE document_id = ? AND id = ?`, float64((idx+1)*1000), docID, bid) + s.replaceDocBlocksInDB(docID, userID, merged, domain.NowISO()) + if req.Title != nil { + s.applyLegacyTitle(docID, userID, strings.TrimSpace(*req.Title)) } - // Reload authoritative block list and update document snapshot - finalBlocks := s.loadDocBlocks(docID, "") - var plainBuilder strings.Builder - for _, b := range finalBlocks { - if b.Content != "" { - plainBuilder.WriteString(b.Content) - plainBuilder.WriteString("\n") - } + updated, err := s.GetDocumentByID(userID, docID) + if err != nil { + return nil, nil, err } - plainContent := plainBuilder.String() - mdCache := domain.CompileBlocksToMarkdown(finalBlocks) - blocksJSON, _ := json.Marshal(finalBlocks) - - _, _ = s.db.Exec(`UPDATE documents SET markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? WHERE id = ?`, - mdCache, plainContent, string(blocksJSON), now, docID) - - doc.Blocks = finalBlocks - doc.MarkdownCache = mdCache - doc.Content = plainContent - doc.UpdatedAt = now - _ = s.indexDocBlockTags(doc.ID, doc.UserID, finalBlocks) - - return doc, mergedOrder, nil + order := make([]string, len(updated.Blocks)) + for i, b := range updated.Blocks { + order[i] = b.ID + } + return updated, order, nil } // GetAuthorizedDocUsers returns all user IDs who have read/write/owner access to docID. @@ -2383,6 +2180,7 @@ func (s *Store) UpdateDocument(userID, docID string, title, icon, folderPath *st if title != nil { doc.Title = strings.TrimSpace(*title) + s.applyLegacyTitle(docID, userID, doc.Title) } if icon != nil { doc.Icon = *icon @@ -3445,8 +3243,7 @@ func (s *Store) CreateFolderShare(currentUserID, folderID, targetType, targetIde } } - // Geteilte Ordner können ausschließlich mit Leserberechtigung geteilt werden (wie Dokumente) - permission = "read" + permission = normalizeSharePermission(permission) var targetID, targetName, email, avatarURL string if cleanType == "team" { @@ -3671,8 +3468,7 @@ func (s *Store) CreateDocTeamShare(currentUserID, docID, teamID, permission stri return nil, fmt.Errorf("team nicht gefunden") } - // Document sharing is strictly read-only - permission = "read" + permission = normalizeSharePermission(permission) now := domain.NowISO() shareID := domain.NewID("share") @@ -3699,6 +3495,15 @@ func (s *Store) CreateDocTeamShare(currentUserID, docID, teamID, permission stri }, nil } +// normalizeSharePermission maps share roles from the API to the stored values: "write" (editor) or "read" (viewer). +func normalizeSharePermission(p string) string { + switch strings.ToLower(strings.TrimSpace(p)) { + case "write", "editor": + return "write" + } + return "read" +} + func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permission string) (*domain.DocShare, error) { doc, err := s.GetDocumentByID(currentUserID, docID) if err != nil { @@ -3712,11 +3517,10 @@ func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permi // If identifier looks like a team or is prefixed with team_, route to CreateDocTeamShare if strings.HasPrefix(cleanTarget, "team_") { - return s.CreateDocTeamShare(currentUserID, docID, cleanTarget, "read") + return s.CreateDocTeamShare(currentUserID, docID, cleanTarget, permission) } - // Document sharing is strictly read-only - permission = "read" + permission = normalizeSharePermission(permission) // Resolve target user by ID, username, or Email var targetUser *domain.User diff --git a/internal/storage/sqlite_test.go b/internal/storage/sqlite_test.go index 1160058..067e4f9 100644 --- a/internal/storage/sqlite_test.go +++ b/internal/storage/sqlite_test.go @@ -574,7 +574,7 @@ func TestDatabaseFirstWorkspaceArchitecture(t *testing.T) { } } - _, err = store.CreateDocShare(userA.ID, doc.ID, userB.ID, "write") + _, err = store.CreateDocShare(userA.ID, doc.ID, userB.ID, "read") if err != nil { t.Fatalf("CreateDocShare failed: %v", err) } From 51f90eb7e4e6fa92fbc809f2a93aa5b35d071a65 Mon Sep 17 00:00:00 2001 From: Sandro Date: Wed, 28 Oct 2026 12:28:15 +0100 Subject: [PATCH 05/11] feat(notes): browser sync engine with offline queue; editor saves through the operation log Phase 2 of the collaboration plan: - frontend/src/lib/docSync: TypeScript port of the block tree, operations, effects and order keys (kept identical to Go via shared test vectors), diff from the editor's flat list to operations, IndexedDB persistence, HTTP transport and DocSyncEngine (one request in flight, idempotent resends, backoff, 3 s/30 s polling, keepalive flush and leave on exit, block leases for focus, 20 s grace and new blocks). - Text changes carry the version the user saw when typing (first change wins when coalescing; a stale focused field keeps its old base), so a concurrent edit becomes a conflict copy instead of a silent overwrite. - DocumentEditor uses the engine instead of the 800 ms whole-document PUT; editors (not only owners) can edit, remote changes appear live. - Server: task IDs of linked blocks are delivered again, linked tasks follow ticked/deleted blocks after sync, note tags are derived, title changes notify other users; block IDs already used in another note are renamed (starter content with fixed IDs was empty for every further user). - Fix: unlinking tasks from blocks wrote to non-existent columns and never worked (error was swallowed). - Tests: Go/TS order key vectors, diff round trips, engine races (lost responses, offline + reload, edits arriving between typing and sending, stale focused field, duplicate first block), plus a Playwright test with two browsers against the real image (e2e/run.sh). Co-Authored-By: Claude Opus 5.5 --- .dockerignore | 1 + .gitignore | 1 + Doku/architecture.md | 2 + Doku/flaws.md | 13 + Doku/modules/workspace.md | 6 +- Doku/plans/kollaboration-notizen.md | 4 +- README.md | 2 + e2e/README.md | 27 + e2e/collab.e2e.mjs | 177 +++++++ e2e/package-lock.json | 57 +++ e2e/package.json | 8 + e2e/run.sh | 18 + frontend/package-lock.json | 11 + frontend/package.json | 1 + .../src/features/workspace/DocumentEditor.tsx | 301 +++++------ .../src/features/workspace/OutlinerTree.tsx | 8 + .../src/lib/docSync/__tests__/diff.test.ts | 96 ++++ .../src/lib/docSync/__tests__/engine.test.ts | 278 ++++++++++ .../lib/docSync/__tests__/orderKey.test.ts | 46 ++ frontend/src/lib/docSync/diff.ts | 139 +++++ frontend/src/lib/docSync/engine.ts | 480 ++++++++++++++++++ frontend/src/lib/docSync/model.ts | 424 ++++++++++++++++ frontend/src/lib/docSync/orderKey.ts | 148 ++++++ frontend/src/lib/docSync/persist.ts | 87 ++++ frontend/src/lib/docSync/transport.ts | 111 ++++ frontend/src/lib/types.ts | 2 + internal/api/docsync.go | 22 + internal/storage/docsync.go | 105 +++- internal/storage/docsync_sim_test.go | 24 +- internal/storage/docsync_test.go | 54 ++ internal/storage/sqlite.go | 22 +- 31 files changed, 2468 insertions(+), 207 deletions(-) create mode 100644 e2e/README.md create mode 100644 e2e/collab.e2e.mjs create mode 100644 e2e/package-lock.json create mode 100644 e2e/package.json create mode 100755 e2e/run.sh create mode 100644 frontend/src/lib/docSync/__tests__/diff.test.ts create mode 100644 frontend/src/lib/docSync/__tests__/engine.test.ts create mode 100644 frontend/src/lib/docSync/__tests__/orderKey.test.ts create mode 100644 frontend/src/lib/docSync/diff.ts create mode 100644 frontend/src/lib/docSync/engine.ts create mode 100644 frontend/src/lib/docSync/model.ts create mode 100644 frontend/src/lib/docSync/orderKey.ts create mode 100644 frontend/src/lib/docSync/persist.ts create mode 100644 frontend/src/lib/docSync/transport.ts diff --git a/.dockerignore b/.dockerignore index fab7364..d6be359 100644 --- a/.dockerignore +++ b/.dockerignore @@ -7,3 +7,4 @@ frontend/dist *.db-shm *.db-wal .env +e2e diff --git a/.gitignore b/.gitignore index 1fc8b72..de8430c 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,4 @@ dev-data/ .DS_Store .idea/ .vscode/ +e2e/node_modules/ diff --git a/Doku/architecture.md b/Doku/architecture.md index 4f5a235..b6a16c6 100644 --- a/Doku/architecture.md +++ b/Doku/architecture.md @@ -39,6 +39,8 @@ Das Frontend liegt per `go:embed` im Binary ([cmd/server/main.go](../cmd/server/ | `internal/config/` | Umgebungsvariablen | | `frontend/src/features/` | `triage`, `workspace`, `lists`, `teams`, `settings`, `capture`, `auth`, `help` | | `frontend/src/lib/` | `api.ts` (REST-Client, Transport-Umschaltung WS/SSE), `types.ts`, `markdown.tsx`, `theme.ts` | +| `frontend/src/lib/docSync/` | Notiz-Synchronisation im Browser: Portierung von `internal/collab` (`model.ts`, `orderKey.ts`), Vergleich Editor-Liste → Operationen (`diff.ts`), `engine.ts`, IndexedDB (`persist.ts`), HTTP (`transport.ts`) | +| `e2e/` | Browser-Tests mit Playwright gegen einen echten Server (`e2e/run.sh`) | | `frontend/src/themes/` | CSS pro Theme | ## Start und Laufzeit diff --git a/Doku/flaws.md b/Doku/flaws.md index f826cef..518f8fc 100644 --- a/Doku/flaws.md +++ b/Doku/flaws.md @@ -34,6 +34,8 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( | FLAW-22 | Netzwerk | Token in der URL (WS/SSE), CORS `*`, `CheckOrigin` immer `true` | Niedrig | | FLAW-23 | SQLite | Single-Writer-Engpass bei vielen gleichzeitigen Schreibern | Niedrig | | FLAW-24 | Gruppen | Organisations-Endpunkte ohne Berechtigungsprüfung | Mittel | +| FLAW-25 | Workspace | Zusammenführen/Einrücken an einem fremd gesperrten Bullet wird nur teilweise ausgeführt | Mittel | +| FLAW-26 | Workspace | Nach Enter landet sehr schnell Getipptes noch im vorherigen Bullet | Niedrig | --- @@ -121,6 +123,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( ### FLAW-14: Verschluckte Fehler - **Fundstellen:** u. a. `PatchDocumentBlocks` (Upserts/Deletes), `AddTaskListShare`, `RemoveTaskListShare`, viele `_, _ = s.db.Exec`. - **Problem:** Scheitert ein Schreibzugriff (z. B. `SQLITE_BUSY`), antwortet die API trotzdem mit Erfolg und broadcastet den Stand, der nie gespeichert wurde. Fehler lassen sich nur schwer nachvollziehen. +- **Beispiel (behoben):** Das Lösen der Verknüpfung zwischen Notiz-Block und Aufgabe („Verknüpfung aufheben“, Löschen eines verknüpften Blocks) schrieb in nicht existierende Spalten `thoughts.doc_id`/`block_id`. Der Fehler wurde verschluckt, die Funktion war nie wirksam. Korrigiert; im neuen Sync-Weg gleicht `ReconcileDocTaskLinks` Häkchen und gelöschte Blöcke ab. - **Maßnahme:** Mehrstufige Änderungen in Transaktionen bündeln und Fehler zurückgeben. ### FLAW-15: WriteTimeout und SSE @@ -172,6 +175,16 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( - **Problem:** Diese Handler prüfen weder Eigentümer noch Mitgliedschaft. Jeder angemeldete Nutzer kann mit bekannter ID eine fremde Organisation löschen, deren Gruppen auflisten oder dort Gruppen anlegen. Gruppenbezogene Endpunkte (`/teams/{id}…`) prüfen dagegen korrekt auf Gruppen-Admin. Das Datenmodell kennt Organisationen nur noch rudimentär; im UI werden Gruppen direkt über `POST /teams` angelegt (`orgId` Standard `default`). - **Maßnahme:** Organisationen entweder konsequent absichern (Eigentümer/Server-Admin) oder entfernen. +### FLAW-25: Teilweise ausgeführte Struktur-Aktionen an gesperrten Bullets +- **Fundstellen:** `handleMergeWithPrevious` und `handleIndentBlock` in [`OutlinerTree.tsx`](../frontend/src/features/workspace/OutlinerTree.tsx); Sperrregeln in `internal/collab` (Verschieben/Löschen gesperrter Blöcke wird abgelehnt). +- **Problem:** Eine Editor-Aktion besteht oft aus mehreren Operationen. Ist einer der beteiligten Bullets von jemand anderem gesperrt, lehnt der Server nur diese Operation ab. Beispiel aus dem Browser-Test: Backspace am Zeilenanfang hängt den Text oben an (angewendet), das Löschen des unteren Bullets wird abgelehnt – der Text steht dann doppelt da. Einrücken eines fremd gesperrten Bullets wird ohne Hinweis zurückgenommen. Es geht kein Text verloren. +- **Maßnahme (Phase 3):** Gesperrte Bullets im Editor anzeigen und Struktur-Aktionen, die sie berühren, gar nicht erst ausführen. + +### FLAW-26: Fokus-Verzögerung nach Enter +- **Fundstelle:** `focusBlockInput` in `OutlinerTree.tsx` setzt den Fokus erst im nächsten Animation-Frame. +- **Problem:** Zeichen, die innerhalb dieses Frames getippt werden (automatisiert oder sehr schnelles Tippen), landen noch im vorherigen Bullet. Bei fremd gesperrtem Bullet entsteht daraus eine Konfliktkopie. +- **Maßnahme:** Fokus nach dem Anlegen synchron setzen (`useLayoutEffect`). + --- ## Aus der früheren Fassung entfernt diff --git a/Doku/modules/workspace.md b/Doku/modules/workspace.md index b384211..4dcc6a1 100644 --- a/Doku/modules/workspace.md +++ b/Doku/modules/workspace.md @@ -17,7 +17,11 @@ Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.ts - **Slash-Befehle:** `/todo`, `/link`, `/h1`, `/h2`, `/h3`, `/code`, `/callout`, `/divider` (Liste in `OutlinerBlock.tsx`). - **Links:** `[[Notiztitel]]`, Markdown-Links `[Titel](url)` und reine URLs werden als Token dargestellt; `LinkModal` hilft beim Verknüpfen von Notizen. - **Einfügen:** Mehrzeiliger Text wird anhand von Markdown-Präfixen und Einrückung in Blöcke zerlegt; Überschriften gruppieren darunterliegenden Text und Code. -- **Speichern:** Der Editor speichert 800 ms nach der letzten Eingabe das gesamte Dokument (`PUT /documents/{id}`). Der schlanke Block-Endpunkt `PATCH /documents/{id}/blocks` wird derzeit vom To-Do-Rollup beim Abhaken genutzt. +- **Speichern:** Jede Notiz hat im Browser eine `DocSyncEngine` ([lib/docSync/](../../frontend/src/lib/docSync/)). Der Editor meldet jede Änderung der Block-Liste; die Engine übersetzt sie in Operationen, speichert sie zuerst in IndexedDB und sendet sie dann gesammelt (300 ms) an `POST /docs/{id}/sync`. Beim Öffnen zeigt sie sofort den gespeicherten Stand, auch offline. +- **Abgleich:** Bei geteilten oder mehrfach geöffneten Notizen fragt die Engine alle 3 s nach Änderungen, sonst alle 30 s, sowie sofort bei Rückkehr ins Netz oder in den Tab. Beim Verlassen sendet sie Ausstehendes mit `keepalive` und gibt ihre Sperren frei. +- **Textkonflikte:** Jede Textänderung trägt die Version, die der Nutzer beim Tippen vor sich hatte. Hat jemand anderes den Bullet inzwischen geändert oder gesperrt, legt der Server eine Konfliktkopie an; es wird nie still überschrieben. +- **Sperren:** Der Bullet mit dem Cursor, verlassene Bullets für 20 s und neu angelegte Bullets werden gehalten. Die Anzeige gesperrter Bullets folgt in Phase 3 des [Plans](../plans/kollaboration-notizen.md). +- Den bisherigen Speicherweg (`PUT /documents/{id}`) nutzt der Editor nur noch für Symbol und Anheften; Titel und Inhalt laufen über die Engine. `PATCH /documents/{id}/blocks` nutzt das To-Do-Rollup; beide Wege landen im selben Änderungs-Log. ## Explorer, Tags, To-Dos diff --git a/Doku/plans/kollaboration-notizen.md b/Doku/plans/kollaboration-notizen.md index 4418fa0..b7e9c1b 100644 --- a/Doku/plans/kollaboration-notizen.md +++ b/Doku/plans/kollaboration-notizen.md @@ -279,8 +279,8 @@ Damit private Notizen keinen unnötigen Aufwand erzeugen, liefert der Server im |---|---|---| | 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Umsortieren (FLAW-03), Stream-Timeout (FLAW-15). **Erledigt.** Die zentrale `docRole` wandert in Phase 1, weil sie auf dem neuen Freigabemodell (`workspace_access`) aufbaut. | Grundlage dicht | | 1 | Operations-Log, `/snapshot`, `/sync`, `/changes`, `/leave`, Konfliktregeln, Leases, zentrale `DocRole`, Simulationstests. **Erledigt.** Abweichungen: Die bestehenden Tabellen `blocks`/`documents` wurden erweitert statt ersetzt (Baum per `parent_id`/`order_key`, alte Daten werden umgewandelt); der bisherige Speicherweg (`PUT`/`PATCH`) läuft als Adapter durch dieselbe Engine. `DocRole` arbeitet auf den vorhandenen Freigabetabellen; deren Zusammenlegung zu `workspace_access` erfolgt mit dem Freigabe-Dialog in Phase 5. `/changes` liefert nur `{headSeq, changed}`, die Daten kommen immer über `/sync`. | Server fertig und abgesichert | -| 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt (Baum statt `level`) | Bearbeiten funktioniert überall, auch offline | -| 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | +| 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt. **Erledigt.** Abweichung: Die Editor-Komponenten arbeiten weiter mit der flachen Liste (Ebene + Reihenfolge); die Engine übersetzt jede Änderung per Vergleich in Operationen (`lib/docSync/diff.ts`). Sperren werden bereits gehalten (Fokus, 20 s Nachlauf, neue Bullets), aber noch nicht angezeigt. Ende-zu-Ende-Test mit zwei Browsern: `e2e/`. | Bearbeiten funktioniert überall, auch offline | +| 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen. Dazu aus Phase 2: gesperrte Bullets schreibgeschützt anzeigen und Struktur-Aktionen auf ihnen gar nicht erst starten (sonst kann Zusammenführen halb ausgeführt werden, siehe FLAW-25); Fokuswechsel nach Enter ohne Frame-Verzögerung (FLAW-26). | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | | 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt | | 5 | Freigabe-Dialog mit Bearbeiter-Rolle, Ordnerfreigaben, Altlasten entfernen (`PUT`-Blockersatz, `PATCH /blocks`, `doc_shares`, `doc_team_shares`, Zwangsmigration auf „read“) | Feature komplett | | 6 | Härtung: Playwright-Tests, Log-Kompaktierung (Snapshot statt Log für Clients mit sehr altem `lastSeq`), Aufräumen der Grabsteine | Langzeitbetrieb | diff --git a/README.md b/README.md index 026fd25..7a22a50 100644 --- a/README.md +++ b/README.md @@ -169,6 +169,8 @@ npm run dev # Vite-Entwicklungsserver npx tsc --noEmit # Typprüfung npm run test:run # Komponententests (Vitest, jsdom) npm run build # Produktions-Build +cd .. +e2e/run.sh # Browser-Tests (Playwright, zwei Benutzer) gegen ein frisch gebautes Image ``` Für das Binary wird das gebaute Frontend nach `cmd/server/dist` kopiert (`go:embed`); das mehrstufige `Dockerfile` erledigt das und führt Typprüfung, Frontend-Tests, Go-Vet und Go-Tests beim Image-Build aus. diff --git a/e2e/README.md b/e2e/README.md new file mode 100644 index 0000000..74f8156 --- /dev/null +++ b/e2e/README.md @@ -0,0 +1,27 @@ +# Ende-zu-Ende-Tests + +Browser-Tests mit Playwright gegen einen echten MindSift-Server. + +```bash +e2e/run.sh # baut das Image, startet es im Demo-Modus und führt die Tests aus +ENGINE=docker e2e/run.sh +``` + +Gegen einen bereits laufenden Server (Demo-Modus mit `admin/admin` und `user/user` nötig): + +```bash +cd e2e && npm install && MINDSIFT_URL=http://localhost:8080 node collab.e2e.mjs +``` + +## collab.e2e.mjs + +Zwei Browser (Eigentümer und Bearbeiter) arbeiten in derselben Notiz. Geprüft wird: + +- Tippen wird gespeichert und erscheint beim anderen ohne Neuladen +- neue Bullets per Enter, in beide Richtungen +- Offline-Phase: Text bleibt lokal und wird nach Rückkehr übertragen +- beide Browser und der Server zeigen denselben Stand, auch nach Neuladen +- Sperren eines geschlossenen Browsers laufen nach spätestens 20 s ab +- Einrücken (Tab) und Zusammenführen (Backspace) bleiben nach Neuladen erhalten + +Der Test dauert wegen des Sperrablaufs etwa eine Minute. Er läuft nicht in der CI, weil er Browser-Images braucht. diff --git a/e2e/collab.e2e.mjs b/e2e/collab.e2e.mjs new file mode 100644 index 0000000..10d6446 --- /dev/null +++ b/e2e/collab.e2e.mjs @@ -0,0 +1,177 @@ +// End-to-end test of collaborative note editing with two real browsers (owner and editor). +// Needs a server in demo mode (accounts admin/admin and user/user); see e2e/README.md. +import { chromium } from 'playwright'; + +const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080'; +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); +let failures = 0; +const check = (cond, msg) => { + console.log(`${cond ? 'OK ' : 'FAIL'} ${msg}`); + if (!cond) failures++; +}; + +async function api(token, method, path, body) { + const res = await fetch(BASE + path, { + method, + headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: `Bearer ${token}` } : {}) }, + body: body ? JSON.stringify(body) : undefined, + }); + const text = await res.text(); + if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`); + return text ? JSON.parse(text) : null; +} + +async function waitFor(fn, ms, label) { + const end = Date.now() + ms; + let last; + while (Date.now() < end) { + try { + last = await fn(); + if (last) return last; + } catch {} + await sleep(250); + } + return null; +} + +const serverBlocks = async (token, docId) => { + const snap = await api(token, 'GET', `/api/v1/docs/${docId}/snapshot`); + return snap.blocks.filter((b) => !b.deleted).map((b) => b.content); +}; + +// Outliner text fields of a page (without the title field) +const editorValues = (page) => + page.$$eval('textarea', (els) => els.filter((e) => !e.placeholder?.startsWith('Titel')).map((e) => e.value)); + +async function openDoc(browser, token, title) { + const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } }); + await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token); + const page = await ctx.newPage(); + page.on('pageerror', (e) => console.log('PAGEERROR', e.message)); + await page.goto(BASE); + await page.getByRole('button', { name: /Workspace/ }).first().click(); + await page.getByText(title, { exact: true }).first().click(); + await page.waitForFunction(() => document.querySelectorAll('textarea').length > 1, null, { timeout: 10000 }); + return { ctx, page }; +} + +const TITLE = `E2E Notiz ${Date.now()}`; +const main = async () => { + const admin = (await api(null, 'POST', '/api/v1/auth/login', { login: 'admin', password: 'admin' })).token; + const user = (await api(null, 'POST', '/api/v1/auth/login', { login: 'user', password: 'user' })).token; + const dir = await api(admin, 'GET', '/api/v1/users/directory'); + const userId = dir.find((u) => u.username === 'user' || u.email?.startsWith('user')).id; + const doc = await api(admin, 'POST', '/api/v1/documents', { + title: TITLE, + blocks: [{ id: 'e2e_1', type: 'bullet', content: 'Erster Punkt', level: 0 }], + }); + await api(admin, 'POST', `/api/v1/documents/${doc.id}/shares`, { userId, permission: 'write' }); + + const browser = await chromium.launch(); + const A = await openDoc(browser, admin, TITLE); + const B = await openDoc(browser, user, TITLE); + check((await editorValues(A.page)).includes('Erster Punkt'), 'Admin sieht die Notiz'); + check((await editorValues(B.page)).includes('Erster Punkt'), 'Bearbeiter sieht die Notiz'); + + // 1. Admin types at the end of the first bullet + await A.page.getByText('Erster Punkt', { exact: true }).first().click(); + await A.page.keyboard.press('End'); + await A.page.keyboard.type(' + A'); + const saved = await waitFor(async () => (await serverBlocks(admin, doc.id)).includes('Erster Punkt + A'), 5000); + check(!!saved, 'Getippter Text liegt auf dem Server'); + const seenByB = await waitFor(async () => (await editorValues(B.page)).includes('Erster Punkt + A'), 8000); + check(!!seenByB, 'Bearbeiter sieht die Änderung ohne Neuladen'); + + // 2. Admin creates a new bullet with Enter + await A.page.keyboard.press('Enter'); + await sleep(150); // a human does not type within the same frame as Enter + await A.page.keyboard.type('Zweiter von A'); + const second = await waitFor(async () => (await editorValues(B.page)).includes('Zweiter von A'), 8000); + check(!!second, 'Neuer Bullet erscheint beim Bearbeiter'); + + // 3. Bearbeiter adds his own bullet at the end + await B.page.getByText('Zweiter von A', { exact: true }).first().click(); + await B.page.keyboard.press('End'); + await B.page.keyboard.press('Enter'); + await sleep(150); + await B.page.keyboard.type('Von B'); + const fromB = await waitFor(async () => (await editorValues(A.page)).includes('Von B'), 8000); + check(!!fromB, 'Bullet des Bearbeiters erscheint beim Admin'); + + // 4. Offline phase on the admin side + await A.ctx.setOffline(true); + await A.page.getByText('Erster Punkt + A', { exact: true }).first().click(); + await A.page.keyboard.press('End'); + await A.page.keyboard.type(' offline'); + await sleep(1500); + check(!(await serverBlocks(admin, doc.id)).includes('Erster Punkt + A offline'), 'Offline getippt: noch nicht auf dem Server'); + await A.ctx.setOffline(false); + await A.page.evaluate(() => window.dispatchEvent(new Event('online'))); + const afterOnline = await waitFor(async () => (await serverBlocks(admin, doc.id)).includes('Erster Punkt + A offline'), 8000); + check(!!afterOnline, 'Nach Rückkehr online wird der Offline-Text übertragen'); + + // 5. Both converge and a reload shows the same state + await sleep(4000); + const final = await serverBlocks(admin, doc.id); + const a = await editorValues(A.page); + const b = await editorValues(B.page); + check(JSON.stringify(a) === JSON.stringify(b), `Beide Browser zeigen dasselbe: ${JSON.stringify(a)}`); + check(final.every((c) => a.includes(c)), `Server-Stand ist sichtbar: ${JSON.stringify(final)}`); + await A.page.reload(); + await A.page.getByRole('button', { name: /Workspace/ }).first().click(); + await A.page.getByText(TITLE, { exact: true }).first().click(); + const reloaded = await waitFor(async () => { + const v = await editorValues(A.page); + return JSON.stringify(v) === JSON.stringify(a) ? v : null; + }, 8000); + check(!!reloaded, 'Nach Neuladen ist alles noch da'); + + // 6. Structure: indent with Tab, merge with Backspace, both survive a reload. B leaves first, so none of + // its block leases are active (with B's cursor in a block the server rightly rejects moving it). + await B.ctx.close(); + // Closing a context abruptly may drop B's "leave" request; its leases then expire after 20 s + const freed = await waitFor(async () => { + const probe = await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'probe', lastSeq: 0 }); + return probe.leases.others.length === 0; + }, 25000); + check(!!freed, 'Sperren eines geschlossenen Browsers laufen nach spätestens 20 s ab'); + await A.page.getByText('Von B', { exact: true }).first().click(); + await A.page.keyboard.press('Home'); + await A.page.keyboard.press('Tab'); + await sleep(1500); + const legacy = await api(admin, 'GET', `/api/v1/documents/${doc.id}`); + const vonB = legacy.blocks.find((b) => b.content === 'Von B'); + check(vonB && vonB.level === 1, `Tab rückt ein (Ebene ${vonB?.level})`); + await A.page.getByText('Zweiter von A', { exact: true }).first().click(); + await A.page.keyboard.press('Home'); + await A.page.keyboard.press('Backspace'); + const merged = await waitFor(async () => (await serverBlocks(admin, doc.id)).includes('Erster Punkt + A offlineZweiter von A'), 5000); + check(!!merged, 'Backspace am Zeilenanfang führt mit dem Bullet darüber zusammen'); + await A.page.reload(); + await A.page.getByRole('button', { name: /Workspace/ }).first().click(); + await A.page.getByText(TITLE, { exact: true }).first().click(); + const afterStructure = await waitFor(async () => { + const v = await editorValues(A.page); + return v.includes('Erster Punkt + A offlineZweiter von A') && v.includes('Von B') && !v.includes('Zweiter von A') ? v : null; + }, 8000); + check(!!afterStructure, `Struktur nach Neuladen erhalten: ${JSON.stringify(afterStructure)}`); + const B2 = await openDoc(browser, user, TITLE); + const bSees = await waitFor(async () => { + const v = await editorValues(B2.page); + return v.includes('Erster Punkt + A offlineZweiter von A') && !v.includes('Zweiter von A') ? v : null; + }, 8000); + check(!!bSees, `Bearbeiter sieht nach erneutem Öffnen denselben Stand: ${JSON.stringify(bSees)}`); + + const log = await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'e2e-check', lastSeq: 0 }); + const conflicts = log.ops.filter((e) => e.effect.kind === 'insert' && e.effect.props?.conflictOf).length; + console.log(`Log: ${log.headSeq} Einträge, ${conflicts} Konfliktkopien`); + + await browser.close(); + console.log(failures ? `${failures} FEHLGESCHLAGEN` : 'ALLE PRÜFUNGEN BESTANDEN'); + process.exit(failures ? 1 : 0); +}; + +main().catch((e) => { + console.error(e); + process.exit(2); +}); diff --git a/e2e/package-lock.json b/e2e/package-lock.json new file mode 100644 index 0000000..d4261ab --- /dev/null +++ b/e2e/package-lock.json @@ -0,0 +1,57 @@ +{ + "name": "mindsift-e2e", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "mindsift-e2e", + "dependencies": { + "playwright": "1.48.2" + } + }, + "node_modules/fsevents": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", + "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==", + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/playwright": { + "version": "1.48.2", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.48.2.tgz", + "integrity": "sha512-NjYvYgp4BPmiwfe31j4gHLa3J7bD2WiBz8Lk2RoSsmX38SVIARZ18VYjxLjAcDsAhA+F4iSEXTSGgjua0rrlgQ==", + "license": "Apache-2.0", + "dependencies": { + "playwright-core": "1.48.2" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "fsevents": "2.3.2" + } + }, + "node_modules/playwright-core": { + "version": "1.48.2", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.48.2.tgz", + "integrity": "sha512-sjjw+qrLFlriJo64du+EK0kJgZzoQPsabGF4lBvsid+3CNIZIYLgnMj9V6JY5VhM2Peh20DJWIVpVljLLnlawA==", + "license": "Apache-2.0", + "bin": { + "playwright-core": "cli.js" + }, + "engines": { + "node": ">=18" + } + } + } +} diff --git a/e2e/package.json b/e2e/package.json new file mode 100644 index 0000000..4aabf5b --- /dev/null +++ b/e2e/package.json @@ -0,0 +1,8 @@ +{ + "name": "mindsift-e2e", + "private": true, + "type": "module", + "dependencies": { + "playwright": "1.48.2" + } +} diff --git a/e2e/run.sh b/e2e/run.sh new file mode 100755 index 0000000..2ac77e9 --- /dev/null +++ b/e2e/run.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# Builds MindSift, starts it in demo mode on port 18080 with a fresh database and runs the browser tests. +# Requires podman (or docker via ENGINE=docker). Usage: e2e/run.sh +set -euo pipefail +ENGINE="${ENGINE:-podman}" +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +WORK="$(mktemp -d)" +trap '"$ENGINE" rm -f mindsift-e2e >/dev/null 2>&1 || true; rm -rf "$WORK"' EXIT + +echo "Baue Image …" +"$ENGINE" build -q -t mindsift-e2e-image "$ROOT" >/dev/null +"$ENGINE" run -d --name mindsift-e2e --network host -e PORT=18080 -e MINDA_DEMO=true \ + -e JWT_SECRET=e2e-only-secret -v "$WORK":/data:Z mindsift-e2e-image >/dev/null +for _ in $(seq 1 60); do curl -s -m 1 http://localhost:18080/api/v1/health >/dev/null && break; sleep 0.5; done + +echo "Starte Browser-Tests …" +"$ENGINE" run --rm --network host -v "$ROOT/e2e":/e2e:Z -w /e2e mcr.microsoft.com/playwright:v1.48.2-jammy \ + sh -c 'npm install --no-audit --no-fund >/dev/null 2>&1 && node collab.e2e.mjs' diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 695fc5d..5fc62ea 100644 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -22,6 +22,7 @@ "@types/react-dom": "^18.3.1", "@vitejs/plugin-react": "^4.3.4", "autoprefixer": "^10.4.20", + "fake-indexeddb": "^6.2.6", "jsdom": "^25.0.1", "postcss": "^8.4.49", "tailwindcss": "^3.4.17", @@ -2238,6 +2239,16 @@ "node": ">=12.0.0" } }, + "node_modules/fake-indexeddb": { + "version": "6.2.6", + "resolved": "https://registry.npmjs.org/fake-indexeddb/-/fake-indexeddb-6.2.6.tgz", + "integrity": "sha512-ihhmr0KoET51jT8R4YhbUYuLLvr1qqO6INISygtN16i8loR8kAxtAUr4+EKHTSMjL+lggaAn4OC7DRqzrNgnuQ==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=18" + } + }, "node_modules/fast-glob": { "version": "3.3.3", "resolved": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.3.3.tgz", diff --git a/frontend/package.json b/frontend/package.json index 09c81cf..a502f5d 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -26,6 +26,7 @@ "@types/react-dom": "^18.3.1", "@vitejs/plugin-react": "^4.3.4", "autoprefixer": "^10.4.20", + "fake-indexeddb": "^6.2.6", "jsdom": "^25.0.1", "postcss": "^8.4.49", "tailwindcss": "^3.4.17", diff --git a/frontend/src/features/workspace/DocumentEditor.tsx b/frontend/src/features/workspace/DocumentEditor.tsx index 8b8c1b7..c533eff 100644 --- a/frontend/src/features/workspace/DocumentEditor.tsx +++ b/frontend/src/features/workspace/DocumentEditor.tsx @@ -16,7 +16,8 @@ import { Search, } from 'lucide-react'; import { Document, DocumentBlock, User } from '../../lib/types'; -import { api, getToken } from '../../lib/api'; +import { api } from '../../lib/api'; +import { DocSyncEngine, EngineState } from '../../lib/docSync/engine'; import { OutlinerTree, OutlinerTreeHandle } from './OutlinerTree'; export type DocSaveStatus = 'saved' | 'saving' | 'unsaved' | 'error'; @@ -119,6 +120,29 @@ function ensureNonEmptyBlocks(docId: string, blocks?: DocumentBlock[]): Document })); } +// True if both lists show the same blocks; avoids re-rendering when the sync engine echoes a local change. +function sameBlocks(a: DocumentBlock[], b: DocumentBlock[]): boolean { + if (a.length !== b.length) return false; + for (let i = 0; i < a.length; i++) { + const x = a[i]; + const y = b[i]; + if ( + x.id !== y.id || + x.content !== y.content || + (x.level || 0) !== (y.level || 0) || + x.type !== y.type || + !!x.checked !== !!y.checked || + !!x.collapsed !== !!y.collapsed || + (x.lang || '') !== (y.lang || '') || + (x.taskId || '') !== (y.taskId || '') || + (x.conflictOf || '') !== (y.conflictOf || '') + ) { + return false; + } + } + return true; +} + export interface BreadcrumbNode { id: string | null; title: string; @@ -224,6 +248,10 @@ export const DocumentEditor: React.FC = ({ const [emojiSearch, setEmojiSearch] = useState(''); const [zoomedBlockId, setZoomedBlockId] = useState(null); + // Sync engine (operation log): stores every change locally and syncs it with the server + const engineRef = useRef(null); + const [engineState, setEngineState] = useState(null); + const filteredEmojis = useMemo(() => { const q = emojiSearch.trim().toLowerCase(); if (!q) return EMOJI_CATALOG; @@ -232,13 +260,6 @@ export const DocumentEditor: React.FC = ({ ); }, [emojiSearch]); - // AutoSave state (REST persistence) - const [, setSaveStatus] = useState('saved'); - const [, setSaveErrorMessage] = useState(null); - const saveTimeoutRef = useRef(null); - const isSavingRef = useRef(false); - const lastSavedSnapshotRef = useRef(''); - const isInitialLoadRef = useRef(true); const titleInputRef = useRef(null); const outlinerTreeRef = useRef(null); @@ -249,7 +270,6 @@ export const DocumentEditor: React.FC = ({ currentTitleRef.current = title; const currentBlocksRef = useRef(blocks); currentBlocksRef.current = blocks; - const lastKnownUpdatedAtRef = useRef(initialDoc?.updatedAt || null); useEffect(() => { if (!loading && (autoFocusTitle || (doc?.title === 'Neue Notiz' && (!doc?.blocks || doc?.blocks.length <= 1)))) { @@ -283,11 +303,12 @@ export const DocumentEditor: React.FC = ({ return computeBreadcrumbs(title || doc?.title || 'Dokument', blocks, zoomedBlockId); }, [title, doc?.title, blocks, zoomedBlockId]); - // Strictly Read-Only for shared documents: only the document owner can edit! + // Owner and editors may edit; the role comes from the server via the sync engine const canEdit = useMemo(() => { - if (!doc) return true; - return Boolean(doc.isOwner || doc.permission === 'owner'); - }, [doc]); + if (engineState?.role) return engineState.role === 'owner' || engineState.role === 'editor'; + if (!doc) return false; + return Boolean(doc.isOwner || doc.permission === 'owner' || doc.permission === 'write'); + }, [doc, engineState?.role]); // Extract all #tags present across the current blocks const extractedTags = useMemo(() => { @@ -303,124 +324,6 @@ export const DocumentEditor: React.FC = ({ return Array.from(tagSet); }, [blocks]); - const currentTagsRef = useRef(extractedTags); - currentTagsRef.current = extractedTags; - - // AutoSave snapshot - const currentSnapshot = useMemo(() => { - return JSON.stringify({ - title, - blocks, - tags: extractedTags, - }); - }, [title, blocks, extractedTags]); - - const currentSnapshotRef = useRef(currentSnapshot); - currentSnapshotRef.current = currentSnapshot; - - const executeAutoSave = useCallback( - async (immediate: boolean = false) => { - if (!docId || !canEdit) return; - if (isSavingRef.current && !immediate) return; - - const snapshot = currentSnapshotRef.current; - if (snapshot === lastSavedSnapshotRef.current) { - setSaveStatus('saved'); - onSaveStatusChange?.('saved'); - return; - } - - isSavingRef.current = true; - setSaveStatus('saving'); - onSaveStatusChange?.('saving'); - - try { - const updated = await api.updateDocument(docId, { - title: currentTitleRef.current, - blocks: currentBlocksRef.current, - tags: currentTagsRef.current, - }); - isSavingRef.current = false; - lastSavedSnapshotRef.current = snapshot; - lastKnownUpdatedAtRef.current = updated.updatedAt || null; - setSaveStatus('saved'); - setSaveErrorMessage(null); - onSaveStatusChange?.('saved'); - if (docRef.current) { - docRef.current = updated; - onDocMetadataChange(updated); - } - } catch (err: any) { - isSavingRef.current = false; - const msg = err?.message || 'Fehler beim Speichern'; - setSaveStatus('error'); - setSaveErrorMessage(msg); - onSaveStatusChange?.('error', msg); - } - }, - [docId, canEdit, onSaveStatusChange, onDocMetadataChange] - ); - - useEffect(() => { - if (isInitialLoadRef.current) return; - if (!canEdit) return; - if (currentSnapshot === lastSavedSnapshotRef.current) return; - - setSaveStatus('unsaved'); - onSaveStatusChange?.('unsaved'); - - if (saveTimeoutRef.current !== null) { - window.clearTimeout(saveTimeoutRef.current); - } - saveTimeoutRef.current = window.setTimeout(() => { - saveTimeoutRef.current = null; - executeAutoSave(); - }, 800); - - return () => { - if (saveTimeoutRef.current !== null) { - window.clearTimeout(saveTimeoutRef.current); - saveTimeoutRef.current = null; - } - }; - }, [currentSnapshot, canEdit, executeAutoSave, onSaveStatusChange]); - - // Flush on page unload / pagehide via keepalive fetch - useEffect(() => { - const handleUnload = () => { - if (saveTimeoutRef.current !== null) { - window.clearTimeout(saveTimeoutRef.current); - saveTimeoutRef.current = null; - } - const snapshot = currentSnapshotRef.current; - if (snapshot !== lastSavedSnapshotRef.current && canEdit) { - const token = getToken(); - if (token) { - fetch(`/api/v1/documents/${docId}`, { - method: 'PUT', - headers: { - 'Content-Type': 'application/json', - Authorization: `Bearer ${token}`, - }, - body: JSON.stringify({ - title: currentTitleRef.current, - blocks: currentBlocksRef.current, - tags: currentTagsRef.current, - }), - keepalive: true, - }).catch(() => {}); - } - } - }; - - window.addEventListener('beforeunload', handleUnload); - window.addEventListener('pagehide', handleUnload); - return () => { - window.removeEventListener('beforeunload', handleUnload); - window.removeEventListener('pagehide', handleUnload); - handleUnload(); - }; - }, [docId, canEdit]); // Fetch document on mount useEffect(() => { @@ -436,29 +339,19 @@ export const DocumentEditor: React.FC = ({ if (isCancelled) return; const normalized = ensureNonEmptyBlocks(docId, fetched.blocks); setDoc(fetched); - setTitle(fetched.title || 'Unbenannte Notiz'); - setBlocks(normalized); + if (!engineRef.current?.getState().ready) { + setTitle(fetched.title || 'Unbenannte Notiz'); + currentBlocksRef.current = normalized; + setBlocks(normalized); + } + const links = new Map(); + for (const b of fetched.blocks || []) { + if (b.taskId) links.set(b.id, b.taskId); + } + engineRef.current?.setTaskLinks(links); setLoading(false); onDocMetadataChange(fetched); - const initialTags: string[] = []; - for (const b of normalized) { - const matches = b.content?.match(/#([a-zA-Z0-9äöüÄÖÜß_-]+)/g); - if (matches) { - for (const m of matches) { - const tag = m.slice(1).toLowerCase(); - if (!initialTags.includes(tag)) initialTags.push(tag); - } - } - } - lastSavedSnapshotRef.current = JSON.stringify({ - title: fetched.title || 'Unbenannte Notiz', - blocks: normalized, - tags: initialTags, - }); - isInitialLoadRef.current = false; - setSaveStatus('saved'); - onSaveStatusChange?.('saved'); }) .catch((err: any) => { if (isCancelled) return; @@ -471,6 +364,60 @@ export const DocumentEditor: React.FC = ({ }; }, [docId]); + // Start the sync engine for this note: shows stored state immediately (also offline), then syncs + useEffect(() => { + const engine = new DocSyncEngine(docId); + engineRef.current = engine; + const unsubscribe = engine.subscribe((state) => { + setEngineState(state); + if (!state.ready) return; + const next = ensureNonEmptyBlocks(docId, state.blocks); + if (!sameBlocks(currentBlocksRef.current, next)) { + currentBlocksRef.current = next; + setBlocks(next); + } + if (state.title && state.title !== currentTitleRef.current) { + setTitle(state.title); + } + }); + void engine.start(); + return () => { + unsubscribe(); + engine.stop(); + if (engineRef.current === engine) engineRef.current = null; + }; + }, [docId]); + + useEffect(() => { + if (!engineState) return; + switch (engineState.status) { + case 'saving': + onSaveStatusChange?.('saving'); + break; + case 'offline': + onSaveStatusChange?.('unsaved', 'Offline – Änderungen sind lokal gesichert und werden gesendet, sobald die Verbindung zurück ist'); + break; + case 'error': + onSaveStatusChange?.('error', engineState.errorMessage); + break; + default: + onSaveStatusChange?.('saved'); + } + }, [engineState?.status, engineState?.errorMessage]); + + // Every local block change goes to the engine, which turns it into operations + const applyLocalBlocks = useCallback( + (update: DocumentBlock[] | ((prev: DocumentBlock[]) => DocumentBlock[])) => { + const prev = currentBlocksRef.current; + const next = typeof update === 'function' ? update(prev) : update; + if (next === prev) return; + currentBlocksRef.current = next; + setBlocks(next); + engineRef.current?.applyFlat(next); + }, + [] + ); + // Keep parent sidebar metadata in sync when blocks/tags change useEffect(() => { if (!docRef.current) return; @@ -488,27 +435,19 @@ export const DocumentEditor: React.FC = ({ const handleTitleChange = (e: React.ChangeEvent) => { if (!canEdit) return; setTitle(e.target.value); + engineRef.current?.setTitle(e.target.value); }; const handleTitleBlur = () => { - executeAutoSave(); + void engineRef.current?.syncNow(); }; const handleUpdateBlock = useCallback( - (updatedBlock: DocumentBlock, immediate: boolean = false) => { + (updatedBlock: DocumentBlock) => { if (!canEdit) return; - setBlocks((prev) => { - const next = prev.map((b) => (b.id === updatedBlock.id ? updatedBlock : b)); - currentBlocksRef.current = next; - return next; - }); - if (immediate) { - window.setTimeout(() => { - executeAutoSave(true); - }, 10); - } + applyLocalBlocks((prev) => prev.map((b) => (b.id === updatedBlock.id ? updatedBlock : b))); }, - [canEdit, executeAutoSave] + [canEdit, applyLocalBlocks] ); const handleUpdateBlocks = useCallback( @@ -518,17 +457,15 @@ export const DocumentEditor: React.FC = ({ for (const b of updatedBlocks) { map.set(b.id, b); } - setBlocks((prev) => - prev.map((b) => map.get(b.id) || b) - ); + applyLocalBlocks((prev) => prev.map((b) => map.get(b.id) || b)); }, - [canEdit] + [canEdit, applyLocalBlocks] ); const handleCreateBlock = useCallback( (newBlock: DocumentBlock, afterBlockId?: string) => { if (!canEdit) return; - setBlocks((prev) => { + applyLocalBlocks((prev) => { const next = [...prev]; if (afterBlockId) { const idx = next.findIndex((b) => b.id === afterBlockId); @@ -541,13 +478,13 @@ export const DocumentEditor: React.FC = ({ return ensureNonEmptyBlocks(docId, next); }); }, - [docId, canEdit] + [docId, canEdit, applyLocalBlocks] ); const handleInsertBlocks = useCallback( (afterBlockId: string, updatedCurrentBlock: DocumentBlock, newBlocks: DocumentBlock[]) => { if (!canEdit) return; - setBlocks((prev) => { + applyLocalBlocks((prev) => { const next = [...prev]; const idx = next.findIndex((b) => b.id === afterBlockId); if (idx >= 0) { @@ -556,32 +493,24 @@ export const DocumentEditor: React.FC = ({ } else { next.push(updatedCurrentBlock, ...newBlocks); } - const normalized = ensureNonEmptyBlocks(docId, next); - currentBlocksRef.current = normalized; - return normalized; + return ensureNonEmptyBlocks(docId, next); }); - window.setTimeout(() => { - executeAutoSave(true); - }, 50); }, - [docId, canEdit, executeAutoSave] + [docId, canEdit, applyLocalBlocks] ); const handleDeleteBlock = useCallback( (blockId: string) => { if (!canEdit) return; - setBlocks((prev) => { - const filtered = prev.filter((b) => b.id !== blockId); - return ensureNonEmptyBlocks(docId, filtered); - }); + applyLocalBlocks((prev) => ensureNonEmptyBlocks(docId, prev.filter((b) => b.id !== blockId))); }, - [docId, canEdit] + [docId, canEdit, applyLocalBlocks] ); const handleMoveBlock = useCallback( (blockId: string, targetIndex: number, newLevel?: number) => { if (!canEdit) return; - setBlocks((prev) => { + applyLocalBlocks((prev) => { const fromIdx = prev.findIndex((b) => b.id === blockId); if (fromIdx === -1) return prev; @@ -632,7 +561,7 @@ export const DocumentEditor: React.FC = ({ return ensureNonEmptyBlocks(docId, copy); }); }, - [docId, canEdit] + [docId, canEdit, applyLocalBlocks] ); const handleTogglePin = async () => { @@ -977,6 +906,7 @@ export const DocumentEditor: React.FC = ({ onTagClick={onSelectTag} onWikiLinkClick={onWikiLinkClick} initialFocusedBlockId={initialFocusedBlockId} + onFocusedBlockChange={(id) => engineRef.current?.focusBlock(id)} /> )} @@ -1113,6 +1043,7 @@ export const DocumentEditor: React.FC = ({ onTagClick={onSelectTag} onWikiLinkClick={onWikiLinkClick} initialFocusedBlockId={initialFocusedBlockId} + onFocusedBlockChange={(id) => engineRef.current?.focusBlock(id)} /> )} diff --git a/frontend/src/features/workspace/OutlinerTree.tsx b/frontend/src/features/workspace/OutlinerTree.tsx index 6254449..26bee05 100644 --- a/frontend/src/features/workspace/OutlinerTree.tsx +++ b/frontend/src/features/workspace/OutlinerTree.tsx @@ -23,6 +23,7 @@ export interface OutlinerTreeProps { onTagClick?: (tag: string) => void; onWikiLinkClick?: (pageTitle: string) => void; initialFocusedBlockId?: string | null; + onFocusedBlockChange?: (blockId: string | null) => void; } export const OutlinerTree = forwardRef(({ @@ -41,8 +42,15 @@ export const OutlinerTree = forwardRef(({ onTagClick, onWikiLinkClick, initialFocusedBlockId, + onFocusedBlockChange, }, ref) => { const [focusedBlockId, setFocusedBlockId] = useState(null); + + const onFocusedBlockChangeRef = useRef(onFocusedBlockChange); + onFocusedBlockChangeRef.current = onFocusedBlockChange; + useEffect(() => { + onFocusedBlockChangeRef.current?.(focusedBlockId); + }, [focusedBlockId]); const [highlightedBlockId, setHighlightedBlockId] = useState( initialFocusedBlockId || null ); diff --git a/frontend/src/lib/docSync/__tests__/diff.test.ts b/frontend/src/lib/docSync/__tests__/diff.test.ts new file mode 100644 index 0000000..54713aa --- /dev/null +++ b/frontend/src/lib/docSync/__tests__/diff.test.ts @@ -0,0 +1,96 @@ +import { describe, it, expect } from 'vitest'; +import type { DocumentBlock } from '../../types'; +import { diffFlat, toDocumentBlocks } from '../diff'; +import { applyEffect, applyOp, cloneDoc, docsEqual, emptyDoc, Doc } from '../model'; + +let opSeq = 0; +const nextOp = () => `op${++opSeq}`; + +function build(blocks: DocumentBlock[]): Doc { + const d = emptyDoc(); + for (const op of diffFlat(d, blocks, nextOp)) applyOp(d, op); + return d; +} + +const B = (id: string, level: number, content = id, extra: Partial = {}): DocumentBlock => ({ + id, + level, + content, + type: 'bullet', + ...extra, +}); + +const shape = (blocks: DocumentBlock[]) => blocks.map((b) => [b.id, b.level || 0, b.content, !!b.checked, b.type]); + +describe('diffFlat', () => { + it('builds the tree from levels and produces no ops for an unchanged list', () => { + const list = [B('a', 0), B('b', 1), B('c', 2), B('d', 1), B('e', 0)]; + const d = build(list); + expect(shape(toDocumentBlocks(d, 'doc'))).toEqual(shape(list)); + expect(diffFlat(d, toDocumentBlocks(d, 'doc'), nextOp)).toEqual([]); + }); + + it('turns typing into a single set_content without moves', () => { + const d = build([B('a', 0), B('b', 0), B('c', 0)]); + const ops = diffFlat(d, [B('a', 0), B('b', 0, 'b geändert'), B('c', 0)], nextOp); + expect(ops.map((o) => o.kind)).toEqual(['set_content']); + }); + + it('inserting in the middle does not move the neighbours', () => { + const d = build([B('a', 0), B('c', 0)]); + const ops = diffFlat(d, [B('a', 0), B('b', 0), B('c', 0)], nextOp); + expect(ops.map((o) => o.kind)).toEqual(['insert']); + }); + + it('reaches any target list (random edits)', () => { + let seed = 99; + const rnd = (n: number) => { + seed = (seed * 1103515245 + 12345) % 2147483648; + return Math.floor((seed / 2147483648) * n); + }; + let current: DocumentBlock[] = [B('x0', 0)]; + let d = build(current); + for (let step = 0; step < 300; step++) { + const next = current.map((b) => ({ ...b })); + const i = rnd(next.length); + switch (rnd(6)) { + case 0: + next.splice(i + 1, 0, B(`x${step + 1}`, next[i] ? next[i].level || 0 : 0)); + break; + case 1: + if (next.length > 1) next.splice(i, 1); + break; + case 2: + next[i].content += '!'; + break; + case 3: + next[i].level = (next[i].level || 0) + 1; + break; + case 4: + next[i].level = Math.max(0, (next[i].level || 0) - 1); + break; + case 5: { + const [moved] = next.splice(i, 1); + next.splice(rnd(next.length + 1), 0, moved); + break; + } + } + // Normalise like the editor would (levels can only grow by one) + let prev = -1; + for (const b of next) { + b.level = Math.min(b.level || 0, prev + 1); + prev = b.level; + } + const before = cloneDoc(d); + const replay = cloneDoc(d); + for (const op of diffFlat(d, next, nextOp)) { + const { effects } = applyOp(d, op); + effects.forEach((e) => applyEffect(replay, e)); + } + expect(shape(toDocumentBlocks(d, 'doc'))).toEqual(shape(next)); + expect(docsEqual(d, replay)).toBe(true); + expect(before).not.toBe(d); + current = next; + } + }); +}); diff --git a/frontend/src/lib/docSync/__tests__/engine.test.ts b/frontend/src/lib/docSync/__tests__/engine.test.ts new file mode 100644 index 0000000..0af7190 --- /dev/null +++ b/frontend/src/lib/docSync/__tests__/engine.test.ts @@ -0,0 +1,278 @@ +import 'fake-indexeddb/auto'; +import { describe, it, expect } from 'vitest'; +import type { DocumentBlock } from '../../types'; +import { DocSyncEngine, formBatch } from '../engine'; +import { applyOp, cloneDoc, Doc, emptyDoc, Op, Result } from '../model'; +import { IndexedDBStorage, MemoryStorage } from '../persist'; +import { LogEntry, SnapshotResponse, SyncRequest, SyncResponse, Transport, TransportError } from '../transport'; + +/** In-memory stand-in for the Go server: same operation rules, log and idempotency. */ +class FakeServer { + doc: Doc = emptyDoc(); + log: LogEntry[] = []; + results = new Map(); + roles = new Map(); // clientId -> role + applyCount = new Map(); + + constructor(initial: { id: string; content: string }[] = [{ id: 'b1', content: 'Start' }]) { + let key = 'a0'; + for (const b of initial) { + this.apply({ opId: `seed-${b.id}`, kind: 'insert', blockId: b.id, parentId: '', orderKey: key, content: b.content }, 'seed'); + key = String.fromCharCode(key.charCodeAt(0)) + String.fromCharCode(key.charCodeAt(1) + 1); + } + } + + apply(op: Op, clientId: string): Result { + const known = this.results.get(op.opId); + if (known) return known; + this.applyCount.set(op.opId, (this.applyCount.get(op.opId) || 0) + 1); + const { effects, result } = applyOp(this.doc, op, clientId); + for (const e of effects) this.log.push({ seq: this.log.length + 1, userId: clientId, clientId, opId: op.opId, effect: e }); + if (effects.length > 0) result.seq = this.log.length; + this.results.set(op.opId, result); + return result; + } + + snapshot(): SnapshotResponse { + return { + docId: 'doc', + title: this.doc.title, + titleVersion: this.doc.titleVersion, + headSeq: this.log.length, + blocks: Array.from(cloneDoc(this.doc).blocks.values()), + role: 'owner', + shared: true, + }; + } + + sync(body: SyncRequest): SyncResponse { + const role = this.roles.get(body.clientId) || 'owner'; + if (body.ops.length > 0 && role === 'viewer') throw new TransportError('nur Lesen', 403); + const results = body.ops.map((op) => this.apply(op, body.clientId)); + return { + results, + ops: this.log.filter((e) => e.seq > body.lastSeq), + headSeq: this.log.length, + hasMore: false, + role, + shared: true, + sessions: 2, + presence: [], + leases: { granted: [], denied: [], others: [] }, + }; + } +} + +/** Network between one client and the fake server, with switchable failures. */ +class FakeNet implements Transport { + mode: 'ok' | 'down' | 'lose-response' = 'ok'; + duringRequest: (() => void) | null = null; // runs while the request is "on the wire" + + constructor(private server: FakeServer, private role = 'owner') {} + async snapshot() { + if (this.mode === 'down') throw new TransportError('offline', 0); + return { ...this.server.snapshot(), role: this.role }; + } + async sync(_docId: string, body: SyncRequest) { + if (this.mode === 'down') throw new TransportError('offline', 0); + this.server.roles.set(body.clientId, this.role); + const hook = this.duringRequest; + this.duringRequest = null; + hook?.(); + const res = this.server.sync(body); + if (this.mode === 'lose-response') throw new TransportError('timeout', 0); + return res; + } + async leave() {} +} + +function flatOf(engine: DocSyncEngine): DocumentBlock[] { + return engine.getState().blocks; +} + +function edit(engine: DocSyncEngine, fn: (blocks: DocumentBlock[]) => DocumentBlock[]) { + engine.applyFlat(fn(flatOf(engine).map((b) => ({ ...b })))); +} + +const contents = (engine: DocSyncEngine) => flatOf(engine).map((b) => b.content); +const serverContents = (server: FakeServer) => docContents(server.doc); +function docContents(d: Doc): string[] { + return Array.from(d.blocks.values()) + .filter((b) => !b.deleted) + .sort((a, b) => (a.orderKey < b.orderKey ? -1 : a.orderKey > b.orderKey ? 1 : a.id < b.id ? -1 : 1)) + .map((b) => b.content); +} + +async function startEngine(net: Transport, storage = new MemoryStorage()) { + const engine = new DocSyncEngine('doc', { transport: net, storage, manual: true }); + await engine.start(); + return engine; +} + +describe('DocSyncEngine', () => { + it('sends edits and confirms them', async () => { + const server = new FakeServer(); + const engine = await startEngine(new FakeNet(server)); + edit(engine, (b) => { + b[0].content = 'Hallo'; + return [...b, { id: 'b2', type: 'bullet', content: 'Neu', level: 1 }]; + }); + expect(engine.getState().pendingCount).toBeGreaterThan(0); + await engine.syncNow(); + expect(engine.getState().status).toBe('saved'); + expect(engine.getState().pendingCount).toBe(0); + expect(serverContents(server)).toEqual(['Hallo', 'Neu']); + }); + + it('coalesces fast typing into one text operation', async () => { + const server = new FakeServer(); + const engine = await startEngine(new FakeNet(server)); + for (const text of ['H', 'Ha', 'Hal', 'Hall', 'Hallo']) edit(engine, (b) => [{ ...b[0], content: text }]); + await engine.syncNow(); + const textOps = server.log.filter((e) => e.effect.kind === 'set_content'); + expect(textOps).toHaveLength(1); + expect(textOps[0].effect.content).toBe('Hallo'); + }); + + it('keeps edits through an offline phase and a reload, and applies them exactly once', async () => { + const server = new FakeServer(); + const storage = new MemoryStorage(); + const net = new FakeNet(server); + const first = await startEngine(net, storage); + + net.mode = 'down'; + edit(first, (b) => [{ ...b[0], content: 'Offline getippt' }, { id: 'b9', type: 'bullet', content: 'Noch einer', level: 0 }]); + await new Promise((r) => setTimeout(r, 0)); // let the queue reach storage + await first.syncNow(); + expect(first.getState().status).toBe('offline'); + expect(contents(first)).toEqual(['Offline getippt', 'Noch einer']); + + // Tab is closed while offline; a new tab opens the note from storage, still offline + const second = await startEngine(net, storage); + expect(contents(second)).toEqual(['Offline getippt', 'Noch einer']); + + net.mode = 'ok'; + await second.syncNow(); + await second.syncNow(); + expect(serverContents(server)).toEqual(['Offline getippt', 'Noch einer']); + for (const count of server.applyCount.values()) expect(count).toBe(1); + }); + + it('retries after a lost response without applying twice', async () => { + const server = new FakeServer(); + const net = new FakeNet(server); + const engine = await startEngine(net); + edit(engine, (b) => [...b, { id: 'b2', type: 'bullet', content: 'Einmal', level: 0 }]); + net.mode = 'lose-response'; + await engine.syncNow(); + expect(engine.getState().status).toBe('offline'); + net.mode = 'ok'; + await engine.syncNow(); + expect(serverContents(server)).toEqual(['Start', 'Einmal']); + expect(server.log.filter((e) => e.effect.kind === 'insert' && e.effect.blockId === 'b2')).toHaveLength(1); + }); + + it('two clients converge, and a conflict keeps both texts', async () => { + const server = new FakeServer(); + const a = await startEngine(new FakeNet(server)); + const b = await startEngine(new FakeNet(server)); + + edit(a, (blocks) => [{ ...blocks[0], content: 'Von A' }]); + edit(b, (blocks) => [{ ...blocks[0], content: 'Von B' }]); + await a.syncNow(); + await b.syncNow(); // B's base version is stale -> conflict copy + await a.syncNow(); + + expect(contents(a)).toEqual(contents(b)); + expect(contents(a)).toEqual(['Von A', 'Von B']); + expect(flatOf(a)[1].conflictOf).toBe('b1'); + expect(b.getState().conflicts).toHaveLength(1); + }); + + it('keeps text when two clients create the same first block of an empty note', async () => { + const server = new FakeServer([]); + const a = await startEngine(new FakeNet(server)); + const b = await startEngine(new FakeNet(server)); + a.applyFlat([{ id: 'doc-b0', type: 'text', content: 'Text von A', level: 0 }]); + b.applyFlat([{ id: 'doc-b0', type: 'text', content: 'Text von B', level: 0 }]); + await a.syncNow(); + await b.syncNow(); // insert rejected: block exists + await b.syncNow(); // text resent as set_content -> conflict copy + await a.syncNow(); + expect(contents(a)).toEqual(contents(b)); + expect(contents(a)).toEqual(expect.arrayContaining(['Text von A', 'Text von B'])); + }); + + it('does not overwrite a change that arrived while the user kept typing', async () => { + const server = new FakeServer([ + { id: 'b1', content: 'Start' }, + { id: 'b2', content: 'Anderer Punkt' }, + ]); + const netA = new FakeNet(server); + const a = await startEngine(netA); + const b = await startEngine(new FakeNet(server)); + + edit(a, (blocks) => [blocks[0], { ...blocks[1], content: 'Anderer Punkt!' }]); + netA.duringRequest = () => { + // While A's request travels: B's change of b1 reaches the server first, and A starts typing into + // b1 (still seeing 'Start') + server.apply({ opId: 'b-op', kind: 'set_content', blockId: 'b1', baseVersion: 1, content: 'B war schneller' }, 'cB'); + edit(a, (blocks) => [{ ...blocks[0], content: 'Start + A' }, blocks[1]]); + }; + await a.syncNow(); // response brings B's change; A's text for b1 is still pending + await a.syncNow(); + await b.syncNow(); + + expect(contents(a)).toEqual(contents(b)); + expect(contents(a)).toEqual(expect.arrayContaining(['B war schneller', 'Start + A', 'Anderer Punkt!'])); + }); + + it('a stale focused text field cannot overwrite a newer remote version', async () => { + const server = new FakeServer(); + const a = await startEngine(new FakeNet(server)); + const b = await startEngine(new FakeNet(server)); + + a.focusBlock('b1'); // A's cursor stays in the block; its field keeps showing 'Start' + edit(b, (blocks) => [{ ...blocks[0], content: 'B hat geändert' }]); + await b.syncNow(); + await a.syncNow(); // A receives B's version, but the focused field still shows the old text + + a.applyFlat([{ id: 'b1', type: 'bullet', content: 'Start + A', level: 0 }]); + await a.syncNow(); + await b.syncNow(); + + expect(contents(a)).toEqual(contents(b)); + expect(contents(a)).toEqual(expect.arrayContaining(['B hat geändert', 'Start + A'])); + }); + + it('viewers cannot change anything', async () => { + const server = new FakeServer(); + const viewer = await startEngine(new FakeNet(server, 'viewer')); + edit(viewer, (b) => [{ ...b[0], content: 'Leser' }]); + await viewer.syncNow(); + expect(viewer.getState().pendingCount).toBe(0); + expect(serverContents(server)).toEqual(['Start']); + }); + + it('formBatch keeps the last text per block with the base of the first', () => { + const batch = formBatch([ + { opId: '1', kind: 'set_content', blockId: 'a', content: 'x', baseVersion: 5 }, + { opId: '2', kind: 'set_title', title: 'T1', baseVersion: 3 }, + { opId: '3', kind: 'set_content', blockId: 'a', content: 'xy', baseVersion: 6 }, + { opId: '4', kind: 'set_title', title: 'T2', baseVersion: 4 }, + ]); + expect(batch).toEqual([ + { opId: '3', kind: 'set_content', blockId: 'a', content: 'xy', baseVersion: 5 }, + { opId: '4', kind: 'set_title', title: 'T2', baseVersion: 3 }, + ]); + }); + + it('stores state in IndexedDB', async () => { + const storage = new IndexedDBStorage(indexedDB); + await storage.save({ docId: 'd', title: 'T', titleVersion: 1, lastSeq: 7, role: 'owner', blocks: [], pending: [], inflight: null, savedAt: 1 }); + const loaded = await storage.load('d'); + expect(loaded?.lastSeq).toBe(7); + expect(await storage.load('missing')).toBeNull(); + }); +}); + diff --git a/frontend/src/lib/docSync/__tests__/orderKey.test.ts b/frontend/src/lib/docSync/__tests__/orderKey.test.ts new file mode 100644 index 0000000..c4afe60 --- /dev/null +++ b/frontend/src/lib/docSync/__tests__/orderKey.test.ts @@ -0,0 +1,46 @@ +import { describe, it, expect } from 'vitest'; +import { keyBetween, keysBetween, validOrderKey } from '../orderKey'; + +// Generated with internal/collab (Go); both implementations must agree exactly. +const VECTORS: [string, string, string][] = [ + ['', '', 'a0'], + ['a0', '', 'a1'], + ['', 'a0', 'Zz'], + ['a0', 'a1', 'a0V'], + ['a0', 'a0V', 'a0F'], + ['a0V', 'a1', 'a0k'], + ['Zz', 'a0', 'ZzV'], + ['az', '', 'b00'], + ['', 'Zz', 'Zy'], + ['a1', 'a1V', 'a1F'], + ['b00', 'b01', 'b00V'], +]; + +describe('order keys', () => { + it('match the Go implementation', () => { + for (const [a, b, want] of VECTORS) expect(keyBetween(a, b)).toBe(want); + expect(keysBetween('a0', 'a1', 5)).toEqual(['a07', 'a0F', 'a0V', 'a0c', 'a0k']); + let last = ''; + for (let i = 0; i < 70; i++) last = keyBetween(last, ''); + expect(last).toBe('b07'); + }); + + it('rejects invalid ranges and keys', () => { + expect(() => keyBetween('zz', '')).toThrow(); + expect(() => keyBetween('a1', 'a0')).toThrow(); + expect(validOrderKey('a0V0')).toBe(false); + }); + + it('stays sorted under random inserts', () => { + const keys: string[] = []; + let seed = 7; + const rnd = () => (seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648; + for (let i = 0; i < 2000; i++) { + const pos = Math.floor(rnd() * (keys.length + 1)); + keys.splice(pos, 0, keyBetween(keys[pos - 1] || '', keys[pos] || '')); + } + const sorted = [...keys].sort(); + expect(keys).toEqual(sorted); + expect(new Set(keys).size).toBe(keys.length); + }); +}); diff --git a/frontend/src/lib/docSync/diff.ts b/frontend/src/lib/docSync/diff.ts new file mode 100644 index 0000000..bc5fbe0 --- /dev/null +++ b/frontend/src/lib/docSync/diff.ts @@ -0,0 +1,139 @@ +// Turns the flat block list the editor works with (level + order) into operations against the tree. +// Same algorithm as legacyOps in internal/storage/docsync.go. +import type { DocumentBlock } from '../types'; +import { keysBetween } from './orderKey'; +import { Doc, Op, flatten, normProps, propsEqual } from './model'; + +/** Parent index (-1 = top level) for every entry of a flat list; impossible levels are clamped. */ +export function parentsFromLevels(levels: number[]): number[] { + const parents: number[] = []; + const stack: number[] = []; + levels.forEach((raw, i) => { + let lvl = Math.max(0, raw || 0); + if (lvl > stack.length) lvl = stack.length; + stack.length = lvl; + parents.push(lvl > 0 ? stack[lvl - 1] : -1); + stack.push(i); + }); + return parents; +} + +/** Marks a longest strictly increasing subsequence of the non-empty keys. */ +export function longestIncreasing(keys: string[]): boolean[] { + const keep = new Array(keys.length).fill(false); + const tails: number[] = []; + const prev = new Array(keys.length).fill(-1); + keys.forEach((k, i) => { + if (!k) return; + let lo = 0; + let hi = tails.length; + while (lo < hi) { + const mid = (lo + hi) >> 1; + if (keys[tails[mid]] < k) lo = mid + 1; + else hi = mid; + } + if (lo > 0) prev[i] = tails[lo - 1]; + if (lo === tails.length) tails.push(i); + else tails[lo] = i; + }); + if (tails.length > 0) { + for (let i = tails[tails.length - 1]; i >= 0; i = prev[i]) keep[i] = true; + } + return keep; +} + +/** + * Computes the operations that turn the tree into the given flat list. newOpId creates unique op IDs. + * Blocks that keep their place keep their order key, so only real changes become operations. + */ +export function diffFlat(d: Doc, blocks: DocumentBlock[], newOpId: () => string): Op[] { + const list = blocks.map((b) => ({ ...b, type: b.type || 'bullet' })); + const parentIdx = parentsFromLevels(list.map((b) => b.level || 0)); + const parentOf = list.map((_, i) => (parentIdx[i] >= 0 ? list[parentIdx[i]].id : '')); + + const byParent = new Map(); + list.forEach((_, i) => { + const p = parentOf[i]; + const arr = byParent.get(p); + if (arr) arr.push(i); + else byParent.set(p, [i]); + }); + + const keys: string[] = new Array(list.length); + byParent.forEach((idxs, parent) => { + const existing = idxs.map((i) => { + const b = d.blocks.get(list[i].id); + return b && !b.deleted && b.parentId === parent ? b.orderKey : ''; + }); + const keep = longestIncreasing(existing); + let j = 0; + while (j < idxs.length) { + if (keep[j]) { + keys[idxs[j]] = existing[j]; + j++; + continue; + } + const start = j; + while (j < idxs.length && !keep[j]) j++; + const left = start > 0 ? keys[idxs[start - 1]] : ''; + const right = j < idxs.length ? existing[j] : ''; + let fresh: string[]; + try { + fresh = keysBetween(left, right, j - start); + } catch { + fresh = keysBetween(left, '', j - start); + } + for (let k = start; k < j; k++) keys[idxs[k]] = fresh[k - start]; + } + }); + + const ops: Op[] = []; + const push = (op: Omit) => ops.push({ opId: newOpId(), ...op } as Op); + const wanted = new Set(); + + list.forEach((b, i) => { + wanted.add(b.id); + const parent = parentOf[i]; + const props = normProps({ checked: b.checked, collapsed: b.collapsed, lang: b.lang }); + const cur = d.blocks.get(b.id); + if (!cur) { + push({ kind: 'insert', blockId: b.id, parentId: parent, orderKey: keys[i], type: b.type, content: b.content || '', props }); + return; + } + if (cur.deleted) push({ kind: 'restore', ids: [b.id] }); + if (cur.deleted || cur.parentId !== parent || cur.orderKey !== keys[i]) { + push({ kind: 'move', blockId: b.id, parentId: parent, orderKey: keys[i] }); + } + if (cur.content !== (b.content || '')) { + push({ kind: 'set_content', blockId: b.id, baseVersion: cur.version, content: b.content || '' }); + } + const curProps = normProps({ checked: cur.props.checked, collapsed: cur.props.collapsed, lang: cur.props.lang }); + if (cur.type !== b.type || !propsEqual(curProps, props)) { + push({ kind: 'set_props', blockId: b.id, patch: { type: b.type, checked: !!b.checked, collapsed: !!b.collapsed, lang: b.lang || '' } }); + } + }); + + // Deletes last, after moves have taken wanted children out of deleted branches + for (const fb of flatten(d)) { + if (!wanted.has(fb.id)) push({ kind: 'delete', blockId: fb.id }); + } + return ops; +} + +/** Converts the tree into the flat list used by the editor components. */ +export function toDocumentBlocks(d: Doc, docId: string, taskLinks?: Map): DocumentBlock[] { + return flatten(d).map((b, i) => ({ + id: b.id, + documentId: docId, + type: b.type as DocumentBlock['type'], + content: b.content, + checked: !!b.props.checked, + collapsed: !!b.props.collapsed, + lang: b.props.lang || undefined, + level: b.level, + position: (i + 1) * 1000, + taskId: taskLinks?.get(b.id), + conflictOf: b.props.conflictOf, + conflictAuthor: b.props.conflictAuthor, + })); +} diff --git a/frontend/src/lib/docSync/engine.ts b/frontend/src/lib/docSync/engine.ts new file mode 100644 index 0000000..ff977b4 --- /dev/null +++ b/frontend/src/lib/docSync/engine.ts @@ -0,0 +1,480 @@ +// DocSyncEngine keeps one note in sync with the server: confirmed state from the server log, plus the +// user's own unconfirmed operations on top. Every change is stored in IndexedDB before it is sent, sent +// with a unique op ID (safe to resend), and the server decides about conflicts. See +// Doku/plans/kollaboration-notizen.md. +import type { DocumentBlock } from '../types'; +import { diffFlat, toDocumentBlocks } from './diff'; +import { applyEffect, applyOp, cloneDoc, Doc, docFromSnapshot, emptyDoc, Op, Result } from './model'; +import { defaultStorage, SyncStorage } from './persist'; +import { httpTransport, LeaseInfo, Presence, Transport, TransportError } from './transport'; + +export type EngineStatus = 'loading' | 'saved' | 'saving' | 'offline' | 'error'; + +export interface EngineState { + ready: boolean; + status: EngineStatus; + errorMessage?: string; + role: string; // owner | editor | viewer | '' (unknown yet) + title: string; + blocks: DocumentBlock[]; + pendingCount: number; + shared: boolean; + sessions: number; + presence: Presence[]; + lockedByOthers: LeaseInfo[]; + conflicts: Result[]; // conflicts of own operations since start +} + +export interface EngineOptions { + transport?: Transport; + storage?: SyncStorage; + now?: () => number; + /** Disables timers and window listeners (tests drive the engine with syncNow()). */ + manual?: boolean; +} + +const LEASE_GRACE_MS = 20_000; // keep a left block for this long +const IDLE_RELEASE_MS = 60_000; // release even the focused block after this long without input +const SYNC_DEBOUNCE_MS = 300; +const POLL_COLLAB_MS = 3_000; +const POLL_SOLO_MS = 30_000; +const BACKOFF_MAX_MS = 30_000; + +function randomHex(bytes: number): string { + const arr = new Uint8Array(bytes); + crypto.getRandomValues(arr); + return Array.from(arr, (b) => b.toString(16).padStart(2, '0')).join(''); +} + +/** Random block ID for new blocks (unique across clients). */ +export function newBlockId(): string { + return `blk_${randomHex(10)}`; +} + +/** + * Keeps only the last text change per block (and the last title change). The base version is the one of + * the first change: all later keystrokes build on what the user saw when starting to type, so a change + * from someone else in between is detected as a conflict instead of being overwritten. + */ +export function formBatch(pending: Op[]): Op[] { + const lastText = new Map(); + const firstTextBase = new Map(); + let lastTitle = -1; + let firstTitleBase: number | undefined; + pending.forEach((op, i) => { + if (op.kind === 'set_content') { + lastText.set(op.blockId!, i); + if (!firstTextBase.has(op.blockId!)) firstTextBase.set(op.blockId!, op.baseVersion); + } + if (op.kind === 'set_title') { + if (lastTitle < 0) firstTitleBase = op.baseVersion; + lastTitle = i; + } + }); + const batch: Op[] = []; + pending.forEach((op, i) => { + if (op.kind === 'set_content') { + if (lastText.get(op.blockId!) !== i) return; + batch.push({ ...op, baseVersion: firstTextBase.get(op.blockId!) ?? 0 }); + return; + } + if (op.kind === 'set_title') { + if (i !== lastTitle) return; + batch.push({ ...op, baseVersion: firstTitleBase ?? 0 }); + return; + } + batch.push(op); + }); + return batch; +} + +export class DocSyncEngine { + readonly clientId = `c_${randomHex(8)}`; + private transport: Transport; + private storage: SyncStorage; + private now: () => number; + private manual: boolean; + + private confirmed: Doc = emptyDoc(); + private lastSeq = 0; + private pending: Op[] = []; + private inflight: Op[] | null = null; + private local: Doc = emptyDoc(); + private opCounter = 0; + + private ready = false; + private status: EngineStatus = 'loading'; + private errorMessage?: string; + private role = ''; + private shared = false; + private sessions = 0; + private presence: Presence[] = []; + private lockedByOthers: LeaseInfo[] = []; + private conflicts: Result[] = []; + private taskLinks = new Map(); + + private focused: string | null = null; + private activity = new Map(); // blockId -> last edit/focus time + // Version of each block the editor last showed in agreement with the engine. The focused text field does + // not take over remote changes while typing, so its edits are based on this version, not the newest one. + private uiSeen = new Map(); + + private syncing = false; + private syncAgain = false; + private stopped = false; + private timer: ReturnType | null = null; + private backoff = 0; + private listeners = new Set<(s: EngineState) => void>(); + private cleanup: Array<() => void> = []; + + constructor(readonly docId: string, opts: EngineOptions = {}) { + this.transport = opts.transport || httpTransport; + this.storage = opts.storage || defaultStorage(); + this.now = opts.now || (() => Date.now()); + this.manual = !!opts.manual; + } + + // --- Public API --- + + /** Loads the stored state (instant, works offline), then syncs with the server. */ + async start(): Promise { + try { + const stored = await this.storage.load(this.docId); + if (stored) { + this.confirmed = docFromSnapshot(stored.title, stored.titleVersion, stored.blocks); + this.lastSeq = stored.lastSeq; + this.role = stored.role || ''; + this.pending = stored.pending || []; + this.inflight = stored.inflight || null; + this.ready = true; + this.recompute(); + } + } catch { + // Storage unavailable: continue with the server state only + } + if (!this.manual) this.attachWindowListeners(); + if (!this.ready) { + try { + await this.loadSnapshot(); + } catch (err) { + this.handleError(err); + this.emit(); + this.schedule(this.nextBackoff()); + return; + } + } + this.emit(); + await this.syncNow(); + } + + /** Flushes pending changes (keepalive) and leaves the note. */ + stop(): void { + if (this.stopped) return; + this.flushOnExit(); + this.stopped = true; + if (this.timer) clearTimeout(this.timer); + this.cleanup.forEach((fn) => fn()); + this.cleanup = []; + this.listeners.clear(); + } + + subscribe(fn: (s: EngineState) => void): () => void { + this.listeners.add(fn); + fn(this.getState()); + return () => this.listeners.delete(fn); + } + + getState(): EngineState { + return { + ready: this.ready, + status: this.status, + errorMessage: this.errorMessage, + role: this.role, + title: this.local.title, + blocks: toDocumentBlocks(this.local, this.docId, this.taskLinks), + pendingCount: this.pending.length + (this.inflight?.length || 0), + shared: this.shared, + sessions: this.sessions, + presence: this.presence, + lockedByOthers: this.lockedByOthers, + conflicts: this.conflicts, + }; + } + + get canEdit(): boolean { + return this.role === 'owner' || this.role === 'editor'; + } + + /** Task IDs of blocks linked to tasks (kept outside the synced tree). */ + setTaskLinks(links: Map): void { + this.taskLinks = new Map(links); + this.emit(); + } + + /** Records the editor's complete block list; differences become operations. */ + applyFlat(blocks: DocumentBlock[]): void { + if (!this.ready || !this.canEdit) return; + for (const b of blocks) { + if (b.taskId) this.taskLinks.set(b.id, b.taskId); + } + for (const b of blocks) { + const cur = this.local.blocks.get(b.id); + if (cur && !cur.deleted && cur.content === (b.content || '')) this.uiSeen.set(b.id, cur.version); + } + const ops = diffFlat(this.local, blocks, () => this.newOpId()); + if (ops.length === 0) return; + const t = this.now(); + for (const op of ops) { + if ((op.kind === 'insert' || op.kind === 'set_content') && op.blockId) this.activity.set(op.blockId, t); + if (op.kind === 'set_content' && this.uiSeen.has(op.blockId!)) op.baseVersion = this.uiSeen.get(op.blockId!); + } + this.enqueue(ops); + // The editor now shows exactly the local state for these blocks + for (const op of ops) { + if (op.kind === 'set_content' || op.kind === 'insert') { + const cur = this.local.blocks.get(op.blockId!); + if (cur) this.uiSeen.set(op.blockId!, cur.version); + } + } + } + + setTitle(title: string): void { + if (!this.ready || !this.canEdit || title === this.local.title) return; + // The title field always shows the engine's title, so the current local version is what the user saw + this.enqueue([{ opId: this.newOpId(), kind: 'set_title', title, baseVersion: this.local.titleVersion }]); + } + + /** Tells the engine which block the cursor is in (null = none); used for block leases. */ + focusBlock(blockId: string | null): void { + const t = this.now(); + if (this.focused) this.activity.set(this.focused, t); + this.focused = blockId; + if (blockId) this.activity.set(blockId, t); + } + + /** Runs one sync round trip now (or right after the running one). */ + async syncNow(): Promise { + if (this.stopped) return; + if (this.syncing) { + this.syncAgain = true; + return; + } + if (this.timer) { + clearTimeout(this.timer); + this.timer = null; + } + this.syncing = true; + let delay = this.pollDelay(); + try { + await this.roundTrip(); + this.backoff = 0; + if (this.syncAgain || this.pending.length > 0) delay = 0; + } catch (err) { + this.handleError(err); + delay = this.status === 'error' ? this.pollDelay() : this.nextBackoff(); + } finally { + this.syncing = false; + this.syncAgain = false; + this.recompute(); + this.emit(); + this.schedule(delay); + } + } + + // --- Internals --- + + private newOpId(): string { + this.opCounter++; + return `${this.clientId}.${this.opCounter}`; + } + + private enqueue(ops: Op[]): void { + this.pending.push(...ops); + this.recompute(); + if (this.status !== 'offline' && this.status !== 'error') this.status = 'saving'; + this.emit(); + // Store first, then send: typed text must survive a crash between the two + void this.persist().finally(() => this.schedule(SYNC_DEBOUNCE_MS)); + } + + private recompute(): void { + const v = cloneDoc(this.confirmed); + for (const op of [...(this.inflight || []), ...this.pending]) { + const local = { ...op }; + if (local.kind === 'set_content') local.baseVersion = v.blocks.get(local.blockId!)?.version ?? 0; + if (local.kind === 'set_title') local.baseVersion = v.titleVersion; + applyOp(v, local); + } + this.local = v; + } + + private emit(): void { + // Unfocused blocks take over whatever the engine shows; only the focused text field may lag behind + this.local.blocks.forEach((b, id) => { + if (id !== this.focused) this.uiSeen.set(id, b.version); + }); + if (this.listeners.size === 0) return; + const s = this.getState(); + this.listeners.forEach((fn) => fn(s)); + } + + private async persist(): Promise { + try { + await this.storage.save({ + docId: this.docId, + title: this.confirmed.title, + titleVersion: this.confirmed.titleVersion, + lastSeq: this.lastSeq, + role: this.role, + blocks: Array.from(this.confirmed.blocks.values()), + pending: this.pending, + inflight: this.inflight, + savedAt: this.now(), + }); + } catch { + // Storage full or unavailable: changes stay in memory and are still sent + } + } + + private async loadSnapshot(): Promise { + const snap = await this.transport.snapshot(this.docId); + this.confirmed = docFromSnapshot(snap.title, snap.titleVersion, snap.blocks || []); + this.lastSeq = snap.headSeq; + this.role = snap.role; + this.shared = snap.shared; + this.ready = true; + this.recompute(); + await this.persist(); + } + + private holds(): string[] { + const t = this.now(); + const out: string[] = []; + this.activity.forEach((at, id) => { + const fresh = t - at < LEASE_GRACE_MS || (id === this.focused && t - at < IDLE_RELEASE_MS); + if (fresh && this.local.blocks.has(id)) out.push(id); + else if (!fresh) this.activity.delete(id); + }); + return out.sort(); + } + + private collaborative(): boolean { + return this.shared || this.sessions > 1; + } + + private async roundTrip(): Promise { + if (!this.ready) await this.loadSnapshot(); + if (!this.inflight && this.pending.length > 0 && this.canEdit) { + this.inflight = formBatch(this.pending); + this.pending = []; + await this.persist(); + } + const ops = this.canEdit ? this.inflight || [] : []; + if (ops.length > 0) this.status = 'saving'; + + for (;;) { + const res = await this.transport.sync(this.docId, { + clientId: this.clientId, + lastSeq: this.lastSeq, + ops: this.inflight ? ops : [], + leases: this.canEdit && this.collaborative() ? { hold: this.holds(), release: [] } : undefined, + }); + if (this.inflight) { + const sent = new Map(this.inflight.map((op) => [op.opId, op])); + for (const r of res.results) { + if (r.status === 'conflict') this.conflicts = [...this.conflicts, r]; + // Someone else created a block with the same ID first (e.g. the first block of an empty note): + // keep the typed text by sending it as a regular text change of that block + const op = sent.get(r.opId); + if (r.status === 'rejected' && r.reason === 'exists' && op?.kind === 'insert' && op.content) { + // Base 0: the user never saw the other client's version of this block + this.pending.unshift({ opId: this.newOpId(), kind: 'set_content', blockId: op.blockId, content: op.content, baseVersion: 0 }); + } + } + this.inflight = null; + } + this.role = res.role; + this.shared = res.shared; + this.sessions = res.sessions; + this.presence = res.presence || []; + this.lockedByOthers = res.leases?.others || []; + + let gap = !!res.resync; + for (const entry of res.ops) { + if (entry.seq <= this.lastSeq) continue; + if (entry.seq !== this.lastSeq + 1) { + gap = true; + break; + } + applyEffect(this.confirmed, entry.effect); + this.lastSeq = entry.seq; + } + if (gap) await this.loadSnapshot(); + await this.persist(); + if (!res.hasMore || gap) break; + } + this.errorMessage = undefined; + this.status = this.pending.length > 0 ? 'saving' : 'saved'; + } + + private handleError(err: unknown): void { + if (err instanceof TransportError && (err.status === 403 || err.status === 404)) { + // Access lost: keep the queue (nothing is thrown away), but stop sending + this.status = 'error'; + this.errorMessage = err.message; + if (err.status === 404) this.role = ''; + else if (this.role !== 'owner') this.role = 'viewer'; + return; + } + this.status = 'offline'; + this.errorMessage = err instanceof Error ? err.message : String(err); + } + + private pollDelay(): number { + return this.collaborative() ? POLL_COLLAB_MS : POLL_SOLO_MS; + } + + private nextBackoff(): number { + this.backoff = this.backoff ? Math.min(this.backoff * 2, BACKOFF_MAX_MS) : 1000; + return this.backoff; + } + + private schedule(delay: number): void { + if (this.manual || this.stopped) return; + if (this.timer) clearTimeout(this.timer); + this.timer = setTimeout(() => { + this.timer = null; + void this.syncNow(); + }, delay); + } + + /** Sends what is pending with keepalive (survives page unload) and releases leases. */ + private flushOnExit(): void { + if (this.canEdit && (this.inflight || this.pending.length > 0)) { + const ops = this.inflight || formBatch(this.pending); + this.transport + .sync(this.docId, { clientId: this.clientId, lastSeq: this.lastSeq, ops }, { keepalive: true }) + .catch(() => { + // Not delivered: the queue is in IndexedDB and is resent on the next visit (same op IDs) + }); + } + this.transport.leave(this.docId, this.clientId).catch(() => {}); + } + + private attachWindowListeners(): void { + const onOnline = () => void this.syncNow(); + const onVisibility = () => { + if (document.visibilityState === 'hidden') this.flushOnExit(); + else void this.syncNow(); + }; + const onPageHide = () => this.flushOnExit(); + window.addEventListener('online', onOnline); + document.addEventListener('visibilitychange', onVisibility); + window.addEventListener('pagehide', onPageHide); + this.cleanup.push( + () => window.removeEventListener('online', onOnline), + () => document.removeEventListener('visibilitychange', onVisibility), + () => window.removeEventListener('pagehide', onPageHide) + ); + } +} diff --git a/frontend/src/lib/docSync/model.ts b/frontend/src/lib/docSync/model.ts new file mode 100644 index 0000000..c7bf319 --- /dev/null +++ b/frontend/src/lib/docSync/model.ts @@ -0,0 +1,424 @@ +// Block tree, operations and effects of collaborative note editing. Port of internal/collab (doc.go, ops.go). +// applyEffect must behave exactly like the server's Doc.ApplyEffect, because clients replay the server log. +import { keyBetween, validOrderKey } from './orderKey'; + +export interface Props { + checked?: boolean; + collapsed?: boolean; + lang?: string; + conflictOf?: string; + conflictAuthor?: string; +} + +export interface Block { + id: string; + parentId: string; // '' = top level + orderKey: string; + type: string; + content: string; + props: Props; + version: number; + deleted?: boolean; + updatedBy?: string; +} + +export interface Doc { + title: string; + titleVersion: number; + blocks: Map; +} + +export interface PropsPatch { + type?: string; + checked?: boolean; + collapsed?: boolean; + lang?: string; +} + +export type OpKind = 'insert' | 'set_content' | 'set_props' | 'move' | 'delete' | 'restore' | 'set_title'; + +export interface Op { + opId: string; + kind: OpKind; + blockId?: string; + baseVersion?: number; + parentId?: string; + orderKey?: string; + type?: string; + content?: string; + props?: Props; + patch?: PropsPatch; + ids?: string[]; + title?: string; +} + +export interface Effect { + kind: OpKind; + blockId?: string; + parentId?: string; + orderKey?: string; + type?: string; + content?: string; + props?: Props; + version?: number; + ids?: string[]; + title?: string; + titleVersion?: number; + by?: string; +} + +export interface Result { + opId: string; + status: 'applied' | 'conflict' | 'rejected'; + reason?: string; + version?: number; + conflictBlockId?: string; + seq?: number; +} + +export interface FlatBlock extends Block { + level: number; +} + +export function emptyDoc(): Doc { + return { title: '', titleVersion: 0, blocks: new Map() }; +} + +export function cloneDoc(d: Doc): Doc { + const blocks = new Map(); + d.blocks.forEach((b, id) => blocks.set(id, { ...b, props: { ...b.props } })); + return { title: d.title, titleVersion: d.titleVersion, blocks }; +} + +/** Normalizes props so that missing and false/empty values compare equal (the server omits them in JSON). */ +export function normProps(p?: Props): Props { + const out: Props = {}; + if (p?.checked) out.checked = true; + if (p?.collapsed) out.collapsed = true; + if (p?.lang) out.lang = p.lang; + if (p?.conflictOf) out.conflictOf = p.conflictOf; + if (p?.conflictAuthor) out.conflictAuthor = p.conflictAuthor; + return out; +} + +export function propsEqual(a?: Props, b?: Props): boolean { + const x = normProps(a); + const y = normProps(b); + return ( + x.checked === y.checked && + x.collapsed === y.collapsed && + x.lang === y.lang && + x.conflictOf === y.conflictOf && + x.conflictAuthor === y.conflictAuthor + ); +} + +function siblingLess(a: Block, b: Block): number { + if (a.orderKey !== b.orderKey) return a.orderKey < b.orderKey ? -1 : 1; + return a.id < b.id ? -1 : a.id > b.id ? 1 : 0; +} + +/** Live children of parentId in display order. */ +export function children(d: Doc, parentId: string): Block[] { + const out: Block[] = []; + d.blocks.forEach((b) => { + if (!b.deleted && b.parentId === parentId) out.push(b); + }); + return out.sort(siblingLess); +} + +/** All live blocks in display order with their nesting level. */ +export function flatten(d: Doc): FlatBlock[] { + const byParent = new Map(); + d.blocks.forEach((b) => { + if (b.deleted) return; + const list = byParent.get(b.parentId); + if (list) list.push(b); + else byParent.set(b.parentId, [b]); + }); + byParent.forEach((list) => list.sort(siblingLess)); + const out: FlatBlock[] = []; + const walk = (parent: string, level: number) => { + for (const b of byParent.get(parent) || []) { + out.push({ ...b, level }); + walk(b.id, level + 1); + } + }; + walk('', 0); + return out; +} + +function subtree(d: Doc, id: string): string[] { + const kids = new Map(); + d.blocks.forEach((b) => { + if (b.deleted) return; + const list = kids.get(b.parentId); + if (list) list.push(b.id); + else kids.set(b.parentId, [b.id]); + }); + const out = [id]; + for (let i = 0; i < out.length; i++) out.push(...(kids.get(out[i]) || [])); + return [out[0], ...out.slice(1).sort()]; +} + +function liveAncestor(d: Doc, id: string): string { + const seen = new Set(); + while (id && !seen.has(id)) { + seen.add(id); + const b = d.blocks.get(id); + if (!b) return ''; + if (!b.deleted) return id; + id = b.parentId; + } + return ''; +} + +function isAncestor(d: Doc, ancestorId: string, id: string): boolean { + const seen = new Set(); + while (id && !seen.has(id)) { + if (id === ancestorId) return true; + seen.add(id); + const b = d.blocks.get(id); + if (!b) return false; + id = b.parentId; + } + return false; +} + +/** Mutates the document with one effect, exactly like the server. */ +export function applyEffect(d: Doc, e: Effect): void { + switch (e.kind) { + case 'insert': + d.blocks.set(e.blockId!, { + id: e.blockId!, + parentId: e.parentId || '', + orderKey: e.orderKey || '', + type: e.type || '', + content: e.content || '', + props: normProps(e.props), + version: e.version || 0, + deleted: false, + updatedBy: e.by || '', + }); + break; + case 'set_content': { + const b = d.blocks.get(e.blockId!); + if (b) { + b.content = e.content || ''; + b.version = e.version || 0; + b.updatedBy = e.by || ''; + } + break; + } + case 'set_props': { + const b = d.blocks.get(e.blockId!); + if (b) { + b.type = e.type || ''; + if (e.props) b.props = normProps(e.props); + } + break; + } + case 'move': { + const b = d.blocks.get(e.blockId!); + if (b) { + b.parentId = e.parentId || ''; + b.orderKey = e.orderKey || ''; + } + break; + } + case 'delete': + for (const id of e.ids || []) { + const b = d.blocks.get(id); + if (b) b.deleted = true; + } + break; + case 'restore': + for (const id of e.ids || []) { + const b = d.blocks.get(id); + if (b) b.deleted = false; + } + break; + case 'set_title': + d.title = e.title || ''; + d.titleVersion = e.titleVersion || 0; + break; + } +} + +/** + * Applies an operation the way the server does (without leases). The client uses it to show its own + * unconfirmed operations on top of the confirmed state; tests use it as a stand-in server. + * conflictId derives the ID of a conflict copy (the server uses a hash of the op ID). + */ +export function applyOp(d: Doc, op: Op, by = '', conflictId: (opId: string) => string = (id) => `blk_cf_${id}`): { effects: Effect[]; result: Result } { + const effects: Effect[] = []; + const emit = (e: Effect) => { + e.by = by; + applyEffect(d, e); + effects.push(e); + }; + const done = (r: Omit) => ({ effects, result: { opId: op.opId, ...r } as Result }); + const reject = (reason: string) => done({ status: 'rejected', reason }); + + const restore = (ids: string[]) => { + const set = new Set(ids); + const sorted = [...ids].sort(); + emit({ kind: 'restore', ids: sorted }); + for (const id of sorted) { + const b = d.blocks.get(id)!; + if (b.parentId === '' || set.has(b.parentId)) continue; + const p = d.blocks.get(b.parentId); + if (p && !p.deleted) continue; + emit({ kind: 'move', blockId: b.id, parentId: liveAncestor(d, b.parentId), orderKey: b.orderKey }); + } + }; + const keyAfter = (b: Block): string => { + const sibs = children(d, b.parentId); + let next = ''; + for (const s of sibs) { + if (s.orderKey > b.orderKey) { + next = s.orderKey; + break; + } + } + try { + return keyBetween(b.orderKey, next); + } catch { + return keyBetween(b.orderKey, ''); + } + }; + + switch (op.kind) { + case 'insert': { + if (!op.blockId || op.blockId.length > 64) return reject('invalid_id'); + if (d.blocks.has(op.blockId)) return reject('exists'); + let parent = op.parentId || ''; + if (parent) { + if (!d.blocks.has(parent)) return reject('parent_missing'); + parent = liveAncestor(d, parent); + } + if (!op.orderKey || !validOrderKey(op.orderKey)) return reject('invalid_order_key'); + emit({ + kind: 'insert', + blockId: op.blockId, + parentId: parent, + orderKey: op.orderKey, + type: op.type || 'bullet', + content: op.content || '', + props: normProps(op.props), + version: 1, + }); + return done({ status: 'applied', version: 1 }); + } + case 'set_content': { + const b = d.blocks.get(op.blockId || ''); + if (!b) return reject('missing'); + if (op.content === undefined) return reject('no_content'); + if (b.deleted) restore([b.id]); + if (op.content === b.content) return done({ status: 'applied', version: b.version }); + if ((op.baseVersion ?? 0) !== b.version) { + const id = conflictId(op.opId); + if (!d.blocks.has(id)) { + emit({ + kind: 'insert', + blockId: id, + parentId: b.parentId, + orderKey: keyAfter(b), + type: b.type, + content: op.content, + props: { conflictOf: b.id }, + version: 1, + }); + } + return done({ status: 'conflict', version: b.version, conflictBlockId: id }); + } + emit({ kind: 'set_content', blockId: b.id, content: op.content, version: b.version + 1 }); + return done({ status: 'applied', version: b.version }); + } + case 'set_props': { + const b = d.blocks.get(op.blockId || ''); + if (!b || b.deleted) return reject('missing'); + if (!op.patch) return reject('no_patch'); + const type = op.patch.type ? op.patch.type : b.type; + const props = normProps({ + ...b.props, + ...(op.patch.checked !== undefined ? { checked: op.patch.checked } : {}), + ...(op.patch.collapsed !== undefined ? { collapsed: op.patch.collapsed } : {}), + ...(op.patch.lang !== undefined ? { lang: op.patch.lang } : {}), + }); + if (type !== b.type || !propsEqual(props, b.props)) emit({ kind: 'set_props', blockId: b.id, type, props }); + return done({ status: 'applied' }); + } + case 'move': { + const b = d.blocks.get(op.blockId || ''); + if (!b || b.deleted) return reject('missing'); + const parent = op.parentId || ''; + if (parent) { + const p = d.blocks.get(parent); + if (!p || p.deleted) return reject('parent_missing'); + if (isAncestor(d, b.id, parent)) return reject('cycle'); + } + if (!op.orderKey || !validOrderKey(op.orderKey)) return reject('invalid_order_key'); + if (parent !== b.parentId || op.orderKey !== b.orderKey) emit({ kind: 'move', blockId: b.id, parentId: parent, orderKey: op.orderKey }); + return done({ status: 'applied' }); + } + case 'delete': { + const b = d.blocks.get(op.blockId || ''); + if (!b) return reject('missing'); + if (!b.deleted) emit({ kind: 'delete', ids: subtree(d, b.id) }); + return done({ status: 'applied' }); + } + case 'restore': { + const ids = (op.ids || []).filter((id) => d.blocks.get(id)?.deleted); + if (ids.length > 0) restore(ids); + return done({ status: 'applied' }); + } + case 'set_title': { + if (op.title === undefined) return reject('no_title'); + if (op.title === d.title) return done({ status: 'applied', version: d.titleVersion }); + if ((op.baseVersion ?? 0) !== d.titleVersion) return done({ status: 'conflict', version: d.titleVersion }); + emit({ kind: 'set_title', title: op.title, titleVersion: d.titleVersion + 1 }); + return done({ status: 'applied', version: d.titleVersion }); + } + } + return reject('unknown_kind'); +} + +/** Builds a Doc from snapshot blocks (as delivered by GET /docs/{id}/snapshot). */ +export function docFromSnapshot(title: string, titleVersion: number, blocks: Block[]): Doc { + const d: Doc = { title, titleVersion, blocks: new Map() }; + for (const b of blocks) { + d.blocks.set(b.id, { + id: b.id, + parentId: b.parentId || '', + orderKey: b.orderKey, + type: b.type || '', + content: b.content || '', + props: normProps(b.props), + version: b.version || 0, + deleted: !!b.deleted, + updatedBy: b.updatedBy || '', + }); + } + return d; +} + +export function docsEqual(a: Doc, b: Doc): boolean { + if (a.title !== b.title || a.titleVersion !== b.titleVersion || a.blocks.size !== b.blocks.size) return false; + for (const [id, x] of a.blocks) { + const y = b.blocks.get(id); + if ( + !y || + x.parentId !== y.parentId || + x.orderKey !== y.orderKey || + x.type !== y.type || + x.content !== y.content || + x.version !== y.version || + !!x.deleted !== !!y.deleted || + !propsEqual(x.props, y.props) + ) { + return false; + } + } + return true; +} diff --git a/frontend/src/lib/docSync/orderKey.ts b/frontend/src/lib/docSync/orderKey.ts new file mode 100644 index 0000000..674aaa8 --- /dev/null +++ b/frontend/src/lib/docSync/orderKey.ts @@ -0,0 +1,148 @@ +// Order keys sort siblings (ties broken by block ID). Port of internal/collab/orderkey.go; both sides must +// produce identical keys, see orderKey.test.ts for shared test vectors. +const DIGITS = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'; +const SMALLEST_INTEGER = 'A00000000000000000000000000'; + +function digitIndex(c: string): number { + return DIGITS.indexOf(c); +} + +function integerLength(head: string): number { + if (head >= 'a' && head <= 'z') return head.charCodeAt(0) - 'a'.charCodeAt(0) + 2; + if (head >= 'A' && head <= 'Z') return 'Z'.charCodeAt(0) - head.charCodeAt(0) + 2; + return 0; +} + +function integerPart(key: string): string | null { + if (!key) return null; + const n = integerLength(key[0]); + if (n === 0 || n > key.length) return null; + return key.slice(0, n); +} + +export function validOrderKey(k: string): boolean { + if (!k || k.length > 256 || k === SMALLEST_INTEGER) return false; + const ip = integerPart(k); + if (ip === null) return false; + for (let i = 1; i < k.length; i++) { + if (digitIndex(k[i]) < 0) return false; + } + const frac = k.slice(ip.length); + return frac === '' || frac[frac.length - 1] !== '0'; +} + +function incrementInteger(x: string): string | null { + const head = x[0]; + const digs = x.slice(1).split(''); + let carry = true; + for (let i = digs.length - 1; carry && i >= 0; i--) { + const d = digitIndex(digs[i]) + 1; + if (d === DIGITS.length) { + digs[i] = '0'; + } else { + digs[i] = DIGITS[d]; + carry = false; + } + } + if (!carry) return head + digs.join(''); + if (head === 'Z') return 'a0'; + if (head === 'z') return null; + const h = String.fromCharCode(head.charCodeAt(0) + 1); + if (h > 'a') digs.push('0'); + else digs.pop(); + return h + digs.join(''); +} + +function decrementInteger(x: string): string | null { + const head = x[0]; + const digs = x.slice(1).split(''); + let borrow = true; + for (let i = digs.length - 1; borrow && i >= 0; i--) { + const d = digitIndex(digs[i]) - 1; + if (d === -1) { + digs[i] = 'z'; + } else { + digs[i] = DIGITS[d]; + borrow = false; + } + } + if (!borrow) return head + digs.join(''); + if (head === 'a') return 'Z' + DIGITS[DIGITS.length - 1]; + if (head === 'A') return null; + const h = String.fromCharCode(head.charCodeAt(0) - 1); + if (h < 'Z') digs.push('z'); + else digs.pop(); + return h + digs.join(''); +} + +function midpoint(a: string, b: string): string { + if (b !== '') { + let n = 0; + while (n < b.length) { + const ca = n < a.length ? a[n] : '0'; + if (ca !== b[n]) break; + n++; + } + if (n > 0) return b.slice(0, n) + midpoint(n < a.length ? a.slice(n) : '', b.slice(n)); + } + const da = a !== '' ? digitIndex(a[0]) : 0; + const db = b !== '' ? digitIndex(b[0]) : DIGITS.length; + if (db - da > 1) return DIGITS[Math.floor((da + db) / 2)]; + if (b !== '' && b.length > 1) return b.slice(0, 1); + return DIGITS[da] + midpoint(a.length > 1 ? a.slice(1) : '', ''); +} + +/** Returns a key strictly between a and b; '' means open end. Throws on an invalid range. */ +export function keyBetween(a: string, b: string): string { + if ((a !== '' && !validOrderKey(a)) || (b !== '' && !validOrderKey(b)) || (a !== '' && b !== '' && a >= b)) { + throw new Error(`invalid order key range ${a}..${b}`); + } + if (a === '' && b === '') return 'a0'; + if (a === '') { + const ib = integerPart(b)!; + const fb = b.slice(ib.length); + if (ib === SMALLEST_INTEGER) return ib + midpoint('', fb); + if (ib < b) return ib; + const res = decrementInteger(ib); + if (res === null) throw new Error('order key range exhausted'); + return res; + } + if (b === '') { + const ia = integerPart(a)!; + const i = incrementInteger(ia); + return i !== null ? i : ia + midpoint(a.slice(ia.length), ''); + } + const ia = integerPart(a)!; + const ib = integerPart(b)!; + if (ia === ib) return ia + midpoint(a.slice(ia.length), b.slice(ib.length)); + const i = incrementInteger(ia); + if (i === null) throw new Error('order key range exhausted'); + return i < b ? i : ia + midpoint(a.slice(ia.length), ''); +} + +/** Returns n increasing keys between a and b, spread out so bulk inserts stay short. */ +export function keysBetween(a: string, b: string, n: number): string[] { + if (n <= 0) return []; + if (n === 1) return [keyBetween(a, b)]; + if (b === '') { + const out: string[] = []; + let prev = a; + for (let i = 0; i < n; i++) { + prev = keyBetween(prev, ''); + out.push(prev); + } + return out; + } + if (a === '') { + const out: string[] = new Array(n); + let next = b; + for (let i = n - 1; i >= 0; i--) { + next = keyBetween('', next); + out[i] = next; + } + return out; + } + const mid = Math.floor(n / 2); + const c = keyBetween(a, b); + return [...keysBetween(a, c, mid), c, ...keysBetween(c, b, n - mid - 1)]; +} diff --git a/frontend/src/lib/docSync/persist.ts b/frontend/src/lib/docSync/persist.ts new file mode 100644 index 0000000..bfc90cb --- /dev/null +++ b/frontend/src/lib/docSync/persist.ts @@ -0,0 +1,87 @@ +// Persists the sync state of each note in IndexedDB so that typed text survives reloads, crashes and +// offline phases. Falls back to memory when IndexedDB is unavailable (private mode, old browsers). +import type { Block, Op } from './model'; + +export interface StoredDocState { + docId: string; + title: string; + titleVersion: number; + lastSeq: number; + role: string; + blocks: Block[]; + pending: Op[]; + inflight: Op[] | null; + savedAt: number; +} + +export interface SyncStorage { + load(docId: string): Promise; + save(state: StoredDocState): Promise; +} + +const DB_NAME = 'mindsift-sync'; +const STORE = 'docs'; + +export class MemoryStorage implements SyncStorage { + private data = new Map(); + async load(docId: string) { + const s = this.data.get(docId); + return s ? JSON.parse(JSON.stringify(s)) : null; + } + async save(state: StoredDocState) { + this.data.set(state.docId, JSON.parse(JSON.stringify(state))); + } +} + +export class IndexedDBStorage implements SyncStorage { + private dbPromise: Promise | null = null; + + constructor(private factory: IDBFactory = indexedDB) {} + + private open(): Promise { + if (!this.dbPromise) { + this.dbPromise = new Promise((resolve, reject) => { + const req = this.factory.open(DB_NAME, 1); + req.onupgradeneeded = () => { + if (!req.result.objectStoreNames.contains(STORE)) req.result.createObjectStore(STORE, { keyPath: 'docId' }); + }; + req.onsuccess = () => resolve(req.result); + req.onerror = () => reject(req.error); + }); + this.dbPromise.catch(() => { + this.dbPromise = null; + }); + } + return this.dbPromise; + } + + async load(docId: string): Promise { + const db = await this.open(); + return new Promise((resolve, reject) => { + const req = db.transaction(STORE, 'readonly').objectStore(STORE).get(docId); + req.onsuccess = () => resolve((req.result as StoredDocState) || null); + req.onerror = () => reject(req.error); + }); + } + + async save(state: StoredDocState): Promise { + const db = await this.open(); + return new Promise((resolve, reject) => { + const tx = db.transaction(STORE, 'readwrite'); + tx.objectStore(STORE).put(state); + tx.oncomplete = () => resolve(); + tx.onerror = () => reject(tx.error); + tx.onabort = () => reject(tx.error); + }); + } +} + +/** IndexedDB when available, otherwise memory (state is then lost on reload). */ +export function defaultStorage(): SyncStorage { + try { + if (typeof indexedDB !== 'undefined') return new IndexedDBStorage(indexedDB); + } catch { + // ignore + } + return new MemoryStorage(); +} diff --git a/frontend/src/lib/docSync/transport.ts b/frontend/src/lib/docSync/transport.ts new file mode 100644 index 0000000..e6de919 --- /dev/null +++ b/frontend/src/lib/docSync/transport.ts @@ -0,0 +1,111 @@ +// HTTP transport of the note sync protocol (see Doku/api/rest.md). Every data exchange is a plain request, +// so it works behind proxies that block WebSockets. +import { getToken } from '../api'; +import type { Block, Effect, Op, Result } from './model'; + +export interface LeaseInfo { + blockId: string; + userId: string; + name: string; +} + +export interface Presence { + userId: string; + name: string; +} + +export interface SnapshotResponse { + docId: string; + title: string; + titleVersion: number; + headSeq: number; + blocks: Block[]; + role: string; + shared: boolean; +} + +export interface LogEntry { + seq: number; + opId?: string; + userId: string; + clientId: string; + effect: Effect; +} + +export interface SyncRequest { + clientId: string; + lastSeq: number; + ops: Op[]; + leases?: { hold: string[]; release: string[] }; +} + +export interface SyncResponse { + results: Result[]; + ops: LogEntry[]; + headSeq: number; + hasMore: boolean; + resync?: boolean; + role: string; + shared: boolean; + sessions: number; + presence: Presence[]; + leases: { granted: string[]; denied: string[]; others: LeaseInfo[] }; +} + +/** status 0 = network error or timeout; otherwise the HTTP status. */ +export class TransportError extends Error { + constructor(message: string, public status: number) { + super(message); + } +} + +export interface Transport { + snapshot(docId: string): Promise; + sync(docId: string, body: SyncRequest, opts?: { keepalive?: boolean }): Promise; + leave(docId: string, clientId: string): Promise; +} + +const TIMEOUT_MS = 15000; + +async function call(path: string, init: RequestInit & { keepalive?: boolean } = {}): Promise { + const headers: Record = { 'Content-Type': 'application/json' }; + const token = getToken(); + if (token) headers.Authorization = `Bearer ${token}`; + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), TIMEOUT_MS); + let res: Response; + try { + res = await fetch(path, { ...init, headers, cache: 'no-store', signal: init.keepalive ? undefined : controller.signal }); + } catch (err: any) { + throw new TransportError(err?.message || 'Netzwerkfehler', 0); + } finally { + clearTimeout(timer); + } + if (res.status === 401) { + window.dispatchEvent(new CustomEvent('mindsift:unauthorized')); + } + if (!res.ok) { + let msg = `HTTP ${res.status}`; + try { + const data = await res.json(); + if (data?.error) msg = data.error; + } catch { + // ignore + } + throw new TransportError(msg, res.status); + } + return res.json(); +} + +export const httpTransport: Transport = { + snapshot: (docId) => call(`/api/v1/docs/${encodeURIComponent(docId)}/snapshot`), + sync: (docId, body, opts) => + call(`/api/v1/docs/${encodeURIComponent(docId)}/sync`, { method: 'POST', body: JSON.stringify(body), keepalive: opts?.keepalive }), + leave: async (docId, clientId) => { + await call(`/api/v1/docs/${encodeURIComponent(docId)}/leave`, { + method: 'POST', + body: JSON.stringify({ clientId }), + keepalive: true, + }); + }, +}; diff --git a/frontend/src/lib/types.ts b/frontend/src/lib/types.ts index c07c235..8e29b00 100644 --- a/frontend/src/lib/types.ts +++ b/frontend/src/lib/types.ts @@ -173,6 +173,8 @@ export interface DocumentBlock { collapsed?: boolean; sortOrder?: number; updatedAt?: string; + conflictOf?: string; // set on conflict copies: ID of the contested block, or 'title' + conflictAuthor?: string; } export interface DocBlocksPatchRequest { diff --git a/internal/api/docsync.go b/internal/api/docsync.go index 91b4fca..fea628f 100644 --- a/internal/api/docsync.go +++ b/internal/api/docsync.go @@ -145,6 +145,9 @@ func (s *Server) handleDocSync(w http.ResponseWriter, r *http.Request) { resp.Leases.Granted = append(resp.Leases.Granted, granted...) } } + if changed { + s.afterDocChange(docID, req.Ops, resp.Results) + } entries, head, err := s.store.GetDocOpsSince(docID, req.LastSeq, syncMaxLog) if err != nil { @@ -168,6 +171,25 @@ func (s *Server) handleDocSync(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusOK, resp) } +// afterDocChange runs side effects of applied operations: linked tasks follow their blocks, and a new +// title reaches the note lists of everyone with access. +func (s *Server) afterDocChange(docID string, ops []collab.Op, results []collab.Result) { + titleChanged := false + for i, res := range results { + if i < len(ops) && ops[i].Kind == collab.OpSetTitle && res.Seq > 0 { + titleChanged = true + } + } + for _, taskID := range s.store.ReconcileDocTaskLinks(docID) { + if th, err := s.store.GetThoughtByIDUnchecked(taskID); err == nil { + s.wsHub.BroadcastToUsers(s.taskAudience(th), "TASK_UPDATED", th) + } + } + if titleChanged { + s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED") + } +} + // notifyDocChanged wakes long-poll requests and pokes WebSocket/SSE clients in the note's room. func (s *Server) notifyDocChanged(docID string, head int64) { s.notifier.Notify(docID) diff --git a/internal/storage/docsync.go b/internal/storage/docsync.go index af01f8a..31dd7fd 100644 --- a/internal/storage/docsync.go +++ b/internal/storage/docsync.go @@ -355,9 +355,21 @@ func persistDocState(tx *sql.Tx, docID string, before, after *collab.Doc, head i } } blocksJSON, _ := json.Marshal(flat) + tagSet := map[string]bool{} + for _, b := range flat { + for _, tag := range domain.ExtractTags(b.Content) { + tagSet[tag] = true + } + } + tags := make([]string, 0, len(tagSet)) + for tag := range tagSet { + tags = append(tags, tag) + } + sort.Strings(tags) + tagsJSON, _ := json.Marshal(tags) if _, err := tx.Exec( - `UPDATE documents SET title = ?, title_version = ?, head_seq = ?, markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? WHERE id = ?`, - after.Title, after.TitleVersion, head, domain.CompileBlocksToMarkdown(flat), plain.String(), string(blocksJSON), now, docID, + `UPDATE documents SET title = ?, title_version = ?, head_seq = ?, markdown_cache = ?, content = ?, blocks_json = ?, tags = ?, updated_at = ? WHERE id = ?`, + after.Title, after.TitleVersion, head, domain.CompileBlocksToMarkdown(flat), plain.String(), string(blocksJSON), string(tagsJSON), now, docID, ); err != nil { return err } @@ -377,6 +389,71 @@ func persistDocState(tx *sql.Tx, docID string, before, after *collab.Doc, head i return nil } +// docTaskLinks maps block IDs of a note to the IDs of tasks created from them ("Erweitert"). +func (s *Store) docTaskLinks(docID string) map[string]string { + links := map[string]string{} + rows, err := s.db.Query( + `SELECT id, converted_ref_id FROM thoughts WHERE converted_type = 'document_block' AND converted_ref_id LIKE ?`, + docID+":%") + if err != nil { + return links + } + defer rows.Close() + for rows.Next() { + var taskID, ref string + if rows.Scan(&taskID, &ref) == nil { + if _, blockID, ok := strings.Cut(ref, ":"); ok { + links[blockID] = taskID + } + } + } + return links +} + +// ReconcileDocTaskLinks brings tasks linked to blocks in line with the note: a ticked block completes its +// task, a deleted block unlinks it (the task itself stays). Returns the IDs of changed tasks. +func (s *Store) ReconcileDocTaskLinks(docID string) []string { + links := s.docTaskLinks(docID) + if len(links) == 0 { + return nil + } + d, _, err := loadCollabDoc(s.db, docID) + if err != nil { + return nil + } + var changed []string + for blockID, taskID := range links { + b, ok := d.Blocks[blockID] + if !ok || b.Deleted { + if _, err := s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '' WHERE id = ?`, taskID); err == nil { + changed = append(changed, taskID) + } + continue + } + kanban := "todo" + if b.Props.Checked { + kanban = "done" + } + res, err := s.db.Exec( + `UPDATE thoughts SET is_completed = ?, kanban_status = ? WHERE id = ? AND is_completed != ?`, + boolInt(b.Props.Checked), kanban, taskID, boolInt(b.Props.Checked)) + if err == nil { + if n, _ := res.RowsAffected(); n > 0 { + changed = append(changed, taskID) + } + } + } + sort.Strings(changed) + return changed +} + +// GetThoughtByIDUnchecked loads a task without access check, for server-internal notifications. +func (s *Store) GetThoughtByIDUnchecked(thoughtID string) (*domain.Thought, error) { + return scanThought(s.db.QueryRow( + `SELECT id, user_id, content, tags, priority, source, status, is_completed, converted_type, converted_ref_id, due_date, task_list_id, kanban_status, notes, alarm_sent, created_at, triaged_at, subtasks, sort_order + FROM thoughts WHERE id = ?`, thoughtID)) +} + // GetDocOpsSince returns up to limit log entries after seq, plus the current head. func (s *Store) GetDocOpsSince(docID string, since int64, limit int) ([]DocLogEntry, int64, error) { var head int64 @@ -495,12 +572,36 @@ const legacyClientID = "legacy" // replaceDocBlocksInDB stores a complete flat block list by diffing it against the current tree and // applying the difference as regular operations, so it shows up in the log like any other change. func (s *Store) replaceDocBlocksInDB(docID, userID string, blocks []domain.DocumentBlock, _ string) { + blocks = s.renameForeignBlockIDs(docID, blocks) a := collab.Actor{UserID: userID, ClientID: legacyClientID} if _, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op { return legacyOps(d, blocks) }); err != nil { slog.Error("Speichern der Notiz fehlgeschlagen", "doc_id", docID, "error", err) } } +// renameForeignBlockIDs gives new IDs to blocks whose ID already belongs to another note (e.g. fixed IDs of +// starter content or copied blocks). Block IDs are global, so keeping them would make the insert fail. +func (s *Store) renameForeignBlockIDs(docID string, blocks []domain.DocumentBlock) []domain.DocumentBlock { + out := blocks + copied := false + for i, b := range blocks { + id := strings.TrimSpace(b.ID) + if id == "" { + continue + } + var owner string + if s.db.QueryRow(`SELECT document_id FROM blocks WHERE id = ?`, id).Scan(&owner) != nil || owner == docID { + continue + } + if !copied { + out = append([]domain.DocumentBlock(nil), blocks...) + copied = true + } + out[i].ID = domain.NewID("blk") + } + return out +} + // applyLegacyTitle records a title change from the whole-document API in the log. func (s *Store) applyLegacyTitle(docID, userID, title string) { a := collab.Actor{UserID: userID, ClientID: legacyClientID} diff --git a/internal/storage/docsync_sim_test.go b/internal/storage/docsync_sim_test.go index e4dc7e5..ae3f6f3 100644 --- a/internal/storage/docsync_sim_test.go +++ b/internal/storage/docsync_sim_test.go @@ -77,7 +77,8 @@ func (w *simWorld) edit(c *simClient) { switch n := w.r.Intn(100); { case n < 45 && target != nil: // typing content := target.Content + fmt.Sprintf(" %s%d", c.actor.ClientID, c.counter) - c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetContent, BlockID: target.ID, Content: &content}) + // Base = the version the user sees while typing (like the browser engine) + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetContent, BlockID: target.ID, BaseVersion: target.Version, Content: &content}) c.holds[target.ID] = true case n < 65: // Enter: new block after target (or first block) parent, after := "", "" @@ -133,7 +134,7 @@ func (w *simWorld) edit(c *simClient) { c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetProps, BlockID: target.ID, Patch: &collab.PropsPatch{Checked: &checked}}) case n < 96: // rename note title := fmt.Sprintf("Titel %s%d", c.actor.ClientID, c.counter) - c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetTitle, Title: &title}) + c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetTitle, BaseVersion: v.TitleVersion, Title: &title}) default: // stop editing some block for id := range c.holds { delete(c.holds, id) @@ -142,15 +143,23 @@ func (w *simWorld) edit(c *simClient) { } } -// formBatch takes pending operations, keeps only the last text change per block and fills in base versions. +// formBatch takes pending operations and keeps only the last text change per block; its base version is +// the one of the first change, i.e. what the user saw when starting to type. func (c *simClient) formBatch() []collab.Op { lastText := map[string]int{} - lastTitle := -1 + firstBase := map[string]int{} + lastTitle, firstTitleBase := -1, 0 for i, op := range c.pending { switch op.Kind { case collab.OpSetContent: + if _, ok := lastText[op.BlockID]; !ok { + firstBase[op.BlockID] = op.BaseVersion + } lastText[op.BlockID] = i case collab.OpSetTitle: + if lastTitle < 0 { + firstTitleBase = op.BaseVersion + } lastTitle = i } } @@ -161,15 +170,12 @@ func (c *simClient) formBatch() []collab.Op { if lastText[op.BlockID] != i { continue } - op.BaseVersion = 1 - if b, ok := c.confirmed.Blocks[op.BlockID]; ok { - op.BaseVersion = b.Version - } + op.BaseVersion = firstBase[op.BlockID] case collab.OpSetTitle: if i != lastTitle { continue } - op.BaseVersion = c.confirmed.TitleVersion + op.BaseVersion = firstTitleBase } batch = append(batch, op) } diff --git a/internal/storage/docsync_test.go b/internal/storage/docsync_test.go index ad438a6..e5318e5 100644 --- a/internal/storage/docsync_test.go +++ b/internal/storage/docsync_test.go @@ -184,3 +184,57 @@ func TestBackfillConvertsFlatBlocksToTree(t *testing.T) { want := flatBlocks([3]interface{}{"x", 0, "x"}, [3]interface{}{"y", 1, "y"}, [3]interface{}{"z", 1, "z"}, [3]interface{}{"w", 0, "w"}) sameOutline(t, got.Blocks, want) } + +func TestTaskLinksFollowTheirBlock(t *testing.T) { + s, u := newSyncTestStore(t) + doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", []domain.DocumentBlock{{ID: "todo1", Type: "todo", Content: "Anrufen"}}, nil, false) + task, err := s.CreateTaskFromDocBlock(u.ID, doc.ID, "todo1", "Anrufen", false) + if err != nil { + t.Fatal(err) + } + got, _ := s.GetDocumentByID(u.ID, doc.ID) + if got.Blocks[0].TaskID != task.ID { + t.Fatalf("block must carry its task ID after loading, got %q", got.Blocks[0].TaskID) + } + + a := collab.Actor{UserID: u.ID, ClientID: "tab"} + yes := true + if _, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "chk", Kind: collab.OpSetProps, BlockID: "todo1", Patch: &collab.PropsPatch{Checked: &yes}}}, nil); err != nil { + t.Fatal(err) + } + if changed := s.ReconcileDocTaskLinks(doc.ID); len(changed) != 1 { + t.Fatalf("ticking the block must complete the task, changed=%v", changed) + } + if th, _ := s.GetThoughtByID(u.ID, task.ID); !th.IsCompleted { + t.Fatal("task not completed") + } + + if _, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "del", Kind: collab.OpDelete, BlockID: "todo1"}}, nil); err != nil { + t.Fatal(err) + } + s.ReconcileDocTaskLinks(doc.ID) + th, err := s.GetThoughtByID(u.ID, task.ID) + if err != nil { + t.Fatal("deleting the block must not delete the task") + } + if th.ConvertedType == "document_block" { + t.Fatal("deleting the block must unlink the task") + } +} + +func TestFixedBlockIDsWorkInSeveralNotes(t *testing.T) { + s, u := newSyncTestStore(t) + other, _ := s.CreateUser("second@example.com", "hash", "Second", "member") + blocks := flatBlocks([3]interface{}{"blk_1", 0, "Willkommen"}, [3]interface{}{"blk_2", 1, "Unterpunkt"}) + first, _ := s.CreateDocument(u.ID, "", "Start", "📄", "", blocks, nil, false) + second, _ := s.CreateDocument(other.ID, "", "Start", "📄", "", blocks, nil, false) + + a, _ := s.GetDocumentByID(u.ID, first.ID) + b, _ := s.GetDocumentByID(other.ID, second.ID) + if len(a.Blocks) != 2 || len(b.Blocks) != 2 || b.Blocks[0].Content != "Willkommen" || b.Blocks[1].Level != 1 { + t.Fatalf("second note must get its own copies of the blocks: %+v", b.Blocks) + } + if a.Blocks[0].ID == b.Blocks[0].ID { + t.Fatal("block IDs must differ between notes") + } +} diff --git a/internal/storage/sqlite.go b/internal/storage/sqlite.go index 6bab8a5..fbcc110 100644 --- a/internal/storage/sqlite.go +++ b/internal/storage/sqlite.go @@ -1448,7 +1448,13 @@ func (s *Store) migrateWorkspaceToPureDB() { func (s *Store) loadDocBlocks(docID string, fallbackJSON string) []domain.DocumentBlock { if d, _, err := loadCollabDoc(s.db, docID); err == nil && len(d.Blocks) > 0 { - return flatDocumentBlocks(docID, d) + blocks := flatDocumentBlocks(docID, d) + if links := s.docTaskLinks(docID); len(links) > 0 { + for i := range blocks { + blocks[i].TaskID = links[blocks[i].ID] + } + } + return blocks } // Very old notes only stored blocks_json; convert them once @@ -2228,18 +2234,18 @@ func (s *Store) UpdateDocument(userID, docID string, title, icon, folderPath *st for _, b := range doc.Blocks { remainingBlockIDs[b.ID] = true } - rows, err := s.db.Query(`SELECT id, converted_ref_id, block_id FROM thoughts WHERE (doc_id = ? OR converted_ref_id LIKE ?) AND converted_type = 'document_block'`, docID, docID+":%") + rows, err := s.db.Query(`SELECT id, converted_ref_id FROM thoughts WHERE converted_ref_id LIKE ? AND converted_type = 'document_block'`, docID+":%") if err == nil { for rows.Next() { - var tid, refID, bID string - if err := rows.Scan(&tid, &refID, &bID); err == nil { - bid := bID - if bid == "" && strings.Contains(refID, ":") { + var tid, refID string + if err := rows.Scan(&tid, &refID); err == nil { + bid := "" + if strings.Contains(refID, ":") { parts := strings.SplitN(refID, ":", 2) bid = parts[1] } if !remainingBlockIDs[bid] { - _, _ = s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '', doc_id = '', block_id = '' WHERE id = ?`, tid) + _, _ = s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '' WHERE id = ?`, tid) } } } @@ -4111,7 +4117,7 @@ func (s *Store) UnlinkTaskFromDocBlock(userID, taskID string) { } } } - _, _ = s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '', doc_id = '', block_id = '' WHERE id = ?`, taskID) + _, _ = s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '' WHERE id = ?`, taskID) } func (s *Store) SyncTaskFromBlockCheck(userID, taskID string, checked bool) { From 08d9b2dc77888a4c8c1baf724b2a21671c553611 Mon Sep 17 00:00:00 2001 From: Sandro Date: Wed, 28 Oct 2026 12:40:22 +0100 Subject: [PATCH 06/11] feat(workspace): allow sharing notes and shared folders as editor The share dialog for notes and shared folders now offers "Bearbeiten" and "Nur Lesen" and stores write/read accordingly. Editors already edit through the sync engine since phase 2. Adds e2e/share.e2e.mjs: shares a note as editor through the dialog and checks that the recipient can edit it; e2e/run.sh runs both browser tests. Co-Authored-By: Claude Opus 5.5 --- Doku/flaws.md | 4 +- Doku/plans/kollaboration-notizen.md | 2 +- Doku/sharing-and-permissions.md | 2 +- e2e/README.md | 6 +- e2e/run.sh | 2 +- e2e/share.e2e.mjs | 108 ++++++++++++++++++ .../src/features/workspace/WorkspaceView.tsx | 11 +- 7 files changed, 122 insertions(+), 13 deletions(-) create mode 100644 e2e/share.e2e.mjs diff --git a/Doku/flaws.md b/Doku/flaws.md index 518f8fc..47b44b3 100644 --- a/Doku/flaws.md +++ b/Doku/flaws.md @@ -26,7 +26,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( | FLAW-14 | Backend | Fehler von Schreiboperationen werden häufig ignoriert (`_, _ =`) | Mittel | | FLAW-15 | Realtime | ~~`WriteTimeout` von 30 s beendet SSE-Streams~~ | **Behoben** | | FLAW-16 | Setup | `/auth/setup` ist bis zum ersten Konto offen; Demo-Modus legt `admin/admin` an | Mittel | -| FLAW-17 | Workspace | Bearbeiter-Freigaben für Notizen erst im Backend aktiv, nicht im UI | Niedrig | +| FLAW-17 | Workspace | ~~Bearbeiter-Freigaben für Notizen nicht wählbar~~ | **Behoben** | | FLAW-18 | Suche | Nur eigene Gedanken und Dokumente, `LIKE` ohne Escaping, keine FTS | Niedrig | | FLAW-19 | Qualität | Frontend-Tests nur für den Outliner, `lint` ist ein Platzhalter | Niedrig | | FLAW-20 | Qualität | Sehr große Dateien, duplizierte Zugriffslogik | Niedrig | @@ -138,7 +138,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( - **Maßnahme:** Setup per einmaligem Token aus dem Server-Log absichern; Demo-Modus nur zusammen mit expliziter Warnung/Bestätigung. ### FLAW-17: Schreibgeschützte Dokumentfreigaben -- **Stand:** Die Zwangsmigration auf `read` ist entfernt, das Backend speichert und prüft Bearbeiter-Freigaben (`DocRole`, `/docs/{id}/sync`). Offen bis Phase 5 des [Kollaborationsplans](plans/kollaboration-notizen.md): Freigabe-Dialog bietet nur „Nur Lesen“, der alte Editor (`PUT /documents/{id}`) schreibt nur für den Eigentümer. +- **Status: Behoben.** Zwangsmigration entfernt, Backend prüft Bearbeiter-Rechte (`DocRole`, `/docs/{id}/sync`), der Editor speichert über die Sync-Engine, und der Freigabe-Dialog bietet „Bearbeiten“ an. Browser-Test: `e2e/share.e2e.mjs`. - **Fundstellen:** Migration in `Store.migrate` (`UPDATE doc_shares SET permission = 'read' …`, ebenso `doc_team_shares`, `workspace_folder_shares`), `PatchDocumentBlocks`/`UpdateDocument` (nur Eigentümer). - **Hinweis:** Das ist offenbar gewollt: Nur der Eigentümer schreibt in ein Dokument, Empfänger lesen. Es gibt daher **kein** kollaboratives Bearbeiten. Das Schema (`CHECK(permission IN ('read','write'))`), die Zweige für `write` im Code und frühere Dokumentation suggerieren etwas anderes. Es gibt weder Sperren noch Konfliktauflösung, noch Cursor-Presence. - **Maßnahme:** Entweder Schreibfreigaben mit Konfliktbehandlung ergänzen oder den toten Code und das Schema bereinigen. diff --git a/Doku/plans/kollaboration-notizen.md b/Doku/plans/kollaboration-notizen.md index b7e9c1b..50422a3 100644 --- a/Doku/plans/kollaboration-notizen.md +++ b/Doku/plans/kollaboration-notizen.md @@ -282,7 +282,7 @@ Damit private Notizen keinen unnötigen Aufwand erzeugen, liefert der Server im | 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt. **Erledigt.** Abweichung: Die Editor-Komponenten arbeiten weiter mit der flachen Liste (Ebene + Reihenfolge); die Engine übersetzt jede Änderung per Vergleich in Operationen (`lib/docSync/diff.ts`). Sperren werden bereits gehalten (Fokus, 20 s Nachlauf, neue Bullets), aber noch nicht angezeigt. Ende-zu-Ende-Test mit zwei Browsern: `e2e/`. | Bearbeiten funktioniert überall, auch offline | | 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen. Dazu aus Phase 2: gesperrte Bullets schreibgeschützt anzeigen und Struktur-Aktionen auf ihnen gar nicht erst starten (sonst kann Zusammenführen halb ausgeführt werden, siehe FLAW-25); Fokuswechsel nach Enter ohne Frame-Verzögerung (FLAW-26). | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | | 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt | -| 5 | Freigabe-Dialog mit Bearbeiter-Rolle, Ordnerfreigaben, Altlasten entfernen (`PUT`-Blockersatz, `PATCH /blocks`, `doc_shares`, `doc_team_shares`, Zwangsmigration auf „read“) | Feature komplett | +| 5 | Freigabe-Dialog mit Bearbeiter-Rolle (**vorgezogen und erledigt**, für Notizen und geteilte Ordner), Ordnerfreigaben, Altlasten entfernen (`PUT`-Blockersatz, `PATCH /blocks`, `doc_shares`, `doc_team_shares`, Zwangsmigration auf „read“) | Feature komplett | | 6 | Härtung: Playwright-Tests, Log-Kompaktierung (Snapshot statt Log für Clients mit sehr altem `lastSeq`), Aufräumen der Grabsteine | Langzeitbetrieb | Phase 2 ist bewusst vor den Push-Kanälen: Steht die Bearbeitung über reine HTTP-Abfragen, ist der schwierigste Fall (Proxy blockt alles außer HTTP) von Anfang an abgedeckt. WebSocket und SSE machen es danach nur schneller. diff --git a/Doku/sharing-and-permissions.md b/Doku/sharing-and-permissions.md index 7e5daa8..157f658 100644 --- a/Doku/sharing-and-permissions.md +++ b/Doku/sharing-and-permissions.md @@ -17,7 +17,7 @@ | **Notiz** (`doc_shares`, `doc_team_shares`) | alles, Freigaben, löschen | Inhalt, Struktur, Titel | nur lesen | Benutzer, Gruppen | | **Geteilter Ordner** (`workspace_folder_shares`) | alles | Notizen darin bearbeiten | nur lesen | Benutzer, Gruppen | -- **Notizen und Ordner:** Die effektive Rolle berechnet `Store.DocRole` (Eigentümer, Eigentümer des geteilten Ordners = Bearbeiter, direkte Freigabe, Gruppenfreigabe, Ordnerfreigabe; höchste gewinnt). Das Backend akzeptiert Bearbeiter-Freigaben (`write`). Der Freigabe-Dialog bietet bis zur Umstellung des Editors (Phase 5 im [Plan](plans/kollaboration-notizen.md)) noch nur „Nur Lesen“ an, und der alte Editor schreibt weiterhin nur für den Eigentümer. +- **Notizen und Ordner:** Die effektive Rolle berechnet `Store.DocRole` (Eigentümer, Eigentümer des geteilten Ordners = Bearbeiter, direkte Freigabe, Gruppenfreigabe, Ordnerfreigabe; höchste gewinnt). Im Freigabe-Dialog sind für Notizen und geteilte Ordner „Bearbeiten“ und „Nur Lesen“ wählbar (gespeichert als `write`/`read`). Bearbeiter ändern Inhalt, Struktur und Titel über die Sync-Engine; Leser sehen Änderungen live. - Bei Aufgabenlisten und Listen gilt: Hat jemand mehrere Zugriffswege (direkt und über Gruppen), zählt die **höchste** Stufe. - Für Gruppen gilt die Freigabe für alle aktuellen und künftigen Mitglieder. - Aufgabenlisten besitzen zusätzlich das Altfeld `team_id`: Mitglieder dieser Gruppe haben ohne Freigabezeile Bearbeiterrechte. Eine Gruppenfreigabe über den Dialog setzt `team_id`, falls es noch leer ist. diff --git a/e2e/README.md b/e2e/README.md index 74f8156..26b1d18 100644 --- a/e2e/README.md +++ b/e2e/README.md @@ -24,4 +24,8 @@ Zwei Browser (Eigentümer und Bearbeiter) arbeiten in derselben Notiz. Geprüft - Sperren eines geschlossenen Browsers laufen nach spätestens 20 s ab - Einrücken (Tab) und Zusammenführen (Backspace) bleiben nach Neuladen erhalten -Der Test dauert wegen des Sperrablaufs etwa eine Minute. Er läuft nicht in der CI, weil er Browser-Images braucht. +## share.e2e.mjs + +Der Eigentümer teilt eine Notiz über den Freigabe-Dialog mit Rolle „Bearbeiten“; geprüft wird, dass die Freigabe als `write` gespeichert, im Dialog als „Bearbeiter“ angezeigt wird und der Empfänger die Notiz bearbeiten kann. + +`collab.e2e.mjs` dauert wegen des Sperrablaufs etwa eine Minute. Er läuft nicht in der CI, weil er Browser-Images braucht. diff --git a/e2e/run.sh b/e2e/run.sh index 2ac77e9..d047786 100755 --- a/e2e/run.sh +++ b/e2e/run.sh @@ -15,4 +15,4 @@ for _ in $(seq 1 60); do curl -s -m 1 http://localhost:18080/api/v1/health >/dev echo "Starte Browser-Tests …" "$ENGINE" run --rm --network host -v "$ROOT/e2e":/e2e:Z -w /e2e mcr.microsoft.com/playwright:v1.48.2-jammy \ - sh -c 'npm install --no-audit --no-fund >/dev/null 2>&1 && node collab.e2e.mjs' + sh -c 'npm install --no-audit --no-fund >/dev/null 2>&1 && node collab.e2e.mjs && node share.e2e.mjs' diff --git a/e2e/share.e2e.mjs b/e2e/share.e2e.mjs new file mode 100644 index 0000000..3ff1337 --- /dev/null +++ b/e2e/share.e2e.mjs @@ -0,0 +1,108 @@ +// Shares a note as editor through the share dialog and checks that the editor can then edit it. +// Needs a server in demo mode (accounts admin/admin and user/user); see e2e/README.md. +import { chromium } from 'playwright'; + +const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080'; +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); +let failures = 0; +const check = (cond, msg) => { + console.log(`${cond ? 'OK ' : 'FAIL'} ${msg}`); + if (!cond) failures++; +}; + +async function api(token, method, path, body) { + const res = await fetch(BASE + path, { + method, + headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: `Bearer ${token}` } : {}) }, + body: body ? JSON.stringify(body) : undefined, + }); + const text = await res.text(); + if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`); + return text ? JSON.parse(text) : null; +} + +async function waitFor(fn, ms) { + const end = Date.now() + ms; + while (Date.now() < end) { + try { + const v = await fn(); + if (v) return v; + } catch {} + await sleep(250); + } + return null; +} + +async function openDoc(browser, token, title) { + const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } }); + await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token); + const page = await ctx.newPage(); + await page.goto(BASE); + await page.getByRole('button', { name: /Workspace/ }).first().click(); + await page.getByText(title, { exact: true }).first().click(); + await page.waitForFunction(() => document.querySelectorAll('textarea').length > 1, null, { timeout: 10000 }); + return { ctx, page }; +} + +const TITLE = `Freigabe-Test ${Date.now()}`; + +const main = async () => { + const admin = (await api(null, 'POST', '/api/v1/auth/login', { login: 'admin', password: 'admin' })).token; + const user = (await api(null, 'POST', '/api/v1/auth/login', { login: 'user', password: 'user' })).token; + const doc = await api(admin, 'POST', '/api/v1/documents', { + title: TITLE, + blocks: [{ id: `fr_${Date.now()}`, type: 'bullet', content: 'Vom Eigentümer', level: 0 }], + }); + + const browser = await chromium.launch(); + const A = await openDoc(browser, admin, TITLE); + + // Share as editor through the dialog + await A.page.getByTitle('Notiz teilen und Berechtigungen').click(); + await A.page.getByRole('button', { name: 'Benutzer', exact: true }).click(); + const search = A.page.getByPlaceholder(/Benutzer suchen/); + await search.click(); + await search.type('user'); + await sleep(300); + await search.press('ArrowDown'); + await search.press('Enter'); + const roleSelect = A.page.locator('select').filter({ has: A.page.locator('option[value="editor"]') }).last(); + check((await roleSelect.isEnabled()) && (await roleSelect.isVisible()), 'Rollenauswahl ist im Dialog verfügbar'); + await roleSelect.selectOption('editor'); + await A.page.getByRole('button', { name: /Benutzer hinzufügen/ }).click(); + const shares = await waitFor(async () => { + const list = await api(admin, 'GET', `/api/v1/documents/${doc.id}/shares`); + return list.length > 0 ? list : null; + }, 5000); + check(shares && shares[0].permission === 'write', `Freigabe gespeichert als ${shares?.[0]?.permission}`); + const badge = await A.page + .getByText('Bearbeiter', { exact: true }) + .first() + .waitFor({ state: 'visible', timeout: 5000 }) + .then(() => true) + .catch(() => false); + check(badge, 'Dialog zeigt die Rolle „Bearbeiter“'); + + // The editor can edit + const snap = await api(user, 'GET', `/api/v1/docs/${doc.id}/snapshot`); + check(snap.role === 'editor', `Empfänger hat die Rolle ${snap.role}`); + const B = await openDoc(browser, user, TITLE); + check(!(await B.page.getByText('schreibgeschützt mit dir geteilt', { exact: false }).first().isVisible().catch(() => false)), 'Kein Nur-Lesen-Hinweis beim Bearbeiter'); + await B.page.getByText('Vom Eigentümer', { exact: true }).first().click(); + await B.page.keyboard.press('End'); + await B.page.keyboard.type(' und vom Bearbeiter'); + const saved = await waitFor(async () => { + const s = await api(admin, 'GET', `/api/v1/docs/${doc.id}/snapshot`); + return s.blocks.some((b) => b.content === 'Vom Eigentümer und vom Bearbeiter'); + }, 6000); + check(!!saved, 'Bearbeiter kann die geteilte Notiz bearbeiten'); + + await browser.close(); + console.log(failures ? `${failures} FEHLGESCHLAGEN` : 'ALLE PRÜFUNGEN BESTANDEN'); + process.exit(failures ? 1 : 0); +}; + +main().catch((e) => { + console.error(e); + process.exit(2); +}); diff --git a/frontend/src/features/workspace/WorkspaceView.tsx b/frontend/src/features/workspace/WorkspaceView.tsx index 60b8650..5759d12 100644 --- a/frontend/src/features/workspace/WorkspaceView.tsx +++ b/frontend/src/features/workspace/WorkspaceView.tsx @@ -2268,15 +2268,12 @@ export const WorkspaceView: React.FC = ({ shares={unifiedShares} teams={teams} collaborators={collaborators} - allowRoleSelection={false} - fixedPermission="viewer" - onAddShare={async (targetType, targetId) => { + onAddShare={async (targetType, targetId, role) => { + const permission = role === 'editor' ? 'write' : 'read'; if (shareModalDoc) { await api.createDocShare( shareModalDoc.id, - targetType === 'team' - ? { teamId: targetId, permission: 'read' } - : { userId: targetId, permission: 'read' } + targetType === 'team' ? { teamId: targetId, permission } : { userId: targetId, permission } ); const updatedShares = await api.getDocShares(shareModalDoc.id); setActiveDocShares(updatedShares); @@ -2284,7 +2281,7 @@ export const WorkspaceView: React.FC = ({ await api.createFolderShare(shareModalFolder.id, { targetType, targetId, - permission: 'read', + permission, }); const updatedFolder = await api.getSharedFolder(shareModalFolder.id); setShareModalFolder(updatedFolder); From aa397c18cd68bf2cf5a4b6d2444d084393bfa079 Mon Sep 17 00:00:00 2001 From: Sandro Date: Wed, 28 Oct 2026 13:44:27 +0100 Subject: [PATCH 07/11] feat(notes): show block locks and presence, conflict copies with accept/discard, undo/redo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 3 of the collaboration plan: - Blocks held by someone else show a lock with the name and are read-only; Enter in such a block adds a new block below. Delete, merge, indent and drag are not started when a involved block (or one in its branch) is locked; a short notice explains why (fixes FLAW-25). - Presence: the editor header lists who else has the note open. - Conflict copies are highlighted ("Konflikt – Version von …") with Übernehmen (replaces the original or the title) and Verwerfen. Both check the locks of the original and of the copy first. - Undo/redo of own changes (Ctrl/Cmd+Z, Ctrl+Shift+Z, Ctrl+Y, beforeinput history events): typing is grouped per block until a 1 s pause, up to 200 steps per tab; a step is skipped with a notice if someone else changed or holds the block meanwhile. New clearConflict props patch on the server. - New blocks get focus synchronously on mount (fixes FLAW-26). - Fix: after reopening a note from local storage the engine polled every 30 s instead of 3 s until the next change, because the delay was decided before the first response told it the note is shared. - Tests: undo/conflict engine tests, lock/undo/conflict UI tests and e2e/lock.e2e.mjs (two browsers); all three browser tests pass against a freshly built image. Co-Authored-By: Claude Opus 5.5 --- Doku/api/rest.md | 2 +- Doku/flaws.md | 6 +- Doku/modules/workspace.md | 5 +- Doku/plans/kollaboration-notizen.md | 2 +- e2e/README.md | 6 +- e2e/collab.e2e.mjs | 1 + e2e/lock.e2e.mjs | 170 ++++++++++++++++ e2e/run.sh | 2 +- .../src/features/workspace/DocumentEditor.tsx | 79 ++++++++ .../src/features/workspace/OutlinerBlock.tsx | 109 ++++++++++- .../src/features/workspace/OutlinerTree.tsx | 97 +++++++-- .../__tests__/OutlinerBlock.caret.test.tsx | 58 +++++- .../__tests__/OutlinerTree.merge.test.tsx | 56 +++++- .../src/lib/docSync/__tests__/engine.test.ts | 29 ++- .../src/lib/docSync/__tests__/undo.test.ts | 184 ++++++++++++++++++ frontend/src/lib/docSync/engine.ts | 124 +++++++++++- frontend/src/lib/docSync/model.ts | 2 + frontend/src/lib/docSync/undo.ts | 150 ++++++++++++++ internal/collab/ops.go | 5 + internal/collab/ops_test.go | 11 ++ 20 files changed, 1060 insertions(+), 38 deletions(-) create mode 100644 e2e/lock.e2e.mjs create mode 100644 frontend/src/lib/docSync/__tests__/undo.test.ts create mode 100644 frontend/src/lib/docSync/undo.ts diff --git a/Doku/api/rest.md b/Doku/api/rest.md index e1bcb8a..e9b630b 100644 --- a/Doku/api/rest.md +++ b/Doku/api/rest.md @@ -104,7 +104,7 @@ Grundlage: [plans/kollaboration-notizen.md](../plans/kollaboration-notizen.md). } ``` -- Operationsarten: `insert`, `set_content`, `set_props` (`patch`: `type`, `checked`, `collapsed`, `lang`), `move`, `delete`, `restore` (`ids`), `set_title` (`baseVersion`, `title`). +- Operationsarten: `insert`, `set_content`, `set_props` (`patch`: `type`, `checked`, `collapsed`, `lang`, `clearConflict`), `move`, `delete`, `restore` (`ids`), `set_title` (`baseVersion`, `title`). - `leases.hold` ist die vollständige Liste der Blöcke, die der Client halten will; alle anderen Leases des Clients werden freigegeben. Neu eingefügte Blöcke und Konfliktkopien bekommt der Autor automatisch. - Höchstens 500 Operationen pro Anfrage. Leser dürfen keine Operationen senden (`403`), aber abfragen. diff --git a/Doku/flaws.md b/Doku/flaws.md index 47b44b3..1dfd6f1 100644 --- a/Doku/flaws.md +++ b/Doku/flaws.md @@ -34,8 +34,8 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( | FLAW-22 | Netzwerk | Token in der URL (WS/SSE), CORS `*`, `CheckOrigin` immer `true` | Niedrig | | FLAW-23 | SQLite | Single-Writer-Engpass bei vielen gleichzeitigen Schreibern | Niedrig | | FLAW-24 | Gruppen | Organisations-Endpunkte ohne Berechtigungsprüfung | Mittel | -| FLAW-25 | Workspace | Zusammenführen/Einrücken an einem fremd gesperrten Bullet wird nur teilweise ausgeführt | Mittel | -| FLAW-26 | Workspace | Nach Enter landet sehr schnell Getipptes noch im vorherigen Bullet | Niedrig | +| FLAW-25 | Workspace | ~~Struktur-Aktionen an fremd gesperrten Bullets nur teilweise ausgeführt~~ | **Behoben** | +| FLAW-26 | Workspace | ~~Nach Enter landet sehr schnell Getipptes noch im vorherigen Bullet~~ | **Behoben** | --- @@ -176,11 +176,13 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** ( - **Maßnahme:** Organisationen entweder konsequent absichern (Eigentümer/Server-Admin) oder entfernen. ### FLAW-25: Teilweise ausgeführte Struktur-Aktionen an gesperrten Bullets +- **Status: Behoben.** Fremd gesperrte Bullets sind im Editor schreibgeschützt und zeigen, wer sie bearbeitet. Löschen, Zusammenführen, Ein-/Ausrücken und Ziehen werden nicht gestartet, wenn ein beteiligter Bullet (oder einer im Ast) gesperrt ist; stattdessen erscheint ein Hinweis. Ebenso für „Übernehmen“/„Verwerfen“ und Rückgängig. Browser-Test: `e2e/lock.e2e.mjs`. - **Fundstellen:** `handleMergeWithPrevious` und `handleIndentBlock` in [`OutlinerTree.tsx`](../frontend/src/features/workspace/OutlinerTree.tsx); Sperrregeln in `internal/collab` (Verschieben/Löschen gesperrter Blöcke wird abgelehnt). - **Problem:** Eine Editor-Aktion besteht oft aus mehreren Operationen. Ist einer der beteiligten Bullets von jemand anderem gesperrt, lehnt der Server nur diese Operation ab. Beispiel aus dem Browser-Test: Backspace am Zeilenanfang hängt den Text oben an (angewendet), das Löschen des unteren Bullets wird abgelehnt – der Text steht dann doppelt da. Einrücken eines fremd gesperrten Bullets wird ohne Hinweis zurückgenommen. Es geht kein Text verloren. - **Maßnahme (Phase 3):** Gesperrte Bullets im Editor anzeigen und Struktur-Aktionen, die sie berühren, gar nicht erst ausführen. ### FLAW-26: Fokus-Verzögerung nach Enter +- **Status: Behoben.** Neu eingehängte Bullets bekommen den Fokus sofort beim Einhängen. Browser-Test: „Sofort nach Enter Getipptes landet im neuen Bullet“. - **Fundstelle:** `focusBlockInput` in `OutlinerTree.tsx` setzt den Fokus erst im nächsten Animation-Frame. - **Problem:** Zeichen, die innerhalb dieses Frames getippt werden (automatisiert oder sehr schnelles Tippen), landen noch im vorherigen Bullet. Bei fremd gesperrtem Bullet entsteht daraus eine Konfliktkopie. - **Maßnahme:** Fokus nach dem Anlegen synchron setzen (`useLayoutEffect`). diff --git a/Doku/modules/workspace.md b/Doku/modules/workspace.md index 4dcc6a1..7d06be3 100644 --- a/Doku/modules/workspace.md +++ b/Doku/modules/workspace.md @@ -20,7 +20,10 @@ Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.ts - **Speichern:** Jede Notiz hat im Browser eine `DocSyncEngine` ([lib/docSync/](../../frontend/src/lib/docSync/)). Der Editor meldet jede Änderung der Block-Liste; die Engine übersetzt sie in Operationen, speichert sie zuerst in IndexedDB und sendet sie dann gesammelt (300 ms) an `POST /docs/{id}/sync`. Beim Öffnen zeigt sie sofort den gespeicherten Stand, auch offline. - **Abgleich:** Bei geteilten oder mehrfach geöffneten Notizen fragt die Engine alle 3 s nach Änderungen, sonst alle 30 s, sowie sofort bei Rückkehr ins Netz oder in den Tab. Beim Verlassen sendet sie Ausstehendes mit `keepalive` und gibt ihre Sperren frei. - **Textkonflikte:** Jede Textänderung trägt die Version, die der Nutzer beim Tippen vor sich hatte. Hat jemand anderes den Bullet inzwischen geändert oder gesperrt, legt der Server eine Konfliktkopie an; es wird nie still überschrieben. -- **Sperren:** Der Bullet mit dem Cursor, verlassene Bullets für 20 s und neu angelegte Bullets werden gehalten. Die Anzeige gesperrter Bullets folgt in Phase 3 des [Plans](../plans/kollaboration-notizen.md). +- **Sperren:** Der Bullet mit dem Cursor, verlassene Bullets für 20 s und neu angelegte Bullets werden gehalten. Bullets, die jemand anderes hält, zeigen ein Schloss mit Namen und sind schreibgeschützt; Enter darin legt einen neuen Bullet darunter an. Löschen, Zusammenführen, Ein-/Ausrücken und Ziehen werden bei gesperrten Bullets nicht ausgeführt (Hinweis unten im Editor). +- **Präsenz:** Im Kopf der Notiz steht, wer sie gerade sonst noch geöffnet hat. +- **Konfliktkopien** sind farbig markiert („Konflikt – Version von …“) mit „Übernehmen“ (Text ersetzt den ursprünglichen Punkt bzw. den Titel, die Kopie verschwindet) und „Verwerfen“. +- **Rückgängig:** `Strg+Z`/`Cmd+Z` nimmt die eigene letzte Änderung zurück, `Strg+Umschalt+Z` oder `Strg+Y` wiederholt (auch im Titel und über das Kontextmenü/mobile Tastaturen). Tippen im selben Punkt bis 1 s Pause ist ein Schritt; bis zu 200 Schritte je Tab. Hat jemand anderes den Punkt inzwischen geändert oder hält ihn gesperrt, wird der Schritt mit Hinweis übersprungen. - Den bisherigen Speicherweg (`PUT /documents/{id}`) nutzt der Editor nur noch für Symbol und Anheften; Titel und Inhalt laufen über die Engine. `PATCH /documents/{id}/blocks` nutzt das To-Do-Rollup; beide Wege landen im selben Änderungs-Log. ## Explorer, Tags, To-Dos diff --git a/Doku/plans/kollaboration-notizen.md b/Doku/plans/kollaboration-notizen.md index 50422a3..e7d194b 100644 --- a/Doku/plans/kollaboration-notizen.md +++ b/Doku/plans/kollaboration-notizen.md @@ -280,7 +280,7 @@ Damit private Notizen keinen unnötigen Aufwand erzeugen, liefert der Server im | 0 | Sicherheitsfixes vorziehen: Raumberechtigung (FLAW-01/02), Umsortieren (FLAW-03), Stream-Timeout (FLAW-15). **Erledigt.** Die zentrale `docRole` wandert in Phase 1, weil sie auf dem neuen Freigabemodell (`workspace_access`) aufbaut. | Grundlage dicht | | 1 | Operations-Log, `/snapshot`, `/sync`, `/changes`, `/leave`, Konfliktregeln, Leases, zentrale `DocRole`, Simulationstests. **Erledigt.** Abweichungen: Die bestehenden Tabellen `blocks`/`documents` wurden erweitert statt ersetzt (Baum per `parent_id`/`order_key`, alte Daten werden umgewandelt); der bisherige Speicherweg (`PUT`/`PATCH`) läuft als Adapter durch dieselbe Engine. `DocRole` arbeitet auf den vorhandenen Freigabetabellen; deren Zusammenlegung zu `workspace_access` erfolgt mit dem Freigabe-Dialog in Phase 5. `/changes` liefert nur `{headSeq, changed}`, die Daten kommen immer über `/sync`. | Server fertig und abgesichert | | 2 | `SyncEngine` mit IndexedDB-Warteschlange, nur Intervall-Abfrage als Transport, Editor umgestellt. **Erledigt.** Abweichung: Die Editor-Komponenten arbeiten weiter mit der flachen Liste (Ebene + Reihenfolge); die Engine übersetzt jede Änderung per Vergleich in Operationen (`lib/docSync/diff.ts`). Sperren werden bereits gehalten (Fokus, 20 s Nachlauf, neue Bullets), aber noch nicht angezeigt. Ende-zu-Ende-Test mit zwei Browsern: `e2e/`. | Bearbeiten funktioniert überall, auch offline | -| 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen. Dazu aus Phase 2: gesperrte Bullets schreibgeschützt anzeigen und Struktur-Aktionen auf ihnen gar nicht erst starten (sonst kann Zusammenführen halb ausgeführt werden, siehe FLAW-25); Fokuswechsel nach Enter ohne Frame-Verzögerung (FLAW-26). | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | +| 3 | Leases und Präsenz in UI, Konfliktblöcke mit Übernehmen/Verwerfen, Rückgängig/Wiederholen, FLAW-25/26. **Erledigt.** Ergänzungen: Enter in einem fremd gesperrten Bullet legt einen neuen Bullet darunter an (der gesperrte bleibt unberührt); „Übernehmen“/„Verwerfen“ prüfen auch die Sperre der Kopie selbst. | Gemeinsames Bearbeiten ohne Kollisionen, Strg+Z | | 4 | Push-Stufen WebSocket → SSE → Long-Polling mit automatischer Erkennung | Live-Gefühl, wo das Netz es zulässt | | 5 | Freigabe-Dialog mit Bearbeiter-Rolle (**vorgezogen und erledigt**, für Notizen und geteilte Ordner), Ordnerfreigaben, Altlasten entfernen (`PUT`-Blockersatz, `PATCH /blocks`, `doc_shares`, `doc_team_shares`, Zwangsmigration auf „read“) | Feature komplett | | 6 | Härtung: Playwright-Tests, Log-Kompaktierung (Snapshot statt Log für Clients mit sehr altem `lastSeq`), Aufräumen der Grabsteine | Langzeitbetrieb | diff --git a/e2e/README.md b/e2e/README.md index 26b1d18..e485079 100644 --- a/e2e/README.md +++ b/e2e/README.md @@ -28,4 +28,8 @@ Zwei Browser (Eigentümer und Bearbeiter) arbeiten in derselben Notiz. Geprüft Der Eigentümer teilt eine Notiz über den Freigabe-Dialog mit Rolle „Bearbeiten“; geprüft wird, dass die Freigabe als `write` gespeichert, im Dialog als „Bearbeiter“ angezeigt wird und der Empfänger die Notiz bearbeiten kann. -`collab.e2e.mjs` dauert wegen des Sperrablaufs etwa eine Minute. Er läuft nicht in der CI, weil er Browser-Images braucht. +## lock.e2e.mjs + +Sperren, Präsenz, Rückgängig und Konfliktkopien mit zwei Browsern: Schloss und Schreibschutz am fremd bearbeiteten Bullet, Hinweis statt halb ausgeführter Aktion, `Strg+Z`/`Strg+Umschalt+Z`, Tippen direkt nach Enter, Markierung von Konfliktkopien sowie „Übernehmen“ (verweigert bei fremder Sperre, erfolgreich für den Sperrinhaber). + +`collab.e2e.mjs` und `lock.e2e.mjs` dauern wegen des Sperrablaufs je etwa eine Minute. Er läuft nicht in der CI, weil er Browser-Images braucht. diff --git a/e2e/collab.e2e.mjs b/e2e/collab.e2e.mjs index 10d6446..577b7b3 100644 --- a/e2e/collab.e2e.mjs +++ b/e2e/collab.e2e.mjs @@ -135,6 +135,7 @@ const main = async () => { return probe.leases.others.length === 0; }, 25000); check(!!freed, 'Sperren eines geschlossenen Browsers laufen nach spätestens 20 s ab'); + await sleep(3500); // the admin's browser learns about it with its next sync (every 3 s in shared notes) await A.page.getByText('Von B', { exact: true }).first().click(); await A.page.keyboard.press('Home'); await A.page.keyboard.press('Tab'); diff --git a/e2e/lock.e2e.mjs b/e2e/lock.e2e.mjs new file mode 100644 index 0000000..f998d74 --- /dev/null +++ b/e2e/lock.e2e.mjs @@ -0,0 +1,170 @@ +// Locks, presence, undo and conflict copies with two real browsers. Needs a server in demo mode +// (accounts admin/admin and user/user); see e2e/README.md. +import { chromium } from 'playwright'; + +const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080'; +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); +let failures = 0; +const check = (cond, msg) => { + console.log(`${cond ? 'OK ' : 'FAIL'} ${msg}`); + if (!cond) failures++; +}; + +async function api(token, method, path, body) { + const res = await fetch(BASE + path, { + method, + headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: `Bearer ${token}` } : {}) }, + body: body ? JSON.stringify(body) : undefined, + }); + const text = await res.text(); + if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`); + return text ? JSON.parse(text) : null; +} + +async function waitFor(fn, ms) { + const end = Date.now() + ms; + while (Date.now() < end) { + try { + const v = await fn(); + if (v) return v; + } catch {} + await sleep(250); + } + return null; +} + +const serverBlocks = async (token, docId) => + (await api(token, 'GET', `/api/v1/docs/${docId}/snapshot`)).blocks.filter((b) => !b.deleted); +const editorValues = (page) => + page.$$eval('textarea', (els) => els.filter((e) => !e.placeholder?.startsWith('Titel')).map((e) => e.value)); + +// Puts the cursor at the end of the text field showing `text` (works whether or not it is focused already) +async function focusField(page, text) { + const handle = await page.waitForFunction( + (t) => Array.from(document.querySelectorAll('textarea')).find((e) => e.value === t) || null, + text, + { timeout: 5000 } + ); + const el = handle.asElement(); + await el.click(); + await page.keyboard.press('End'); +} + +async function openDoc(browser, token, title) { + const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } }); + await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token); + const page = await ctx.newPage(); + await page.goto(BASE); + await page.getByRole('button', { name: /Workspace/ }).first().click(); + await page.getByText(title, { exact: true }).first().click(); + await page.waitForFunction(() => document.querySelectorAll('textarea').length > 1, null, { timeout: 10000 }); + return { ctx, page }; +} + +const TITLE = `Sperr-Test ${Date.now()}`; +const id = (n) => `lk_${n}_${Date.now()}`; + +const main = async () => { + const admin = (await api(null, 'POST', '/api/v1/auth/login', { login: 'admin', password: 'admin' })).token; + const user = (await api(null, 'POST', '/api/v1/auth/login', { login: 'user', password: 'user' })).token; + const dir = await api(admin, 'GET', '/api/v1/users/directory'); + const userId = dir.find((u) => u.username === 'user' || u.email?.startsWith('user')).id; + const ids = { a: id('a'), b: id('b') }; + const doc = await api(admin, 'POST', '/api/v1/documents', { + title: TITLE, + blocks: [ + { id: ids.a, type: 'bullet', content: 'Admin schreibt hier', level: 0 }, + { id: ids.b, type: 'bullet', content: 'Darunter', level: 0 }, + ], + }); + await api(admin, 'POST', `/api/v1/documents/${doc.id}/shares`, { userId, permission: 'write' }); + + const browser = await chromium.launch(); + const A = await openDoc(browser, admin, TITLE); + const B = await openDoc(browser, user, TITLE); + + // Presence + const presence = await waitFor(async () => B.page.getByTitle('Hat diese Notiz gerade geöffnet').innerText(), 8000); + check(!!presence && /Admin/i.test(presence), `Bearbeiter sieht, wer noch in der Notiz ist: ${presence}`); + + // Admin clicks into the first bullet -> user sees the lock + await A.page.getByText('Admin schreibt hier', { exact: true }).first().click(); + const lockBadge = await waitFor(async () => (await B.page.getByTitle(/bearbeitet diesen Punkt gerade/).count()) > 0, 10000); + check(!!lockBadge, 'Bearbeiter sieht das Schloss am gesperrten Bullet'); + const readOnly = await B.page.$$eval('textarea', (els) => els.find((e) => e.value === 'Admin schreibt hier')?.readOnly); + check(readOnly === true, 'Gesperrter Bullet ist beim Bearbeiter schreibgeschützt'); + + // User tries to merge "Darunter" into the locked bullet + await B.page.getByText('Darunter', { exact: true }).first().click(); + await B.page.keyboard.press('Home'); + await B.page.keyboard.press('Backspace'); + const notice = await waitFor(async () => B.page.getByRole('status').innerText(), 3000); + check(!!notice && /bearbeitet diesen Punkt gerade/.test(notice), `Hinweis statt halber Aktion: ${notice}`); + await sleep(1500); + const afterMerge = (await serverBlocks(admin, doc.id)).map((b) => b.content).sort(); + check(JSON.stringify(afterMerge) === JSON.stringify(['Admin schreibt hier', 'Darunter']), `Nichts wurde teilweise ausgeführt: ${JSON.stringify(afterMerge)}`); + + // Undo on the admin side + await focusField(A.page, 'Admin schreibt hier'); + await A.page.keyboard.type(' und noch mehr'); + await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Admin schreibt hier und noch mehr'), 5000); + await A.page.keyboard.press('Control+z'); + const undone = await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Admin schreibt hier'), 5000); + check(!!undone, 'Strg+Z nimmt den getippten Text zurück (auch auf dem Server)'); + const fieldUndone = await waitFor(async () => (await editorValues(A.page)).includes('Admin schreibt hier'), 3000); + check(!!fieldUndone, 'Das fokussierte Feld zeigt den zurückgenommenen Text'); + await A.page.keyboard.press('Control+Shift+z'); + const redone = await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Admin schreibt hier und noch mehr'), 5000); + check(!!redone, 'Strg+Umschalt+Z stellt ihn wieder her'); + + // Typing right after Enter lands in the new bullet (no pause) + await A.page.keyboard.press('End'); + await A.page.keyboard.press('Enter'); + await A.page.keyboard.type('Direkt nach Enter'); + const newBullet = await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Direkt nach Enter'), 5000); + check(!!newBullet, 'Sofort nach Enter Getipptes landet im neuen Bullet'); + + // Conflict copy: two writes on the same base version via the API + const before = (await serverBlocks(admin, doc.id)).find((b) => b.id === ids.b); + await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'api-1', lastSeq: 0, ops: [{ opId: `${ids.b}-x`, kind: 'set_content', blockId: ids.b, baseVersion: before.version, content: 'Gewinner-Text' }] }); + await api(user, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'api-2', lastSeq: 0, ops: [{ opId: `${ids.b}-y`, kind: 'set_content', blockId: ids.b, baseVersion: before.version, content: 'Verlierer-Text' }] }); + const conflictShown = await waitFor(async () => (await A.page.getByText(/Version von/).count()) > 0, 8000); + check(!!conflictShown, 'Konfliktkopie wird markiert angezeigt'); + // The user's cursor is still in the original bullet (it holds the lock), so the admin may not overwrite it + await A.page.getByRole('button', { name: 'Übernehmen' }).first().click(); + const refused = await waitFor(async () => A.page.getByRole('status').innerText(), 3000); + check(!!refused && /bearbeitet/.test(refused), `Übernehmen in einen fremd gesperrten Punkt wird verweigert: ${refused}`); + // The user holding the original can accept once the copy's author (here a separate API client that never + // renews its lease) no longer holds the copy, i.e. after at most 20 s + await waitFor(async () => (await B.page.getByText(/Version von/).count()) > 0, 8000); + const copyId = (await serverBlocks(admin, doc.id)).find((b) => b.props?.conflictOf)?.id; + const copyFree = await waitFor(async () => { + const probe = await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'probe', lastSeq: 0 }); + return !probe.leases.others.some((l) => l.blockId === copyId); + }, 25000); + check(!!copyFree, 'Sperre der Konfliktkopie läuft ab'); + await sleep(3500); // the user's browser learns about it with its next sync + // Both API writes became conflict copies (the user's cursor locked the original); accept the first one + const copiesBefore = (await serverBlocks(admin, doc.id)).filter((b) => b.props?.conflictOf === ids.b); + const firstCopyText = await B.page.$$eval( + 'textarea', + (els, texts) => els.map((e) => e.value).find((v) => texts.includes(v)), + copiesBefore.map((c) => c.content) + ); + await B.page.getByRole('button', { name: 'Übernehmen' }).first().click(); + const accepted = await waitFor(async () => { + const blocks = await serverBlocks(admin, doc.id); + const copiesNow = blocks.filter((b) => b.props?.conflictOf === ids.b); + return blocks.find((b) => b.id === ids.b)?.content === firstCopyText && copiesNow.length === copiesBefore.length - 1; + }, 6000); + check(!!accepted, `„Übernehmen“ ersetzt den ursprünglichen Text durch „${firstCopyText}“ und entfernt diese Kopie`); + + await browser.close(); + console.log(failures ? `${failures} FEHLGESCHLAGEN` : 'ALLE PRÜFUNGEN BESTANDEN'); + process.exit(failures ? 1 : 0); +}; + +main().catch((e) => { + console.error(e); + process.exit(2); +}); diff --git a/e2e/run.sh b/e2e/run.sh index d047786..64efd7b 100755 --- a/e2e/run.sh +++ b/e2e/run.sh @@ -15,4 +15,4 @@ for _ in $(seq 1 60); do curl -s -m 1 http://localhost:18080/api/v1/health >/dev echo "Starte Browser-Tests …" "$ENGINE" run --rm --network host -v "$ROOT/e2e":/e2e:Z -w /e2e mcr.microsoft.com/playwright:v1.48.2-jammy \ - sh -c 'npm install --no-audit --no-fund >/dev/null 2>&1 && node collab.e2e.mjs && node share.e2e.mjs' + sh -c 'npm install --no-audit --no-fund >/dev/null 2>&1 && node collab.e2e.mjs && node share.e2e.mjs && node lock.e2e.mjs' diff --git a/frontend/src/features/workspace/DocumentEditor.tsx b/frontend/src/features/workspace/DocumentEditor.tsx index c533eff..e2af139 100644 --- a/frontend/src/features/workspace/DocumentEditor.tsx +++ b/frontend/src/features/workspace/DocumentEditor.tsx @@ -405,6 +405,50 @@ export const DocumentEditor: React.FC = ({ } }, [engineState?.status, engineState?.errorMessage]); + // Blocks other users are editing right now (blockId -> name) + const lockedBlocks = useMemo(() => { + const m = new Map(); + for (const l of engineState?.lockedByOthers || []) m.set(l.blockId, l.name || 'Jemand'); + return m; + }, [engineState?.lockedByOthers]); + + const otherPeople = useMemo( + () => (engineState?.presence || []).filter((p) => p.userId !== currentUser?.id), + [engineState?.presence, currentUser?.id] + ); + + // Short hint at the bottom of the editor (blocked actions, skipped undo) + const [notice, setNotice] = useState(null); + const noticeTimerRef = useRef(null); + const showNotice = useCallback((message: string) => { + setNotice(message); + if (noticeTimerRef.current !== null) window.clearTimeout(noticeTimerRef.current); + noticeTimerRef.current = window.setTimeout(() => setNotice(null), 3500); + }, []); + useEffect(() => () => { + if (noticeTimerRef.current !== null) window.clearTimeout(noticeTimerRef.current); + }, []); + + const handleLockedAction = useCallback( + (holder: string) => showNotice(`${holder} bearbeitet diesen Punkt gerade – bitte kurz warten`), + [showNotice] + ); + const handleUndo = useCallback(() => { + const res = engineRef.current?.undo(); + if (res?.message) showNotice(res.message); + }, [showNotice]); + const handleRedo = useCallback(() => { + const res = engineRef.current?.redo(); + if (res?.message) showNotice(res.message); + }, [showNotice]); + const handleResolveConflict = useCallback( + (blockId: string, accept: boolean) => { + const res = engineRef.current?.resolveConflict(blockId, accept); + if (res?.message) showNotice(res.message); + }, + [showNotice] + ); + // Every local block change goes to the engine, which turns it into operations const applyLocalBlocks = useCallback( (update: DocumentBlock[] | ((prev: DocumentBlock[]) => DocumentBlock[])) => { @@ -722,6 +766,16 @@ export const DocumentEditor: React.FC = ({ {/* Right Action Buttons */}
+ {/* Who else has this note open */} + {otherPeople.length > 0 && ( + + + {otherPeople.map((p) => p.name).join(', ')} + + )} {/* Sharing Section */} {doc && (() => { const isOwner = doc.isOwner !== false && (doc.userId === currentUser?.id || !doc.userId || currentUser?.role === 'admin'); @@ -907,6 +961,11 @@ export const DocumentEditor: React.FC = ({ onWikiLinkClick={onWikiLinkClick} initialFocusedBlockId={initialFocusedBlockId} onFocusedBlockChange={(id) => engineRef.current?.focusBlock(id)} + lockedBlocks={lockedBlocks} + onLockedAction={handleLockedAction} + onResolveConflict={handleResolveConflict} + onUndo={handleUndo} + onRedo={handleRedo} /> )}
@@ -1014,6 +1073,12 @@ export const DocumentEditor: React.FC = ({ e.target.style.height = `${e.target.scrollHeight}px`; }} onKeyDown={(e) => { + if ((e.ctrlKey || e.metaKey) && !e.altKey && ['z', 'y'].includes(e.key.toLowerCase())) { + e.preventDefault(); + if (e.key.toLowerCase() === 'y' || e.shiftKey) handleRedo(); + else handleUndo(); + return; + } if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); outlinerTreeRef.current?.focusFirstBlock(); @@ -1044,10 +1109,24 @@ export const DocumentEditor: React.FC = ({ onWikiLinkClick={onWikiLinkClick} initialFocusedBlockId={initialFocusedBlockId} onFocusedBlockChange={(id) => engineRef.current?.focusBlock(id)} + lockedBlocks={lockedBlocks} + onLockedAction={handleLockedAction} + onResolveConflict={handleResolveConflict} + onUndo={handleUndo} + onRedo={handleRedo} /> )} + + {notice && ( +
+ {notice} +
+ )} ); }; diff --git a/frontend/src/features/workspace/OutlinerBlock.tsx b/frontend/src/features/workspace/OutlinerBlock.tsx index b0cdc06..9a28d14 100644 --- a/frontend/src/features/workspace/OutlinerBlock.tsx +++ b/frontend/src/features/workspace/OutlinerBlock.tsx @@ -18,6 +18,8 @@ import { Maximize2, Link as LinkIcon, FileText, + Lock, + AlertTriangle, } from 'lucide-react'; import { DocumentBlock, BlockType } from '../../lib/types'; import { LinkModal } from './LinkModal'; @@ -70,6 +72,11 @@ export interface OutlinerBlockProps { isTargeted?: boolean; registerTextareaRef: (blockId: string, el: HTMLTextAreaElement | null) => void; onPasteMultiLines?: (blockId: string, textBefore: string, textAfter: string, rawLines: string[]) => void; + lockedBy?: string; // someone else is editing this block + canAppendBelow?: boolean; // the user may edit the note, so Enter in a locked block still adds one below + onResolveConflict?: (blockId: string, accept: boolean) => void; + onUndo?: () => void; + onRedo?: () => void; } // Rendered markdown only drops syntax characters (**, `, [..](url) …), so its text is a subsequence of the raw @@ -112,10 +119,17 @@ export const OutlinerBlock: React.FC = ({ isTargeted, registerTextareaRef, onPasteMultiLines, + lockedBy, + canAppendBelow, + onResolveConflict, + onUndo, + onRedo, }) => { const [localContent, setLocalContent] = useState(block.content || ''); const [isLocallyFocused, setIsLocallyFocused] = useState(false); const activeFocus = isFocused || isLocallyFocused; + // A block someone else is editing can still get the cursor (read-only), e.g. to press Enter at its end + const canFocus = canEdit || (!!lockedBy && !!canAppendBelow); const [slashSearch, setSlashSearch] = useState(null); const [slashIndex, setSlashIndex] = useState(0); const [showLinkModal, setShowLinkModal] = useState(false); @@ -164,6 +178,13 @@ export const OutlinerBlock: React.FC = ({ [] ); + // Right after undo/redo the focused field must show the engine's text, not keep its own + const acceptRemoteUntilRef = useRef(0); + const onUndoRef = useRef(onUndo); + onUndoRef.current = onUndo; + const onRedoRef = useRef(onRedo); + onRedoRef.current = onRedo; + // Sync incoming remote changes ONLY when this textarea is NOT actively being edited // Condition: isFocusedRef.current === true && document.hasFocus() useEffect(() => { @@ -174,7 +195,8 @@ export const OutlinerBlock: React.FC = ({ textareaRef.current !== null && document.activeElement === textareaRef.current; - if (!isActivelyEditing && block.content !== localContentRef.current) { + const acceptAfterUndo = Date.now() < acceptRemoteUntilRef.current; + if ((!isActivelyEditing || acceptAfterUndo) && block.content !== localContentRef.current) { setLocalContent(block.content || ''); localContentRef.current = block.content || ''; } @@ -406,7 +428,44 @@ export const OutlinerBlock: React.FC = ({ onPasteMultiLinesRef.current?.(block.id, textBefore, textAfter, rawLines); }; + // Ctrl/Cmd+Z undoes the user's own changes in the whole note (not the browser's per-field undo) + const runHistory = (redo: boolean) => { + if (debounceTimerRef.current !== null) { + flushDebounce(localContentRef.current); + } + acceptRemoteUntilRef.current = Date.now() + 500; + if (redo) onRedoRef.current?.(); + else onUndoRef.current?.(); + }; + + useEffect(() => { + const el = textareaRef.current; + if (!el) return; + // Mobile keyboards and the context menu undo via beforeinput instead of key events + const onBeforeInput = (e: InputEvent) => { + if (e.inputType === 'historyUndo' || e.inputType === 'historyRedo') { + e.preventDefault(); + runHistory(e.inputType === 'historyRedo'); + } + }; + el.addEventListener('beforeinput', onBeforeInput); + return () => el.removeEventListener('beforeinput', onBeforeInput); + }); + const handleKeyDown = (e: React.KeyboardEvent) => { + if ((e.ctrlKey || e.metaKey) && !e.altKey && ['z', 'y'].includes(e.key.toLowerCase())) { + e.preventDefault(); + e.stopPropagation(); + runHistory(e.key.toLowerCase() === 'y' || e.shiftKey); + return; + } + // Enter in a block someone else is editing cannot split it (read-only), so it adds a new block below + // without touching the locked one. (Chromium does not move the caret with End in read-only fields.) + if (lockedBy && canAppendBelow && e.key === 'Enter' && !e.shiftKey) { + e.preventDefault(); + onCreateBlockBelowRef.current(block.id, ''); + return; + } if (!canEdit) return; // Slash command navigation @@ -577,7 +636,7 @@ export const OutlinerBlock: React.FC = ({ : isTargeted ? 'bg-panel-elevated' : '' - }`} + } ${block.conflictOf ? 'border-l-2 border-status-warn bg-status-warn/5' : ''}`} style={{ paddingLeft: `${visualLevel * 22 + 4}px`, }} @@ -585,7 +644,7 @@ export const OutlinerBlock: React.FC = ({ onDrop={() => onDropBlock?.(index)} onClick={(e) => { if ((e.target as HTMLElement).closest('button, a, input, select')) return; - if (!activeFocus && canEdit && textareaRef.current) { + if (!activeFocus && canFocus && textareaRef.current) { textareaRef.current.focus(); const len = textareaRef.current.value.length; textareaRef.current.setSelectionRange(len, len); @@ -728,7 +787,7 @@ export const OutlinerBlock: React.FC = ({ {/* View Layer with Inline Markdown & Actions (3-dots or Erweitert Badge) directly behind text */}
{ - if (!activeFocus && canEdit && textareaRef.current) { + if (!activeFocus && canFocus && textareaRef.current) { // Keep the row's click handler from moving the caret back to the end e.stopPropagation(); focusAtPoint(e.clientX, e.clientY); @@ -875,6 +934,37 @@ export const OutlinerBlock: React.FC = ({
)} + {/* Conflict copy: text that lost against someone else's change */} + {block.conflictOf && ( +
+ + + {block.conflictOf === 'title' ? 'Konflikt beim Titel' : 'Konflikt'} + {block.conflictAuthor ? ` – Version von ${block.conflictAuthor}` : ''} + + {onResolveConflict && ( + <> + + + + )} +
+ )} + {/* Rich Tokens Bar (Wiki-Links & #Tags) if present */} {hasSpecialTokens && !isFocused && (
@@ -980,6 +1070,17 @@ export const OutlinerBlock: React.FC = ({ + {/* Someone else is editing this block */} + {lockedBy && ( + + + {lockedBy} + + )} + {/* Delete button on hover (Trash2 icon) */} {canEdit && (