17 Commits
Author SHA1 Message Date
sandrodach 1220919257 Merge dev into main for v0.2
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m21s
2026-10-28 15:32:39 +01:00
sandrodachandClaude Opus 5.5 4e39b367bd release: v0.2 – collaboration docs, richer demo data, new screenshots
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m30s
- Turn the collaboration plan into module docs (Doku/modules/collaboration.md)
  describing the implemented state; remove Doku/plans
- Header version comes from the build: git tag via APP_VERSION (CI passes
  it for tag builds), otherwise "<package version>-dev"; package 0.2.0
- Sample data (setup option and demo mode): six inbox thoughts, two task
  lists with Kanban states, subtasks and due dates, shopping and packing
  lists, three notes in folders incl. a long nested outline; demo mode
  shares the note and two lists with user/user
- e2e/screenshots.sh renews the README screenshots from demo data
- README: live collaboration, editor sharing, image tags, version badge

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 15:32:31 +01:00
sandrodachandClaude Opus 5.5 f26f286fb8 feat(notes): compact the operation log and purge old tombstones
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m17s
- Background job at start and every 6 h: log entries older than 30 days
  (private notes: 7) are removed, documents.log_floor marks the boundary;
  clients behind it get resync and keep their queued changes
- Deleted blocks are purged 30 days after deleted_at via a "purge" log
  effect, so open editors drop them without reloading; a tombstone stays
  while a deleted child still points to it
- Text for a purged block becomes a new block at the end; inserts under a
  purged parent go to the top level (server and browser model)
- doc_op_results kept for 90 days; expired lease/session maps are swept
- Tests: storage, API, engine resync, simulation with compactions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 15:08:13 +01:00
sandrodachandClaude Opus 5.5 aea4324c20 refactor(notes): remove legacy whole-document saves; all block changes go through sync
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m14s
Phase 5 of the collaboration plan:
- Remove PATCH /documents/{id}/blocks, PatchDocumentBlocks, the
  DOC_BLOCKS_PATCHED event and its types, FlushDocumentState and
  LoadDocumentState. PUT /documents/{id} rejects blocks (400) and only
  changes title, icon, folder and pin; editors may now change title/icon.
- The to-do overview ticks via a single operation on /docs/{id}/sync
  (new skipLog flag). Task links set the block tick with a targeted
  operation and notify open editors; creating a linked task needs edit
  rights.
- Keep the three share tables: DocRole already decides centrally, and
  rewriting the visibility queries would only add risk (documented in plan).
- Tests adapted to the new paths; share e2e test now also ticks a to-do in
  the overview.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 14:35:59 +01:00
sandrodachandClaude Opus 5.5 8b7a53daab feat(notes): push, long polling and fallback detection for note sync
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m16s
Phase 4 of the collaboration plan:
- The sync engine listens for DOC_SEQ in the note's realtime room and syncs
  right away. Without WebSocket/SSE it long-polls /docs/{id}/changes (25 s);
  after three failures it falls back to polling every 3 s and retries long
  polling when the browser comes back online.
- With push or long polling the engine only syncs every 7 s while holding
  leases (renewal), otherwise every 30 s as a safety net. Focusing a block
  announces its lease immediately (otherwise it could take up to 30 s).
- Realtime connection: SSE counts as connected only after its first event;
  a stream that stays silent for 5 s (buffering proxy) switches SSE off for
  the tab. App-level PING/PONG detects WebSockets that are accepted but
  forward nothing and switches to SSE after 35 s.
- The editor shows the active channel for shared notes.
- Tests: engine channel tests (push, heartbeat, lease on focus, long
  polling, interval fallback, switching), realtime fallback tests with fake
  WebSocket/EventSource, Go tests for PONG and DOC_SEQ delivery, and
  e2e/channels.e2e.mjs measuring latency behind six simulated proxy setups
  (0.2-0.3 s on all live channels, ~3 s with plain polling).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 14:10:29 +01:00
sandrodachandClaude Opus 5.5 aa397c18cd feat(notes): show block locks and presence, conflict copies with accept/discard, undo/redo
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m21s
Phase 3 of the collaboration plan:
- Blocks held by someone else show a lock with the name and are read-only;
  Enter in such a block adds a new block below. Delete, merge, indent and
  drag are not started when a involved block (or one in its branch) is
  locked; a short notice explains why (fixes FLAW-25).
- Presence: the editor header lists who else has the note open.
- Conflict copies are highlighted ("Konflikt – Version von …") with
  Übernehmen (replaces the original or the title) and Verwerfen. Both check
  the locks of the original and of the copy first.
- Undo/redo of own changes (Ctrl/Cmd+Z, Ctrl+Shift+Z, Ctrl+Y, beforeinput
  history events): typing is grouped per block until a 1 s pause, up to 200
  steps per tab; a step is skipped with a notice if someone else changed or
  holds the block meanwhile. New clearConflict props patch on the server.
- New blocks get focus synchronously on mount (fixes FLAW-26).
- Fix: after reopening a note from local storage the engine polled every
  30 s instead of 3 s until the next change, because the delay was decided
  before the first response told it the note is shared.
- Tests: undo/conflict engine tests, lock/undo/conflict UI tests and
  e2e/lock.e2e.mjs (two browsers); all three browser tests pass against a
  freshly built image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 13:44:27 +01:00
sandrodachandClaude Opus 5.5 08d9b2dc77 feat(workspace): allow sharing notes and shared folders as editor
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m13s
The share dialog for notes and shared folders now offers "Bearbeiten" and
"Nur Lesen" and stores write/read accordingly. Editors already edit through
the sync engine since phase 2.

Adds e2e/share.e2e.mjs: shares a note as editor through the dialog and checks
that the recipient can edit it; e2e/run.sh runs both browser tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 12:40:22 +01:00
sandrodachandClaude Opus 5.5 51f90eb7e4 feat(notes): browser sync engine with offline queue; editor saves through the operation log
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m26s
Phase 2 of the collaboration plan:
- frontend/src/lib/docSync: TypeScript port of the block tree, operations,
  effects and order keys (kept identical to Go via shared test vectors),
  diff from the editor's flat list to operations, IndexedDB persistence,
  HTTP transport and DocSyncEngine (one request in flight, idempotent
  resends, backoff, 3 s/30 s polling, keepalive flush and leave on exit,
  block leases for focus, 20 s grace and new blocks).
- Text changes carry the version the user saw when typing (first change
  wins when coalescing; a stale focused field keeps its old base), so a
  concurrent edit becomes a conflict copy instead of a silent overwrite.
- DocumentEditor uses the engine instead of the 800 ms whole-document PUT;
  editors (not only owners) can edit, remote changes appear live.
- Server: task IDs of linked blocks are delivered again, linked tasks follow
  ticked/deleted blocks after sync, note tags are derived, title changes
  notify other users; block IDs already used in another note are renamed
  (starter content with fixed IDs was empty for every further user).
- Fix: unlinking tasks from blocks wrote to non-existent columns and never
  worked (error was swallowed).
- Tests: Go/TS order key vectors, diff round trips, engine races (lost
  responses, offline + reload, edits arriving between typing and sending,
  stale focused field, duplicate first block), plus a Playwright test with
  two browsers against the real image (e2e/run.sh).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 12:28:15 +01:00
sandrodachandClaude Opus 5.5 1ea0c78b41 feat(notes): operation log, block leases and sync API for collaborative editing
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 50s
Phase 1 of the collaboration plan (server side):
- internal/collab: block tree with parent IDs and fractional order keys,
  operations with conflict rules (version check, block leases, conflict
  copies instead of lost text, restore on late edits, cycle checks),
  in-memory leases/sessions and long-poll notifier.
- Storage: per-note lock + transaction, append-only effect log (doc_ops),
  idempotent results per opId (doc_op_results), snapshot, log paging,
  central DocRole. Old flat block data is converted to the tree.
- The existing whole-document save (PUT/PATCH) runs as an adapter through
  the same engine, so the current editor keeps working and writes to the log.
- API: GET /docs/{id}/snapshot, POST /docs/{id}/sync, GET /docs/{id}/changes
  (long poll), POST /docs/{id}/leave; DOC_SEQ poke for WebSocket/SSE rooms.
- Backend accepts editor (write) shares for notes and folders; the forced
  read-only migration is gone. The share dialog still offers read only.
- Tests: conflict matrix, order keys, leases, storage adapter round trip,
  idempotency, API protocol incl. long poll, and a randomized simulation
  (4 clients, lost/duplicated requests and responses, offline phases,
  lease expiry) checking convergence, no lost text and exactly-once results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 11:47:56 +01:00
sandrodachandClaude Opus 5.5 bcc06da3ee fix(realtime): scope events to authorized users, authorize rooms, keep SSE alive
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m6s
Phase 0 of the collaboration plan:
- Remove BroadcastToAll; task, task list and workspace folder events now
  only reach users who can see the resource. Task list events carry only
  the list ID, clients reload with their own permissions.
- Reject unauthenticated WebSocket/SSE connections (401). Every room join
  (URL room, JOIN_ROOM, SSE rooms) is checked via a room authorizer;
  /events/rooms requires login and only accepts the caller's own sseId.
- Reorder rejects tasks the caller cannot access or edit (IDOR).
- Lift the server WriteTimeout for SSE streams so they are not cut after 30s.
- Fix folder share removal sending DOC_DELETED to the share ID.
- Tests for all of the above; docs and flaw list updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 11:19:14 +01:00
sandrodachandClaude Opus 5.5 74797e3f44 docs(plan): block leases with auto-lease on insert and release grace period
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 8s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 11:10:15 +01:00
sandrodachandClaude Opus 5.5 af578d8237 docs: add plan for collaborative note editing
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 8s
Server-authoritative operation log with per-block versions and 20s
block leases, conflict copies instead of data loss, IndexedDB queue
for offline use, HTTP-based sync with WebSocket/SSE/long-poll wakeups,
unified note/folder permissions, and per-user undo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 11:03:45 +01:00
sandrodach 8a4b4820df Merge pull request 'Dev' (#9) from dev into main
Reviewed-on: #9
2026-10-28 10:38:28 +01:00
sandrodachandClaude Opus 5.5 106a34d260 test(frontend): set up Vitest with jsdom and run tests in the image build
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 43s
- Add vitest, jsdom and Testing Library as dev dependencies,
  replace the placeholder test:run script with vitest run.
- Add outliner tests for Backspace merging and caret placement on click.
- Run the frontend tests in the Dockerfile before the Vite build.
- Update docs (README, architecture, FLAW-19).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 09:25:10 +01:00
sandrodachandClaude Opus 5.5 2b5291dc7b fix(workspace): caret at click position, no grey row highlight, hover-only collapse toggles
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 59s
- Clicking into a bullet places the caret where the mouse was, mapped
  from the rendered markdown layer back to the raw text, instead of
  always jumping to the end.
- Remove the grey background on focused and hovered rows and on the
  bullet handle.
- Collapse/expand chevrons are only shown while hovering the row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 09:19:35 +01:00
sandrodachandClaude Sonnet 5.5 b4c8577a61 fix(workspace): make Backspace merge work for untyped blocks and blocks with children
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 57s
The merge only applied to a fixed list of block types, so legacy blocks
without a type were ignored, and blocks with children were rejected.
Now only code/divider/table/progress are excluded and children move up
under the previous block. The focused bullet dot is also neutral now.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 08:53:35 +01:00
sandrodachandClaude Sonnet 5.5 c7e05b385c feat(workspace): merge bullet on Backspace, render hashtags inline
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m8s
- Backspace at the start of a non-empty bullet appends its text to the
  previous visible bullet (skipped for blocks with children and for
  code/divider/table/progress blocks).
- Hashtags are rendered inline as clickable chips instead of a badge row
  below the text, so lines no longer grow taller.
- Targeted/highlighted bullets use a neutral background instead of accent.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 08:31:25 +01:00
82 changed files with 11055 additions and 1071 deletions

No files matched your search

+1
View File
@@ -7,3 +7,4 @@ frontend/dist
*.db-shm *.db-shm
*.db-wal *.db-wal
.env .env
e2e
+7 -1
View File
@@ -37,6 +37,12 @@ jobs:
echo "image=${IMAGE_NAME}" >> "$GITHUB_OUTPUT" echo "image=${IMAGE_NAME}" >> "$GITHUB_OUTPUT"
echo "tag=${DOCKER_TAG}" >> "$GITHUB_OUTPUT" echo "tag=${DOCKER_TAG}" >> "$GITHUB_OUTPUT"
echo "actor=${ACTOR_NAME}" >> "$GITHUB_OUTPUT" echo "actor=${ACTOR_NAME}" >> "$GITHUB_OUTPUT"
# Releases (tags) show the tag as version in the app header
APP_VERSION=""
case "${GITHUB_REF:-}" in
refs/tags/*) APP_VERSION="${REF_NAME}" ;;
esac
echo "app_version=${APP_VERSION}" >> "$GITHUB_OUTPUT"
echo "Target Docker image: ${IMAGE_NAME}:${DOCKER_TAG} (Registry: ${REGISTRY_HOST}, Actor: ${ACTOR_NAME})" echo "Target Docker image: ${IMAGE_NAME}:${DOCKER_TAG} (Registry: ${REGISTRY_HOST}, Actor: ${ACTOR_NAME})"
- name: Login to Gitea Container Registry - name: Login to Gitea Container Registry
@@ -50,6 +56,6 @@ jobs:
run: | run: |
FULL_IMAGE="${{ steps.prep.outputs.image }}:${{ steps.prep.outputs.tag }}" FULL_IMAGE="${{ steps.prep.outputs.image }}:${{ steps.prep.outputs.tag }}"
echo "🚀 Building container & executing Frontend/Backend checks inside image: ${FULL_IMAGE}" echo "🚀 Building container & executing Frontend/Backend checks inside image: ${FULL_IMAGE}"
DOCKER_BUILDKIT=1 docker build --progress=plain -t "${FULL_IMAGE}" . DOCKER_BUILDKIT=1 docker build --progress=plain --build-arg APP_VERSION="${{ steps.prep.outputs.app_version }}" -t "${FULL_IMAGE}" .
echo "✅ All checks & tests passed inside container! Pushing ${FULL_IMAGE}..." echo "✅ All checks & tests passed inside container! Pushing ${FULL_IMAGE}..."
docker push "${FULL_IMAGE}" docker push "${FULL_IMAGE}"
+1
View File
@@ -17,3 +17,4 @@ dev-data/
.DS_Store .DS_Store
.idea/ .idea/
.vscode/ .vscode/
e2e/node_modules/
+3 -1
View File
@@ -6,7 +6,9 @@ WORKDIR /app/frontend
COPY frontend/package*.json ./ COPY frontend/package*.json ./
RUN npm install --no-audit --no-fund RUN npm install --no-audit --no-fund
COPY frontend/ ./ COPY frontend/ ./
RUN npx tsc --noEmit && npm run build # Version shown in the header: the git tag for releases (set by CI), otherwise "<package version>-dev"
ARG APP_VERSION=""
RUN npx tsc --noEmit && npm run test:run && APP_VERSION="${APP_VERSION}" npm run build
# ============================================================================== # ==============================================================================
# Stage 2: Go Backend Lint, Unit Tests & Binary Build (Cached Go module & sqlite layer) # Stage 2: Go Backend Lint, Unit Tests & Binary Build (Cached Go module & sqlite layer)
+5 -3
View File
@@ -13,6 +13,7 @@ Referenz für Entwickler und Administratoren. Alle Angaben wurden gegen den Code
| [modules/triage.md](modules/triage.md) | Schnellerfassung und Triage | | [modules/triage.md](modules/triage.md) | Schnellerfassung und Triage |
| [modules/tasks.md](modules/tasks.md) | Aufgaben, Kanban, Alarme | | [modules/tasks.md](modules/tasks.md) | Aufgaben, Kanban, Alarme |
| [modules/workspace.md](modules/workspace.md) | Outliner-Notizen, Ordner, Tags | | [modules/workspace.md](modules/workspace.md) | Outliner-Notizen, Ordner, Tags |
| [modules/collaboration.md](modules/collaboration.md) | Gemeinsames Bearbeiten von Notizen: Log, Sperren, Konflikte, Offline, Transport-Stufen |
| [modules/lists.md](modules/lists.md) | Geteilte Listen | | [modules/lists.md](modules/lists.md) | Geteilte Listen |
| [sharing-and-permissions.md](sharing-and-permissions.md) | Rollen, Gruppen, Freigaben | | [sharing-and-permissions.md](sharing-and-permissions.md) | Rollen, Gruppen, Freigaben |
| [reverse-proxy.md](reverse-proxy.md) | Nginx, Traefik, Caddy, Cloudflare | | [reverse-proxy.md](reverse-proxy.md) | Nginx, Traefik, Caddy, Cloudflare |
@@ -23,9 +24,10 @@ Referenz für Entwickler und Administratoren. Alle Angaben wurden gegen den Code
- **Ein Go-Binary** liefert REST-API, WebSocket/SSE und das eingebettete React-Frontend (`go:embed`). Es gibt genau einen Container und eine SQLite-Datei (`/data/mindsift.db`). - **Ein Go-Binary** liefert REST-API, WebSocket/SSE und das eingebettete React-Frontend (`go:embed`). Es gibt genau einen Container und eine SQLite-Datei (`/data/mindsift.db`).
- **Datenhaltung:** vollständig in SQLite. Es gibt keine `.md`-Dateien auf der Platte; Markdown wird bei Bedarf aus den Blöcken erzeugt (`documents.markdown_cache`, Export als `.md`). - **Datenhaltung:** vollständig in SQLite. Es gibt keine `.md`-Dateien auf der Platte; Markdown wird bei Bedarf aus den Blöcken erzeugt (`documents.markdown_cache`, Export als `.md`).
- **Echtzeit:** ein Hub mit Räumen und Benutzer-Kanälen. Verteilt werden Ereignisse nach REST-Änderungen. Es gibt keine CRDTs, Sperren oder Cursor-Presence. - **Echtzeit:** ein Hub mit Räumen und Benutzer-Kanälen (WebSocket, SSE als Ersatz). Verteilt werden Ereignisse nach Änderungen.
- **Notizen gemeinsam bearbeiten:** server-autoritatives Operations-Log mit Versionen und Sperren pro Bullet, Konfliktkopien statt Überschreiben, Offline-Warteschlange in IndexedDB, Abgleich auch hinter restriktiven Proxys ([modules/collaboration.md](modules/collaboration.md)).
- **Auth:** JWT (HS256, 30 Tage) mit `token_version` und Sperrliste, dazu API-Keys (`mindsift_sec_…`) für Webhooks. - **Auth:** JWT (HS256, 30 Tage) mit `token_version` und Sperrliste, dazu API-Keys (`mindsift_sec_…`) für Webhooks.
- **Freigaben:** Aufgabenlisten und geteilte Listen mit Rolle Bearbeiter oder Leser, Notizen und Ordner nur lesend. - **Freigaben:** Aufgabenlisten, geteilte Listen, Notizen und Ordner jeweils mit Rolle Bearbeiter oder Leser.
## Entwickeln ## Entwickeln
@@ -34,7 +36,7 @@ Referenz für Entwickler und Administratoren. Alle Angaben wurden gegen den Code
go vet ./... && go test ./... go vet ./... && go test ./...
# Frontend (Node 22) # Frontend (Node 22)
cd frontend && npm ci && npx tsc --noEmit && npm run build cd frontend && npm ci && npx tsc --noEmit && npm run test:run && npm run build
``` ```
Das Frontend wird für das Binary nach `cmd/server/dist` gebaut (siehe `Dockerfile`). Die CI läuft unter `.gitea/workflows/build-and-publish.yaml`. Das Frontend wird für das Binary nach `cmd/server/dist` gebaut (siehe `Dockerfile`). Die CI läuft unter `.gitea/workflows/build-and-publish.yaml`.
+41 -8
View File
@@ -13,15 +13,15 @@ Basis: `/api/v1`. Antworten sind JSON; Fehler haben die Form `{"error": "Text"}`
| POST | `/auth/login` | Anmeldung (`login`/`email`/`username`, `password`) → `{token, user}` | | POST | `/auth/login` | Anmeldung (`login`/`email`/`username`, `password`) → `{token, user}` |
| POST | `/auth/quickstart` | Immer `403` (Gastzugang deaktiviert) | | POST | `/auth/quickstart` | Immer `403` (Gastzugang deaktiviert) |
| POST | `/webhooks/ingest` | Aufgabe per Webhook anlegen, Token als `?token=`, `X-API-Key` oder Bearer. Body: `title`/`content`/`text`/`message`, `priority`, `source`, `tags`, `dueDate`; Klartext-Body wird als Inhalt genommen | | POST | `/webhooks/ingest` | Aufgabe per Webhook anlegen, Token als `?token=`, `X-API-Key` oder Bearer. Body: `title`/`content`/`text`/`message`, `priority`, `source`, `tags`, `dueDate`; Klartext-Body wird als Inhalt genommen |
| GET | `/ws`, `/ws/spaces/{id}`, `/ws/lists/{id}` | WebSocket, siehe [websocket.md](websocket.md) | | GET | `/ws`, `/ws/spaces/{id}`, `/ws/lists/{id}` | WebSocket, siehe [websocket.md](websocket.md); Token als `?token=` nötig |
| GET | `/events` | SSE-Fallback | | GET | `/events` | SSE-Fallback; Token als `?token=` nötig |
| POST | `/events/rooms` | SSE-Räume beitreten/verlassen ([FLAW-02](../flaws.md)) |
## Konto und Allgemeines (angemeldet) ## Konto und Allgemeines (angemeldet)
| Methode | Pfad | Zweck | | Methode | Pfad | Zweck |
|---|---|---| |---|---|---|
| GET | `/auth/me` | Benutzer und Kennzahlen | | GET | `/auth/me` | Benutzer und Kennzahlen |
| POST | `/events/rooms` | SSE-Raum beitreten/verlassen (`sseId`, `room`, `action`); nur eigene `sseId`, nur Räume mit Lesezugriff |
| POST | `/auth/logout` | Token sperren | | POST | `/auth/logout` | Token sperren |
| POST | `/auth/revoke-all` | Alle Sitzungen beenden (`token_version` +1) | | POST | `/auth/revoke-all` | Alle Sitzungen beenden (`token_version` +1) |
| PATCH | `/auth/settings` | `displayName`, `email`, `password`, `avatarUrl`, `alarmChannels`, `alarmEmail`, `discordWebhookUrl`, `customWebhookUrl`, `notifyBeforeMinutes` | | PATCH | `/auth/settings` | `displayName`, `email`, `password`, `avatarUrl`, `alarmChannels`, `alarmEmail`, `discordWebhookUrl`, `customWebhookUrl`, `notifyBeforeMinutes` |
@@ -67,17 +67,50 @@ Schreibzugriffe von Lesern auf Listen oder deren Aufgaben liefern `403` („Kein
| GET/POST | `/workspace/folders/shared` | Geteilte Ordner; anlegen (`name`, `mountPath`) | | GET/POST | `/workspace/folders/shared` | Geteilte Ordner; anlegen (`name`, `mountPath`) |
| GET/DELETE | `/workspace/folders/shared/{id}` | Abruf; Löschen | | GET/DELETE | `/workspace/folders/shared/{id}` | Abruf; Löschen |
| PUT | `/workspace/folders/shared/{id}/mount` | Eigenen Einhängepfad ändern (`mountPath`) | | PUT | `/workspace/folders/shared/{id}/mount` | Eigenen Einhängepfad ändern (`mountPath`) |
| GET/POST/DELETE | `/workspace/folders/shared/{id}/shares[/{shareId}]` | Freigaben (nur Lesen wirksam) | | GET/POST/DELETE | `/workspace/folders/shared/{id}/shares[/{shareId}]` | Freigaben (`permission`: `read`/`write`) |
| GET | `/workspace/tags`, `/workspace/tags/{tag}` | Tag-Übersicht und -Details | | GET | `/workspace/tags`, `/workspace/tags/{tag}` | Tag-Übersicht und -Details |
| PUT | `/workspace/tags/{tag}/note` | Tag-Seite speichern (`content`) | | PUT | `/workspace/tags/{tag}/note` | Tag-Seite speichern (`content`) |
| GET/POST | `/documents` | Notizen auflisten; anlegen (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`) | | GET/POST | `/documents` | Notizen auflisten; anlegen (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`) |
| GET/PUT/DELETE | `/documents/{id}` | Abruf; Ändern (`title`, `icon`, `folderPath`, `blocks`, `tags`, `isPinned`); Löschen | | GET/PUT/DELETE | `/documents/{id}` | Abruf; Ändern von `title`, `icon`, `folderPath`, `isPinned` (Blöcke → `400`, sie laufen über `/docs/{id}/sync`); Löschen |
| PATCH | `/documents/{id}/blocks` | Teil-Update der Blöcke (`upserted`, `deletedIds`, Reihenfolge); löst `DOC_BLOCKS_PATCHED` aus |
| PUT | `/documents/{id}/move` | Ordner wechseln (`folderPath`) | | PUT | `/documents/{id}/move` | Ordner wechseln (`folderPath`) |
| POST | `/documents/{id}/blocks/{blockId}/enhanced-task` | Aus einem Block eine verknüpfte Aufgabe machen (`content`, `checked`) | | POST | `/documents/{id}/blocks/{blockId}/enhanced-task` | Aus einem Block eine verknüpfte Aufgabe machen (`content`, `checked`) |
| GET/POST/DELETE | `/documents/{id}/shares[/{userId}]` | Notiz-Freigaben (`userId`/`email`/`teamId`; wirksam nur Lesen) | | GET/POST/DELETE | `/documents/{id}/shares[/{userId}]` | Notiz-Freigaben (`userId`/`email`/`teamId`, `permission`: `read`/`write`) |
Nur der Eigentümer schreibt in eine Notiz; Empfänger erhalten `403`/„nur Lesezugriff“ ([FLAW-17](../flaws.md)). Der Inhalt einer Notiz (Blöcke) wird ausschließlich über `POST /docs/{id}/sync` geändert. `POST /documents` nimmt beim Anlegen weiterhin eine Start-Blockliste an; sie wird intern in Operationen umgerechnet.
## Gemeinsames Bearbeiten von Notizen
Konzept und Regeln: [modules/collaboration.md](../modules/collaboration.md). Zugriff über die Rolle `owner`/`editor`/`viewer` (`Store.DocRole`); ohne Rolle `404`.
| Methode | Pfad | Zweck |
|---|---|---|
| GET | `/docs/{id}/snapshot` | Kompletter Blockbaum inkl. Grabsteinen, `title`, `titleVersion`, `headSeq`, `role`, `shared` |
| POST | `/docs/{id}/sync` | Ein Rundlauf: eigene Operationen senden, Leases halten/abgeben, Log ab `lastSeq` abholen |
| GET | `/docs/{id}/changes?since=N&wait=S` | Long-Polling: antwortet sofort, wenn `headSeq > N`, sonst bei der nächsten Änderung (auch Leases/Präsenz) oder nach höchstens 25 s. Antwort `{headSeq, changed}`; Daten holt der Client danach per `/sync` |
| POST | `/docs/{id}/leave` | Sitzung beenden, alle Leases des Clients freigeben (`{clientId}`) |
`/sync`-Anfrage:
```json
{
"clientId": "tab-7f3…",
"lastSeq": 412,
"ops": [
{ "opId": "c1a…", "kind": "set_content", "blockId": "b9…", "baseVersion": 7, "content": "Milch und Brot" },
{ "opId": "c1b…", "kind": "insert", "blockId": "bA…", "parentId": "b2…", "orderKey": "a0V", "type": "bullet", "content": "" }
],
"leases": { "hold": ["b9…", "bA…"], "release": [] }
}
```
- Operationsarten: `insert`, `set_content`, `set_props` (`patch`: `type`, `checked`, `collapsed`, `lang`, `clearConflict`), `move`, `delete`, `restore` (`ids`), `set_title` (`baseVersion`, `title`).
- `leases.hold` ist die vollständige Liste der Blöcke, die der Client halten will; alle anderen Leases des Clients werden freigegeben. Neu eingefügte Blöcke und Konfliktkopien bekommt der Autor automatisch.
- Höchstens 500 Operationen pro Anfrage. Leser dürfen keine Operationen senden (`403`), aber abfragen.
- `skipLog: true` liefert kein Log zurück; gedacht für einzelne Änderungen ohne geöffneten Editor (z. B. Abhaken in der To-Do-Übersicht).
Antwort: `results` (je Operation `status` `applied`/`conflict`/`rejected`, ggf. `conflictBlockId`, `version`, `seq`), `ops` (Log-Einträge `{seq, opId, userId, clientId, effect}` ab `lastSeq`, höchstens 1000; `hasMore` bei mehr), `headSeq`, `resync` (Snapshot neu laden: der Client ist dem Server voraus, oder sein `lastSeq` liegt vor der verdichteten Grenze `log_floor`; die gesendeten Operationen werden trotzdem angewendet), `role`, `shared`, `sessions`, `presence`, `leases` (`granted`, `denied`, `others`).
Wiederholte Operationen (gleiche `opId`) werden nicht erneut angewendet; die Antwort enthält das gespeicherte Ergebnis.
## Geteilte Listen ## Geteilte Listen
+21 -12
View File
@@ -11,9 +11,18 @@ Der Server verteilt Änderungen als JSON-Ereignisse an verbundene Clients. Die E
| WebSocket | `GET /api/v1/ws` (auch `/ws/lists/{id}`, `/ws/spaces/{id}`) | Ping alle 30 s, Lesefrist 60 s, Nachrichten bis 4 MB | | WebSocket | `GET /api/v1/ws` (auch `/ws/lists/{id}`, `/ws/spaces/{id}`) | Ping alle 30 s, Lesefrist 60 s, Nachrichten bis 4 MB |
| SSE | `GET /api/v1/events` | Kommentar-Ping alle 15 s, Header `X-Accel-Buffering: no` | | SSE | `GET /api/v1/events` | Kommentar-Ping alle 15 s, Header `X-Accel-Buffering: no` |
Der Client versucht zuerst WebSocket. Gelingt der Aufbau nicht innerhalb von 2,5 s oder bricht die Verbindung unsauber ab, wechselt er auf SSE und bleibt dabei (Firmen-Proxys). Dort liefert der Server `SSE_CONNECTED` mit einer `sseId`; Räume werden über `POST /api/v1/events/rooms` (`sseId`, `room`, `action`: `join`/`leave`) abonniert. Der Client versucht zuerst WebSocket und wechselt in folgenden Fällen für die Sitzung des Tabs auf SSE:
**Authentifizierung:** Token als `?token=<JWT|API-Key>` oder `Authorization`-Header. Ohne gültiges Token wird die Verbindung als `anonymous` akzeptiert ([FLAW-01](../flaws.md), [FLAW-02](../flaws.md)). Das Token wird nur beim Verbinden geprüft. - Aufbau gelingt nicht innerhalb von 2,5 s, oder die Verbindung bricht unsauber ab (z. B. Code 1006/1008).
- **Herzschlag:** Alle 15 s schickt der Client `{"event":"PING"}`, der Server antwortet `PONG`. Kommt 35 s lang keine Nachricht, gilt der WebSocket als stumm (Proxy nimmt das Upgrade an, leitet aber nichts weiter).
Über SSE liefert der Server zuerst `SSE_CONNECTED` mit einer `sseId`; erst dann gilt SSE als verbunden. Bleibt dieses erste Ereignis 5 s nach dem Öffnen aus, puffert ein Proxy den Stream: SSE wird für den Tab abgeschaltet. Räume werden über `POST /api/v1/events/rooms` (`sseId`, `room`, `action`: `join`/`leave`) abonniert.
Ohne WebSocket und SSE nutzt die Notiz-Synchronisation Long-Polling (`GET /docs/{id}/changes`, bis 25 s), und wenn auch das dreimal scheitert, eine Abfrage alle 3 s (siehe [modules/workspace.md](../modules/workspace.md)).
**Authentifizierung:** Token als `?token=<JWT|API-Key>` oder `Authorization`-Header. Ohne gültiges Token antworten `/ws` und `/events` mit `401`. Das Token wird nur beim Verbinden geprüft ([FLAW-22](../flaws.md)).
**Räume:** Jeder Beitritt (Raum in der URL, `JOIN_ROOM`, SSE-Startparameter `rooms`/`docId`, `POST /events/rooms`) wird geprüft: `doc:<id>` nur mit Lesezugriff auf die Notiz, sonst muss die ID eine lesbare geteilte Liste sein. Nicht erlaubte Räume werden ignoriert, `POST /events/rooms` antwortet dann mit `403`. SSE-Räume kann nur der Besitzer der jeweiligen `sseId` ändern.
## Nachrichtenformat ## Nachrichtenformat
@@ -21,7 +30,7 @@ Der Client versucht zuerst WebSocket. Gelingt der Aufbau nicht innerhalb von 2,5
{ "event": "TASK_UPDATED", "listId": "", "payload": { … }, "timestamp": "2026-10-27T12:00:00Z" } { "event": "TASK_UPDATED", "listId": "", "payload": { … }, "timestamp": "2026-10-27T12:00:00Z" }
``` ```
Client → Server versteht der Server nur: `{"event":"JOIN_ROOM","room":"<id>"}` und `{"event":"LEAVE_ROOM","room":"<id>"}`. Räume heißen `doc:<docId>` für Notizen oder `<listId>` für geteilte Listen. Client → Server versteht der Server nur: `{"event":"JOIN_ROOM","room":"<id>"}`, `{"event":"LEAVE_ROOM","room":"<id>"}` und `{"event":"PING"}` (Antwort `PONG`). Räume heißen `doc:<docId>` für Notizen oder `<listId>` für geteilte Listen.
## Zustellung ## Zustellung
@@ -29,20 +38,20 @@ Client → Server versteht der Server nur: `{"event":"JOIN_ROOM","room":"<id>"}`
|---|---| |---|---|
| Benutzer-Kanal (`BroadcastToUser`) | alle Verbindungen eines Benutzers | | Benutzer-Kanal (`BroadcastToUser`) | alle Verbindungen eines Benutzers |
| Raum (`BroadcastRoom`, `Broadcast`) | Verbindungen im Raum | | Raum (`BroadcastRoom`, `Broadcast`) | Verbindungen im Raum |
| Alle (`BroadcastToAll`) | **jede** Verbindung, auch unangemeldete | | Mehrere Benutzer (`BroadcastToUsers`) | alle Verbindungen einer berechneten Benutzergruppe, z. B. Eigentümer und Mitglieder einer Aufgabenliste |
## Ereignisse ## Ereignisse
| Ereignis | Auslöser | Kanal | | Ereignis | Auslöser | Kanal |
|---|---|---| |---|---|---|
| `TASK_CREATED` | `POST /inbox` | Benutzer | | `TASK_CREATED` | `POST /inbox`, Webhook | Eigentümer und Benutzer der Liste |
| `TASK_UPDATED` | `PATCH /thoughts/{id}` (Triage: Benutzer) | alle | | `TASK_UPDATED` | `PATCH /thoughts/{id}`, Triage | Eigentümer und Benutzer der alten und neuen Liste |
| `TASK_DELETED` | `DELETE /thoughts/{id}` | alle | | `TASK_DELETED` | `DELETE /thoughts/{id}` | Eigentümer und Benutzer der Liste |
| `TASK_REORDERED` | Aufgaben umsortieren | alle | | `TASK_REORDERED` | Aufgaben umsortieren | Benutzer aller betroffenen Listen |
| `TASK_LIST_UPDATED` / `TASK_LIST_DELETED` | Liste ändern, (un)teilen, löschen | alle | | `TASK_LIST_UPDATED` / `TASK_LIST_DELETED` | Liste ändern, (un)teilen, löschen; Payload nur `{id}`, Clients laden ihre Listen neu | Benutzer vor und nach der Änderung |
| `DOC_CREATED`, `DOC_UPDATED`, `DOC_DELETED` | Notiz anlegen/ändern/löschen, Freigaben | berechtigte Benutzer | | `DOC_CREATED`, `DOC_UPDATED`, `DOC_DELETED` | Notiz anlegen/ändern/löschen, Freigaben | berechtigte Benutzer |
| `DOC_BLOCKS_PATCHED` | `PATCH /documents/{id}/blocks` (`docId`, `updatedBy`, `upserted`, `deletedIds`, `blockOrder`) | Raum `doc:<id>` und berechtigte Benutzer | | `DOC_SEQ` | Änderung an Notiz, Leases oder Präsenz (`docId`, `headSeq`); Clients holen die Daten per `/docs/{id}/sync` | Raum `doc:<id>` |
| `WORKSPACE_FOLDERS_UPDATED` | Ordner/Mounts ändern | Benutzer | | `WORKSPACE_FOLDERS_UPDATED` | Ordner/Mounts ändern | Benutzer selbst; bei geteilten Ordnern Eigentümer und alle Beteiligten |
| `LIST_CREATED`, `LIST_UPDATED`, `LIST_DELETED`, `LIST_SHARED`, `LIST_UNSHARED`, `LIST_CLEARED` | Listenverwaltung | Benutzer bzw. Raum der Liste | | `LIST_CREATED`, `LIST_UPDATED`, `LIST_DELETED`, `LIST_SHARED`, `LIST_UNSHARED`, `LIST_CLEARED` | Listenverwaltung | Benutzer bzw. Raum der Liste |
| `ITEM_CREATED`, `ITEM_UPDATED`, `ITEM_TOGGLED`, `ITEM_DELETED`, `ITEM_REORDERED` | Listeneinträge | Raum der Liste | | `ITEM_CREATED`, `ITEM_UPDATED`, `ITEM_TOGGLED`, `ITEM_DELETED`, `ITEM_REORDERED` | Listeneinträge | Raum der Liste |
| `USER_PRESENCE` | Verbindung auf-/abgebaut (`onlineCount`) | Raum | | `USER_PRESENCE` | Verbindung auf-/abgebaut (`onlineCount`) | Raum |
@@ -52,4 +61,4 @@ Client → Server versteht der Server nur: `{"event":"JOIN_ROOM","room":"<id>"}`
- Die globale Verbindung (ein Singleton) leitet jede Nachricht als Browser-Ereignis `mindsift:ws` weiter; Ansichten (`TasksView`, `WorkspaceView`, `SharedListsView`) hören darauf und laden bzw. patchen ihren Zustand. - Die globale Verbindung (ein Singleton) leitet jede Nachricht als Browser-Ereignis `mindsift:ws` weiter; Ansichten (`TasksView`, `WorkspaceView`, `SharedListsView`) hören darauf und laden bzw. patchen ihren Zustand.
- Nach einem Abbruch verbindet sich der Client mit Verzögerung neu. - Nach einem Abbruch verbindet sich der Client mit Verzögerung neu.
- Weil Notizen nur vom Eigentümer geschrieben werden, dient `DOC_BLOCKS_PATCHED` vor allem dazu, Lesenden und weiteren Geräten des Eigentümers Änderungen zu zeigen. Es gibt keine Konfliktauflösung (letzter Schreiber gewinnt). - Notiz-Inhalte kommen nie per Push, sondern immer über `POST /docs/{id}/sync`; `DOC_SEQ` ist nur der Weckruf. Konflikte entscheidet der Server (Versionen und Sperren, siehe [modules/collaboration.md](../modules/collaboration.md)).
+12 -3
View File
@@ -34,10 +34,13 @@ Das Frontend liegt per `go:embed` im Binary ([cmd/server/main.go](../cmd/server/
| `internal/storage/` | `sqlite.go` (Schema, Migrationen, gesamte Datenlogik) | | `internal/storage/` | `sqlite.go` (Schema, Migrationen, gesamte Datenlogik) |
| `internal/domain/` | Strukturen (`models.go`), ID-Erzeugung, Tag-Extraktion | | `internal/domain/` | Strukturen (`models.go`), ID-Erzeugung, Tag-Extraktion |
| `internal/ws/` | `hub.go` | | `internal/ws/` | `hub.go` |
| `internal/collab/` | Kern der Notiz-Synchronisation ohne Datenbank: Blockbaum, Operationen und Konfliktregeln, Sortierschlüssel, Leases, Long-Poll-Benachrichtigung |
| `internal/auth/` | Tokens und Passwörter | | `internal/auth/` | Tokens und Passwörter |
| `internal/config/` | Umgebungsvariablen | | `internal/config/` | Umgebungsvariablen |
| `frontend/src/features/` | `triage`, `workspace`, `lists`, `teams`, `settings`, `capture`, `auth`, `help` | | `frontend/src/features/` | `triage`, `workspace`, `lists`, `teams`, `settings`, `capture`, `auth`, `help` |
| `frontend/src/lib/` | `api.ts` (REST-Client, Transport-Umschaltung WS/SSE), `types.ts`, `markdown.tsx`, `theme.ts` | | `frontend/src/lib/` | `api.ts` (REST-Client, Transport-Umschaltung WS/SSE), `types.ts`, `markdown.tsx`, `theme.ts` |
| `frontend/src/lib/docSync/` | Notiz-Synchronisation im Browser: Portierung von `internal/collab` (`model.ts`, `orderKey.ts`), Vergleich Editor-Liste → Operationen (`diff.ts`), `engine.ts`, IndexedDB (`persist.ts`), HTTP (`transport.ts`) |
| `e2e/` | Browser-Tests mit Playwright gegen einen echten Server (`e2e/run.sh`) |
| `frontend/src/themes/` | CSS pro Theme | | `frontend/src/themes/` | CSS pro Theme |
## Start und Laufzeit ## Start und Laufzeit
@@ -46,7 +49,7 @@ Das Frontend liegt per `go:embed` im Binary ([cmd/server/main.go](../cmd/server/
2. `storage.NewStore()` öffnet SQLite (`busy_timeout`, WAL, max. 8 Verbindungen) und führt `migrate()` aus. 2. `storage.NewStore()` öffnet SQLite (`busy_timeout`, WAL, max. 8 Verbindungen) und führt `migrate()` aus.
3. SMTP-Werte aus der Umgebung werden in `system_settings` übernommen, sofern dort noch nichts steht. 3. SMTP-Werte aus der Umgebung werden in `system_settings` übernommen, sofern dort noch nichts steht.
4. Mit Demo-Flag legt `EnsureDemoAccounts` die Konten `admin/admin` und `user/user` an. 4. Mit Demo-Flag legt `EnsureDemoAccounts` die Konten `admin/admin` und `user/user` an.
5. `api.NewServer` startet den **Alarm-Ticker** (jede Minute) und baut den Router. Server-Timeouts: Lesen 15 s, Schreiben 30 s, Leerlauf 60 s. Beim Beenden wird sauber heruntergefahren. 5. `api.NewServer` startet den **Alarm-Ticker** (jede Minute), die **Verdichtung des Notiz-Logs** (beim Start, dann alle 6 Stunden) und baut den Router. Server-Timeouts: Lesen 15 s, Schreiben 30 s, Leerlauf 60 s. Beim Beenden wird sauber heruntergefahren.
## Konfiguration ## Konfiguration
@@ -76,12 +79,18 @@ Hinweis: `DATABASE_URL` mit `postgres…` führt dazu, dass der Standardpfad in
`startAlarmTicker` prüft jede Minute alle offenen, noch nicht gemeldeten Aufgaben mit Fälligkeit. Ist `due_date <= heute` (Serverzeit), sendet er über die Kanäle des Benutzers (`email`, `discord`, `webhook`) und setzt `alarm_sent`, sobald mindestens ein Kanal erfolgreich war. Details und Einschränkungen: [modules/tasks.md](modules/tasks.md), [FLAW-12](flaws.md). `startAlarmTicker` prüft jede Minute alle offenen, noch nicht gemeldeten Aufgaben mit Fälligkeit. Ist `due_date <= heute` (Serverzeit), sendet er über die Kanäle des Benutzers (`email`, `discord`, `webhook`) und setzt `alarm_sent`, sobald mindestens ein Kanal erfolgreich war. Details und Einschränkungen: [modules/tasks.md](modules/tasks.md), [FLAW-12](flaws.md).
## Hintergrundjob: Verdichtung der Notiz-Synchronisation
`startCompaction` räumt beim Start und dann alle 6 Stunden auf: abgelaufene Leases und Sitzungen im Speicher, Log-Einträge älter als 30 Tage (private Notizen: 7 Tage), Grabsteine älter als 30 Tage und Idempotenz-Ergebnisse älter als 90 Tage. Details: [database.md](database.md#verdichtung-des-notiz-logs).
## Build ## Build
``` ```
Stage 1 node:22-alpine npm ci && npm run build → frontend/dist Stage 1 node:22-alpine tsc, vitest, vite build → frontend/dist
Stage 2 golang:1.23 go vet, go test, go build → /app/mindsift (dist wird eingebettet) Stage 2 golang:1.23 go vet, go test, go build → /app/mindsift (dist wird eingebettet)
Stage 3 alpine:3.21 nur Binary, ca-certificates, tzdata Stage 3 alpine:3.21 nur Binary, ca-certificates, tzdata
``` ```
Die Tests laufen beim Image-Build (`go test -v ./...`); ein Fehler bricht den Build ab. Frontend-Tests existieren nicht ([FLAW-19](flaws.md)). Die angezeigte Version (Kopf der App) setzt der Build-Parameter `APP_VERSION`: Die CI übergibt bei Git-Tags den Tag-Namen (`v0.2`), sonst zeigt die App die Version aus `frontend/package.json` mit `-dev` (`v0.2-dev`). Umgesetzt in `frontend/vite.config.ts` und `frontend/src/version.ts`.
Die Tests laufen beim Image-Build (Frontend: `npm run test:run`, Backend: `go test -v ./...`); ein Fehler bricht den Build ab. Frontend-Tests liegen unter `frontend/src/**/__tests__/` (Vitest + jsdom + Testing Library, Setup in `frontend/src/test/setup.ts`).
+19 -4
View File
@@ -27,12 +27,14 @@ Eine Aufgabe ist ein `thought` mit `status='kept'` oder `status='converted' AND
### Workspace ### Workspace
| Tabelle | Zweck | | Tabelle | Zweck |
|---|---| |---|---|
| `documents` | Notizen: `title`, `icon`, `folder_path`, `tags` (JSON), `is_archived`, `markdown_cache`, `shared_folder_id`, `sub_folder_path`, Altspalten `content`, `blocks_json` | | `documents` | Notizen: `title`, `icon`, `folder_path`, `tags` (JSON), `is_archived`, `markdown_cache`, `shared_folder_id`, `sub_folder_path`, `head_seq` (letzte Log-Position), `title_version`, `log_floor` (bis hierhin ist das Log verdichtet), Altspalten `content`, `blocks_json` |
| `blocks` | Outliner-Blöcke: `document_id`, `type`, `content`, `level`, `position` (REAL), `checked`, `collapsed`, `lang` | | `blocks` | Outliner-Blöcke als Baum: `document_id`, `parent_id` (`''` = oberste Ebene), `order_key` (Sortierschlüssel unter Geschwistern), `type`, `content`, `checked`, `collapsed`, `lang`, `version` (Textversion), `conflict_of`/`conflict_author` (Konfliktkopien), `updated_by`, `deleted` (Grabstein) und `deleted_at` (Zeitpunkt des Löschens, für das Aufräumen). `level`/`position` sind Altspalten und werden nicht mehr gepflegt; Ebene und Reihenfolge berechnet der Server beim Lesen aus dem Baum. |
| `doc_ops` | Änderungs-Log je Notiz: `seq` (fortlaufend pro Notiz), `op_id`, `user_id`, `client_id`, `kind`, `block_id`, `payload` (Effekt als JSON), `created_at`. Abspielen aller Einträge ab einem Snapshot ergibt den Stand. Wird verdichtet (siehe unten). |
| `doc_op_results` | Ergebnis je Client-Operation (`op_id` eindeutig) für idempotente Wiederholungen; wird nach 90 Tagen gelöscht |
| `block_tags` | `#Hashtags` je Block (`tag`, `document_id`, `block_id`) | | `block_tags` | `#Hashtags` je Block (`tag`, `document_id`, `block_id`) |
| `tag_pages` | Freitext-Notiz je Benutzer und Tag | | `tag_pages` | Freitext-Notiz je Benutzer und Tag |
| `doc_pins` | Favoriten je Benutzer | | `doc_pins` | Favoriten je Benutzer |
| `doc_shares`, `doc_team_shares` | Notiz-Freigaben an Benutzer bzw. Gruppen (`permission` effektiv immer `read`) | | `doc_shares`, `doc_team_shares` | Notiz-Freigaben an Benutzer bzw. Gruppen (`permission`: `read` = Leser, `write` = Bearbeiter) |
| `workspace_folders` | Explizit angelegte Ordnerpfade | | `workspace_folders` | Explizit angelegte Ordnerpfade |
| `workspace_shared_folders` | Geteilte Ordner (Entität mit Eigentümer) | | `workspace_shared_folders` | Geteilte Ordner (Entität mit Eigentümer) |
| `workspace_folder_shares` | Freigaben geteilter Ordner | | `workspace_folder_shares` | Freigaben geteilter Ordner |
@@ -74,10 +76,23 @@ teams 1─n team_members n─1 users
- Spalten/Tabellen/Indizes werden nachgezogen (`ALTER TABLE … ADD COLUMN`, Fehler werden ignoriert). - Spalten/Tabellen/Indizes werden nachgezogen (`ALTER TABLE … ADD COLUMN`, Fehler werden ignoriert).
- Veraltete Tabellen werden gelöscht: `spaces`, `space_members`, `space_team_shares`, `document_crdt_state`, `document_crdt_updates`, `document_blocks`, `doc_block_tags`. - Veraltete Tabellen werden gelöscht: `spaces`, `space_members`, `space_team_shares`, `document_crdt_state`, `document_crdt_updates`, `document_blocks`, `doc_block_tags`.
- `users.username` wird aus dem E-Mail-Präfix aufgefüllt. - `users.username` wird aus dem E-Mail-Präfix aufgefüllt.
- Alle Dokument-, Team- und Ordnerfreigaben werden auf `read` gesetzt (bei **jedem** Start, siehe [FLAW-17](flaws.md)). - Notiz-Synchronisation (`migrateDocSync`): neue Spalten an `blocks` (inkl. `deleted_at`; vorhandene Grabsteine erhalten den Migrationszeitpunkt) und `documents` (`head_seq`, `title_version`, `log_floor`), Tabellen `doc_ops` und `doc_op_results`. Blöcke im alten flachen Format (nur `level`/`position`) werden einmalig in den Baum umgewandelt.
- Standardwerte in `system_settings` (`INSERT OR IGNORE`): `sharing_mode=teams_only`, `allow_registration=false`, `instance_name=MindSift`, SMTP leer. - Standardwerte in `system_settings` (`INSERT OR IGNORE`): `sharing_mode=teams_only`, `allow_registration=false`, `instance_name=MindSift`, SMTP leer.
- Die Tabelle `teams` wird einmalig ohne Fremdschlüssel auf `organizations` neu aufgebaut. - Die Tabelle `teams` wird einmalig ohne Fremdschlüssel auf `organizations` neu aufgebaut.
## Sicherung ## Sicherung
Die gesamte Instanz besteht aus der Datei `mindsift.db` (plus `-wal`/`-shm` im laufenden Betrieb). Konsistent sichern per `sqlite3 mindsift.db ".backup 'backup.db'"` oder durch Stoppen des Containers und Kopieren des `data/`-Verzeichnisses. Benutzerexport: `GET /api/v1/export` (JSON mit Gedanken, Notizen, Listen, Aufgabenlisten). Die gesamte Instanz besteht aus der Datei `mindsift.db` (plus `-wal`/`-shm` im laufenden Betrieb). Konsistent sichern per `sqlite3 mindsift.db ".backup 'backup.db'"` oder durch Stoppen des Containers und Kopieren des `data/`-Verzeichnisses. Benutzerexport: `GET /api/v1/export` (JSON mit Gedanken, Notizen, Listen, Aufgabenlisten).
## Verdichtung des Notiz-Logs
`Store.CompactDocs` läuft beim Start und danach alle 6 Stunden (`startCompaction` in `internal/api/docsync.go`):
| Was | Aufbewahrung | Folge |
|---|---|---|
| `doc_ops` geteilter Notizen | 30 Tage | ältere Einträge werden gelöscht, `documents.log_floor` merkt sich die Grenze |
| `doc_ops` privater Notizen | 7 Tage | wie oben |
| Grabsteine in `blocks` | 30 Tage nach `deleted_at` | endgültig gelöscht; das ist selbst ein Log-Eintrag `purge`, geöffnete Editoren übernehmen ihn ohne Neuladen. Ein Grabstein bleibt, solange ein jüngerer gelöschter Unterblock auf ihn zeigt. |
| `doc_op_results` | 90 Tage | gelöscht |
Ein Client, dessen `lastSeq` unter `log_floor` liegt (z. B. Gerät wochenlang offline), bekommt von `/sync` `resync: true` und lädt den Snapshot neu; seine noch nicht gesendeten Änderungen bleiben in der Warteschlange und werden danach angewendet. Text für einen inzwischen endgültig gelöschten Block wird als neuer Block am Ende der Notiz angelegt, ein neuer Block unter einem endgültig gelöschten Elternblock landet auf der obersten Ebene.
+38 -12
View File
@@ -10,9 +10,9 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
| ID | Bereich | Kurzbeschreibung | Schwere | | ID | Bereich | Kurzbeschreibung | Schwere |
|---|---|---|---| |---|---|---|---|
| FLAW-01 | Realtime | `BroadcastToAll` sendet Aufgaben- und Listendaten an alle Verbindungen, auch unauthentifizierte | **Hoch** | | FLAW-01 | Realtime | ~~`BroadcastToAll` sendet Aufgaben- und Listendaten an alle Verbindungen~~ | **Behoben** |
| FLAW-02 | Realtime | Räume (WS `JOIN_ROOM`, SSE `/events/rooms`, `?listId=`) ohne Berechtigungsprüfung | **Hoch** | | FLAW-02 | Realtime | ~~Räume ohne Berechtigungsprüfung~~ | **Behoben** |
| FLAW-03 | Aufgaben | `ReorderTasks` ändert beliebige Aufgaben per ID, ohne Zugriffsprüfung (IDOR) | **Hoch** | | FLAW-03 | Aufgaben | ~~`ReorderTasks` ändert beliebige Aufgaben per ID (IDOR)~~ | **Behoben** |
| FLAW-04 | Auth | Fest eincodiertes Fallback-`JWT_SECRET` | **Hoch** | | FLAW-04 | Auth | Fest eincodiertes Fallback-`JWT_SECRET` | **Hoch** |
| FLAW-05 | Benachrichtigungen | Discord-/Webhook-URLs ungeprüft (SSRF) | Mittel | | FLAW-05 | Benachrichtigungen | Discord-/Webhook-URLs ungeprüft (SSRF) | Mittel |
| FLAW-06 | Aufgaben | Jeder Editor einer Aufgabenliste kann sie per `teamId` an weitere Gruppen freigeben | Mittel | | FLAW-06 | Aufgaben | Jeder Editor einer Aufgabenliste kann sie per `teamId` an weitere Gruppen freigeben | Mittel |
@@ -20,36 +20,41 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
| FLAW-08 | Auth | Webhook-Ingest mit JWT prüft weder Sperrliste noch `token_version` noch `is_active` | Mittel | | FLAW-08 | Auth | Webhook-Ingest mit JWT prüft weder Sperrliste noch `token_version` noch `is_active` | Mittel |
| FLAW-09 | Auth | Kein Rate-Limit, keine Sperre bei Fehlversuchen, Mindestpasswort 4 Zeichen | Mittel | | FLAW-09 | Auth | Kein Rate-Limit, keine Sperre bei Fehlversuchen, Mindestpasswort 4 Zeichen | Mittel |
| FLAW-10 | Freigaben | `sharing_mode` wird bei Freigaben nicht durchgesetzt; Benutzerverzeichnis leakt alle Konten | Mittel | | FLAW-10 | Freigaben | `sharing_mode` wird bei Freigaben nicht durchgesetzt; Benutzerverzeichnis leakt alle Konten | Mittel |
| FLAW-11 | Workspace | Block-Upsert (`ON CONFLICT(id)`) prüft das Dokument nicht | Mittel | | FLAW-11 | Workspace | ~~Block-Upsert (`ON CONFLICT(id)`) prüft das Dokument nicht~~ | **Behoben** |
| FLAW-12 | Benachrichtigungen | `notify_before_minutes` wirkungslos; Alarm feuert nach Umplanung nie wieder | Mittel | | FLAW-12 | Benachrichtigungen | `notify_before_minutes` wirkungslos; Alarm feuert nach Umplanung nie wieder | Mittel |
| FLAW-13 | Datenbank | Keine Schema-Versionierung; Migrationsfehler werden verschluckt; `migrations/001_init.sql` ist toter Code | Mittel | | FLAW-13 | Datenbank | Keine Schema-Versionierung; Migrationsfehler werden verschluckt; `migrations/001_init.sql` ist toter Code | Mittel |
| FLAW-14 | Backend | Fehler von Schreiboperationen werden häufig ignoriert (`_, _ =`) | Mittel | | FLAW-14 | Backend | Fehler von Schreiboperationen werden häufig ignoriert (`_, _ =`) | Mittel |
| FLAW-15 | Realtime | `WriteTimeout` von 30 s trifft vermutlich auch SSE-Streams *(nicht live verifiziert)* | Mittel | | FLAW-15 | Realtime | ~~`WriteTimeout` von 30 s beendet SSE-Streams~~ | **Behoben** |
| FLAW-16 | Setup | `/auth/setup` ist bis zum ersten Konto offen; Demo-Modus legt `admin/admin` an | Mittel | | FLAW-16 | Setup | `/auth/setup` ist bis zum ersten Konto offen; Demo-Modus legt `admin/admin` an | Mittel |
| FLAW-17 | Workspace | Dokument- und Ordnerfreigaben sind per Migration erzwungen schreibgeschützt; tote Schreibpfade | Niedrig | | FLAW-17 | Workspace | ~~Bearbeiter-Freigaben für Notizen nicht wählbar~~ | **Behoben** |
| FLAW-18 | Suche | Nur eigene Gedanken und Dokumente, `LIKE` ohne Escaping, keine FTS | Niedrig | | FLAW-18 | Suche | Nur eigene Gedanken und Dokumente, `LIKE` ohne Escaping, keine FTS | Niedrig |
| FLAW-19 | Qualität | Keine Frontend-Tests, `lint` und `test:run` sind Platzhalter | Niedrig | | FLAW-19 | Qualität | Frontend-Tests nur für den Outliner, `lint` ist ein Platzhalter | Niedrig |
| FLAW-20 | Qualität | Sehr große Dateien, duplizierte Zugriffslogik | Niedrig | | FLAW-20 | Qualität | Sehr große Dateien, duplizierte Zugriffslogik | Niedrig |
| FLAW-21 | Repo | `LICENSE` fehlt, obwohl README MIT angibt | Niedrig | | FLAW-21 | Repo | `LICENSE` fehlt, obwohl README MIT angibt | Niedrig |
| FLAW-22 | Netzwerk | Token in der URL (WS/SSE), CORS `*`, `CheckOrigin` immer `true` | Niedrig | | FLAW-22 | Netzwerk | Token in der URL (WS/SSE), CORS `*`, `CheckOrigin` immer `true` | Niedrig |
| FLAW-23 | SQLite | Single-Writer-Engpass bei vielen gleichzeitigen Schreibern | Niedrig | | FLAW-23 | SQLite | Single-Writer-Engpass bei vielen gleichzeitigen Schreibern | Niedrig |
| FLAW-24 | Gruppen | Organisations-Endpunkte ohne Berechtigungsprüfung | Mittel | | FLAW-24 | Gruppen | Organisations-Endpunkte ohne Berechtigungsprüfung | Mittel |
| FLAW-25 | Workspace | ~~Struktur-Aktionen an fremd gesperrten Bullets nur teilweise ausgeführt~~ | **Behoben** |
| FLAW-26 | Workspace | ~~Nach Enter landet sehr schnell Getipptes noch im vorherigen Bullet~~ | **Behoben** |
--- ---
## Details ## Details
### FLAW-01: Broadcast an alle Verbindungen (Datenabfluss) ### FLAW-01: Broadcast an alle Verbindungen (Datenabfluss)
- **Status: Behoben.** `BroadcastToAll` existiert nicht mehr. Aufgabenereignisse gehen an Eigentümer und Benutzer der Liste (`GetTaskListUserIDs`), Listenereignisse tragen nur noch die ID, Ordnerereignisse gehen an Benutzer bzw. Ordnerbeteiligte (`GetSharedFolderUserIDs`). Test: `TestTaskEventsOnlyReachAuthorizedUsers`.
- **Fundstellen:** [`internal/ws/hub.go`](../internal/ws/hub.go) `BroadcastToAll`; Aufrufer in [`internal/api/server.go`](../internal/api/server.go): `handleUpdateThought` (`TASK_UPDATED`), `handleDeleteThought` (`TASK_DELETED`), `handleReorderTasks` (`TASK_REORDERED`), `handleUpdateTaskList`, `handleCreateTaskListShare`, `handleDeleteTaskListShare` (`TASK_LIST_UPDATED`), `handleDeleteTaskList`. - **Fundstellen:** [`internal/ws/hub.go`](../internal/ws/hub.go) `BroadcastToAll`; Aufrufer in [`internal/api/server.go`](../internal/api/server.go): `handleUpdateThought` (`TASK_UPDATED`), `handleDeleteThought` (`TASK_DELETED`), `handleReorderTasks` (`TASK_REORDERED`), `handleUpdateTaskList`, `handleCreateTaskListShare`, `handleDeleteTaskListShare` (`TASK_LIST_UPDATED`), `handleDeleteTaskList`.
- **Problem:** `BroadcastToAll` schreibt an `allClients` und `sseClients`. Dazu zählen auch Verbindungen mit `userID = "anonymous"`, denn `handleWebSocket` und `handleSSE` liegen außerhalb der Auth-Gruppe und lassen jede Verbindung zu. Der vollständige Aufgabeninhalt (Titel, Notizen, Unteraufgaben) und die Listen samt Mitgliedern gehen so an jeden, der `/api/v1/events` öffnet. - **Problem:** `BroadcastToAll` schreibt an `allClients` und `sseClients`. Dazu zählen auch Verbindungen mit `userID = "anonymous"`, denn `handleWebSocket` und `handleSSE` liegen außerhalb der Auth-Gruppe und lassen jede Verbindung zu. Der vollständige Aufgabeninhalt (Titel, Notizen, Unteraufgaben) und die Listen samt Mitgliedern gehen so an jeden, der `/api/v1/events` öffnet.
- **Maßnahme:** Ereignisse an die berechtigten Nutzer adressieren (wie bei `DOC_*` über `GetAuthorizedDocUsers`), unauthentifizierte Verbindungen ablehnen. - **Maßnahme:** Ereignisse an die berechtigten Nutzer adressieren (wie bei `DOC_*` über `GetAuthorizedDocUsers`), unauthentifizierte Verbindungen ablehnen.
### FLAW-02: Räume ohne Berechtigungsprüfung ### FLAW-02: Räume ohne Berechtigungsprüfung
- **Status: Behoben.** `/ws` und `/events` verlangen ein gültiges Token (sonst `401`). Jeder Raumbeitritt läuft über `Hub.SetRoomAuthorizer` → `Server.canJoinRoom` (Notiz bzw. Liste muss lesbar sein). `/events/rooms` liegt hinter der Anmeldung und akzeptiert nur die eigene `sseId`. Tests: `TestRealtimeEndpointsRequireAuth`, `TestSSERoomJoinRequiresAccess`, `TestRoomJoinsRequireAuthorization`.
- **Fundstellen:** `Hub.readPump` (`JOIN_ROOM`), `handleSSERooms` (Route `/api/v1/events/rooms` liegt **außerhalb** der Auth-Gruppe), `Hub.Register` (Raum aus `?listId=`/`?spaceId=`). - **Fundstellen:** `Hub.readPump` (`JOIN_ROOM`), `handleSSERooms` (Route `/api/v1/events/rooms` liegt **außerhalb** der Auth-Gruppe), `Hub.Register` (Raum aus `?listId=`/`?spaceId=`).
- **Problem:** Wer eine Raum-ID kennt (`doc:<id>`, Listen-ID), tritt bei und erhält Inhalte wie `DOC_BLOCKS_PATCHED` oder `ITEM_*`. Raum-IDs tauchen in Ereignissen auf und sind nicht geheim. Für den SSE-Beitritt reicht die `sseId`, die der Server selbst im Stream mitsendet. - **Problem:** Wer eine Raum-ID kennt (`doc:<id>`, Listen-ID), tritt bei und erhält Inhalte wie `DOC_BLOCKS_PATCHED` oder `ITEM_*`. Raum-IDs tauchen in Ereignissen auf und sind nicht geheim. Für den SSE-Beitritt reicht die `sseId`, die der Server selbst im Stream mitsendet.
- **Maßnahme:** Beitritt nur nach Zugriffsprüfung (`GetDocumentByID`, `GetListByID`); `/events/rooms` hinter `authMiddleware`. - **Maßnahme:** Beitritt nur nach Zugriffsprüfung (`GetDocumentByID`, `GetListByID`); `/events/rooms` hinter `authMiddleware`.
### FLAW-03: `ReorderTasks` ohne Zugriffsprüfung (IDOR) ### FLAW-03: `ReorderTasks` ohne Zugriffsprüfung (IDOR)
- **Status: Behoben.** `handleReorderTasks` lehnt die ganze Anfrage ab, sobald eine Aufgabe nicht zugreifbar (`404`) oder schreibgeschützt (`403`) ist. Test: `TestReorderRejectsForeignTasks`.
- **Fundstelle:** `Store.ReorderTasks` (`UPDATE thoughts SET sort_order … WHERE id = ?`) und `handleReorderTasks`. - **Fundstelle:** `Store.ReorderTasks` (`UPDATE thoughts SET sort_order … WHERE id = ?`) und `handleReorderTasks`.
- **Problem:** Die SQL-Anweisung ist weder auf `user_id` noch auf die Liste eingeschränkt. Jeder angemeldete Nutzer kann `sortOrder`, `kanbanStatus` und damit `isCompleted` fremder Aufgaben ändern, sofern er die ID kennt. Die Leser-Prüfung für Aufgabenlisten überspringt Aufgaben, auf die der Nutzer gar keinen Zugriff hat (`GetThoughtByID` liefert dann einen Fehler), und schützt davor nicht. - **Problem:** Die SQL-Anweisung ist weder auf `user_id` noch auf die Liste eingeschränkt. Jeder angemeldete Nutzer kann `sortOrder`, `kanbanStatus` und damit `isCompleted` fremder Aufgaben ändern, sofern er die ID kennt. Die Leser-Prüfung für Aufgabenlisten überspringt Aufgaben, auf die der Nutzer gar keinen Zugriff hat (`GetThoughtByID` liefert dann einen Fehler), und schützt davor nicht.
- **Maßnahme:** Jede ID gegen `accessibleTaskListIDsQuery` bzw. `user_id` prüfen und bei Verstoß ablehnen. - **Maßnahme:** Jede ID gegen `accessibleTaskListIDsQuery` bzw. `user_id` prüfen und bei Verstoß ablehnen.
@@ -93,6 +98,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
- **Maßnahme:** Eine zentrale Funktion `canShareWith(owner, target)` und in allen Share-Handlern verwenden; Verzeichnisse darauf einschränken. - **Maßnahme:** Eine zentrale Funktion `canShareWith(owner, target)` und in allen Share-Handlern verwenden; Verzeichnisse darauf einschränken.
### FLAW-11: Block-Upsert prüft das Dokument nicht ### FLAW-11: Block-Upsert prüft das Dokument nicht
- **Status: Behoben.** Alle Blockänderungen laufen über `Store.applyDocChange`: Einfügen mit einer Block-ID aus einer anderen Notiz wird abgelehnt (`exists`), und der Upsert ändert nur Zeilen derselben Notiz (`WHERE blocks.document_id = excluded.document_id`). Test: `TestInsertCannotReuseBlockIDOfAnotherNote`.
- **Fundstelle:** `Store.PatchDocumentBlocks`, `INSERT INTO blocks … ON CONFLICT(id) DO UPDATE`. - **Fundstelle:** `Store.PatchDocumentBlocks`, `INSERT INTO blocks … ON CONFLICT(id) DO UPDATE`.
- **Problem:** `blocks.id` ist global eindeutig. Existiert die ID bereits in einem **anderen** Dokument, überschreibt der Konflikt-Zweig dessen Inhalt, ohne `document_id` zu prüfen. Voraussetzung: Schreibrecht im eigenen Dokument und Kenntnis einer fremden Block-ID. - **Problem:** `blocks.id` ist global eindeutig. Existiert die ID bereits in einem **anderen** Dokument, überschreibt der Konflikt-Zweig dessen Inhalt, ohne `document_id` zu prüfen. Voraussetzung: Schreibrecht im eigenen Dokument und Kenntnis einer fremden Block-ID.
- **Maßnahme:** `DO UPDATE … WHERE blocks.document_id = excluded.document_id`; fremde IDs ablehnen. - **Maßnahme:** `DO UPDATE … WHERE blocks.document_id = excluded.document_id`; fremde IDs ablehnen.
@@ -117,9 +123,11 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
### FLAW-14: Verschluckte Fehler ### FLAW-14: Verschluckte Fehler
- **Fundstellen:** u. a. `PatchDocumentBlocks` (Upserts/Deletes), `AddTaskListShare`, `RemoveTaskListShare`, viele `_, _ = s.db.Exec`. - **Fundstellen:** u. a. `PatchDocumentBlocks` (Upserts/Deletes), `AddTaskListShare`, `RemoveTaskListShare`, viele `_, _ = s.db.Exec`.
- **Problem:** Scheitert ein Schreibzugriff (z. B. `SQLITE_BUSY`), antwortet die API trotzdem mit Erfolg und broadcastet den Stand, der nie gespeichert wurde. Fehler lassen sich nur schwer nachvollziehen. - **Problem:** Scheitert ein Schreibzugriff (z. B. `SQLITE_BUSY`), antwortet die API trotzdem mit Erfolg und broadcastet den Stand, der nie gespeichert wurde. Fehler lassen sich nur schwer nachvollziehen.
- **Beispiel (behoben):** Das Lösen der Verknüpfung zwischen Notiz-Block und Aufgabe („Verknüpfung aufheben“, Löschen eines verknüpften Blocks) schrieb in nicht existierende Spalten `thoughts.doc_id`/`block_id`. Der Fehler wurde verschluckt, die Funktion war nie wirksam. Korrigiert; im neuen Sync-Weg gleicht `ReconcileDocTaskLinks` Häkchen und gelöschte Blöcke ab.
- **Maßnahme:** Mehrstufige Änderungen in Transaktionen bündeln und Fehler zurückgeben. - **Maßnahme:** Mehrstufige Änderungen in Transaktionen bündeln und Fehler zurückgeben.
### FLAW-15: WriteTimeout und SSE *(nicht live verifiziert)* ### FLAW-15: WriteTimeout und SSE
- **Status: Behoben.** `ServeSSE` hebt die Schreibfrist für die eigene Antwort auf (`http.ResponseController.SetWriteDeadline`). Test mit 1-s-Timeout: `TestSSEStreamOutlivesWriteTimeout`. Der Test hat den Abbruch im alten Code bestätigt.
- **Fundstelle:** [`cmd/server/main.go`](../cmd/server/main.go) `WriteTimeout: 30 * time.Second`. - **Fundstelle:** [`cmd/server/main.go`](../cmd/server/main.go) `WriteTimeout: 30 * time.Second`.
- **Problem:** Go beendet nach diesem Zeitraum das Schreiben der Antwort. Für den SSE-Fallback (`ServeSSE`) bedeutet das voraussichtlich einen Verbindungsabbruch alle 30 s. WebSockets sind durch Hijacking nicht betroffen. Der Client verbindet sich neu, daher fällt es kaum auf. - **Problem:** Go beendet nach diesem Zeitraum das Schreiben der Antwort. Für den SSE-Fallback (`ServeSSE`) bedeutet das voraussichtlich einen Verbindungsabbruch alle 30 s. WebSockets sind durch Hijacking nicht betroffen. Der Client verbindet sich neu, daher fällt es kaum auf.
- **Maßnahme:** Im SSE-Handler `http.ResponseController.SetWriteDeadline(time.Time{})` setzen oder `WriteTimeout` für diese Route aufheben. - **Maßnahme:** Im SSE-Handler `http.ResponseController.SetWriteDeadline(time.Time{})` setzen oder `WriteTimeout` für diese Route aufheben.
@@ -130,6 +138,7 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
- **Maßnahme:** Setup per einmaligem Token aus dem Server-Log absichern; Demo-Modus nur zusammen mit expliziter Warnung/Bestätigung. - **Maßnahme:** Setup per einmaligem Token aus dem Server-Log absichern; Demo-Modus nur zusammen mit expliziter Warnung/Bestätigung.
### FLAW-17: Schreibgeschützte Dokumentfreigaben ### FLAW-17: Schreibgeschützte Dokumentfreigaben
- **Status: Behoben.** Zwangsmigration entfernt, Backend prüft Bearbeiter-Rechte (`DocRole`, `/docs/{id}/sync`), der Editor speichert über die Sync-Engine, und der Freigabe-Dialog bietet „Bearbeiten“ an. Browser-Test: `e2e/share.e2e.mjs`.
- **Fundstellen:** Migration in `Store.migrate` (`UPDATE doc_shares SET permission = 'read' …`, ebenso `doc_team_shares`, `workspace_folder_shares`), `PatchDocumentBlocks`/`UpdateDocument` (nur Eigentümer). - **Fundstellen:** Migration in `Store.migrate` (`UPDATE doc_shares SET permission = 'read' …`, ebenso `doc_team_shares`, `workspace_folder_shares`), `PatchDocumentBlocks`/`UpdateDocument` (nur Eigentümer).
- **Hinweis:** Das ist offenbar gewollt: Nur der Eigentümer schreibt in ein Dokument, Empfänger lesen. Es gibt daher **kein** kollaboratives Bearbeiten. Das Schema (`CHECK(permission IN ('read','write'))`), die Zweige für `write` im Code und frühere Dokumentation suggerieren etwas anderes. Es gibt weder Sperren noch Konfliktauflösung, noch Cursor-Presence. - **Hinweis:** Das ist offenbar gewollt: Nur der Eigentümer schreibt in ein Dokument, Empfänger lesen. Es gibt daher **kein** kollaboratives Bearbeiten. Das Schema (`CHECK(permission IN ('read','write'))`), die Zweige für `write` im Code und frühere Dokumentation suggerieren etwas anderes. Es gibt weder Sperren noch Konfliktauflösung, noch Cursor-Presence.
- **Maßnahme:** Entweder Schreibfreigaben mit Konfliktbehandlung ergänzen oder den toten Code und das Schema bereinigen. - **Maßnahme:** Entweder Schreibfreigaben mit Konfliktbehandlung ergänzen oder den toten Code und das Schema bereinigen.
@@ -139,10 +148,10 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
- **Probleme:** Durchsucht nur eigene Gedanken (30) und eigene Dokumente (20); geteilte Dokumente, Aufgaben aus geteilten Listen, Einkaufslisten und Blöcke bleiben außen vor. `LIKE '%…%'` ohne Escaping von `%`/`_`, keine Indizes, keine Relevanz. - **Probleme:** Durchsucht nur eigene Gedanken (30) und eigene Dokumente (20); geteilte Dokumente, Aufgaben aus geteilten Listen, Einkaufslisten und Blöcke bleiben außen vor. `LIKE '%…%'` ohne Escaping von `%`/`_`, keine Indizes, keine Relevanz.
- **Maßnahme:** FTS5-Index über Blöcke/Gedanken; Berechtigungen mitberücksichtigen. - **Maßnahme:** FTS5-Index über Blöcke/Gedanken; Berechtigungen mitberücksichtigen.
### FLAW-19: Keine automatisierten Frontend-Prüfungen ### FLAW-19: Geringe Frontend-Testabdeckung
- **Fundstelle:** [`frontend/package.json`](../frontend/package.json): `"lint": "echo 'Linting passed'"`, `"test:run": "echo 'Frontend checks passed'"`. - **Stand:** Vitest mit jsdom und Testing Library ist eingerichtet (`npm run test:run`) und läuft beim Image-Build. Getestet sind bisher nur Outliner-Funktionen (Backspace-Zusammenführen, Cursor-Position beim Klicken).
- **Problem:** Die Skripte melden immer Erfolg. Die CI (`.gitea/workflows/build-and-publish.yaml`) prüft daher nur Go-Tests und den Vite-Build. Regressionen im UI (z. B. Berechtigungsanzeigen) fallen nicht auf. - **Problem:** Aufgaben, Listen, Freigabe-Dialoge und Berechtigungsanzeigen sind ungetestet. `"lint"` in [`frontend/package.json`](../frontend/package.json) ist weiterhin ein Platzhalter (`echo`).
- **Maßnahme:** `tsc --noEmit` und ESLint als echte Skripte, Komponententests für kritische Pfade. - **Maßnahme:** Tests für kritische Pfade ergänzen, ESLint als echtes `lint`-Skript.
### FLAW-20: Dateigröße und Duplikate ### FLAW-20: Dateigröße und Duplikate
- `internal/storage/sqlite.go` (≈ 5.600 Zeilen), `internal/api/server.go` (≈ 3.000), `WorkspaceView.tsx` (≈ 2.300), `TasksView.tsx` (≈ 2.500), `api.ts` (≈ 1.500). - `internal/storage/sqlite.go` (≈ 5.600 Zeilen), `internal/api/server.go` (≈ 3.000), `WorkspaceView.tsx` (≈ 2.300), `TasksView.tsx` (≈ 2.500), `api.ts` (≈ 1.500).
@@ -166,6 +175,23 @@ Schweregrade: **Hoch** (Datenabfluss oder Rechteumgehung möglich), **Mittel** (
- **Problem:** Diese Handler prüfen weder Eigentümer noch Mitgliedschaft. Jeder angemeldete Nutzer kann mit bekannter ID eine fremde Organisation löschen, deren Gruppen auflisten oder dort Gruppen anlegen. Gruppenbezogene Endpunkte (`/teams/{id}…`) prüfen dagegen korrekt auf Gruppen-Admin. Das Datenmodell kennt Organisationen nur noch rudimentär; im UI werden Gruppen direkt über `POST /teams` angelegt (`orgId` Standard `default`). - **Problem:** Diese Handler prüfen weder Eigentümer noch Mitgliedschaft. Jeder angemeldete Nutzer kann mit bekannter ID eine fremde Organisation löschen, deren Gruppen auflisten oder dort Gruppen anlegen. Gruppenbezogene Endpunkte (`/teams/{id}…`) prüfen dagegen korrekt auf Gruppen-Admin. Das Datenmodell kennt Organisationen nur noch rudimentär; im UI werden Gruppen direkt über `POST /teams` angelegt (`orgId` Standard `default`).
- **Maßnahme:** Organisationen entweder konsequent absichern (Eigentümer/Server-Admin) oder entfernen. - **Maßnahme:** Organisationen entweder konsequent absichern (Eigentümer/Server-Admin) oder entfernen.
### FLAW-25: Teilweise ausgeführte Struktur-Aktionen an gesperrten Bullets
- **Status: Behoben.** Fremd gesperrte Bullets sind im Editor schreibgeschützt und zeigen, wer sie bearbeitet. Löschen, Zusammenführen, Ein-/Ausrücken und Ziehen werden nicht gestartet, wenn ein beteiligter Bullet (oder einer im Ast) gesperrt ist; stattdessen erscheint ein Hinweis. Ebenso für „Übernehmen“/„Verwerfen“ und Rückgängig. Browser-Test: `e2e/lock.e2e.mjs`.
- **Fundstellen:** `handleMergeWithPrevious` und `handleIndentBlock` in [`OutlinerTree.tsx`](../frontend/src/features/workspace/OutlinerTree.tsx); Sperrregeln in `internal/collab` (Verschieben/Löschen gesperrter Blöcke wird abgelehnt).
- **Problem:** Eine Editor-Aktion besteht oft aus mehreren Operationen. Ist einer der beteiligten Bullets von jemand anderem gesperrt, lehnt der Server nur diese Operation ab. Beispiel aus dem Browser-Test: Backspace am Zeilenanfang hängt den Text oben an (angewendet), das Löschen des unteren Bullets wird abgelehnt – der Text steht dann doppelt da. Einrücken eines fremd gesperrten Bullets wird ohne Hinweis zurückgenommen. Es geht kein Text verloren.
- **Maßnahme (Phase 3):** Gesperrte Bullets im Editor anzeigen und Struktur-Aktionen, die sie berühren, gar nicht erst ausführen.
### FLAW-26: Fokus-Verzögerung nach Enter
- **Status: Behoben.** Neu eingehängte Bullets bekommen den Fokus sofort beim Einhängen. Browser-Test: „Sofort nach Enter Getipptes landet im neuen Bullet“.
- **Fundstelle:** `focusBlockInput` in `OutlinerTree.tsx` setzt den Fokus erst im nächsten Animation-Frame.
- **Problem:** Zeichen, die innerhalb dieses Frames getippt werden (automatisiert oder sehr schnelles Tippen), landen noch im vorherigen Bullet. Bei fremd gesperrtem Bullet entsteht daraus eine Konfliktkopie.
- **Maßnahme:** Fokus nach dem Anlegen synchron setzen (`useLayoutEffect`).
### FLAW-27: Grenzen nach der Verdichtung
- **Status: Bekannt, bewusst so.**
- **Fundstellen:** `CompactDocs` in [`internal/storage/docsync.go`](../internal/storage/docsync.go), `reviveAsNewBlock` in [`internal/collab/ops.go`](../internal/collab/ops.go).
- **Verhalten:** Nach 30 Tagen sind gelöschte Blöcke endgültig weg. Rückgängig für ein so altes Löschen ist nicht mehr möglich (Hinweis im Editor). Text, der wochenlang offline in einem inzwischen endgültig gelöschten Block getippt wurde, kommt als neuer Block am Ende der Notiz zurück, nicht an der alten Stelle. Wer eine Operation erst nach mehr als 90 Tagen erneut sendet (Antwort ging verloren, Gerät so lange offline), wird nicht mehr als Wiederholung erkannt; Textänderungen erzeugen dann höchstens eine Konfliktkopie, kein stilles Überschreiben.
--- ---
## Aus der früheren Fassung entfernt ## Aus der früheren Fassung entfernt
+162
View File
@@ -0,0 +1,162 @@
# Modul: Gemeinsames Bearbeiten von Notizen
Code: [internal/collab/](../../internal/collab/) (Regeln, ohne Datenbank), [internal/storage/docsync.go](../../internal/storage/docsync.go) (Log, Snapshot, Rollen, Verdichtung), [internal/api/docsync.go](../../internal/api/docsync.go) (Endpunkte, Hintergrundjob), [frontend/src/lib/docSync/](../../frontend/src/lib/docSync/) (Engine im Browser), Anzeige in [DocumentEditor.tsx](../../frontend/src/features/workspace/DocumentEditor.tsx), [OutlinerTree.tsx](../../frontend/src/features/workspace/OutlinerTree.tsx) und [OutlinerBlock.tsx](../../frontend/src/features/workspace/OutlinerBlock.tsx).
Mehrere Personen und Geräte bearbeiten dieselbe Notiz gleichzeitig. Jede Notiz, auch eine private, speichert über diesen Weg; es gibt keinen zweiten Speicherweg.
## Anforderungen
1. **Kein Datenverlust.** Was jemand getippt hat, steht danach in der Notiz, auch nach Verbindungsabbruch, Neuladen oder gleichzeitigem Bearbeiten desselben Bullets.
2. **Funktioniert ohne WebSocket.** Hinter Proxys, die WebSockets blocken oder Streams puffern, funktioniert die Bearbeitung voll, nur mit etwas mehr Verzögerung.
3. **Funktioniert bei schlechtem Netz.** Eingaben werden lokal gesichert und später zuverlässig übertragen, auch nach Tagen offline.
4. **Klare Regel bei Konflikten.** Bearbeiten zwei Personen denselben Bullet, bekommt eine das Recht. Der Text der anderen bleibt als Konfliktkopie erhalten.
5. **Berechtigungen.** Notizen und Ordner lassen sich als *Bearbeiter* oder *Leser* teilen; der Server prüft das bei jeder Änderung.
## Ansatz
**Server-autoritatives Operations-Log mit Versionen pro Block und Block-Sperren (Leases).** Daten fließen immer über gewöhnliche HTTP-Anfragen; WebSocket, SSE und Long-Polling sind nur der Weckruf.
| Baustein | Aufgabe |
|---|---|
| **Operations-Log** | Jede Änderung ist eine kleine Operation (Text ändern, Block einfügen, verschieben, löschen …). Der Server prüft und wendet sie an und schreibt die Auswirkungen (Effekte) mit fortlaufender Nummer `seq` pro Notiz ins Log. Wer seine `seq` kennt, holt alles danach. |
| **Version pro Block** | Jede Textänderung nennt die Blockversion, die der Nutzer beim Tippen vor sich hatte. Passt sie nicht mehr, liegt ein Konflikt vor. |
| **Lease pro Block** | Wer in einen Bullet klickt, hält ihn für 20 s und verlängert das, solange er dort arbeitet. Andere sehen „Anna bearbeitet …“ und können genau diesen Bullet so lange nicht ändern, alle anderen schon, auch im selben Ast. |
| **Konfliktkopie** | Kommt es trotzdem zum Konflikt (Lease abgelaufen, jemand war offline), behält der Block die zuerst gespeicherte Fassung; der andere Text wird als markierter Block direkt darunter eingefügt. |
| **Lokale Warteschlange** | Jede Änderung landet zuerst in IndexedDB und wird von dort gesendet. Jede Operation hat eine eindeutige ID; doppeltes Senden ist harmlos. |
### Warum nicht CRDT (Yjs/Automerge) oder OT
- **CRDT** führt Zeichen für Zeichen zusammen. Dafür bräuchte das Go-Backend eine vollständige Yjs-Implementierung, um Rechte, Hashtags und Markdown serverseitig auszuwerten; die gibt es in Go nicht ausgereift. Ohne sie wäre der Server nur ein Speicher für Binärdaten, den er nicht prüfen kann. Ein früherer Versuch ist daran gescheitert.
- **OT** braucht einen fehlerfreien Transformations-Kern für jede Operationsart; schwer korrekt zu bekommen und schwer zu testen.
- Gewollt ist, dass bei einem Bullet **einer das Recht bekommt**. Sperren plus Versionsprüfung sind einfach zu verstehen, vollständig serverseitig prüfbar und gegen Netzfehler testbar.
- Bewusster Verzicht: Zwei Personen können nicht gleichzeitig im **selben** Bullet tippen. In **verschiedenen** Bullets, auch im selben Ast, geht es.
## Datenmodell
Details in [database.md](../database.md).
- **Baum statt flacher Liste.** `blocks.parent_id` (leer = oberste Ebene) und `blocks.order_key` sortieren Geschwister. Einrücken oder Verschieben eines Astes ist *eine* Operation auf *einem* Block. Ebene und Position für die Oberfläche berechnet der Server beim Lesen.
- **Sortierschlüssel (Fractional Index).** Ein neuer Block zwischen `a0` und `a1` bekommt einen Schlüssel dazwischen; umnummeriert wird nie. Gleiche Schlüssel (zwei gleichzeitige Einfügungen) sortiert die Block-ID. Go (`orderkey.go`) und TypeScript (`orderKey.ts`) erzeugen identische Schlüssel, geprüft mit gemeinsamen Testvektoren.
- **Grabsteine.** Gelöschte Blöcke bleiben mit `deleted = 1` und `deleted_at` erhalten. Eine verspätete Textänderung stellt den Block wieder her, statt verloren zu gehen. Nach 30 Tagen werden sie endgültig entfernt (siehe [Verdichtung](#verdichtung)).
- **Log** `doc_ops` (Effekte je `seq`) und **Idempotenz** `doc_op_results` (Ergebnis je Operations-ID). Das Abspielen des Logs ab einem Snapshot ergibt exakt den Serverstand.
- **Leases und Sitzungen liegen im Arbeitsspeicher** (`collab.LeaseManager`). Sie sind kurzlebig, nach einem Neustart fordern Clients sie automatisch neu an. MindSift läuft als eine Instanz, das passt.
- **Abgeleitete Daten** (`markdown_cache`, Suchtext, `tags`, `block_tags`) entstehen in derselben Transaktion wie die Änderung.
## Protokoll
Endpunkte und Felder: [api/rest.md](../api/rest.md#gemeinsames-bearbeiten-von-notizen).
| Methode | Pfad | Zweck |
|---|---|---|
| GET | `/api/v1/docs/{id}/snapshot` | Alle Blöcke inkl. Grabsteine, Titel, `headSeq`, eigene Rolle, `shared` |
| POST | `/api/v1/docs/{id}/sync` | **Ein Aufruf für alles:** eigene Operationen senden, Leases halten, neue Log-Einträge abholen |
| GET | `/api/v1/docs/{id}/changes?since=N&wait=25` | Long-Polling: antwortet, sobald sich die Notiz (oder Sperren/Präsenz) ändert, spätestens nach `wait` Sekunden |
| POST | `/api/v1/docs/{id}/leave` | Sitzung beenden, Leases freigeben |
Operationen: `insert`, `set_content`, `set_props` (Typ, abgehakt, eingeklappt, Sprache, Konflikt erledigt), `move`, `delete`, `restore`, `set_title`. Im Log steht zusätzlich `purge` (Verdichtung).
### Regeln auf dem Server
Alle Operationen einer Notiz laufen nacheinander: ein Mutex pro Notiz und eine SQLite-Transaktion pro `/sync`. „Wer zuerst kommt“ ist dadurch immer eindeutig.
1. **Idempotenz:** Ist die Operations-ID bekannt, kommt das gespeicherte Ergebnis zurück, ohne erneut anzuwenden. Verlorene Antworten sind harmlos.
2. **Berechtigung:** Operationen nur mit Rolle `owner` oder `editor`, sonst `403`. Block-IDs anderer Notizen sind tabu.
3. **`set_content`:** Hält ein *anderer* Client eine Lease oder passt die Basisversion nicht → **Konflikt**. Gleicher Text → angewendet ohne neue Version. Gelöschter Block → wiederherstellen (bei gelöschtem Elternblock unter dem nächsten lebenden Vorfahren) und anwenden. Endgültig entfernter Block → Text wird als neuer Block am Ende angelegt.
4. **Konflikt:** Der Block behält seinen Inhalt. Direkt danach entsteht ein Block mit dem abgelehnten Text, `conflictOf` und `conflictAuthor` gesetzt, mit deterministischer ID (`blk_cf_` + Hash der Operations-ID), damit Wiederholungen keine zweite Kopie erzeugen. Für den Titel gilt dasselbe mit eigener Titelversion.
5. **`move`:** abgelehnt bei Zyklus, bei gelöschtem Ziel-Elternblock oder wenn jemand anders den Block selbst hält. Unterpunkte wandern mit; wer darin tippt, tippt einfach an der neuen Stelle weiter.
6. **`delete`:** abgelehnt, wenn jemand anders einen Block im Ast hält; sonst Grabstein für den ganzen Ast.
7. **`insert`:** Ist der Elternblock gelöscht, wird unter dem nächsten lebenden Vorfahren eingefügt, ist er endgültig entfernt, auf oberster Ebene.
8. **Leases:** Vergabe nur, wenn frei, abgelaufen oder schon im Besitz desselben Clients; gültig 20 s ab letzter Bestätigung. `leases.hold` ist immer die vollständige Liste. Neu eingefügte Blöcke und Konfliktkopien bekommt ihr Autor automatisch.
9. **Keine Lease auf Elternblöcke:** Einfügen, Ein- und Ausrücken unter einem Elternblock brauchen keine Sperre auf diesen.
### Ablauf beim Tippen
```mermaid
sequenceDiagram
participant A as Client Anna
participant S as Server
participant B as Client Ben
A->>S: /sync leases.hold=[b9]
S-->>A: granted b9
S--)B: DOC_SEQ (Weckruf)
B->>S: /sync lastSeq=412
S-->>B: leases.others: b9 = Anna (Bullet gesperrt)
A->>S: /sync op set_content b9 base=7
S-->>A: applied, seq 414
S--)B: DOC_SEQ headSeq=414
B->>S: /sync lastSeq=413
S-->>B: ops [set_content b9 v8]
```
## Client (`DocSyncEngine`)
- **Zustände:** *bestätigter Stand* (vom Server, mit `lastSeq`) und *Warteschlange* (eigene, unbestätigte Operationen). Angezeigt wird der bestätigte Stand mit der Warteschlange darübergelegt; das macht `model.ts` mit denselben Regeln wie der Server.
- **Editor-Anbindung:** Die Editor-Komponenten arbeiten mit einer flachen Liste (Ebene + Reihenfolge). `diff.ts` übersetzt jede Änderung per Vergleich in Operationen.
- **Persistenz:** Warteschlange und bestätigter Stand liegen in IndexedDB. Beim Öffnen erscheint sofort der gespeicherte Stand, auch offline.
- **Senden:** gesammelt nach 300 ms, immer nur ein `/sync` gleichzeitig; Wiederholung mit Backoff 1 s, 2 s, 4 s … höchstens 30 s. Mehrere Textänderungen eines Blocks werden zusammengefasst; die Basisversion bleibt die der ersten (die der Nutzer beim Tippen sah), damit nie eine fremde Änderung überschrieben wird.
- **Leases:**
- Tippen wartet nie auf eine Sperre. Beim Fokussieren wird sie sofort angemeldet.
- Neue Bullets (Enter, Einfügen) gehören sofort dem Ersteller.
- Verlassene Bullets bleiben 20 s gehalten; nach 60 s ohne Eingabe wird auch der fokussierte Bullet freigegeben.
- Fremd gesperrte Bullets zeigen ein Schloss mit Namen und sind schreibgeschützt; Enter darin legt einen neuen Bullet darunter an. Löschen, Zusammenführen, Ein-/Ausrücken und Ziehen, die einen gesperrten Bullet berühren, werden gar nicht erst ausgeführt (Hinweis unten im Editor).
- **Konflikte:** Konfliktkopien sind farbig markiert, mit „Übernehmen“ (ersetzt den Originaltext) und „Verwerfen“. Beide prüfen vorher die Sperren von Original und Kopie.
- **Hintergrund und Netz:** Beim Verbergen der Seite sendet die Engine Ausstehendes mit `keepalive` und gibt ihre Sperren frei; bei `online` und Rückkehr in den Tab gleicht sie sofort ab.
- **Rechteentzug:** Der nächste `/sync` liefert `403` bzw. `404`. Die Engine verwirft die Warteschlange nicht, hört aber auf zu senden und zeigt den Fehler.
- **Neu laden:** Meldet der Server `resync` (Client voraus oder hinter der verdichteten Log-Grenze) oder fehlt eine `seq`, lädt die Engine den Snapshot; die Warteschlange bleibt erhalten und wird danach angewendet.
## Transport: robust auch hinter Proxys
Daten fließen nur über `/sync` und `/snapshot`. Push-Kanäle melden nur „Notiz X hat jetzt `headSeq` N“ (`DOC_SEQ` im Raum `doc:<id>`, siehe [api/websocket.md](../api/websocket.md)).
| Stufe | Ausfall erkannt durch | Wechsel zu |
|---|---|---|
| 1. WebSocket | Verbindungsfehler oder 35 s ohne `PONG` auf das `PING` alle 15 s (WebSocket angenommen, aber nichts kommt durch) | SSE |
| 2. SSE | Erstes Ereignis kommt nicht innerhalb von 5 s (Proxy puffert); gilt dann für den ganzen Tab | Long-Polling |
| 3. Long-Polling (`/changes`, 25 s) | 3 Fehlschläge hintereinander | Intervall |
| 4. Intervall | – | Abfrage alle 3 s; bei `online` wird Long-Polling erneut versucht |
Zusätzlich, solange Stufe 1–3 läuft: Abgleich alle 7 s, wenn die Engine Sperren hält (Verlängerung), sonst alle 30 s zur Sicherheit. Der Editor zeigt den Weg an („Live“, „Live (Long-Polling)“, „Abgleich alle 3 s“). Gemessen im Browser-Test: 0,2–0,3 s über WebSocket, SSE und Long-Polling, rund 3 s im Intervall.
Long-Polling bleibt mit 25 s unter üblichen Proxy-Grenzen (Nginx 60 s, Cloudflare 100 s). Den Schreib-Timeout hebt der Server für Streams und Long-Polling auf. Räume betritt nur, wer die Notiz lesen darf. Proxy-Beispiele: [reverse-proxy.md](../reverse-proxy.md).
## Berechtigungen
- Rollen je Notiz: `owner`, `editor`, `viewer`. Die effektive Rolle ist die höchste aus Eigentum, direkter Freigabe, Gruppenfreigabe und Freigabe des geteilten Ordners. Eine zentrale Funktion `Store.DocRole` entscheidet für Snapshot, Sync, Long-Polling, Raumbeitritt und Aufgaben-Verknüpfungen.
- Nur der Eigentümer verwaltet Freigaben und löscht die Notiz. Bearbeiter dürfen sonst alles: Inhalt, Struktur, Titel und Symbol.
- Freigaben liegen in `doc_shares`, `doc_team_shares` und `workspace_folder_shares` (`read`/`write`). Eine Zusammenlegung zu einer Tabelle wurde bewusst nicht umgesetzt: `DocRole` entscheidet bereits zentral, das Umschreiben der rund 20 Sichtbarkeitsabfragen brächte nur Risiko. Mehr: [sharing-and-permissions.md](../sharing-and-permissions.md).
## Private Notizen
Gleiches Modell, aber sparsam: Solange eine Notiz nicht geteilt ist (`shared = false`) und nur in einem Tab offen ist (`sessions = 1`), gibt es keine Leases, keinen Herzschlag und nur die 30-s-Abfrage. Öffnet ein zweites Gerät die Notiz oder wird sie geteilt, schalten alle offenen Clients mit der nächsten Antwort auf den vollen Ablauf um. Gegenüber dem früheren Speichern des ganzen Dokuments alle 800 ms überträgt und schreibt das Modell nur geänderte Blöcke.
## Rückgängig und Wiederholen
- `Strg+Z`/`Cmd+Z` nimmt die letzte **eigene** Änderung zurück, `Strg+Umschalt+Z` bzw. `Strg+Y` wiederholt. Fremde Änderungen bleiben unberührt.
- Zu jeder eigenen Änderung merkt sich die Engine den Gegenschritt (`undo.ts`) und sendet ihn als normale Operation; Server und Log brauchen dafür nichts Besonderes.
- Tippen im selben Bullet ist ein Schritt bis zu einer Pause von 1 s; Einfügen, Löschen, Einrücken, Verschieben, Abhaken und mehrzeiliges Einfügen sind jeweils ein Schritt. Bis zu 200 Schritte pro Tab und Notiz.
- Hat inzwischen jemand anders den Bullet geändert oder hält ihn gerade, wird der Schritt mit Hinweis übersprungen, nie blind überschrieben.
- Das Undo des Textfelds selbst wird abgefangen (`keydown` und `beforeinput` mit `historyUndo`), damit Browser-Undo und eigenes Undo nicht gegeneinander arbeiten.
## Verdichtung
Ein Hintergrundjob (`startCompaction`) läuft beim Start und danach alle 6 Stunden:
| Was | Aufbewahrung |
|---|---|
| Log geteilter Notizen | 30 Tage; die Grenze steht in `documents.log_floor` |
| Log privater Notizen | 7 Tage |
| Grabsteine | 30 Tage nach `deleted_at`, dann endgültig entfernt per Log-Eintrag `purge` (offene Editoren übernehmen das ohne Neuladen) |
| Idempotenz-Ergebnisse | 90 Tage |
| Leases und Sitzungen im Speicher | abgelaufene werden entfernt |
Clients hinter der Grenze laden den Snapshot neu und behalten ihre Warteschlange. Grenzen dieses Vorgehens: [FLAW-27](../flaws.md).
## Tests
| Ebene | Was | Wo |
|---|---|---|
| Regeln | Konfliktmatrix aus Lease, Version, Grabstein, Verschieben, Löschen, Purge | `internal/collab/ops_test.go` |
| Simulation | 4 Clients, 25 Seeds × 600 Schritte über ein Netz, das Anfragen und Antworten verliert und verdoppelt; Clients gehen offline; zwischendurch Verdichtung. Geprüft: Konvergenz, gültiger Baum, jede Operation genau einmal angewendet, kein Text verloren. Per Gegenprobe (absichtlich eingebaute Fehler) abgesichert. | `internal/storage/docsync_sim_test.go` |
| Speicher und API | Log ↔ Snapshot, Rollen, Long-Polling, Verdichtung | `internal/storage/docsync_test.go`, `internal/api/docsync_test.go` |
| Browser-Engine | Vitest mit gefälschtem Server und Netz, `fake-indexeddb`: offline, Neuladen, verlorene Antworten, Push-Wechsel, Verdichtung | `frontend/src/lib/docSync/__tests__/` |
| Ende-zu-Ende | Playwright mit zwei Benutzern: gemeinsames Tippen, Freigabe, Sperren, Konflikte, Undo, alle Transport-Stufen mit blockiertem WebSocket/SSE | `e2e/` (`e2e/run.sh`) |
+17 -4
View File
@@ -1,10 +1,10 @@
# Modul: Workspace (Outliner-Notizen) # Modul: Workspace (Outliner-Notizen)
Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.tsx) (Explorer, Tags, To-Do-Rollup), [DocumentEditor.tsx](../../frontend/src/features/workspace/DocumentEditor.tsx), [OutlinerTree.tsx](../../frontend/src/features/workspace/OutlinerTree.tsx), [OutlinerBlock.tsx](../../frontend/src/features/workspace/OutlinerBlock.tsx), [CodeBlockItem.tsx](../../frontend/src/features/workspace/CodeBlockItem.tsx), [LinkModal.tsx](../../frontend/src/features/workspace/LinkModal.tsx); Backend: Dokument-, Ordner- und Tag-Handler, `Store.PatchDocumentBlocks`. Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.tsx) (Explorer, Tags, To-Do-Rollup), [DocumentEditor.tsx](../../frontend/src/features/workspace/DocumentEditor.tsx), [OutlinerTree.tsx](../../frontend/src/features/workspace/OutlinerTree.tsx), [OutlinerBlock.tsx](../../frontend/src/features/workspace/OutlinerBlock.tsx), [CodeBlockItem.tsx](../../frontend/src/features/workspace/CodeBlockItem.tsx), [LinkModal.tsx](../../frontend/src/features/workspace/LinkModal.tsx); Backend: Dokument-, Ordner- und Tag-Handler, Sync-Handler in `internal/api/docsync.go`, `internal/collab`, `internal/storage/docsync.go`.
## Datenhaltung ## Datenhaltung
- Eine Notiz ist ein Datensatz in `documents` plus geordnete Zeilen in `blocks` (`position` als Kommazahl, `level` für die Einrückung). Alles liegt in SQLite; es gibt keine Dateien auf der Platte. - Eine Notiz ist ein Datensatz in `documents` plus ein Baum von Blöcken in `blocks` (`parent_id`, Sortierschlüssel `order_key`). Jede Änderung läuft über `internal/collab` und wird im Log `doc_ops` festgehalten. Ebene und Position für die Oberfläche berechnet der Server beim Lesen. Das Log und gelöschte Blöcke werden regelmäßig verdichtet ([database.md](../database.md#verdichtung-des-notiz-logs)). Alles liegt in SQLite; es gibt keine Dateien auf der Platte.
- Bei jeder Änderung wird `documents.markdown_cache` neu erzeugt; `block_tags` wird aus `#Hashtags` der Blöcke neu aufgebaut. - Bei jeder Änderung wird `documents.markdown_cache` neu erzeugt; `block_tags` wird aus `#Hashtags` der Blöcke neu aufgebaut.
- Der Export als `.md` erfolgt im Browser (Button im Editor, Blob-Download). - Der Export als `.md` erfolgt im Browser (Button im Editor, Blob-Download).
- Blocktypen: Überschriften `h1`–`h3`, `paragraph`/`text`, `bullet`, `todo`, `code`, `callout`, `divider`, `table`, `progress`. - Blocktypen: Überschriften `h1`–`h3`, `paragraph`/`text`, `bullet`, `todo`, `code`, `callout`, `divider`, `table`, `progress`.
@@ -17,7 +17,20 @@ Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.ts
- **Slash-Befehle:** `/todo`, `/link`, `/h1`, `/h2`, `/h3`, `/code`, `/callout`, `/divider` (Liste in `OutlinerBlock.tsx`). - **Slash-Befehle:** `/todo`, `/link`, `/h1`, `/h2`, `/h3`, `/code`, `/callout`, `/divider` (Liste in `OutlinerBlock.tsx`).
- **Links:** `[[Notiztitel]]`, Markdown-Links `[Titel](url)` und reine URLs werden als Token dargestellt; `LinkModal` hilft beim Verknüpfen von Notizen. - **Links:** `[[Notiztitel]]`, Markdown-Links `[Titel](url)` und reine URLs werden als Token dargestellt; `LinkModal` hilft beim Verknüpfen von Notizen.
- **Einfügen:** Mehrzeiliger Text wird anhand von Markdown-Präfixen und Einrückung in Blöcke zerlegt; Überschriften gruppieren darunterliegenden Text und Code. - **Einfügen:** Mehrzeiliger Text wird anhand von Markdown-Präfixen und Einrückung in Blöcke zerlegt; Überschriften gruppieren darunterliegenden Text und Code.
- **Speichern:** Der Editor speichert 800 ms nach der letzten Eingabe das gesamte Dokument (`PUT /documents/{id}`). Der schlanke Block-Endpunkt `PATCH /documents/{id}/blocks` wird derzeit vom To-Do-Rollup beim Abhaken genutzt. - **Speichern:** Jede Notiz hat im Browser eine `DocSyncEngine` ([lib/docSync/](../../frontend/src/lib/docSync/)). Der Editor meldet jede Änderung der Block-Liste; die Engine übersetzt sie in Operationen, speichert sie zuerst in IndexedDB und sendet sie dann gesammelt (300 ms) an `POST /docs/{id}/sync`. Beim Öffnen zeigt sie sofort den gespeicherten Stand, auch offline.
- **Abgleich bei geteilten oder mehrfach geöffneten Notizen**, nach Verfügbarkeit:
1. **Push** über WebSocket oder SSE: Der Server meldet `DOC_SEQ`, die Engine gleicht sofort ab (im Test 0,2–0,3 s).
2. **Long-Polling** (`/docs/{id}/changes`), wenn ein Proxy WebSocket und SSE blockiert oder SSE puffert: ebenso schnell.
3. **Abfrage alle 3 s**, wenn auch Long-Polling dreimal scheitert (bei `online` wird Long-Polling erneut versucht).
Solange 1. oder 2. läuft, gleicht die Engine zusätzlich alle 7 s ab, wenn sie Sperren hält (Verlängerung), sonst alle 30 s zur Sicherheit. Ein Fokuswechsel meldet die Sperre sofort an. Der Editor zeigt den Weg an („Live“, „Live (Long-Polling)“, „Abgleich alle 3 s“).
- **Private Notizen**, die nur einmal geöffnet sind, gleichen alle 30 s ab, außerdem sofort bei Rückkehr ins Netz oder in den Tab. Beim Verlassen sendet die Engine Ausstehendes mit `keepalive` und gibt ihre Sperren frei.
- **Textkonflikte:** Jede Textänderung trägt die Version, die der Nutzer beim Tippen vor sich hatte. Hat jemand anderes den Bullet inzwischen geändert oder gesperrt, legt der Server eine Konfliktkopie an; es wird nie still überschrieben.
- **Sperren:** Der Bullet mit dem Cursor, verlassene Bullets für 20 s und neu angelegte Bullets werden gehalten. Bullets, die jemand anderes hält, zeigen ein Schloss mit Namen und sind schreibgeschützt; Enter darin legt einen neuen Bullet darunter an. Löschen, Zusammenführen, Ein-/Ausrücken und Ziehen werden bei gesperrten Bullets nicht ausgeführt (Hinweis unten im Editor).
- **Präsenz:** Im Kopf der Notiz steht, wer sie gerade sonst noch geöffnet hat.
- **Konfliktkopien** sind farbig markiert („Konflikt – Version von …“) mit „Übernehmen“ (Text ersetzt den ursprünglichen Punkt bzw. den Titel, die Kopie verschwindet) und „Verwerfen“.
- **Rückgängig:** `Strg+Z`/`Cmd+Z` nimmt die eigene letzte Änderung zurück, `Strg+Umschalt+Z` oder `Strg+Y` wiederholt (auch im Titel und über das Kontextmenü/mobile Tastaturen). Tippen im selben Punkt bis 1 s Pause ist ein Schritt; bis zu 200 Schritte je Tab. Hat jemand anderes den Punkt inzwischen geändert oder hält ihn gesperrt, wird der Schritt mit Hinweis übersprungen.
- `PUT /documents/{id}` ändert nur noch Symbol, Anheften und Ordner (Titel wird ins Log übernommen); Inhalte laufen ausschließlich über die Engine. Das To-Do-Rollup hakt per einzelner Operation über denselben Sync-Endpunkt ab.
## Explorer, Tags, To-Dos ## Explorer, Tags, To-Dos
@@ -30,7 +43,7 @@ Code: [WorkspaceView.tsx](../../frontend/src/features/workspace/WorkspaceView.ts
- Einzelne Notizen und ganze **geteilte Ordner** lassen sich an Benutzer oder Gruppen geben. Empfänger haben **nur Leserechte**; nur der Eigentümer schreibt ([sharing-and-permissions.md](../sharing-and-permissions.md), [FLAW-17](../flaws.md)). - Einzelne Notizen und ganze **geteilte Ordner** lassen sich an Benutzer oder Gruppen geben. Empfänger haben **nur Leserechte**; nur der Eigentümer schreibt ([sharing-and-permissions.md](../sharing-and-permissions.md), [FLAW-17](../flaws.md)).
- Empfänger ordnen geteilte Notizen und Ordner in ihrem eigenen Baum ein (`doc_user_folders`, `workspace_folder_mounts`), ohne dass der Eigentümer etwas davon merkt. - Empfänger ordnen geteilte Notizen und Ordner in ihrem eigenen Baum ein (`doc_user_folders`, `workspace_folder_mounts`), ohne dass der Eigentümer etwas davon merkt.
- Änderungen erreichen offene Fenster über `DOC_BLOCKS_PATCHED`/`DOC_UPDATED` ([websocket.md](../api/websocket.md)). - Änderungen erreichen offene Fenster über den Weckruf `DOC_SEQ` und anschließenden Sync; Notizlisten über `DOC_UPDATED` ([websocket.md](../api/websocket.md)).
## Grenzen ## Grenzen
+4
View File
@@ -193,6 +193,10 @@ VirtualHost-Konfiguration (`/etc/apache2/sites-available/mindsift.conf`):
--- ---
## Wenn der Proxy nicht angepasst werden kann
MindSift funktioniert auch hinter Proxys, die WebSocket blocken, Streams puffern oder nur kurze Anfragen erlauben. Die Notiz-Synchronisation wählt automatisch den besten verfügbaren Weg (WebSocket → SSE → Long-Polling → Abfrage alle 3 s); der Editor zeigt ihn an. Long-Polling-Anfragen dauern höchstens 25 s und bleiben damit unter üblichen Proxy-Grenzen. Details: [api/websocket.md](api/websocket.md).
## 🔍 Diagnose & Verifikation ## 🔍 Diagnose & Verifikation
Mit den folgenden Befehlen lässt sich die Erreichbarkeit und Streaming-Funktion prüfen: Mit den folgenden Befehlen lässt sich die Erreichbarkeit und Streaming-Funktion prüfen:
+3 -3
View File
@@ -14,10 +14,10 @@
|---|---|---|---|---| |---|---|---|---|---|
| **Aufgabenliste** (`task_list_shares.permission`) | alles, löschen, (un)teilen | Aufgaben anlegen/ändern/abhaken/verschieben/löschen, Liste umbenennen, Spalten ändern | nur ansehen | Benutzer, Gruppen | | **Aufgabenliste** (`task_list_shares.permission`) | alles, löschen, (un)teilen | Aufgaben anlegen/ändern/abhaken/verschieben/löschen, Liste umbenennen, Spalten ändern | nur ansehen | Benutzer, Gruppen |
| **Geteilte Liste** (`list_shares.role`) | alles, löschen, (un)teilen | Einträge anlegen/ändern/abhaken/löschen | nur ansehen | Benutzer, Gruppen | | **Geteilte Liste** (`list_shares.role`) | alles, löschen, (un)teilen | Einträge anlegen/ändern/abhaken/löschen | nur ansehen | Benutzer, Gruppen |
| **Notiz** (`doc_shares`, `doc_team_shares`) | schreiben | – | nur lesen | Benutzer, Gruppen | | **Notiz** (`doc_shares`, `doc_team_shares`) | alles, Freigaben, löschen | Inhalt, Struktur, Titel | nur lesen | Benutzer, Gruppen |
| **Geteilter Ordner** (`workspace_folder_shares`) | schreiben | – | nur lesen | Benutzer, Gruppen | | **Geteilter Ordner** (`workspace_folder_shares`) | alles | Notizen darin bearbeiten | nur lesen | Benutzer, Gruppen |
- **Notizen und Ordner sind absichtlich schreibgeschützt für Empfänger.** Das erzwingt eine Migration beim Start (`permission = 'read'`); Schreibpfade im Code sind tot ([FLAW-17](flaws.md)). - **Notizen und Ordner:** Die effektive Rolle berechnet `Store.DocRole` (Eigentümer, Eigentümer des geteilten Ordners = Bearbeiter, direkte Freigabe, Gruppenfreigabe, Ordnerfreigabe; höchste gewinnt). Im Freigabe-Dialog sind für Notizen und geteilte Ordner „Bearbeiten“ und „Nur Lesen“ wählbar (gespeichert als `write`/`read`). Bearbeiter ändern Inhalt, Struktur und Titel über die Sync-Engine; Leser sehen Änderungen live.
- Bei Aufgabenlisten und Listen gilt: Hat jemand mehrere Zugriffswege (direkt und über Gruppen), zählt die **höchste** Stufe. - Bei Aufgabenlisten und Listen gilt: Hat jemand mehrere Zugriffswege (direkt und über Gruppen), zählt die **höchste** Stufe.
- Für Gruppen gilt die Freigabe für alle aktuellen und künftigen Mitglieder. - Für Gruppen gilt die Freigabe für alle aktuellen und künftigen Mitglieder.
- Aufgabenlisten besitzen zusätzlich das Altfeld `team_id`: Mitglieder dieser Gruppe haben ohne Freigabezeile Bearbeiterrechte. Eine Gruppenfreigabe über den Dialog setzt `team_id`, falls es noch leer ist. - Aufgabenlisten besitzen zusätzlich das Altfeld `team_id`: Mitglieder dieser Gruppe haben ohne Freigabezeile Bearbeiterrechte. Eine Gruppenfreigabe über den Dialog setzt `team_id`, falls es noch leer ist.
+24 -9
View File
@@ -6,8 +6,9 @@
[![Self-Hosted](https://img.shields.io/badge/100%25-Self--Hosted-2563eb?style=flat-square)](#) [![Self-Hosted](https://img.shields.io/badge/100%25-Self--Hosted-2563eb?style=flat-square)](#)
[![Docker](https://img.shields.io/badge/Deployment-Single%20Container-2496ed?style=flat-square&logo=docker&logoColor=white)](#-schnellstart) [![Docker](https://img.shields.io/badge/Deployment-Single%20Container-2496ed?style=flat-square&logo=docker&logoColor=white)](#-schnellstart)
[![SQLite](https://img.shields.io/badge/Daten-SQLite-003b57?style=flat-square&logo=sqlite&logoColor=white)](#-deine-daten-gehören-dir) [![SQLite](https://img.shields.io/badge/Daten-SQLite-003b57?style=flat-square&logo=sqlite&logoColor=white)](#-deine-daten-gehören-dir)
[![Version](https://img.shields.io/badge/Version-v0.2-16a34a?style=flat-square)](#-schnellstart)
**MindSift** ist eine selbst gehostete Schaltstelle für spontane Gedanken, Aufgaben mit Kanban-Board, hierarchische Outliner-Notizen und gemeinsame Listen. Alles läuft als ein einzelner Container mit eingebetteter SQLite-Datenbank, auf deinem eigenen Server. **MindSift** ist eine selbst gehostete Schaltstelle für spontane Gedanken, Aufgaben mit Kanban-Board, hierarchische Outliner-Notizen, die mehrere Personen gleichzeitig bearbeiten können, und gemeinsame Listen. Alles läuft als ein einzelner Container mit eingebetteter SQLite-Datenbank, auf deinem eigenen Server.
<p align="center"> <p align="center">
<img src="screenshots/Triage_desktop.png" alt="MindSift Triage Desktop" width="85%" /> <img src="screenshots/Triage_desktop.png" alt="MindSift Triage Desktop" width="85%" />
@@ -61,8 +62,17 @@ Die meisten Notiz- und Aufgaben-Apps verlangen Struktur im falschen Moment: Wenn
* Blocktypen und Slash-Befehle: `/todo`, `/link`, `/h1`–`/h3`, `/code` (mit Syntax-Hervorhebung), `/callout`, `/divider`. * Blocktypen und Slash-Befehle: `/todo`, `/link`, `/h1`–`/h3`, `/code` (mit Syntax-Hervorhebung), `/callout`, `/divider`.
* Wiki-Links `[[Notiz]]`, Links und `#Hashtags`; Tag-Seiten bündeln alle Fundstellen. * Wiki-Links `[[Notiz]]`, Links und `#Hashtags`; Tag-Seiten bündeln alle Fundstellen.
* Ordner, Favoriten, **To-Do-Rollup** über alle Notizen und Export einer Notiz als `.md`. * Ordner, Favoriten, **To-Do-Rollup** über alle Notizen und Export einer Notiz als `.md`.
* Einfügen von mehrzeiligem Markdown wird in Blöcke zerlegt. * Einfügen von mehrzeiligem Markdown wird in Blöcke zerlegt; `Strg+Z` / `Strg+Umschalt+Z` nimmt eigene Änderungen zurück bzw. stellt sie wieder her.
* **Teilen:** Notizen und ganze Ordner lassen sich Benutzern oder Gruppen **zum Lesen** freigeben. Empfänger sortieren sie in ihren eigenen Baum ein. * **Teilen:** Notizen und ganze Ordner lassen sich Benutzern oder Gruppen als **Bearbeiter** oder **Leser** freigeben. Empfänger sortieren sie in ihren eigenen Baum ein.
#### 🤝 Gemeinsam bearbeiten
* **Live:** Mehrere Personen schreiben gleichzeitig in derselben Notiz; Änderungen erscheinen bei den anderen in Bruchteilen einer Sekunde. Der Kopf der Notiz zeigt, wer gerade mitliest.
* **Kein Überschreiben:** Wer in einem Bullet schreibt, hält ihn kurz für sich; andere sehen ein Schloss mit Namen und arbeiten in den übrigen Bullets weiter. Kommt es doch zu einem Konflikt (z. B. nach Offline-Arbeit), bleibt der zweite Text als markierte Kopie mit „Übernehmen“/„Verwerfen“ erhalten.
* **Offline-fest:** Jede Eingabe wird zuerst im Browser gesichert und später übertragen, auch nach Neuladen oder Tagen ohne Netz.
* **Hinter strengen Proxys:** WebSocket → SSE → Long-Polling → Abfrage alle 3 s; MindSift wählt automatisch, was durchkommt. Der aktive Weg steht im Editor („Live“, „Abgleich alle 3 s“).
Technik dahinter: [Doku/modules/collaboration.md](Doku/modules/collaboration.md).
<p align="center"> <p align="center">
<img src="screenshots/workspace_desktop.png" alt="MindSift Workspace Outliner" width="90%" /> <img src="screenshots/workspace_desktop.png" alt="MindSift Workspace Outliner" width="90%" />
@@ -118,7 +128,7 @@ Minimal:
```yaml ```yaml
services: services:
mindsift: mindsift:
image: git.dsit.me/sandrodach/mindsift:dev image: git.dsit.me/sandrodach/mindsift:v0.2 # feste Version; :dev = neuester Entwicklungsstand
restart: unless-stopped restart: unless-stopped
ports: ["8080:8080"] ports: ["8080:8080"]
volumes: ["./data:/data"] volumes: ["./data:/data"]
@@ -146,11 +156,12 @@ Rufe `http://localhost:8080` auf. Beim ersten Start führt ein Assistent durch d
| `ALLOW_REGISTRATION` | `false` | Offene Registrierung (im Admin-UI änderbar) | | `ALLOW_REGISTRATION` | `false` | Offene Registrierung (im Admin-UI änderbar) |
| `LOG_LEVEL` / `DEBUG` | `info` / `false` | Logging | | `LOG_LEVEL` / `DEBUG` | `info` / `false` | Logging |
| `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASS`, `SMTP_FROM`, `SMTP_TLS` | leer | E-Mail-Alarme (auch im Admin-UI) | | `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASS`, `SMTP_FROM`, `SMTP_TLS` | leer | E-Mail-Alarme (auch im Admin-UI) |
| `MINDA_DEMO` | `false` | Legt Demo-Konten `admin/admin`, `user/user` an. **Nie auf öffentlichen Servern!** | | `MINDA_DEMO` | `false` | Legt Demo-Konten `admin/admin` (mit Beispieldaten) und `user/user` an; eine Notiz und zwei Listen sind zwischen beiden geteilt, um das gemeinsame Bearbeiten auszuprobieren. **Nie auf öffentlichen Servern!** |
* **Reverse-Proxy:** WebSocket-Upgrade und ungepufferte SSE-Streams müssen durchgereicht werden. Beispiele für Nginx, Traefik, Caddy und Cloudflare: [Doku/reverse-proxy.md](Doku/reverse-proxy.md). * **Reverse-Proxy:** Am besten WebSocket-Upgrade und ungepufferte SSE-Streams durchreichen. Blockiert ein Proxy beides, funktioniert MindSift trotzdem über Long-Polling bzw. regelmäßige Abfragen, nur etwas träger. Beispiele für Nginx, Traefik, Caddy und Cloudflare: [Doku/reverse-proxy.md](Doku/reverse-proxy.md).
* **Datensicherung:** Alle Daten liegen in `data/mindsift.db` (+ `-wal`/`-shm`). Sichere sie mit `sqlite3 mindsift.db ".backup 'backup.db'"` oder bei gestopptem Container. Zusätzlich gibt es einen JSON-Export pro Benutzer. * **Datensicherung:** Alle Daten liegen in `data/mindsift.db` (+ `-wal`/`-shm`). Sichere sie mit `sqlite3 mindsift.db ".backup 'backup.db'"` oder bei gestopptem Container. Zusätzlich gibt es einen JSON-Export pro Benutzer.
* **Updates:** Neues Image ziehen und Container neu starten (`docker compose pull && docker compose up -d`). Das Schema wird beim Start automatisch angepasst. * **Updates:** Neues Image ziehen und Container neu starten (`docker compose pull && docker compose up -d`). Das Schema wird beim Start automatisch angepasst. Die laufende Version steht im Kopf der App unter dem Namen.
* **Image-Tags:** Jeder Git-Tag erzeugt ein gleichnamiges Image (`:v0.2`), jeder Branch eines mit dem Branch-Namen (`:dev`).
--- ---
@@ -167,10 +178,14 @@ cd frontend
npm ci npm ci
npm run dev # Vite-Entwicklungsserver npm run dev # Vite-Entwicklungsserver
npx tsc --noEmit # Typprüfung npx tsc --noEmit # Typprüfung
npm run test:run # Komponententests (Vitest, jsdom)
npm run build # Produktions-Build npm run build # Produktions-Build
cd ..
e2e/run.sh # Browser-Tests (Playwright, zwei Benutzer) gegen ein frisch gebautes Image
APP_VERSION=v0.2 e2e/screenshots.sh # Screenshots dieser README neu erzeugen (Demo-Daten)
``` ```
Für das Binary wird das gebaute Frontend nach `cmd/server/dist` kopiert (`go:embed`); das mehrstufige `Dockerfile` erledigt das und führt Vet und Tests beim Image-Build aus. Für das Binary wird das gebaute Frontend nach `cmd/server/dist` kopiert (`go:embed`); das mehrstufige `Dockerfile` erledigt das und führt Typprüfung, Frontend-Tests, Go-Vet und Go-Tests beim Image-Build aus.
**Stack:** Go (chi, gorilla/websocket, JWT, modernc SQLite) · React 18, TypeScript, Vite, Tailwind. **Stack:** Go (chi, gorilla/websocket, JWT, modernc SQLite) · React 18, TypeScript, Vite, Tailwind.
@@ -178,7 +193,7 @@ Für das Binary wird das gebaute Frontend nach `cmd/server/dist` kopiert (`go:em
## 📚 Dokumentation ## 📚 Dokumentation
Die technische Doku liegt unter [Doku/](Doku/README.md): Architektur, Datenbank, REST- und Echtzeit-API, Module, Freigaben und Berechtigungen, Themes. Bekannte Schwachstellen und offene Punkte stehen in [Doku/flaws.md](Doku/flaws.md). Die technische Doku liegt unter [Doku/](Doku/README.md): Architektur, Datenbank, REST- und Echtzeit-API, Module (inkl. gemeinsamem Bearbeiten), Freigaben und Berechtigungen, Reverse-Proxy, Themes. Bekannte Schwachstellen und offene Punkte stehen in [Doku/flaws.md](Doku/flaws.md).
--- ---
+43
View File
@@ -0,0 +1,43 @@
# Ende-zu-Ende-Tests
Browser-Tests mit Playwright gegen einen echten MindSift-Server.
```bash
e2e/run.sh # baut das Image, startet es im Demo-Modus und führt die Tests aus
ENGINE=docker e2e/run.sh
```
Gegen einen bereits laufenden Server (Demo-Modus mit `admin/admin` und `user/user` nötig):
```bash
cd e2e && npm install && MINDSIFT_URL=http://localhost:8080 node collab.e2e.mjs
```
## collab.e2e.mjs
Zwei Browser (Eigentümer und Bearbeiter) arbeiten in derselben Notiz. Geprüft wird:
- Tippen wird gespeichert und erscheint beim anderen ohne Neuladen
- neue Bullets per Enter, in beide Richtungen
- Offline-Phase: Text bleibt lokal und wird nach Rückkehr übertragen
- beide Browser und der Server zeigen denselben Stand, auch nach Neuladen
- Sperren eines geschlossenen Browsers laufen nach spätestens 20 s ab
- Einrücken (Tab) und Zusammenführen (Backspace) bleiben nach Neuladen erhalten
## share.e2e.mjs
Der Eigentümer teilt eine Notiz über den Freigabe-Dialog mit Rolle „Bearbeiten“; geprüft wird, dass die Freigabe als `write` gespeichert, im Dialog als „Bearbeiter“ angezeigt wird und der Empfänger die Notiz bearbeiten kann. Zum Schluss hakt der Eigentümer ein To-Do in der To-Do-Übersicht ab.
## lock.e2e.mjs
Sperren, Präsenz, Rückgängig und Konfliktkopien mit zwei Browsern: Schloss und Schreibschutz am fremd bearbeiteten Bullet, Hinweis statt halb ausgeführter Aktion, `Strg+Z`/`Strg+Umschalt+Z`, Tippen direkt nach Enter, Markierung von Konfliktkopien sowie „Übernehmen“ (verweigert bei fremder Sperre, erfolgreich für den Sperrinhaber).
## channels.e2e.mjs
Der Eigentümer tippt mit normaler Verbindung, der Bearbeiter sitzt hinter einem simulierten Proxy. Für jede Situation wird der gewählte Weg und die Zeit bis zur Anzeige gemessen: nichts blockiert (WebSocket), WebSocket abgelehnt (SSE), WebSocket angenommen aber stumm (SSE nach Herzschlag-Ausfall), WebSocket und SSE blockiert sowie SSE hängend (Long-Polling), nur normale Anfragen (Abfrage alle 3 s).
`collab.e2e.mjs`, `lock.e2e.mjs` und `channels.e2e.mjs` dauern je etwa eine Minute. Die Tests laufen nicht in der CI, weil sie Browser-Images brauchen.
## Screenshots
`e2e/screenshots.sh` baut das Image, startet es im Demo-Modus mit frischer Datenbank und erzeugt die Bilder in `screenshots/` neu (`screenshots.mjs`). Für die Versionsangabe im Kopf: `APP_VERSION=v0.2 e2e/screenshots.sh`. In der Workspace-Aufnahme bearbeitet der Demo-Benutzer `user` gleichzeitig einen Bullet, damit Sperre und Präsenz zu sehen sind.
+119
View File
@@ -0,0 +1,119 @@
// How fast changes arrive behind different (simulated) proxies. The admin edits with a normal connection;
// the editor's browser blocks WebSocket, SSE and/or long polling. Needs a server in demo mode.
import { chromium } from 'playwright';
const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
let failures = 0;
const check = (cond, msg) => {
console.log(`${cond ? 'OK ' : 'FAIL'} ${msg}`);
if (!cond) failures++;
};
async function api(token, method, path, body) {
const res = await fetch(BASE + path, {
method,
headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: `Bearer ${token}` } : {}) },
body: body ? JSON.stringify(body) : undefined,
});
const text = await res.text();
if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`);
return text ? JSON.parse(text) : null;
}
async function waitFor(fn, ms) {
const end = Date.now() + ms;
while (Date.now() < end) {
try {
const v = await fn();
if (v) return v;
} catch {}
await sleep(100);
}
return null;
}
const editorValues = (page) =>
page.$$eval('textarea', (els) => els.filter((e) => !e.placeholder?.startsWith('Titel')).map((e) => e.value));
const channelOf = (page) => page.getAttribute('[data-sync-channel]', 'data-sync-channel');
async function openDoc(browser, token, title, block = {}) {
const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } });
// A blocking proxy refuses the connection (policy violation); a clean close would just mean "reconnect"
if (block.ws === 'reject') await ctx.routeWebSocket(/\/api\/v1\/ws/, (ws) => ws.close({ code: 1008, reason: 'blocked by proxy' }));
if (block.ws === 'silent') await ctx.routeWebSocket(/\/api\/v1\/ws/, () => {}); // accepted, nothing forwarded
if (block.sse === 'abort') await ctx.route('**/api/v1/events**', (r) => r.abort());
if (block.sse === 'hang') await ctx.route('**/api/v1/events**', () => {}); // never answered
if (block.longpoll) await ctx.route('**/changes**', (r) => r.abort());
await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token);
const page = await ctx.newPage();
await page.goto(BASE);
await page.getByRole('button', { name: /Workspace/ }).first().click();
await page.getByText(title, { exact: true }).first().click();
await page.waitForFunction(() => document.querySelectorAll('textarea').length > 1, null, { timeout: 10000 });
return { ctx, page };
}
async function typeAtEnd(page, text) {
await page.bringToFront();
// Place the cursor via script: Playwright's actionability checks wait for animation frames, which
// pages in the background do not get while the other browser window is in front
const ok = await page.evaluate(() => {
const el = Array.from(document.querySelectorAll('textarea')).find((e) => e.value.startsWith('Live-Test'));
if (!el) return false;
el.focus();
el.setSelectionRange(el.value.length, el.value.length);
return true;
});
if (!ok) throw new Error('Live-Test field not found');
await page.keyboard.type(text);
}
const TITLE = `Kanal-Test ${Date.now()}`;
const MODES = [
{ name: 'nichts blockiert', block: {}, channel: 'ws', maxMs: 2500 },
{ name: 'WebSocket-Handshake abgelehnt', block: { ws: 'reject' }, channel: 'sse', maxMs: 2500 },
{ name: 'WebSocket angenommen, leitet nichts weiter', block: { ws: 'silent' }, channel: 'sse', maxMs: 2500, switchMs: 60000 },
{ name: 'WebSocket und SSE blockiert', block: { ws: 'reject', sse: 'abort' }, channel: 'longpoll', maxMs: 2500 },
{ name: 'SSE-Verbindung hängt', block: { ws: 'reject', sse: 'hang' }, channel: 'longpoll', maxMs: 2500 },
{ name: 'nur normale Anfragen möglich', block: { ws: 'reject', sse: 'abort', longpoll: true }, channel: 'interval', maxMs: 6000, switchMs: 15000 },
];
const main = async () => {
const admin = (await api(null, 'POST', '/api/v1/auth/login', { login: 'admin', password: 'admin' })).token;
const user = (await api(null, 'POST', '/api/v1/auth/login', { login: 'user', password: 'user' })).token;
const dir = await api(admin, 'GET', '/api/v1/users/directory');
const userId = dir.find((u) => u.username === 'user' || u.email?.startsWith('user')).id;
const doc = await api(admin, 'POST', '/api/v1/documents', {
title: TITLE,
blocks: [{ id: `ch_${Date.now()}`, type: 'bullet', content: 'Live-Test', level: 0 }],
});
await api(admin, 'POST', `/api/v1/documents/${doc.id}/shares`, { userId, permission: 'write' });
const browser = await chromium.launch();
const A = await openDoc(browser, admin, TITLE);
for (const [i, mode] of MODES.entries()) {
const B = await openDoc(browser, user, TITLE, mode.block);
const channel = await waitFor(async () => ((await channelOf(B.page)) === mode.channel ? mode.channel : null), mode.switchMs || 10000);
check(channel === mode.channel, `${mode.name}: Weg ${(await channelOf(B.page)) || '?'} (erwartet ${mode.channel})`);
const marker = ` r${i}`;
const start = Date.now();
await typeAtEnd(A.page, marker);
const arrived = await waitFor(async () => (await editorValues(B.page)).some((v) => v.endsWith(marker)), mode.maxMs + 4000);
const ms = Date.now() - start;
check(!!arrived && ms <= mode.maxMs, `${mode.name}: Änderung kam nach ${arrived ? ms : '–'} ms an (Ziel ≤ ${mode.maxMs} ms)`);
await B.ctx.close();
}
await browser.close();
console.log(failures ? `${failures} FEHLGESCHLAGEN` : 'ALLE PRÜFUNGEN BESTANDEN');
process.exit(failures ? 1 : 0);
};
main().catch((e) => {
console.error(e);
process.exit(2);
});
+178
View File
@@ -0,0 +1,178 @@
// End-to-end test of collaborative note editing with two real browsers (owner and editor).
// Needs a server in demo mode (accounts admin/admin and user/user); see e2e/README.md.
import { chromium } from 'playwright';
const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
let failures = 0;
const check = (cond, msg) => {
console.log(`${cond ? 'OK ' : 'FAIL'} ${msg}`);
if (!cond) failures++;
};
async function api(token, method, path, body) {
const res = await fetch(BASE + path, {
method,
headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: `Bearer ${token}` } : {}) },
body: body ? JSON.stringify(body) : undefined,
});
const text = await res.text();
if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`);
return text ? JSON.parse(text) : null;
}
async function waitFor(fn, ms, label) {
const end = Date.now() + ms;
let last;
while (Date.now() < end) {
try {
last = await fn();
if (last) return last;
} catch {}
await sleep(250);
}
return null;
}
const serverBlocks = async (token, docId) => {
const snap = await api(token, 'GET', `/api/v1/docs/${docId}/snapshot`);
return snap.blocks.filter((b) => !b.deleted).map((b) => b.content);
};
// Outliner text fields of a page (without the title field)
const editorValues = (page) =>
page.$$eval('textarea', (els) => els.filter((e) => !e.placeholder?.startsWith('Titel')).map((e) => e.value));
async function openDoc(browser, token, title) {
const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } });
await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token);
const page = await ctx.newPage();
page.on('pageerror', (e) => console.log('PAGEERROR', e.message));
await page.goto(BASE);
await page.getByRole('button', { name: /Workspace/ }).first().click();
await page.getByText(title, { exact: true }).first().click();
await page.waitForFunction(() => document.querySelectorAll('textarea').length > 1, null, { timeout: 10000 });
return { ctx, page };
}
const TITLE = `E2E Notiz ${Date.now()}`;
const main = async () => {
const admin = (await api(null, 'POST', '/api/v1/auth/login', { login: 'admin', password: 'admin' })).token;
const user = (await api(null, 'POST', '/api/v1/auth/login', { login: 'user', password: 'user' })).token;
const dir = await api(admin, 'GET', '/api/v1/users/directory');
const userId = dir.find((u) => u.username === 'user' || u.email?.startsWith('user')).id;
const doc = await api(admin, 'POST', '/api/v1/documents', {
title: TITLE,
blocks: [{ id: 'e2e_1', type: 'bullet', content: 'Erster Punkt', level: 0 }],
});
await api(admin, 'POST', `/api/v1/documents/${doc.id}/shares`, { userId, permission: 'write' });
const browser = await chromium.launch();
const A = await openDoc(browser, admin, TITLE);
const B = await openDoc(browser, user, TITLE);
check((await editorValues(A.page)).includes('Erster Punkt'), 'Admin sieht die Notiz');
check((await editorValues(B.page)).includes('Erster Punkt'), 'Bearbeiter sieht die Notiz');
// 1. Admin types at the end of the first bullet
await A.page.getByText('Erster Punkt', { exact: true }).first().click();
await A.page.keyboard.press('End');
await A.page.keyboard.type(' + A');
const saved = await waitFor(async () => (await serverBlocks(admin, doc.id)).includes('Erster Punkt + A'), 5000);
check(!!saved, 'Getippter Text liegt auf dem Server');
const seenByB = await waitFor(async () => (await editorValues(B.page)).includes('Erster Punkt + A'), 8000);
check(!!seenByB, 'Bearbeiter sieht die Änderung ohne Neuladen');
// 2. Admin creates a new bullet with Enter
await A.page.keyboard.press('Enter');
await sleep(150); // a human does not type within the same frame as Enter
await A.page.keyboard.type('Zweiter von A');
const second = await waitFor(async () => (await editorValues(B.page)).includes('Zweiter von A'), 8000);
check(!!second, 'Neuer Bullet erscheint beim Bearbeiter');
// 3. Bearbeiter adds his own bullet at the end
await B.page.getByText('Zweiter von A', { exact: true }).first().click();
await B.page.keyboard.press('End');
await B.page.keyboard.press('Enter');
await sleep(150);
await B.page.keyboard.type('Von B');
const fromB = await waitFor(async () => (await editorValues(A.page)).includes('Von B'), 8000);
check(!!fromB, 'Bullet des Bearbeiters erscheint beim Admin');
// 4. Offline phase on the admin side
await A.ctx.setOffline(true);
await A.page.getByText('Erster Punkt + A', { exact: true }).first().click();
await A.page.keyboard.press('End');
await A.page.keyboard.type(' offline');
await sleep(1500);
check(!(await serverBlocks(admin, doc.id)).includes('Erster Punkt + A offline'), 'Offline getippt: noch nicht auf dem Server');
await A.ctx.setOffline(false);
await A.page.evaluate(() => window.dispatchEvent(new Event('online')));
const afterOnline = await waitFor(async () => (await serverBlocks(admin, doc.id)).includes('Erster Punkt + A offline'), 8000);
check(!!afterOnline, 'Nach Rückkehr online wird der Offline-Text übertragen');
// 5. Both converge and a reload shows the same state
await sleep(4000);
const final = await serverBlocks(admin, doc.id);
const a = await editorValues(A.page);
const b = await editorValues(B.page);
check(JSON.stringify(a) === JSON.stringify(b), `Beide Browser zeigen dasselbe: ${JSON.stringify(a)}`);
check(final.every((c) => a.includes(c)), `Server-Stand ist sichtbar: ${JSON.stringify(final)}`);
await A.page.reload();
await A.page.getByRole('button', { name: /Workspace/ }).first().click();
await A.page.getByText(TITLE, { exact: true }).first().click();
const reloaded = await waitFor(async () => {
const v = await editorValues(A.page);
return JSON.stringify(v) === JSON.stringify(a) ? v : null;
}, 8000);
check(!!reloaded, 'Nach Neuladen ist alles noch da');
// 6. Structure: indent with Tab, merge with Backspace, both survive a reload. B leaves first, so none of
// its block leases are active (with B's cursor in a block the server rightly rejects moving it).
await B.ctx.close();
// Closing a context abruptly may drop B's "leave" request; its leases then expire after 20 s
const freed = await waitFor(async () => {
const probe = await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'probe', lastSeq: 0 });
return probe.leases.others.length === 0;
}, 25000);
check(!!freed, 'Sperren eines geschlossenen Browsers laufen nach spätestens 20 s ab');
await sleep(3500); // the admin's browser learns about it with its next sync (every 3 s in shared notes)
await A.page.getByText('Von B', { exact: true }).first().click();
await A.page.keyboard.press('Home');
await A.page.keyboard.press('Tab');
await sleep(1500);
const legacy = await api(admin, 'GET', `/api/v1/documents/${doc.id}`);
const vonB = legacy.blocks.find((b) => b.content === 'Von B');
check(vonB && vonB.level === 1, `Tab rückt ein (Ebene ${vonB?.level})`);
await A.page.getByText('Zweiter von A', { exact: true }).first().click();
await A.page.keyboard.press('Home');
await A.page.keyboard.press('Backspace');
const merged = await waitFor(async () => (await serverBlocks(admin, doc.id)).includes('Erster Punkt + A offlineZweiter von A'), 5000);
check(!!merged, 'Backspace am Zeilenanfang führt mit dem Bullet darüber zusammen');
await A.page.reload();
await A.page.getByRole('button', { name: /Workspace/ }).first().click();
await A.page.getByText(TITLE, { exact: true }).first().click();
const afterStructure = await waitFor(async () => {
const v = await editorValues(A.page);
return v.includes('Erster Punkt + A offlineZweiter von A') && v.includes('Von B') && !v.includes('Zweiter von A') ? v : null;
}, 8000);
check(!!afterStructure, `Struktur nach Neuladen erhalten: ${JSON.stringify(afterStructure)}`);
const B2 = await openDoc(browser, user, TITLE);
const bSees = await waitFor(async () => {
const v = await editorValues(B2.page);
return v.includes('Erster Punkt + A offlineZweiter von A') && !v.includes('Zweiter von A') ? v : null;
}, 8000);
check(!!bSees, `Bearbeiter sieht nach erneutem Öffnen denselben Stand: ${JSON.stringify(bSees)}`);
const log = await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'e2e-check', lastSeq: 0 });
const conflicts = log.ops.filter((e) => e.effect.kind === 'insert' && e.effect.props?.conflictOf).length;
console.log(`Log: ${log.headSeq} Einträge, ${conflicts} Konfliktkopien`);
await browser.close();
console.log(failures ? `${failures} FEHLGESCHLAGEN` : 'ALLE PRÜFUNGEN BESTANDEN');
process.exit(failures ? 1 : 0);
};
main().catch((e) => {
console.error(e);
process.exit(2);
});
+170
View File
@@ -0,0 +1,170 @@
// Locks, presence, undo and conflict copies with two real browsers. Needs a server in demo mode
// (accounts admin/admin and user/user); see e2e/README.md.
import { chromium } from 'playwright';
const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
let failures = 0;
const check = (cond, msg) => {
console.log(`${cond ? 'OK ' : 'FAIL'} ${msg}`);
if (!cond) failures++;
};
async function api(token, method, path, body) {
const res = await fetch(BASE + path, {
method,
headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: `Bearer ${token}` } : {}) },
body: body ? JSON.stringify(body) : undefined,
});
const text = await res.text();
if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`);
return text ? JSON.parse(text) : null;
}
async function waitFor(fn, ms) {
const end = Date.now() + ms;
while (Date.now() < end) {
try {
const v = await fn();
if (v) return v;
} catch {}
await sleep(250);
}
return null;
}
const serverBlocks = async (token, docId) =>
(await api(token, 'GET', `/api/v1/docs/${docId}/snapshot`)).blocks.filter((b) => !b.deleted);
const editorValues = (page) =>
page.$$eval('textarea', (els) => els.filter((e) => !e.placeholder?.startsWith('Titel')).map((e) => e.value));
// Puts the cursor at the end of the text field showing `text` (works whether or not it is focused already)
async function focusField(page, text) {
const handle = await page.waitForFunction(
(t) => Array.from(document.querySelectorAll('textarea')).find((e) => e.value === t) || null,
text,
{ timeout: 5000, polling: 100 } // rAF polling stalls in background pages
);
const el = handle.asElement();
await el.click();
await page.keyboard.press('End');
}
async function openDoc(browser, token, title) {
const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } });
await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token);
const page = await ctx.newPage();
await page.goto(BASE);
await page.getByRole('button', { name: /Workspace/ }).first().click();
await page.getByText(title, { exact: true }).first().click();
await page.waitForFunction(() => document.querySelectorAll('textarea').length > 1, null, { timeout: 10000 });
return { ctx, page };
}
const TITLE = `Sperr-Test ${Date.now()}`;
const id = (n) => `lk_${n}_${Date.now()}`;
const main = async () => {
const admin = (await api(null, 'POST', '/api/v1/auth/login', { login: 'admin', password: 'admin' })).token;
const user = (await api(null, 'POST', '/api/v1/auth/login', { login: 'user', password: 'user' })).token;
const dir = await api(admin, 'GET', '/api/v1/users/directory');
const userId = dir.find((u) => u.username === 'user' || u.email?.startsWith('user')).id;
const ids = { a: id('a'), b: id('b') };
const doc = await api(admin, 'POST', '/api/v1/documents', {
title: TITLE,
blocks: [
{ id: ids.a, type: 'bullet', content: 'Admin schreibt hier', level: 0 },
{ id: ids.b, type: 'bullet', content: 'Darunter', level: 0 },
],
});
await api(admin, 'POST', `/api/v1/documents/${doc.id}/shares`, { userId, permission: 'write' });
const browser = await chromium.launch();
const A = await openDoc(browser, admin, TITLE);
const B = await openDoc(browser, user, TITLE);
// Presence
const presence = await waitFor(async () => B.page.getByTitle('Hat diese Notiz gerade geöffnet').innerText(), 8000);
check(!!presence && /Admin/i.test(presence), `Bearbeiter sieht, wer noch in der Notiz ist: ${presence}`);
// Admin clicks into the first bullet -> user sees the lock
await A.page.getByText('Admin schreibt hier', { exact: true }).first().click();
const lockBadge = await waitFor(async () => (await B.page.getByTitle(/bearbeitet diesen Punkt gerade/).count()) > 0, 10000);
check(!!lockBadge, 'Bearbeiter sieht das Schloss am gesperrten Bullet');
const readOnly = await B.page.$$eval('textarea', (els) => els.find((e) => e.value === 'Admin schreibt hier')?.readOnly);
check(readOnly === true, 'Gesperrter Bullet ist beim Bearbeiter schreibgeschützt');
// User tries to merge "Darunter" into the locked bullet
await B.page.getByText('Darunter', { exact: true }).first().click();
await B.page.keyboard.press('Home');
await B.page.keyboard.press('Backspace');
const notice = await waitFor(async () => B.page.getByRole('status').innerText(), 3000);
check(!!notice && /bearbeitet diesen Punkt gerade/.test(notice), `Hinweis statt halber Aktion: ${notice}`);
await sleep(1500);
const afterMerge = (await serverBlocks(admin, doc.id)).map((b) => b.content).sort();
check(JSON.stringify(afterMerge) === JSON.stringify(['Admin schreibt hier', 'Darunter']), `Nichts wurde teilweise ausgeführt: ${JSON.stringify(afterMerge)}`);
// Undo on the admin side
await focusField(A.page, 'Admin schreibt hier');
await A.page.keyboard.type(' und noch mehr');
await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Admin schreibt hier und noch mehr'), 5000);
await A.page.keyboard.press('Control+z');
const undone = await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Admin schreibt hier'), 5000);
check(!!undone, 'Strg+Z nimmt den getippten Text zurück (auch auf dem Server)');
const fieldUndone = await waitFor(async () => (await editorValues(A.page)).includes('Admin schreibt hier'), 3000);
check(!!fieldUndone, 'Das fokussierte Feld zeigt den zurückgenommenen Text');
await A.page.keyboard.press('Control+Shift+z');
const redone = await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Admin schreibt hier und noch mehr'), 5000);
check(!!redone, 'Strg+Umschalt+Z stellt ihn wieder her');
// Typing right after Enter lands in the new bullet (no pause)
await A.page.keyboard.press('End');
await A.page.keyboard.press('Enter');
await A.page.keyboard.type('Direkt nach Enter');
const newBullet = await waitFor(async () => (await serverBlocks(admin, doc.id)).some((b) => b.content === 'Direkt nach Enter'), 5000);
check(!!newBullet, 'Sofort nach Enter Getipptes landet im neuen Bullet');
// Conflict copy: two writes on the same base version via the API
const before = (await serverBlocks(admin, doc.id)).find((b) => b.id === ids.b);
await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'api-1', lastSeq: 0, ops: [{ opId: `${ids.b}-x`, kind: 'set_content', blockId: ids.b, baseVersion: before.version, content: 'Gewinner-Text' }] });
await api(user, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'api-2', lastSeq: 0, ops: [{ opId: `${ids.b}-y`, kind: 'set_content', blockId: ids.b, baseVersion: before.version, content: 'Verlierer-Text' }] });
const conflictShown = await waitFor(async () => (await A.page.getByText(/Version von/).count()) > 0, 8000);
check(!!conflictShown, 'Konfliktkopie wird markiert angezeigt');
// The user's cursor is still in the original bullet (it holds the lock), so the admin may not overwrite it
await A.page.getByRole('button', { name: 'Übernehmen' }).first().click();
const refused = await waitFor(async () => A.page.getByRole('status').innerText(), 3000);
check(!!refused && /bearbeitet/.test(refused), `Übernehmen in einen fremd gesperrten Punkt wird verweigert: ${refused}`);
// The user holding the original can accept once the copy's author (here a separate API client that never
// renews its lease) no longer holds the copy, i.e. after at most 20 s
await waitFor(async () => (await B.page.getByText(/Version von/).count()) > 0, 8000);
const copyId = (await serverBlocks(admin, doc.id)).find((b) => b.props?.conflictOf)?.id;
const copyFree = await waitFor(async () => {
const probe = await api(admin, 'POST', `/api/v1/docs/${doc.id}/sync`, { clientId: 'probe', lastSeq: 0 });
return !probe.leases.others.some((l) => l.blockId === copyId);
}, 25000);
check(!!copyFree, 'Sperre der Konfliktkopie läuft ab');
await sleep(3500); // the user's browser learns about it with its next sync
// Both API writes became conflict copies (the user's cursor locked the original); accept the first one
const copiesBefore = (await serverBlocks(admin, doc.id)).filter((b) => b.props?.conflictOf === ids.b);
const firstCopyText = await B.page.$$eval(
'textarea',
(els, texts) => els.map((e) => e.value).find((v) => texts.includes(v)),
copiesBefore.map((c) => c.content)
);
await B.page.getByRole('button', { name: 'Übernehmen' }).first().click();
const accepted = await waitFor(async () => {
const blocks = await serverBlocks(admin, doc.id);
const copiesNow = blocks.filter((b) => b.props?.conflictOf === ids.b);
return blocks.find((b) => b.id === ids.b)?.content === firstCopyText && copiesNow.length === copiesBefore.length - 1;
}, 6000);
check(!!accepted, `„Übernehmen“ ersetzt den ursprünglichen Text durch „${firstCopyText}“ und entfernt diese Kopie`);
await browser.close();
console.log(failures ? `${failures} FEHLGESCHLAGEN` : 'ALLE PRÜFUNGEN BESTANDEN');
process.exit(failures ? 1 : 0);
};
main().catch((e) => {
console.error(e);
process.exit(2);
});
+57
View File
@@ -0,0 +1,57 @@
{
"name": "mindsift-e2e",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "mindsift-e2e",
"dependencies": {
"playwright": "1.48.2"
}
},
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/playwright": {
"version": "1.48.2",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.48.2.tgz",
"integrity": "sha512-NjYvYgp4BPmiwfe31j4gHLa3J7bD2WiBz8Lk2RoSsmX38SVIARZ18VYjxLjAcDsAhA+F4iSEXTSGgjua0rrlgQ==",
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.48.2"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.48.2",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.48.2.tgz",
"integrity": "sha512-sjjw+qrLFlriJo64du+EK0kJgZzoQPsabGF4lBvsid+3CNIZIYLgnMj9V6JY5VhM2Peh20DJWIVpVljLLnlawA==",
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
}
}
}
+8
View File
@@ -0,0 +1,8 @@
{
"name": "mindsift-e2e",
"private": true,
"type": "module",
"dependencies": {
"playwright": "1.48.2"
}
}
Executable
+18
View File
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
# Builds MindSift, starts it in demo mode on port 18080 with a fresh database and runs the browser tests.
# Requires podman (or docker via ENGINE=docker). Usage: e2e/run.sh
set -euo pipefail
ENGINE="${ENGINE:-podman}"
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
WORK="$(mktemp -d)"
trap '"$ENGINE" rm -f mindsift-e2e >/dev/null 2>&1 || true; rm -rf "$WORK"' EXIT
echo "Baue Image …"
"$ENGINE" build -q -t mindsift-e2e-image "$ROOT" >/dev/null
"$ENGINE" run -d --name mindsift-e2e --network host -e PORT=18080 -e MINDA_DEMO=true \
-e JWT_SECRET=e2e-only-secret -v "$WORK":/data:Z mindsift-e2e-image >/dev/null
for _ in $(seq 1 60); do curl -s -m 1 http://localhost:18080/api/v1/health >/dev/null && break; sleep 0.5; done
echo "Starte Browser-Tests …"
"$ENGINE" run --rm --network host -v "$ROOT/e2e":/e2e:Z -w /e2e mcr.microsoft.com/playwright:v1.48.2-jammy \
sh -c 'npm install --no-audit --no-fund >/dev/null 2>&1 && node collab.e2e.mjs && node share.e2e.mjs && node lock.e2e.mjs && node channels.e2e.mjs'
+84
View File
@@ -0,0 +1,84 @@
// Takes the README screenshots from a fresh server in demo mode (sample data of admin/admin).
// Usage: e2e/screenshots.sh (builds the image) or MINDSIFT_URL=… SHOTS_DIR=… node screenshots.mjs
import { chromium } from 'playwright';
const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080';
const OUT = process.env.SHOTS_DIR || '../screenshots';
const DESKTOP = { viewport: { width: 1518, height: 886 }, deviceScaleFactor: 2 };
const MOBILE = { viewport: { width: 390, height: 844 }, deviceScaleFactor: 2, isMobile: true, hasTouch: true };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function login(user) {
const res = await fetch(`${BASE}/api/v1/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ login: user, password: user }),
});
if (!res.ok) throw new Error(`login ${user}: ${res.status}`);
return (await res.json()).token;
}
async function open(browser, token, options) {
const ctx = await browser.newContext({ ...options, colorScheme: 'dark', locale: 'de-DE' });
await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token);
const page = await ctx.newPage();
page.on('pageerror', (e) => console.log('PAGEERROR', e.message));
await page.goto(BASE);
await page.waitForLoadState('networkidle');
return page;
}
const tab = (page, name) => page.getByRole('button', { name: new RegExp(name) }).first().click();
async function shot(page, file) {
await sleep(800); // animations
await page.mouse.move(0, 0);
await page.screenshot({ path: `${OUT}/${file}` });
console.log(`OK ${file}`);
}
const main = async () => {
const admin = await login('admin');
const user = await login('user');
const browser = await chromium.launch({ args: ['--lang=de-DE'] });
const page = await open(browser, admin, DESKTOP);
await tab(page, 'Triage');
await shot(page, 'Triage_desktop.png');
await tab(page, 'Aufgaben');
await page.getByText('Homelab', { exact: true }).first().click().catch(() => {});
await page.getByRole('button', { name: /Kanban|Board/ }).first().click().catch(() => {});
await shot(page, 'aufgaben_desktop.png');
// The demo user edits a bullet of the shared note, so the screenshot shows lock and presence
const other = await open(browser, user, DESKTOP);
await tab(other, 'Workspace');
await other.getByText(/^Homelab 2\.0/).first().click();
await other.waitForFunction(() => document.querySelectorAll('textarea').length > 5, null, { timeout: 10000 });
await other.getByText('Endlich Platz für die Dienste', { exact: false }).first().click();
await other.keyboard.press('End');
await other.keyboard.type(': Paperless und Immich', { delay: 20 });
await tab(page, 'Workspace');
await page.getByText(/^Homelab 2\.0/).first().click();
await page.waitForFunction(() => document.querySelectorAll('textarea').length > 5, null, { timeout: 10000 });
await page.getByText('User', { exact: false }).first().waitFor({ timeout: 15000 }).catch(() => {});
await sleep(2000);
await shot(page, 'workspace_desktop.png');
await tab(page, 'Listen');
await page.getByText('Wocheneinkauf', { exact: true }).first().click().catch(() => {});
await shot(page, 'Listen_desktop.png');
const mobile = await open(browser, admin, MOBILE);
await mobile.getByText('Später im Browser weiter nutzen').click({ timeout: 5000 }).catch(() => {}); // install prompt
await shot(mobile, 'Mobile_Triage.png');
await browser.close();
};
main().catch((e) => {
console.error(e);
process.exit(1);
});
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
# Builds MindSift, starts it in demo mode with a fresh database and renews the README screenshots
# in screenshots/. Requires podman (or docker via ENGINE=docker). Usage: e2e/screenshots.sh
set -euo pipefail
ENGINE="${ENGINE:-podman}"
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
WORK="$(mktemp -d)"
trap '"$ENGINE" rm -f mindsift-shots >/dev/null 2>&1 || true; rm -rf "$WORK"' EXIT
echo "Baue Image …"
# Version in the header, e.g. APP_VERSION=v0.2 e2e/screenshots.sh
"$ENGINE" build -q --build-arg APP_VERSION="${APP_VERSION:-}" -t mindsift-e2e-image "$ROOT" >/dev/null
"$ENGINE" run -d --name mindsift-shots --network host -e PORT=18081 -e MINDA_DEMO=true \
-e JWT_SECRET=screenshots-only-secret -v "$WORK":/data:Z mindsift-e2e-image >/dev/null
for _ in $(seq 1 60); do curl -s -m 1 http://localhost:18081/api/v1/health >/dev/null && break; sleep 0.5; done
echo "Erstelle Screenshots …"
"$ENGINE" run --rm --network host -e MINDSIFT_URL=http://localhost:18081 -e SHOTS_DIR=/shots -e LANG=de_DE.UTF-8 \
-v "$ROOT/e2e":/e2e:Z -v "$ROOT/screenshots":/shots:Z -w /e2e mcr.microsoft.com/playwright:v1.48.2-jammy \
sh -c 'apt-get update -qq >/dev/null && apt-get install -y -qq fonts-inter >/dev/null 2>&1; npm install --no-audit --no-fund >/dev/null 2>&1 && node screenshots.mjs'
+125
View File
@@ -0,0 +1,125 @@
// Shares a note as editor through the share dialog and checks that the editor can then edit it.
// Needs a server in demo mode (accounts admin/admin and user/user); see e2e/README.md.
import { chromium } from 'playwright';
const BASE = process.env.MINDSIFT_URL || 'http://localhost:18080';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
let failures = 0;
const check = (cond, msg) => {
console.log(`${cond ? 'OK ' : 'FAIL'} ${msg}`);
if (!cond) failures++;
};
async function api(token, method, path, body) {
const res = await fetch(BASE + path, {
method,
headers: { 'Content-Type': 'application/json', ...(token ? { Authorization: `Bearer ${token}` } : {}) },
body: body ? JSON.stringify(body) : undefined,
});
const text = await res.text();
if (!res.ok) throw new Error(`${method} ${path}: ${res.status} ${text}`);
return text ? JSON.parse(text) : null;
}
async function waitFor(fn, ms) {
const end = Date.now() + ms;
while (Date.now() < end) {
try {
const v = await fn();
if (v) return v;
} catch {}
await sleep(250);
}
return null;
}
async function openDoc(browser, token, title) {
const ctx = await browser.newContext({ viewport: { width: 1280, height: 900 } });
await ctx.addInitScript((t) => localStorage.setItem('mindsift_jwt_token', t), token);
const page = await ctx.newPage();
await page.goto(BASE);
await page.getByRole('button', { name: /Workspace/ }).first().click();
await page.getByText(title, { exact: true }).first().click();
await page.waitForFunction(() => document.querySelectorAll('textarea').length > 1, null, { timeout: 10000 });
return { ctx, page };
}
const TITLE = `Freigabe-Test ${Date.now()}`;
const TODO_ID = `fr_todo_${Date.now()}`;
const TODO_TEXT = `Rollup-Aufgabe ${Date.now()}`;
const main = async () => {
const admin = (await api(null, 'POST', '/api/v1/auth/login', { login: 'admin', password: 'admin' })).token;
const user = (await api(null, 'POST', '/api/v1/auth/login', { login: 'user', password: 'user' })).token;
const doc = await api(admin, 'POST', '/api/v1/documents', {
title: TITLE,
blocks: [
{ id: `fr_${Date.now()}`, type: 'bullet', content: 'Vom Eigentümer', level: 0 },
{ id: TODO_ID, type: 'todo', content: TODO_TEXT, level: 0 },
],
});
const browser = await chromium.launch();
const A = await openDoc(browser, admin, TITLE);
// Share as editor through the dialog
await A.page.getByTitle('Notiz teilen und Berechtigungen').click();
await A.page.getByRole('button', { name: 'Benutzer', exact: true }).click();
const search = A.page.getByPlaceholder(/Benutzer suchen/);
await search.click();
await search.type('user');
await sleep(300);
await search.press('ArrowDown');
await search.press('Enter');
const roleSelect = A.page.locator('select').filter({ has: A.page.locator('option[value="editor"]') }).last();
check((await roleSelect.isEnabled()) && (await roleSelect.isVisible()), 'Rollenauswahl ist im Dialog verfügbar');
await roleSelect.selectOption('editor');
await A.page.getByRole('button', { name: /Benutzer hinzufügen/ }).click();
const shares = await waitFor(async () => {
const list = await api(admin, 'GET', `/api/v1/documents/${doc.id}/shares`);
return list.length > 0 ? list : null;
}, 5000);
check(shares && shares[0].permission === 'write', `Freigabe gespeichert als ${shares?.[0]?.permission}`);
const badge = await A.page
.getByText('Bearbeiter', { exact: true })
.first()
.waitFor({ state: 'visible', timeout: 5000 })
.then(() => true)
.catch(() => false);
check(badge, 'Dialog zeigt die Rolle „Bearbeiter“');
// The editor can edit
const snap = await api(user, 'GET', `/api/v1/docs/${doc.id}/snapshot`);
check(snap.role === 'editor', `Empfänger hat die Rolle ${snap.role}`);
const B = await openDoc(browser, user, TITLE);
check(!(await B.page.getByText('schreibgeschützt mit dir geteilt', { exact: false }).first().isVisible().catch(() => false)), 'Kein Nur-Lesen-Hinweis beim Bearbeiter');
await B.page.getByText('Vom Eigentümer', { exact: true }).first().click();
await B.page.keyboard.press('End');
await B.page.keyboard.type(' und vom Bearbeiter');
const saved = await waitFor(async () => {
const s = await api(admin, 'GET', `/api/v1/docs/${doc.id}/snapshot`);
return s.blocks.some((b) => b.content === 'Vom Eigentümer und vom Bearbeiter');
}, 6000);
check(!!saved, 'Bearbeiter kann die geteilte Notiz bearbeiten');
// Ticking a to-do in the overview goes through the sync endpoint as well
await A.page.bringToFront();
await A.page.getByRole('button', { name: 'Schließen', exact: true }).last().click(); // the share dialog is still open
await A.page.locator('button:visible', { hasText: 'To-Dos' }).first().click();
await A.page.getByTitle('Alle To-Dos als einfache Liste').click();
await A.page.locator('div', { hasText: TODO_TEXT }).last().locator('button').first().click();
const ticked = await waitFor(async () => {
const s = await api(admin, 'GET', `/api/v1/docs/${doc.id}/snapshot`);
return s.blocks.find((b) => b.id === TODO_ID)?.props?.checked === true;
}, 5000);
check(!!ticked, 'Abhaken in der To-Do-Übersicht wird gespeichert');
await browser.close();
console.log(failures ? `${failures} FEHLGESCHLAGEN` : 'ALLE PRÜFUNGEN BESTANDEN');
process.exit(failures ? 1 : 0);
};
main().catch((e) => {
console.error(e);
process.exit(2);
});
+1299 -63
View File
File diff suppressed because it is too large. Load diff
+9 -3
View File
@@ -1,14 +1,15 @@
{ {
"name": "mindsift-frontend", "name": "mindsift-frontend",
"private": true, "private": true,
"version": "1.0.0", "version": "0.2.0",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite", "dev": "vite",
"build": "vite build", "build": "vite build",
"preview": "vite preview", "preview": "vite preview",
"lint": "echo 'Linting passed'", "lint": "echo 'Linting passed'",
"test:run": "echo 'Frontend checks passed'" "test": "vitest",
"test:run": "vitest run"
}, },
"dependencies": { "dependencies": {
"@types/prismjs": "^1.26.6", "@types/prismjs": "^1.26.6",
@@ -19,13 +20,18 @@
"react-dom": "^18.3.1" "react-dom": "^18.3.1"
}, },
"devDependencies": { "devDependencies": {
"@testing-library/dom": "^10.4.2",
"@testing-library/react": "^16.3.3",
"@types/react": "^18.3.12", "@types/react": "^18.3.12",
"@types/react-dom": "^18.3.1", "@types/react-dom": "^18.3.1",
"@vitejs/plugin-react": "^4.3.4", "@vitejs/plugin-react": "^4.3.4",
"autoprefixer": "^10.4.20", "autoprefixer": "^10.4.20",
"fake-indexeddb": "^6.2.6",
"jsdom": "^25.0.1",
"postcss": "^8.4.49", "postcss": "^8.4.49",
"tailwindcss": "^3.4.17", "tailwindcss": "^3.4.17",
"typescript": "^5.6.3", "typescript": "^5.6.3",
"vite": "^5.4.11" "vite": "^5.4.11",
"vitest": "^2.1.9"
} }
} }
+3 -2
View File
@@ -52,6 +52,7 @@ import { GroupsView } from './features/teams/GroupsView';
import { useMobileActions } from './context/MobileActionContext'; import { useMobileActions } from './context/MobileActionContext';
import { MobileHeaderMenu } from './components/MobileHeaderMenu'; import { MobileHeaderMenu } from './components/MobileHeaderMenu';
import { UserAvatar } from './components/UserAvatar'; import { UserAvatar } from './components/UserAvatar';
import { APP_VERSION } from './version';
type ActiveTab = type ActiveTab =
| 'triage' | 'triage'
@@ -648,7 +649,7 @@ export const App: React.FC = () => {
<span className="font-bold text-base text-main tracking-tight leading-none"> <span className="font-bold text-base text-main tracking-tight leading-none">
{authStatus.instanceName || 'MindSift'} {authStatus.instanceName || 'MindSift'}
</span> </span>
<span className="text-[9px] text-muted font-mono leading-tight mt-0.5">v0.1</span> <span className="text-[9px] text-muted font-mono leading-tight mt-0.5">{APP_VERSION}</span>
</div> </div>
<span className="text-[11px] font-semibold text-accent bg-accent/15 px-1.5 py-0.5 rounded-md"> <span className="text-[11px] font-semibold text-accent bg-accent/15 px-1.5 py-0.5 rounded-md">
{activeTabLabelMap[activeTab]} {activeTabLabelMap[activeTab]}
@@ -713,7 +714,7 @@ export const App: React.FC = () => {
<span className="font-bold text-sm sm:text-base text-main tracking-tight leading-none"> <span className="font-bold text-sm sm:text-base text-main tracking-tight leading-none">
{authStatus.instanceName || 'MindSift'} {authStatus.instanceName || 'MindSift'}
</span> </span>
<span className="text-[10px] text-muted font-mono leading-tight mt-0.5">v0.1</span> <span className="text-[10px] text-muted font-mono leading-tight mt-0.5">{APP_VERSION}</span>
</div> </div>
</div> </div>
@@ -220,6 +220,9 @@ export const TasksView: React.FC<TasksViewProps> = ({
} }
} else if (msg.event === 'TASK_REORDERED') { } else if (msg.event === 'TASK_REORDERED') {
loadData(); loadData();
} else if (msg.event === 'TASK_LIST_UPDATED' || msg.event === 'TASK_LIST_DELETED') {
// Payload only carries the list ID; reload so permissions reflect this user's view
api.getTaskLists().then((lists) => setTaskLists(Array.isArray(lists) ? lists : [])).catch(() => {});
} }
}; };
+217 -184
View File
@@ -16,7 +16,8 @@ import {
Search, Search,
} from 'lucide-react'; } from 'lucide-react';
import { Document, DocumentBlock, User } from '../../lib/types'; import { Document, DocumentBlock, User } from '../../lib/types';
import { api, getToken } from '../../lib/api'; import { api } from '../../lib/api';
import { DocSyncEngine, EngineState } from '../../lib/docSync/engine';
import { OutlinerTree, OutlinerTreeHandle } from './OutlinerTree'; import { OutlinerTree, OutlinerTreeHandle } from './OutlinerTree';
export type DocSaveStatus = 'saved' | 'saving' | 'unsaved' | 'error'; export type DocSaveStatus = 'saved' | 'saving' | 'unsaved' | 'error';
@@ -119,6 +120,29 @@ function ensureNonEmptyBlocks(docId: string, blocks?: DocumentBlock[]): Document
})); }));
} }
// True if both lists show the same blocks; avoids re-rendering when the sync engine echoes a local change.
function sameBlocks(a: DocumentBlock[], b: DocumentBlock[]): boolean {
if (a.length !== b.length) return false;
for (let i = 0; i < a.length; i++) {
const x = a[i];
const y = b[i];
if (
x.id !== y.id ||
x.content !== y.content ||
(x.level || 0) !== (y.level || 0) ||
x.type !== y.type ||
!!x.checked !== !!y.checked ||
!!x.collapsed !== !!y.collapsed ||
(x.lang || '') !== (y.lang || '') ||
(x.taskId || '') !== (y.taskId || '') ||
(x.conflictOf || '') !== (y.conflictOf || '')
) {
return false;
}
}
return true;
}
export interface BreadcrumbNode { export interface BreadcrumbNode {
id: string | null; id: string | null;
title: string; title: string;
@@ -224,6 +248,10 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
const [emojiSearch, setEmojiSearch] = useState<string>(''); const [emojiSearch, setEmojiSearch] = useState<string>('');
const [zoomedBlockId, setZoomedBlockId] = useState<string | null>(null); const [zoomedBlockId, setZoomedBlockId] = useState<string | null>(null);
// Sync engine (operation log): stores every change locally and syncs it with the server
const engineRef = useRef<DocSyncEngine | null>(null);
const [engineState, setEngineState] = useState<EngineState | null>(null);
const filteredEmojis = useMemo(() => { const filteredEmojis = useMemo(() => {
const q = emojiSearch.trim().toLowerCase(); const q = emojiSearch.trim().toLowerCase();
if (!q) return EMOJI_CATALOG; if (!q) return EMOJI_CATALOG;
@@ -232,13 +260,6 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
); );
}, [emojiSearch]); }, [emojiSearch]);
// AutoSave state (REST persistence)
const [, setSaveStatus] = useState<DocSaveStatus>('saved');
const [, setSaveErrorMessage] = useState<string | null>(null);
const saveTimeoutRef = useRef<number | null>(null);
const isSavingRef = useRef<boolean>(false);
const lastSavedSnapshotRef = useRef<string>('');
const isInitialLoadRef = useRef<boolean>(true);
const titleInputRef = useRef<HTMLTextAreaElement | HTMLInputElement | null>(null); const titleInputRef = useRef<HTMLTextAreaElement | HTMLInputElement | null>(null);
const outlinerTreeRef = useRef<OutlinerTreeHandle | null>(null); const outlinerTreeRef = useRef<OutlinerTreeHandle | null>(null);
@@ -249,7 +270,6 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
currentTitleRef.current = title; currentTitleRef.current = title;
const currentBlocksRef = useRef<DocumentBlock[]>(blocks); const currentBlocksRef = useRef<DocumentBlock[]>(blocks);
currentBlocksRef.current = blocks; currentBlocksRef.current = blocks;
const lastKnownUpdatedAtRef = useRef<string | null>(initialDoc?.updatedAt || null);
useEffect(() => { useEffect(() => {
if (!loading && (autoFocusTitle || (doc?.title === 'Neue Notiz' && (!doc?.blocks || doc?.blocks.length <= 1)))) { if (!loading && (autoFocusTitle || (doc?.title === 'Neue Notiz' && (!doc?.blocks || doc?.blocks.length <= 1)))) {
@@ -283,11 +303,12 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
return computeBreadcrumbs(title || doc?.title || 'Dokument', blocks, zoomedBlockId); return computeBreadcrumbs(title || doc?.title || 'Dokument', blocks, zoomedBlockId);
}, [title, doc?.title, blocks, zoomedBlockId]); }, [title, doc?.title, blocks, zoomedBlockId]);
// Strictly Read-Only for shared documents: only the document owner can edit! // Owner and editors may edit; the role comes from the server via the sync engine
const canEdit = useMemo(() => { const canEdit = useMemo(() => {
if (!doc) return true; if (engineState?.role) return engineState.role === 'owner' || engineState.role === 'editor';
return Boolean(doc.isOwner || doc.permission === 'owner'); if (!doc) return false;
}, [doc]); return Boolean(doc.isOwner || doc.permission === 'owner' || doc.permission === 'write');
}, [doc, engineState?.role]);
// Extract all #tags present across the current blocks // Extract all #tags present across the current blocks
const extractedTags = useMemo(() => { const extractedTags = useMemo(() => {
@@ -303,124 +324,6 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
return Array.from(tagSet); return Array.from(tagSet);
}, [blocks]); }, [blocks]);
const currentTagsRef = useRef<string[]>(extractedTags);
currentTagsRef.current = extractedTags;
// AutoSave snapshot
const currentSnapshot = useMemo(() => {
return JSON.stringify({
title,
blocks,
tags: extractedTags,
});
}, [title, blocks, extractedTags]);
const currentSnapshotRef = useRef<string>(currentSnapshot);
currentSnapshotRef.current = currentSnapshot;
const executeAutoSave = useCallback(
async (immediate: boolean = false) => {
if (!docId || !canEdit) return;
if (isSavingRef.current && !immediate) return;
const snapshot = currentSnapshotRef.current;
if (snapshot === lastSavedSnapshotRef.current) {
setSaveStatus('saved');
onSaveStatusChange?.('saved');
return;
}
isSavingRef.current = true;
setSaveStatus('saving');
onSaveStatusChange?.('saving');
try {
const updated = await api.updateDocument(docId, {
title: currentTitleRef.current,
blocks: currentBlocksRef.current,
tags: currentTagsRef.current,
});
isSavingRef.current = false;
lastSavedSnapshotRef.current = snapshot;
lastKnownUpdatedAtRef.current = updated.updatedAt || null;
setSaveStatus('saved');
setSaveErrorMessage(null);
onSaveStatusChange?.('saved');
if (docRef.current) {
docRef.current = updated;
onDocMetadataChange(updated);
}
} catch (err: any) {
isSavingRef.current = false;
const msg = err?.message || 'Fehler beim Speichern';
setSaveStatus('error');
setSaveErrorMessage(msg);
onSaveStatusChange?.('error', msg);
}
},
[docId, canEdit, onSaveStatusChange, onDocMetadataChange]
);
useEffect(() => {
if (isInitialLoadRef.current) return;
if (!canEdit) return;
if (currentSnapshot === lastSavedSnapshotRef.current) return;
setSaveStatus('unsaved');
onSaveStatusChange?.('unsaved');
if (saveTimeoutRef.current !== null) {
window.clearTimeout(saveTimeoutRef.current);
}
saveTimeoutRef.current = window.setTimeout(() => {
saveTimeoutRef.current = null;
executeAutoSave();
}, 800);
return () => {
if (saveTimeoutRef.current !== null) {
window.clearTimeout(saveTimeoutRef.current);
saveTimeoutRef.current = null;
}
};
}, [currentSnapshot, canEdit, executeAutoSave, onSaveStatusChange]);
// Flush on page unload / pagehide via keepalive fetch
useEffect(() => {
const handleUnload = () => {
if (saveTimeoutRef.current !== null) {
window.clearTimeout(saveTimeoutRef.current);
saveTimeoutRef.current = null;
}
const snapshot = currentSnapshotRef.current;
if (snapshot !== lastSavedSnapshotRef.current && canEdit) {
const token = getToken();
if (token) {
fetch(`/api/v1/documents/${docId}`, {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`,
},
body: JSON.stringify({
title: currentTitleRef.current,
blocks: currentBlocksRef.current,
tags: currentTagsRef.current,
}),
keepalive: true,
}).catch(() => {});
}
}
};
window.addEventListener('beforeunload', handleUnload);
window.addEventListener('pagehide', handleUnload);
return () => {
window.removeEventListener('beforeunload', handleUnload);
window.removeEventListener('pagehide', handleUnload);
handleUnload();
};
}, [docId, canEdit]);
// Fetch document on mount // Fetch document on mount
useEffect(() => { useEffect(() => {
@@ -436,29 +339,19 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
if (isCancelled) return; if (isCancelled) return;
const normalized = ensureNonEmptyBlocks(docId, fetched.blocks); const normalized = ensureNonEmptyBlocks(docId, fetched.blocks);
setDoc(fetched); setDoc(fetched);
if (!engineRef.current?.getState().ready) {
setTitle(fetched.title || 'Unbenannte Notiz'); setTitle(fetched.title || 'Unbenannte Notiz');
currentBlocksRef.current = normalized;
setBlocks(normalized); setBlocks(normalized);
}
const links = new Map<string, string>();
for (const b of fetched.blocks || []) {
if (b.taskId) links.set(b.id, b.taskId);
}
engineRef.current?.setTaskLinks(links);
setLoading(false); setLoading(false);
onDocMetadataChange(fetched); onDocMetadataChange(fetched);
const initialTags: string[] = [];
for (const b of normalized) {
const matches = b.content?.match(/#([a-zA-Z0-9äöüÄÖÜß_-]+)/g);
if (matches) {
for (const m of matches) {
const tag = m.slice(1).toLowerCase();
if (!initialTags.includes(tag)) initialTags.push(tag);
}
}
}
lastSavedSnapshotRef.current = JSON.stringify({
title: fetched.title || 'Unbenannte Notiz',
blocks: normalized,
tags: initialTags,
});
isInitialLoadRef.current = false;
setSaveStatus('saved');
onSaveStatusChange?.('saved');
}) })
.catch((err: any) => { .catch((err: any) => {
if (isCancelled) return; if (isCancelled) return;
@@ -471,6 +364,104 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
}; };
}, [docId]); }, [docId]);
// Start the sync engine for this note: shows stored state immediately (also offline), then syncs
useEffect(() => {
const engine = new DocSyncEngine(docId);
engineRef.current = engine;
const unsubscribe = engine.subscribe((state) => {
setEngineState(state);
if (!state.ready) return;
const next = ensureNonEmptyBlocks(docId, state.blocks);
if (!sameBlocks(currentBlocksRef.current, next)) {
currentBlocksRef.current = next;
setBlocks(next);
}
if (state.title && state.title !== currentTitleRef.current) {
setTitle(state.title);
}
});
void engine.start();
return () => {
unsubscribe();
engine.stop();
if (engineRef.current === engine) engineRef.current = null;
};
}, [docId]);
useEffect(() => {
if (!engineState) return;
switch (engineState.status) {
case 'saving':
onSaveStatusChange?.('saving');
break;
case 'offline':
onSaveStatusChange?.('unsaved', 'Offline – Änderungen sind lokal gesichert und werden gesendet, sobald die Verbindung zurück ist');
break;
case 'error':
onSaveStatusChange?.('error', engineState.errorMessage);
break;
default:
onSaveStatusChange?.('saved');
}
}, [engineState?.status, engineState?.errorMessage]);
// Blocks other users are editing right now (blockId -> name)
const lockedBlocks = useMemo(() => {
const m = new Map<string, string>();
for (const l of engineState?.lockedByOthers || []) m.set(l.blockId, l.name || 'Jemand');
return m;
}, [engineState?.lockedByOthers]);
const otherPeople = useMemo(
() => (engineState?.presence || []).filter((p) => p.userId !== currentUser?.id),
[engineState?.presence, currentUser?.id]
);
// Short hint at the bottom of the editor (blocked actions, skipped undo)
const [notice, setNotice] = useState<string | null>(null);
const noticeTimerRef = useRef<number | null>(null);
const showNotice = useCallback((message: string) => {
setNotice(message);
if (noticeTimerRef.current !== null) window.clearTimeout(noticeTimerRef.current);
noticeTimerRef.current = window.setTimeout(() => setNotice(null), 3500);
}, []);
useEffect(() => () => {
if (noticeTimerRef.current !== null) window.clearTimeout(noticeTimerRef.current);
}, []);
const handleLockedAction = useCallback(
(holder: string) => showNotice(`${holder} bearbeitet diesen Punkt gerade – bitte kurz warten`),
[showNotice]
);
const handleUndo = useCallback(() => {
const res = engineRef.current?.undo();
if (res?.message) showNotice(res.message);
}, [showNotice]);
const handleRedo = useCallback(() => {
const res = engineRef.current?.redo();
if (res?.message) showNotice(res.message);
}, [showNotice]);
const handleResolveConflict = useCallback(
(blockId: string, accept: boolean) => {
const res = engineRef.current?.resolveConflict(blockId, accept);
if (res?.message) showNotice(res.message);
},
[showNotice]
);
// Every local block change goes to the engine, which turns it into operations
const applyLocalBlocks = useCallback(
(update: DocumentBlock[] | ((prev: DocumentBlock[]) => DocumentBlock[])) => {
const prev = currentBlocksRef.current;
const next = typeof update === 'function' ? update(prev) : update;
if (next === prev) return;
currentBlocksRef.current = next;
setBlocks(next);
engineRef.current?.applyFlat(next);
},
[]
);
// Keep parent sidebar metadata in sync when blocks/tags change // Keep parent sidebar metadata in sync when blocks/tags change
useEffect(() => { useEffect(() => {
if (!docRef.current) return; if (!docRef.current) return;
@@ -488,27 +479,19 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
const handleTitleChange = (e: React.ChangeEvent<HTMLInputElement | HTMLTextAreaElement>) => { const handleTitleChange = (e: React.ChangeEvent<HTMLInputElement | HTMLTextAreaElement>) => {
if (!canEdit) return; if (!canEdit) return;
setTitle(e.target.value); setTitle(e.target.value);
engineRef.current?.setTitle(e.target.value);
}; };
const handleTitleBlur = () => { const handleTitleBlur = () => {
executeAutoSave(); void engineRef.current?.syncNow();
}; };
const handleUpdateBlock = useCallback( const handleUpdateBlock = useCallback(
(updatedBlock: DocumentBlock, immediate: boolean = false) => { (updatedBlock: DocumentBlock) => {
if (!canEdit) return; if (!canEdit) return;
setBlocks((prev) => { applyLocalBlocks((prev) => prev.map((b) => (b.id === updatedBlock.id ? updatedBlock : b)));
const next = prev.map((b) => (b.id === updatedBlock.id ? updatedBlock : b));
currentBlocksRef.current = next;
return next;
});
if (immediate) {
window.setTimeout(() => {
executeAutoSave(true);
}, 10);
}
}, },
[canEdit, executeAutoSave] [canEdit, applyLocalBlocks]
); );
const handleUpdateBlocks = useCallback( const handleUpdateBlocks = useCallback(
@@ -518,17 +501,15 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
for (const b of updatedBlocks) { for (const b of updatedBlocks) {
map.set(b.id, b); map.set(b.id, b);
} }
setBlocks((prev) => applyLocalBlocks((prev) => prev.map((b) => map.get(b.id) || b));
prev.map((b) => map.get(b.id) || b)
);
}, },
[canEdit] [canEdit, applyLocalBlocks]
); );
const handleCreateBlock = useCallback( const handleCreateBlock = useCallback(
(newBlock: DocumentBlock, afterBlockId?: string) => { (newBlock: DocumentBlock, afterBlockId?: string) => {
if (!canEdit) return; if (!canEdit) return;
setBlocks((prev) => { applyLocalBlocks((prev) => {
const next = [...prev]; const next = [...prev];
if (afterBlockId) { if (afterBlockId) {
const idx = next.findIndex((b) => b.id === afterBlockId); const idx = next.findIndex((b) => b.id === afterBlockId);
@@ -541,13 +522,13 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
return ensureNonEmptyBlocks(docId, next); return ensureNonEmptyBlocks(docId, next);
}); });
}, },
[docId, canEdit] [docId, canEdit, applyLocalBlocks]
); );
const handleInsertBlocks = useCallback( const handleInsertBlocks = useCallback(
(afterBlockId: string, updatedCurrentBlock: DocumentBlock, newBlocks: DocumentBlock[]) => { (afterBlockId: string, updatedCurrentBlock: DocumentBlock, newBlocks: DocumentBlock[]) => {
if (!canEdit) return; if (!canEdit) return;
setBlocks((prev) => { applyLocalBlocks((prev) => {
const next = [...prev]; const next = [...prev];
const idx = next.findIndex((b) => b.id === afterBlockId); const idx = next.findIndex((b) => b.id === afterBlockId);
if (idx >= 0) { if (idx >= 0) {
@@ -556,32 +537,24 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
} else { } else {
next.push(updatedCurrentBlock, ...newBlocks); next.push(updatedCurrentBlock, ...newBlocks);
} }
const normalized = ensureNonEmptyBlocks(docId, next); return ensureNonEmptyBlocks(docId, next);
currentBlocksRef.current = normalized;
return normalized;
}); });
window.setTimeout(() => {
executeAutoSave(true);
}, 50);
}, },
[docId, canEdit, executeAutoSave] [docId, canEdit, applyLocalBlocks]
); );
const handleDeleteBlock = useCallback( const handleDeleteBlock = useCallback(
(blockId: string) => { (blockId: string) => {
if (!canEdit) return; if (!canEdit) return;
setBlocks((prev) => { applyLocalBlocks((prev) => ensureNonEmptyBlocks(docId, prev.filter((b) => b.id !== blockId)));
const filtered = prev.filter((b) => b.id !== blockId);
return ensureNonEmptyBlocks(docId, filtered);
});
}, },
[docId, canEdit] [docId, canEdit, applyLocalBlocks]
); );
const handleMoveBlock = useCallback( const handleMoveBlock = useCallback(
(blockId: string, targetIndex: number, newLevel?: number) => { (blockId: string, targetIndex: number, newLevel?: number) => {
if (!canEdit) return; if (!canEdit) return;
setBlocks((prev) => { applyLocalBlocks((prev) => {
const fromIdx = prev.findIndex((b) => b.id === blockId); const fromIdx = prev.findIndex((b) => b.id === blockId);
if (fromIdx === -1) return prev; if (fromIdx === -1) return prev;
@@ -632,7 +605,7 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
return ensureNonEmptyBlocks(docId, copy); return ensureNonEmptyBlocks(docId, copy);
}); });
}, },
[docId, canEdit] [docId, canEdit, applyLocalBlocks]
); );
const handleTogglePin = async () => { const handleTogglePin = async () => {
@@ -714,7 +687,7 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
} }
return ( return (
<div className="flex-1 flex flex-col h-full min-h-0 bg-canvas overflow-y-auto"> <div className="flex-1 flex flex-col h-full min-h-0 bg-canvas overflow-y-auto" data-sync-channel={engineState?.channel}>
{/* Top Document Action Bar */} {/* Top Document Action Bar */}
<div className="sticky top-0 z-20 flex flex-wrap items-center justify-between gap-2 px-4 sm:px-6 py-2 bg-canvas border-b border-border"> <div className="sticky top-0 z-20 flex flex-wrap items-center justify-between gap-2 px-4 sm:px-6 py-2 bg-canvas border-b border-border">
<div className="flex items-center gap-2 min-w-0"> <div className="flex items-center gap-2 min-w-0">
@@ -793,6 +766,39 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
{/* Right Action Buttons */} {/* Right Action Buttons */}
<div className="flex items-center gap-1.5"> <div className="flex items-center gap-1.5">
{/* How changes of others arrive (only relevant for shared notes) */}
{engineState && (engineState.shared || engineState.sessions > 1) && (
<span
className="hidden sm:inline-flex items-center gap-1 px-2 py-0.5 rounded-btn text-[11px] text-muted bg-panel border border-border"
title={
engineState.channel === 'ws'
? 'Live-Verbindung über WebSocket'
: engineState.channel === 'sse'
? 'Live-Verbindung über Server-Sent Events (WebSocket wird blockiert)'
: engineState.channel === 'longpoll'
? 'Live über Long-Polling (WebSocket und SSE werden blockiert)'
: 'Keine Live-Verbindung möglich – Änderungen werden alle 3 Sekunden abgefragt'
}
>
<span
className={`w-1.5 h-1.5 rounded-full ${
engineState.channel === 'interval' ? 'bg-status-warn' : 'bg-status-ok'
}`}
/>
{engineState.channel === 'longpoll' ? 'Live (Long-Polling)' : engineState.channel === 'interval' ? 'Abgleich alle 3 s' : 'Live'}
</span>
)}
{/* Who else has this note open */}
{otherPeople.length > 0 && (
<span
className="hidden sm:inline-flex items-center gap-1 px-2 py-0.5 rounded-btn text-[11px] text-muted bg-panel border border-border"
title="Hat diese Notiz gerade geöffnet"
>
<Users className="w-3 h-3 text-accent" />
{otherPeople.map((p) => p.name).join(', ')}
</span>
)}
{/* Sharing Section */} {/* Sharing Section */}
{doc && (() => { {doc && (() => {
const isOwner = doc.isOwner !== false && (doc.userId === currentUser?.id || !doc.userId || currentUser?.role === 'admin'); const isOwner = doc.isOwner !== false && (doc.userId === currentUser?.id || !doc.userId || currentUser?.role === 'admin');
@@ -977,6 +983,12 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
onTagClick={onSelectTag} onTagClick={onSelectTag}
onWikiLinkClick={onWikiLinkClick} onWikiLinkClick={onWikiLinkClick}
initialFocusedBlockId={initialFocusedBlockId} initialFocusedBlockId={initialFocusedBlockId}
onFocusedBlockChange={(id) => engineRef.current?.focusBlock(id)}
lockedBlocks={lockedBlocks}
onLockedAction={handleLockedAction}
onResolveConflict={handleResolveConflict}
onUndo={handleUndo}
onRedo={handleRedo}
/> />
)} )}
</div> </div>
@@ -1084,6 +1096,12 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
e.target.style.height = `${e.target.scrollHeight}px`; e.target.style.height = `${e.target.scrollHeight}px`;
}} }}
onKeyDown={(e) => { onKeyDown={(e) => {
if ((e.ctrlKey || e.metaKey) && !e.altKey && ['z', 'y'].includes(e.key.toLowerCase())) {
e.preventDefault();
if (e.key.toLowerCase() === 'y' || e.shiftKey) handleRedo();
else handleUndo();
return;
}
if (e.key === 'Enter' && !e.shiftKey) { if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault(); e.preventDefault();
outlinerTreeRef.current?.focusFirstBlock(); outlinerTreeRef.current?.focusFirstBlock();
@@ -1113,10 +1131,25 @@ export const DocumentEditor: React.FC<DocumentEditorProps> = ({
onTagClick={onSelectTag} onTagClick={onSelectTag}
onWikiLinkClick={onWikiLinkClick} onWikiLinkClick={onWikiLinkClick}
initialFocusedBlockId={initialFocusedBlockId} initialFocusedBlockId={initialFocusedBlockId}
onFocusedBlockChange={(id) => engineRef.current?.focusBlock(id)}
lockedBlocks={lockedBlocks}
onLockedAction={handleLockedAction}
onResolveConflict={handleResolveConflict}
onUndo={handleUndo}
onRedo={handleRedo}
/> />
</> </>
)} )}
</div> </div>
{notice && (
<div
role="status"
className="fixed bottom-6 left-1/2 -translate-x-1/2 z-50 max-w-[90vw] px-3 py-2 rounded-card text-xs text-main bg-panel-elevated border border-border shadow-panel"
>
{notice}
</div>
)}
</div> </div>
); );
}; };
+189 -32
View File
@@ -14,11 +14,12 @@ import {
ListTodo, ListTodo,
Minus, Minus,
MessageSquare, MessageSquare,
Hash,
ExternalLink, ExternalLink,
Maximize2, Maximize2,
Link as LinkIcon, Link as LinkIcon,
FileText, FileText,
Lock,
AlertTriangle,
} from 'lucide-react'; } from 'lucide-react';
import { DocumentBlock, BlockType } from '../../lib/types'; import { DocumentBlock, BlockType } from '../../lib/types';
import { LinkModal } from './LinkModal'; import { LinkModal } from './LinkModal';
@@ -57,6 +58,7 @@ export interface OutlinerBlockProps {
onUpdateBlock: (updated: DocumentBlock, immediate?: boolean) => void; onUpdateBlock: (updated: DocumentBlock, immediate?: boolean) => void;
onCreateBlockBelow: (afterBlockId: string, newContent?: string) => void; onCreateBlockBelow: (afterBlockId: string, newContent?: string) => void;
onDeleteBlock: (blockId: string, deleteSubtree?: boolean) => void; onDeleteBlock: (blockId: string, deleteSubtree?: boolean) => void;
onMergeWithPrevious?: (blockId: string, content: string) => boolean;
onIndentBlock: (blockId: string, delta: 1 | -1) => void; onIndentBlock: (blockId: string, delta: 1 | -1) => void;
onNavigateFocus: (blockId: string, direction: 'up' | 'down') => void; onNavigateFocus: (blockId: string, direction: 'up' | 'down') => void;
onFocusBlock?: (blockId: string) => void; onFocusBlock?: (blockId: string) => void;
@@ -70,6 +72,23 @@ export interface OutlinerBlockProps {
isTargeted?: boolean; isTargeted?: boolean;
registerTextareaRef: (blockId: string, el: HTMLTextAreaElement | null) => void; registerTextareaRef: (blockId: string, el: HTMLTextAreaElement | null) => void;
onPasteMultiLines?: (blockId: string, textBefore: string, textAfter: string, rawLines: string[]) => void; onPasteMultiLines?: (blockId: string, textBefore: string, textAfter: string, rawLines: string[]) => void;
lockedBy?: string; // someone else is editing this block
canAppendBelow?: boolean; // the user may edit the note, so Enter in a locked block still adds one below
onResolveConflict?: (blockId: string, accept: boolean) => void;
onUndo?: () => void;
onRedo?: () => void;
}
// Rendered markdown only drops syntax characters (**, `, [..](url) …), so its text is a subsequence of the raw
// text. Walking both strings in parallel maps a caret offset in the rendered text back to the raw text.
export function mapRenderedOffsetToRaw(raw: string, rendered: string, renderedOffset: number): number {
let j = 0;
for (let i = 0; i < renderedOffset && i < rendered.length; i++) {
const idx = raw.indexOf(rendered[i], j);
if (idx === -1) break;
j = idx + 1;
}
return Math.min(j, raw.length);
} }
export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
@@ -86,6 +105,7 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
onUpdateBlock, onUpdateBlock,
onCreateBlockBelow, onCreateBlockBelow,
onDeleteBlock, onDeleteBlock,
onMergeWithPrevious,
onIndentBlock, onIndentBlock,
onNavigateFocus, onNavigateFocus,
onFocusBlock, onFocusBlock,
@@ -99,10 +119,17 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
isTargeted, isTargeted,
registerTextareaRef, registerTextareaRef,
onPasteMultiLines, onPasteMultiLines,
lockedBy,
canAppendBelow,
onResolveConflict,
onUndo,
onRedo,
}) => { }) => {
const [localContent, setLocalContent] = useState<string>(block.content || ''); const [localContent, setLocalContent] = useState<string>(block.content || '');
const [isLocallyFocused, setIsLocallyFocused] = useState<boolean>(false); const [isLocallyFocused, setIsLocallyFocused] = useState<boolean>(false);
const activeFocus = isFocused || isLocallyFocused; const activeFocus = isFocused || isLocallyFocused;
// A block someone else is editing can still get the cursor (read-only), e.g. to press Enter at its end
const canFocus = canEdit || (!!lockedBy && !!canAppendBelow);
const [slashSearch, setSlashSearch] = useState<string | null>(null); const [slashSearch, setSlashSearch] = useState<string | null>(null);
const [slashIndex, setSlashIndex] = useState(0); const [slashIndex, setSlashIndex] = useState(0);
const [showLinkModal, setShowLinkModal] = useState<boolean>(false); const [showLinkModal, setShowLinkModal] = useState<boolean>(false);
@@ -124,6 +151,8 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
onPasteMultiLinesRef.current = onPasteMultiLines; onPasteMultiLinesRef.current = onPasteMultiLines;
const onDeleteBlockRef = useRef(onDeleteBlock); const onDeleteBlockRef = useRef(onDeleteBlock);
onDeleteBlockRef.current = onDeleteBlock; onDeleteBlockRef.current = onDeleteBlock;
const onMergeWithPreviousRef = useRef(onMergeWithPrevious);
onMergeWithPreviousRef.current = onMergeWithPrevious;
const onIndentBlockRef = useRef(onIndentBlock); const onIndentBlockRef = useRef(onIndentBlock);
onIndentBlockRef.current = onIndentBlock; onIndentBlockRef.current = onIndentBlock;
const onNavigateFocusRef = useRef(onNavigateFocus); const onNavigateFocusRef = useRef(onNavigateFocus);
@@ -149,6 +178,13 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
[] []
); );
// Right after undo/redo the focused field must show the engine's text, not keep its own
const acceptRemoteUntilRef = useRef<number>(0);
const onUndoRef = useRef(onUndo);
onUndoRef.current = onUndo;
const onRedoRef = useRef(onRedo);
onRedoRef.current = onRedo;
// Sync incoming remote changes ONLY when this textarea is NOT actively being edited // Sync incoming remote changes ONLY when this textarea is NOT actively being edited
// Condition: isFocusedRef.current === true && document.hasFocus() // Condition: isFocusedRef.current === true && document.hasFocus()
useEffect(() => { useEffect(() => {
@@ -159,7 +195,8 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
textareaRef.current !== null && textareaRef.current !== null &&
document.activeElement === textareaRef.current; document.activeElement === textareaRef.current;
if (!isActivelyEditing && block.content !== localContentRef.current) { const acceptAfterUndo = Date.now() < acceptRemoteUntilRef.current;
if ((!isActivelyEditing || acceptAfterUndo) && block.content !== localContentRef.current) {
setLocalContent(block.content || ''); setLocalContent(block.content || '');
localContentRef.current = block.content || ''; localContentRef.current = block.content || '';
} }
@@ -203,6 +240,41 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
}; };
}, []); }, []);
const renderedTextRef = useRef<HTMLSpanElement | null>(null);
// Focus the textarea and place the caret where the user clicked in the rendered (markdown) text layer
const focusAtPoint = (clientX: number, clientY: number) => {
const ta = textareaRef.current;
if (!ta) return;
const raw = ta.value;
let pos = raw.length;
const span = renderedTextRef.current;
const doc = document as any;
let node: Node | null = null;
let offset = 0;
if (typeof doc.caretPositionFromPoint === 'function') {
const p = doc.caretPositionFromPoint(clientX, clientY);
if (p) {
node = p.offsetNode;
offset = p.offset;
}
} else if (typeof doc.caretRangeFromPoint === 'function') {
const r = doc.caretRangeFromPoint(clientX, clientY);
if (r) {
node = r.startContainer;
offset = r.startOffset;
}
}
if (span && node && span.contains(node)) {
const range = document.createRange();
range.setStart(span, 0);
range.setEnd(node, offset);
pos = mapRenderedOffsetToRaw(raw, span.textContent || '', range.toString().length);
}
ta.focus();
ta.setSelectionRange(pos, pos);
};
const adjustHeight = useCallback(() => { const adjustHeight = useCallback(() => {
const el = textareaRef.current; const el = textareaRef.current;
if (!el) return; if (!el) return;
@@ -356,7 +428,44 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
onPasteMultiLinesRef.current?.(block.id, textBefore, textAfter, rawLines); onPasteMultiLinesRef.current?.(block.id, textBefore, textAfter, rawLines);
}; };
// Ctrl/Cmd+Z undoes the user's own changes in the whole note (not the browser's per-field undo)
const runHistory = (redo: boolean) => {
if (debounceTimerRef.current !== null) {
flushDebounce(localContentRef.current);
}
acceptRemoteUntilRef.current = Date.now() + 500;
if (redo) onRedoRef.current?.();
else onUndoRef.current?.();
};
useEffect(() => {
const el = textareaRef.current;
if (!el) return;
// Mobile keyboards and the context menu undo via beforeinput instead of key events
const onBeforeInput = (e: InputEvent) => {
if (e.inputType === 'historyUndo' || e.inputType === 'historyRedo') {
e.preventDefault();
runHistory(e.inputType === 'historyRedo');
}
};
el.addEventListener('beforeinput', onBeforeInput);
return () => el.removeEventListener('beforeinput', onBeforeInput);
});
const handleKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => { const handleKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
if ((e.ctrlKey || e.metaKey) && !e.altKey && ['z', 'y'].includes(e.key.toLowerCase())) {
e.preventDefault();
e.stopPropagation();
runHistory(e.key.toLowerCase() === 'y' || e.shiftKey);
return;
}
// Enter in a block someone else is editing cannot split it (read-only), so it adds a new block below
// without touching the locked one. (Chromium does not move the caret with End in read-only fields.)
if (lockedBy && canAppendBelow && e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
onCreateBlockBelowRef.current(block.id, '');
return;
}
if (!canEdit) return; if (!canEdit) return;
// Slash command navigation // Slash command navigation
@@ -455,6 +564,28 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
return; return;
} }
// 3b. Backspace at the very start of a non-empty block -> append its text to the previous block
if (
e.key === 'Backspace' &&
localContent !== '' &&
block.type !== 'code' &&
onMergeWithPreviousRef.current &&
e.currentTarget.selectionStart === 0 &&
e.currentTarget.selectionEnd === 0
) {
if (onMergeWithPreviousRef.current(block.id, localContent)) {
// Text lives in the previous block afterwards, so drop this block's pending save
if (debounceTimerRef.current !== null) {
window.clearTimeout(debounceTimerRef.current);
debounceTimerRef.current = null;
}
e.preventDefault();
e.stopPropagation();
setSlashSearch(null);
}
return;
}
// 4. Arrow navigation between blocks // 4. Arrow navigation between blocks
if (e.key === 'ArrowUp' && !e.shiftKey && !e.altKey) { if (e.key === 'ArrowUp' && !e.shiftKey && !e.altKey) {
const el = textareaRef.current; const el = textareaRef.current;
@@ -489,9 +620,9 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
return parts.filter(Boolean); return parts.filter(Boolean);
}, [localContent]); }, [localContent]);
// #Hashtags are rendered inline inside the text, so only links need the badge bar below
const hasSpecialTokens = tokens.some( const hasSpecialTokens = tokens.some(
(t) => (t) =>
t.startsWith('#') ||
(t.startsWith('[[') && t.endsWith(']]')) || (t.startsWith('[[') && t.endsWith(']]')) ||
/^\[(.*?)\]\((https?:\/\/.*?)\)$/.test(t) || /^\[(.*?)\]\((https?:\/\/.*?)\)$/.test(t) ||
/^https?:\/\//.test(t) /^https?:\/\//.test(t)
@@ -503,11 +634,9 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
isDragTarget isDragTarget
? 'border-t-2 border-accent bg-accent/10' ? 'border-t-2 border-accent bg-accent/10'
: isTargeted : isTargeted
? 'bg-accent/15 ring-2 ring-accent'
: activeFocus
? 'bg-panel-elevated' ? 'bg-panel-elevated'
: 'hover:bg-panel' : ''
}`} } ${block.conflictOf ? 'border-l-2 border-status-warn bg-status-warn/5' : ''}`}
style={{ style={{
paddingLeft: `${visualLevel * 22 + 4}px`, paddingLeft: `${visualLevel * 22 + 4}px`,
}} }}
@@ -515,7 +644,7 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
onDrop={() => onDropBlock?.(index)} onDrop={() => onDropBlock?.(index)}
onClick={(e) => { onClick={(e) => {
if ((e.target as HTMLElement).closest('button, a, input, select')) return; if ((e.target as HTMLElement).closest('button, a, input, select')) return;
if (!activeFocus && canEdit && textareaRef.current) { if (!activeFocus && canFocus && textareaRef.current) {
textareaRef.current.focus(); textareaRef.current.focus();
const len = textareaRef.current.value.length; const len = textareaRef.current.value.length;
textareaRef.current.setSelectionRange(len, len); textareaRef.current.setSelectionRange(len, len);
@@ -540,7 +669,7 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
onClick={() => onClick={() =>
flushDebounce(localContent, { collapsed: !block.collapsed }) flushDebounce(localContent, { collapsed: !block.collapsed })
} }
className="p-0.5 text-muted hover:text-accent transition-colors" className="p-0.5 text-muted hover:text-accent transition-[color,opacity] opacity-0 group-hover:opacity-100 focus-visible:opacity-100"
title={block.collapsed ? 'Unterpunkte ausklappen' : 'Unterpunkte einklappen'} title={block.collapsed ? 'Unterpunkte ausklappen' : 'Unterpunkte einklappen'}
> >
{block.collapsed ? ( {block.collapsed ? (
@@ -559,7 +688,7 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
draggable={canEdit} draggable={canEdit}
onDragStart={() => canEdit && onDragStartBlock?.(index)} onDragStart={() => canEdit && onDragStartBlock?.(index)}
onClick={() => onZoomBlock?.(block.id)} onClick={() => onZoomBlock?.(block.id)}
className={`w-4 h-4 flex items-center justify-center rounded-btn hover:bg-panel-hover group/bullet transition-transform ${ className={`w-4 h-4 flex items-center justify-center rounded-btn group/bullet transition-transform ${
canEdit ? 'cursor-grab active:cursor-grabbing' : 'cursor-pointer' canEdit ? 'cursor-grab active:cursor-grabbing' : 'cursor-pointer'
}`} }`}
title={ title={
@@ -573,7 +702,7 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
block.collapsed && hasChildren block.collapsed && hasChildren
? 'bg-accent ring-2 ring-accent/30' ? 'bg-accent ring-2 ring-accent/30'
: activeFocus : activeFocus
? 'bg-accent scale-125' ? 'bg-main scale-125'
: 'bg-faint group-hover/bullet:bg-accent group-hover/bullet:scale-125' : 'bg-faint group-hover/bullet:bg-accent group-hover/bullet:scale-125'
} ${ } ${
isEmpty isEmpty
@@ -657,11 +786,11 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
<div className="relative w-full min-h-[1.5rem]"> <div className="relative w-full min-h-[1.5rem]">
{/* View Layer with Inline Markdown & Actions (3-dots or Erweitert Badge) directly behind text */} {/* View Layer with Inline Markdown & Actions (3-dots or Erweitert Badge) directly behind text */}
<div <div
onClick={() => { onClick={(e) => {
if (!activeFocus && canEdit && textareaRef.current) { if (!activeFocus && canFocus && textareaRef.current) {
textareaRef.current.focus(); // Keep the row's click handler from moving the caret back to the end
const len = textareaRef.current.value.length; e.stopPropagation();
textareaRef.current.setSelectionRange(len, len); focusAtPoint(e.clientX, e.clientY);
} }
}} }}
className={`w-full min-h-[1.5rem] leading-relaxed whitespace-pre-wrap break-words [overflow-wrap:anywhere] ${ className={`w-full min-h-[1.5rem] leading-relaxed whitespace-pre-wrap break-words [overflow-wrap:anywhere] ${
@@ -679,8 +808,8 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
}`} }`}
> >
{/* When actively focused, ghost text is invisible so textarea renders text/caret directly */} {/* When actively focused, ghost text is invisible so textarea renders text/caret directly */}
<span className={activeFocus ? 'invisible' : ''}> <span ref={renderedTextRef} className={activeFocus ? 'invisible' : ''}>
{localContent ? renderInlineMarkdown(localContent) : '\u200b'} {localContent ? renderInlineMarkdown(localContent, { onTagClick }) : '\u200b'}
</span> </span>
{/* Inline Badge and 3-Dots directly behind the text */} {/* Inline Badge and 3-Dots directly behind the text */}
@@ -805,24 +934,41 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
</div> </div>
)} )}
{/* Conflict copy: text that lost against someone else's change */}
{block.conflictOf && (
<div className="flex flex-wrap items-center gap-1.5 mt-1 text-[11px] text-status-warn select-none">
<AlertTriangle className="w-3 h-3 shrink-0" />
<span>
{block.conflictOf === 'title' ? 'Konflikt beim Titel' : 'Konflikt'}
{block.conflictAuthor ? ` – Version von ${block.conflictAuthor}` : ''}
</span>
{onResolveConflict && (
<>
<button
type="button"
onClick={() => onResolveConflict(block.id, true)}
className="px-1.5 py-0.5 rounded-btn border border-status-warn/40 hover:bg-status-warn/15 cursor-pointer"
title={block.conflictOf === 'title' ? 'Diesen Text als Titel übernehmen' : 'Diesen Text in den ursprünglichen Punkt übernehmen'}
>
Übernehmen
</button>
<button
type="button"
onClick={() => onResolveConflict(block.id, false)}
className="px-1.5 py-0.5 rounded-btn border border-border text-muted hover:text-main hover:bg-panel cursor-pointer"
title="Diese Version verwerfen"
>
Verwerfen
</button>
</>
)}
</div>
)}
{/* Rich Tokens Bar (Wiki-Links & #Tags) if present */} {/* Rich Tokens Bar (Wiki-Links & #Tags) if present */}
{hasSpecialTokens && !isFocused && ( {hasSpecialTokens && !isFocused && (
<div className="flex flex-wrap items-center gap-1 mt-0.5"> <div className="flex flex-wrap items-center gap-1 mt-0.5">
{tokens.map((part, idx) => { {tokens.map((part, idx) => {
if (part.startsWith('#') && part.length > 1) {
const tag = part.slice(1);
return (
<button
key={idx}
type="button"
onClick={() => onTagClick?.(tag)}
className="inline-flex items-center gap-0.5 px-1.5 py-0.5 text-[10px] font-medium text-accent bg-accent-dim hover:bg-accent/20 rounded-btn transition-colors"
>
<Hash className="w-2.5 h-2.5 opacity-80" />
<span>{tag}</span>
</button>
);
}
if (part.startsWith('[[') && part.endsWith(']]')) { if (part.startsWith('[[') && part.endsWith(']]')) {
const page = part.slice(2, -2).trim(); const page = part.slice(2, -2).trim();
return ( return (
@@ -924,6 +1070,17 @@ export const OutlinerBlock: React.FC<OutlinerBlockProps> = ({
<Maximize2 className="w-3.5 h-3.5" /> <Maximize2 className="w-3.5 h-3.5" />
</button> </button>
{/* Someone else is editing this block */}
{lockedBy && (
<span
className="shrink-0 self-start mt-1 inline-flex items-center gap-1 px-1.5 py-0.5 rounded-btn text-[10px] text-muted bg-panel border border-border select-none"
title={`${lockedBy} bearbeitet diesen Punkt gerade`}
>
<Lock className="w-2.5 h-2.5" />
{lockedBy}
</span>
)}
{/* Delete button on hover (Trash2 icon) */} {/* Delete button on hover (Trash2 icon) */}
{canEdit && ( {canEdit && (
<button <button
+123 -13
View File
@@ -23,6 +23,12 @@ export interface OutlinerTreeProps {
onTagClick?: (tag: string) => void; onTagClick?: (tag: string) => void;
onWikiLinkClick?: (pageTitle: string) => void; onWikiLinkClick?: (pageTitle: string) => void;
initialFocusedBlockId?: string | null; initialFocusedBlockId?: string | null;
onFocusedBlockChange?: (blockId: string | null) => void;
lockedBlocks?: Map<string, string>; // blockId -> name of the user editing it (someone else)
onLockedAction?: (holderName: string) => void;
onResolveConflict?: (blockId: string, accept: boolean) => void;
onUndo?: () => void;
onRedo?: () => void;
} }
export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
@@ -41,8 +47,20 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
onTagClick, onTagClick,
onWikiLinkClick, onWikiLinkClick,
initialFocusedBlockId, initialFocusedBlockId,
onFocusedBlockChange,
lockedBlocks,
onLockedAction,
onResolveConflict,
onUndo,
onRedo,
}, ref) => { }, ref) => {
const [focusedBlockId, setFocusedBlockId] = useState<string | null>(null); const [focusedBlockId, setFocusedBlockId] = useState<string | null>(null);
const onFocusedBlockChangeRef = useRef(onFocusedBlockChange);
onFocusedBlockChangeRef.current = onFocusedBlockChange;
useEffect(() => {
onFocusedBlockChangeRef.current?.(focusedBlockId);
}, [focusedBlockId]);
const [highlightedBlockId, setHighlightedBlockId] = useState<string | null>( const [highlightedBlockId, setHighlightedBlockId] = useState<string | null>(
initialFocusedBlockId || null initialFocusedBlockId || null
); );
@@ -50,27 +68,34 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
const [dragOverIndex, setDragOverIndex] = useState<number | null>(null); const [dragOverIndex, setDragOverIndex] = useState<number | null>(null);
const textareaMapRef = useRef<Map<string, HTMLTextAreaElement>>(new Map()); const textareaMapRef = useRef<Map<string, HTMLTextAreaElement>>(new Map());
const pendingFocusRef = useRef<{ blockId: string; cursor: 'start' | 'end' | 'select' } | null>(null); const pendingFocusRef = useRef<{ blockId: string; cursor: 'start' | 'end' | 'select' | number } | null>(null);
const focusBlockInput = useCallback( const focusBlockInput = useCallback(
(blockId: string, cursor: 'start' | 'end' | 'select' = 'end') => { (blockId: string, cursor: 'start' | 'end' | 'select' | number = 'end') => {
setFocusedBlockId(blockId); setFocusedBlockId(blockId);
const el = textareaMapRef.current.get(blockId); const el = textareaMapRef.current.get(blockId);
if (el) { if (el) {
requestAnimationFrame(() => { const apply = () => {
el.focus(); el.focus();
try { try {
if (cursor === 'select') { if (cursor === 'select') {
el.select(); el.select();
} else { } else {
const pos = cursor === 'end' ? el.value.length : 0; const pos =
typeof cursor === 'number'
? Math.min(cursor, el.value.length)
: cursor === 'end'
? el.value.length
: 0;
el.setSelectionRange(pos, pos); el.setSelectionRange(pos, pos);
} }
} catch { } catch {
// ignore // ignore
} }
el.scrollIntoView({ behavior: 'smooth', block: 'center' }); el.scrollIntoView({ behavior: 'smooth', block: 'center' });
}); };
// A numeric position refers to content that was just changed, so wait one extra frame for it to render
requestAnimationFrame(() => (typeof cursor === 'number' ? requestAnimationFrame(apply) : apply()));
} else { } else {
pendingFocusRef.current = { blockId, cursor }; pendingFocusRef.current = { blockId, cursor };
} }
@@ -110,21 +135,26 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
if (pendingFocusRef.current && pendingFocusRef.current.blockId === blockId) { if (pendingFocusRef.current && pendingFocusRef.current.blockId === blockId) {
const target = pendingFocusRef.current; const target = pendingFocusRef.current;
pendingFocusRef.current = null; pendingFocusRef.current = null;
requestAnimationFrame(() => { // Focus right away (the element is already in the DOM): keystrokes typed directly after Enter
// must land in the new block, not in the previous one
setFocusedBlockId(blockId); setFocusedBlockId(blockId);
el.focus(); el.focus();
try { try {
if (target.cursor === 'select') { if (target.cursor === 'select') {
el.select(); el.select();
} else { } else {
const pos = target.cursor === 'end' ? el.value.length : 0; const pos =
typeof target.cursor === 'number'
? Math.min(target.cursor, el.value.length)
: target.cursor === 'end'
? el.value.length
: 0;
el.setSelectionRange(pos, pos); el.setSelectionRange(pos, pos);
} }
} catch { } catch {
// ignore // ignore
} }
el.scrollIntoView({ behavior: 'smooth', block: 'center' }); el.scrollIntoView({ behavior: 'smooth', block: 'center' });
});
} }
} else { } else {
textareaMapRef.current.delete(blockId); textareaMapRef.current.delete(blockId);
@@ -467,9 +497,33 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
[blocks, canEdit, docId, focusBlockInput, onCreateBlock, onInsertBlocks, onUpdateBlock] [blocks, canEdit, docId, focusBlockInput, onCreateBlock, onInsertBlocks, onUpdateBlock]
); );
// Name of the other user holding the block (or, withBranch, any block below it); structural actions on
// such blocks are not started at all, so they cannot end up half applied
const lockHolder = useCallback(
(blockId: string, withBranch = false): string | undefined => {
if (!lockedBlocks || lockedBlocks.size === 0) return undefined;
const direct = lockedBlocks.get(blockId);
if (direct || !withBranch) return direct;
const idx = blocks.findIndex((b) => b.id === blockId);
if (idx < 0) return undefined;
const level = blocks[idx].level || 0;
for (let i = idx + 1; i < blocks.length && (blocks[i].level || 0) > level; i++) {
const name = lockedBlocks.get(blocks[i].id);
if (name) return name;
}
return undefined;
},
[blocks, lockedBlocks]
);
const handleDeleteBlock = useCallback( const handleDeleteBlock = useCallback(
(blockId: string, deleteSubtree = false) => { (blockId: string, deleteSubtree = false) => {
if (!canEdit) return; if (!canEdit) return;
const holder = lockHolder(blockId, true);
if (holder) {
onLockedAction?.(holder);
return;
}
const idx = blocks.findIndex((b) => b.id === blockId); const idx = blocks.findIndex((b) => b.id === blockId);
if (idx === -1) return; if (idx === -1) return;
@@ -511,7 +565,49 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
onDeleteBlock(blockId); onDeleteBlock(blockId);
} }
}, },
[blocks, canEdit, focusBlockInput, levelOffset, onDeleteBlock, onUpdateBlock] [blocks, canEdit, focusBlockInput, levelOffset, onDeleteBlock, onUpdateBlock, lockHolder, onLockedAction]
);
// Backspace at the start of a non-empty block: append its text to the previous visible block
const handleMergeWithPrevious = useCallback(
(blockId: string, content: string) => {
if (!canEdit) return false;
const notMergeable = ['code', 'divider', 'table', 'progress'];
const visIdx = visibleEntries.findIndex((e) => e.block.id === blockId);
if (visIdx <= 0) return false;
const prev = visibleEntries[visIdx - 1].block;
const idx = blocks.findIndex((b) => b.id === blockId);
const cur = blocks[idx];
if (!cur || notMergeable.includes(prev.type) || notMergeable.includes(cur.type)) return false;
const holder = lockHolder(cur.id, true) || lockHolder(prev.id);
if (holder) {
onLockedAction?.(holder);
return true;
}
// Children of the merged block move up so they become children of the previous block
const curLevel = Math.max(0, cur.level || 0);
const delta = Math.max(0, prev.level || 0) - curLevel;
const movedChildren: DocumentBlock[] = [];
for (let j = idx + 1; j < blocks.length; j++) {
const lvl = Math.max(0, blocks[j].level || 0);
if (lvl <= curLevel) break;
movedChildren.push({ ...blocks[j], level: Math.max(levelOffset || 0, lvl + delta) });
}
const joinPos = (prev.content || '').length;
const mergedPrev: DocumentBlock = { ...prev, content: (prev.content || '') + content };
if (movedChildren.length > 0 && onUpdateBlocks) {
onUpdateBlocks([mergedPrev, ...movedChildren]);
} else {
onUpdateBlock(mergedPrev, true);
movedChildren.forEach((c) => onUpdateBlock(c));
}
onDeleteBlock(cur.id);
focusBlockInput(prev.id, joinPos);
return true;
},
[blocks, canEdit, focusBlockInput, levelOffset, onDeleteBlock, onUpdateBlock, onUpdateBlocks, visibleEntries, lockHolder, onLockedAction]
); );
const handleIndentBlock = useCallback( const handleIndentBlock = useCallback(
@@ -519,6 +615,11 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
if (!canEdit) return; if (!canEdit) return;
const idx = blocks.findIndex((b) => b.id === blockId); const idx = blocks.findIndex((b) => b.id === blockId);
if (idx === -1) return; if (idx === -1) return;
const holder = lockHolder(blockId);
if (holder) {
onLockedAction?.(holder);
return;
}
const current = blocks[idx]; const current = blocks[idx];
const currentLevel = Math.max(0, current.level || 0); const currentLevel = Math.max(0, current.level || 0);
@@ -564,7 +665,7 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
} }
} }
}, },
[blocks, canEdit, levelOffset, onUpdateBlock, onUpdateBlocks] [blocks, canEdit, levelOffset, onUpdateBlock, onUpdateBlocks, lockHolder, onLockedAction]
); );
const handleNavigateFocus = useCallback( const handleNavigateFocus = useCallback(
@@ -633,13 +734,16 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
return; return;
} }
const movingBlock = blocks[draggedIndex]; const movingBlock = blocks[draggedIndex];
if (movingBlock) { const holder = movingBlock ? lockHolder(movingBlock.id) : undefined;
if (holder) {
onLockedAction?.(holder);
} else if (movingBlock) {
onMoveBlock(movingBlock.id, targetIndex, movingBlock.level); onMoveBlock(movingBlock.id, targetIndex, movingBlock.level);
} }
setDraggedIndex(null); setDraggedIndex(null);
setDragOverIndex(null); setDragOverIndex(null);
}, },
[blocks, draggedIndex, onMoveBlock] [blocks, draggedIndex, onMoveBlock, lockHolder, onLockedAction]
); );
useEffect(() => { useEffect(() => {
@@ -665,13 +769,19 @@ export const OutlinerTree = forwardRef<OutlinerTreeHandle, OutlinerTreeProps>(({
maxAllowedLevel={maxAllowedLevel} maxAllowedLevel={maxAllowedLevel}
hasChildren={hasChildren} hasChildren={hasChildren}
isFocused={focusedBlockId === block.id} isFocused={focusedBlockId === block.id}
canEdit={canEdit} canEdit={canEdit && !lockedBlocks?.has(block.id)}
lockedBy={lockedBlocks?.get(block.id)}
canAppendBelow={canEdit}
onResolveConflict={canEdit ? onResolveConflict : undefined}
onUndo={onUndo}
onRedo={onRedo}
levelOffset={levelOffset} levelOffset={levelOffset}
onZoomBlock={onZoomBlock} onZoomBlock={onZoomBlock}
onUpdateBlock={onUpdateBlock} onUpdateBlock={onUpdateBlock}
onCreateBlockBelow={handleCreateBlockBelow} onCreateBlockBelow={handleCreateBlockBelow}
onPasteMultiLines={handlePasteMultiLines} onPasteMultiLines={handlePasteMultiLines}
onDeleteBlock={handleDeleteBlock} onDeleteBlock={handleDeleteBlock}
onMergeWithPrevious={handleMergeWithPrevious}
onIndentBlock={handleIndentBlock} onIndentBlock={handleIndentBlock}
onNavigateFocus={handleNavigateFocus} onNavigateFocus={handleNavigateFocus}
onFocusBlock={setFocusedBlockId} onFocusBlock={setFocusedBlockId}
@@ -49,6 +49,7 @@ import {
RealtimeTransport, RealtimeTransport,
} from '../../lib/api'; } from '../../lib/api';
import { UnifiedShareModal } from '../../components/UnifiedShareModal'; import { UnifiedShareModal } from '../../components/UnifiedShareModal';
import { sendDocOps } from '../../lib/docSync/oneoff';
import { useMobileActions } from '../../context/MobileActionContext'; import { useMobileActions } from '../../context/MobileActionContext';
import { DocumentEditor, DocSaveStatus } from './DocumentEditor'; import { DocumentEditor, DocSaveStatus } from './DocumentEditor';
@@ -789,9 +790,8 @@ export const WorkspaceView: React.FC<WorkspaceViewProps> = ({
prev.map((d) => (d.id === doc.id ? { ...d, blocks: updatedBlocks } : d)) prev.map((d) => (d.id === doc.id ? { ...d, blocks: updatedBlocks } : d))
); );
try { try {
await api.patchDocumentBlocks(doc.id, { const [result] = await sendDocOps(doc.id, [{ kind: 'set_props', blockId, patch: { checked: !currentChecked } }]);
upserted: [{ id: blockId, checked: !currentChecked, type: 'todo', content: '' }], if (result?.status !== 'applied') loadWorkspaceData();
});
} catch { } catch {
loadWorkspaceData(); loadWorkspaceData();
} }
@@ -2268,15 +2268,12 @@ export const WorkspaceView: React.FC<WorkspaceViewProps> = ({
shares={unifiedShares} shares={unifiedShares}
teams={teams} teams={teams}
collaborators={collaborators} collaborators={collaborators}
allowRoleSelection={false} onAddShare={async (targetType, targetId, role) => {
fixedPermission="viewer" const permission = role === 'editor' ? 'write' : 'read';
onAddShare={async (targetType, targetId) => {
if (shareModalDoc) { if (shareModalDoc) {
await api.createDocShare( await api.createDocShare(
shareModalDoc.id, shareModalDoc.id,
targetType === 'team' targetType === 'team' ? { teamId: targetId, permission } : { userId: targetId, permission }
? { teamId: targetId, permission: 'read' }
: { userId: targetId, permission: 'read' }
); );
const updatedShares = await api.getDocShares(shareModalDoc.id); const updatedShares = await api.getDocShares(shareModalDoc.id);
setActiveDocShares(updatedShares); setActiveDocShares(updatedShares);
@@ -2284,7 +2281,7 @@ export const WorkspaceView: React.FC<WorkspaceViewProps> = ({
await api.createFolderShare(shareModalFolder.id, { await api.createFolderShare(shareModalFolder.id, {
targetType, targetType,
targetId, targetId,
permission: 'read', permission,
}); });
const updatedFolder = await api.getSharedFolder(shareModalFolder.id); const updatedFolder = await api.getSharedFolder(shareModalFolder.id);
setShareModalFolder(updatedFolder); setShareModalFolder(updatedFolder);
@@ -0,0 +1,120 @@
import React from 'react';
import { describe, it, expect, afterEach, vi } from 'vitest';
import { render, fireEvent } from '@testing-library/react';
import { OutlinerBlock, mapRenderedOffsetToRaw } from '../OutlinerBlock';
import { DocumentBlock } from '../../../lib/types';
describe('mapRenderedOffsetToRaw', () => {
it('is the identity for plain text', () => {
expect(mapRenderedOffsetToRaw('Hallo Welt', 'Hallo Welt', 4)).toBe(4);
});
it('skips bold markers', () => {
expect(mapRenderedOffsetToRaw('Hallo **Welt** du', 'Hallo Welt du', 8)).toBe(10);
});
it('maps behind closing markers', () => {
expect(mapRenderedOffsetToRaw('a **b** c', 'a b c', 4)).toBe(8);
});
it('maps into a link label', () => {
expect(mapRenderedOffsetToRaw('x [Seite](https://a.de) y', 'x Seite y', 4)).toBe(5);
});
it('handles the empty placeholder', () => {
expect(mapRenderedOffsetToRaw('', '​', 1)).toBe(0);
});
});
describe('clicking into an unfocused bullet', () => {
afterEach(() => {
delete (document as any).caretRangeFromPoint;
});
it('places the caret at the click position instead of the end', () => {
const block = { id: 'b1', type: 'bullet', content: 'Hallo **Welt** du', level: 0, position: 1 } as DocumentBlock;
const { container } = render(
<OutlinerBlock
docId="d"
block={block}
index={0}
isLast
maxAllowedLevel={1}
hasChildren={false}
isFocused={false}
canEdit
onUpdateBlock={() => {}}
onCreateBlockBelow={() => {}}
onDeleteBlock={() => {}}
onIndentBlock={() => {}}
onNavigateFocus={() => {}}
registerTextareaRef={() => {}}
/>
);
const ta = container.querySelector('textarea') as HTMLTextAreaElement;
const strong = container.querySelector('strong') as HTMLElement; // rendered "Welt"
// jsdom has no layout: pretend the click hit between "We" and "lt"
(document as any).caretRangeFromPoint = () => {
const r = document.createRange();
r.setStart(strong.firstChild!, 2);
return r;
};
fireEvent.click(strong);
expect(document.activeElement).toBe(ta);
expect(ta.selectionStart).toBe(10); // "Hallo **We|lt** du"
});
});
function renderBlock(block: DocumentBlock, extra: Record<string, unknown> = {}) {
return render(
<OutlinerBlock
docId="d"
block={block}
index={0}
isLast
maxAllowedLevel={1}
hasChildren={false}
isFocused={false}
canEdit
onUpdateBlock={() => {}}
onCreateBlockBelow={() => {}}
onDeleteBlock={() => {}}
onIndentBlock={() => {}}
onNavigateFocus={() => {}}
registerTextareaRef={() => {}}
{...extra}
/>
);
}
describe('undo shortcut', () => {
it('saves pending typing first, then undoes', () => {
const calls: string[] = [];
const block = { id: 'b1', type: 'bullet', content: 'Alt', level: 0 } as DocumentBlock;
const { container } = renderBlock(block, {
onUpdateBlock: (b: DocumentBlock) => calls.push(`update:${b.content}`),
onUndo: () => calls.push('undo'),
onRedo: () => calls.push('redo'),
});
const ta = container.querySelector('textarea') as HTMLTextAreaElement;
ta.focus();
fireEvent.change(ta, { target: { value: 'Alt neu' } });
fireEvent.keyDown(ta, { key: 'z', ctrlKey: true });
fireEvent.keyDown(ta, { key: 'z', ctrlKey: true, shiftKey: true });
fireEvent.keyDown(ta, { key: 'y', metaKey: true });
expect(calls).toEqual(['update:Alt neu', 'undo', 'redo', 'redo']);
});
});
describe('conflict copies', () => {
it('show who lost and offer accept/discard', () => {
const onResolve = vi.fn();
const block = { id: 'cf', type: 'bullet', content: 'Meine Version', level: 0, conflictOf: 'b1', conflictAuthor: 'Ben' } as DocumentBlock;
const { getByText } = renderBlock(block, { onResolveConflict: onResolve });
expect(getByText(/Version von Ben/)).toBeTruthy();
fireEvent.click(getByText('Übernehmen'));
fireEvent.click(getByText('Verwerfen'));
expect(onResolve.mock.calls).toEqual([
['cf', true],
['cf', false],
]);
});
});
@@ -0,0 +1,131 @@
import React, { useState } from 'react';
import { describe, it, expect } from 'vitest';
import { render, fireEvent, screen, act } from '@testing-library/react';
import { OutlinerTree } from '../OutlinerTree';
import { DocumentBlock } from '../../../lib/types';
// Holds the latest block state rendered by the harness
let live: DocumentBlock[] = [];
let lockedNotices: string[] = [];
const Harness: React.FC<{ initial: DocumentBlock[]; locked?: Map<string, string> }> = ({ initial, locked }) => {
const [blocks, setBlocks] = useState(initial);
live = blocks;
return (
<OutlinerTree
docId="d1"
blocks={blocks}
lockedBlocks={locked}
onLockedAction={(name) => lockedNotices.push(name)}
canEdit
onUpdateBlock={(b) => setBlocks((p) => p.map((x) => (x.id === b.id ? b : x)))}
onUpdateBlocks={(u) => setBlocks((p) => p.map((x) => u.find((y) => y.id === x.id) || x))}
onCreateBlock={(nb, after) =>
setBlocks((p) => {
const i = p.findIndex((x) => x.id === after);
return i < 0 ? [...p, nb] : [...p.slice(0, i + 1), nb, ...p.slice(i + 1)];
})
}
onDeleteBlock={(id) => setBlocks((p) => p.filter((x) => x.id !== id))}
onMoveBlock={() => {}}
/>
);
};
const mk = (id: string, content: string, level = 0, type: any = 'bullet'): DocumentBlock =>
({ id, type, content, level, position: 1 }) as DocumentBlock;
const area = (text: string) => screen.getAllByDisplayValue(text)[0] as HTMLTextAreaElement;
const backspaceAt = async (ta: HTMLTextAreaElement, pos = 0) => {
ta.focus();
ta.setSelectionRange(pos, pos);
await act(async () => {
fireEvent.keyDown(ta, { key: 'Backspace' });
});
};
describe('Backspace at the start of a bullet', () => {
it('appends its text to the previous bullet', async () => {
render(<Harness initial={[mk('a', 'Hallo '), mk('b', 'Welt')]} />);
await backspaceAt(area('Welt'));
expect(live.map((b) => b.content)).toEqual(['Hallo Welt']);
});
it('works for legacy blocks without a type', async () => {
render(<Harness initial={[mk('a', 'A', 0, undefined), mk('b', 'B', 0, '')]} />);
await backspaceAt(area('B'));
expect(live.map((b) => b.content)).toEqual(['AB']);
});
it('moves children up under the previous bullet', async () => {
render(<Harness initial={[mk('a', 'A'), mk('b', 'B'), mk('c', 'C', 1)]} />);
await backspaceAt(area('B'));
expect(live.map((b) => [b.content, b.level])).toEqual([
['AB', 0],
['C', 1],
]);
});
it('does nothing when the caret is not at the start', async () => {
render(<Harness initial={[mk('a', 'A'), mk('b', 'BB')]} />);
await backspaceAt(area('BB'), 1);
expect(live).toHaveLength(2);
});
it('does not merge the first bullet', async () => {
render(<Harness initial={[mk('a', 'A'), mk('b', 'B')]} />);
await backspaceAt(area('A'));
expect(live).toHaveLength(2);
});
it('does not merge into a code block', async () => {
render(<Harness initial={[mk('a', 'x = 1', 0, 'code'), mk('b', 'B')]} />);
await backspaceAt(area('B'));
expect(live).toHaveLength(2);
});
});
describe('blocks locked by someone else', () => {
it('are read-only and show who edits them', () => {
render(<Harness initial={[mk('a', 'Frei'), mk('b', 'Gesperrt')]} locked={new Map([['b', 'Anna']])} />);
expect(area('Gesperrt').readOnly).toBe(true);
expect(area('Frei').readOnly).toBe(false);
expect(screen.getByTitle('Anna bearbeitet diesen Punkt gerade')).toBeTruthy();
});
it('merging into a locked bullet is not started at all', async () => {
lockedNotices = [];
render(<Harness initial={[mk('a', 'Oben'), mk('b', 'Unten')]} locked={new Map([['a', 'Anna']])} />);
await backspaceAt(area('Unten'));
expect(live.map((b) => b.content)).toEqual(['Oben', 'Unten']);
expect(lockedNotices).toEqual(['Anna']);
});
it('a branch with a locked child cannot be merged away', async () => {
lockedNotices = [];
render(<Harness initial={[mk('a', 'A'), mk('b', 'B'), mk('c', 'C', 1)]} locked={new Map([['c', 'Ben']])} />);
await backspaceAt(area('B'));
expect(live.map((b) => b.content)).toEqual(['A', 'B', 'C']);
expect(lockedNotices).toEqual(['Ben']);
});
});
describe('Enter on a locked bullet', () => {
for (const [label, pos] of [
['at the end', 8],
['in the middle', 3],
] as const) {
it(`${label} adds a new bullet below and never splits the locked one`, async () => {
render(<Harness initial={[mk('a', 'Gesperrt')]} locked={new Map([['a', 'Anna']])} />);
const ta = area('Gesperrt');
ta.focus();
ta.setSelectionRange(pos, pos);
await act(async () => {
fireEvent.keyDown(ta, { key: 'Enter' });
});
expect(live.map((b) => b.content)).toEqual(['Gesperrt', '']);
});
}
});
+132
View File
@@ -0,0 +1,132 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
// Minimal stand-ins: tests decide when connections open and which messages arrive.
class FakeWebSocket {
static CONNECTING = 0;
static OPEN = 1;
static CLOSING = 2;
static CLOSED = 3;
static instances: FakeWebSocket[] = [];
readyState = FakeWebSocket.CONNECTING;
sent: string[] = [];
onopen: (() => void) | null = null;
onmessage: ((e: { data: string }) => void) | null = null;
onclose: ((e: { wasClean: boolean; code: number }) => void) | null = null;
onerror: ((e: unknown) => void) | null = null;
constructor(public url: string) {
FakeWebSocket.instances.push(this);
}
open() {
this.readyState = FakeWebSocket.OPEN;
this.onopen?.();
}
send(data: string) {
this.sent.push(data);
}
close() {
this.readyState = FakeWebSocket.CLOSED;
}
}
class FakeEventSource {
static instances: FakeEventSource[] = [];
closed = false;
onopen: (() => void) | null = null;
onmessage: ((e: { data: string }) => void) | null = null;
onerror: ((e: unknown) => void) | null = null;
constructor(public url: string) {
FakeEventSource.instances.push(this);
}
open() {
this.onopen?.();
}
emit(event: string, payload: unknown = {}) {
this.onmessage?.({ data: JSON.stringify({ event, payload }) });
}
close() {
this.closed = true;
}
}
async function loadApi() {
vi.resetModules(); // fresh module state (transport flags) for every test
return import('../api');
}
describe('realtime fallback', () => {
beforeEach(() => {
vi.useFakeTimers();
FakeWebSocket.instances = [];
FakeEventSource.instances = [];
vi.stubGlobal('WebSocket', FakeWebSocket);
vi.stubGlobal('EventSource', FakeEventSource);
sessionStorage.clear();
localStorage.setItem('mindsift_jwt_token', 'test-token');
vi.stubGlobal('fetch', vi.fn(async () => new Response('{}', { status: 200 })));
});
afterEach(() => {
vi.useRealTimers();
vi.unstubAllGlobals();
});
it('a WebSocket that stays silent is replaced by SSE', async () => {
const api = await loadApi();
const states: string[] = [];
api.onRealtimeStatus((connected, transport) => states.push(`${connected}:${transport}`));
api.connectWebSocket();
FakeWebSocket.instances[0].open();
expect(api.getRealtimeTransport()).toBe('ws');
await vi.advanceTimersByTimeAsync(15_000);
expect(FakeWebSocket.instances[0].sent).toContain(JSON.stringify({ event: 'PING' }));
await vi.advanceTimersByTimeAsync(40_000); // no message at all, not even PONG
expect(FakeEventSource.instances).toHaveLength(1);
FakeEventSource.instances[0].open();
FakeEventSource.instances[0].emit('SSE_CONNECTED', { sseId: 'sse_1' });
expect(api.getRealtimeTransport()).toBe('sse');
api.disconnectWebSocket();
});
it('a WebSocket answering PONG stays in use', async () => {
const api = await loadApi();
api.connectWebSocket();
const ws = FakeWebSocket.instances[0];
ws.open();
for (let i = 0; i < 6; i++) {
await vi.advanceTimersByTimeAsync(15_000);
ws.onmessage?.({ data: JSON.stringify({ event: 'PONG' }) });
}
expect(FakeEventSource.instances).toHaveLength(0);
expect(api.getRealtimeTransport()).toBe('ws');
api.disconnectWebSocket();
});
it('SSE only counts as connected once the first event arrives', async () => {
sessionStorage.setItem('mindsift_ws_blocked', 'true');
const api = await loadApi();
api.connectWebSocket();
const es = FakeEventSource.instances[0];
es.open();
expect(api.isRealtimeConnected()).toBe(false);
es.emit('SSE_CONNECTED', { sseId: 'sse_1' });
expect(api.isRealtimeConnected()).toBe(true);
await vi.advanceTimersByTimeAsync(10_000);
expect(api.isRealtimeConnected()).toBe(true);
api.disconnectWebSocket();
});
it('a buffering proxy (stream opens, no events) switches SSE off for this tab', async () => {
sessionStorage.setItem('mindsift_ws_blocked', 'true');
const api = await loadApi();
api.connectWebSocket();
const es = FakeEventSource.instances[0];
es.open();
await vi.advanceTimersByTimeAsync(5_500);
expect(es.closed).toBe(true);
expect(api.isRealtimeConnected()).toBe(false);
expect(sessionStorage.getItem('mindsift_sse_buffered')).toBe('true');
await vi.advanceTimersByTimeAsync(60_000);
expect(FakeEventSource.instances).toHaveLength(1); // no reconnect loop: note sync uses long polling
api.disconnectWebSocket();
});
});
+51 -27
View File
@@ -595,11 +595,10 @@ export const api = {
title?: string; title?: string;
icon?: string; icon?: string;
folderPath?: string; folderPath?: string;
blocks?: DocumentBlock[];
tags?: string[];
isPinned?: boolean; isPinned?: boolean;
} }
): Promise<Document> { ): Promise<Document> {
// Block content is saved through the note sync engine (lib/docSync), not here
const d = await request<Document>(`/api/v1/documents/${id}`, { const d = await request<Document>(`/api/v1/documents/${id}`, {
method: 'PUT', method: 'PUT',
body: JSON.stringify(patch), body: JSON.stringify(patch),
@@ -611,25 +610,6 @@ export const api = {
}; };
}, },
async patchDocumentBlocks(
id: string,
payload: {
upserted?: DocumentBlock[];
deletedIds?: string[];
blockOrder?: string[];
}
): Promise<Document> {
const d = await request<Document>(`/api/v1/documents/${id}/blocks`, {
method: 'PATCH',
body: JSON.stringify(payload),
});
return {
...d,
blocks: ensureArray(d.blocks),
tags: ensureArray(d.tags),
};
},
async joinSSERoom(sseId: string, room: string, action: 'join' | 'leave' = 'join'): Promise<{ success: boolean }> { async joinSSERoom(sseId: string, room: string, action: 'join' | 'leave' = 'join'): Promise<{ success: boolean }> {
return request<{ success: boolean }>('/api/v1/events/rooms', { return request<{ success: boolean }>('/api/v1/events/rooms', {
method: 'POST', method: 'POST',
@@ -1134,6 +1114,16 @@ export function onRealtimeStatus(callback: (connected: boolean, transport: Realt
} }
let isWsBlockedByProxy = typeof window !== 'undefined' && sessionStorage.getItem('mindsift_ws_blocked') === 'true'; let isWsBlockedByProxy = typeof window !== 'undefined' && sessionStorage.getItem('mindsift_ws_blocked') === 'true';
// Some proxies buffer event streams: the connection opens, but events only arrive much later (or never).
// Then SSE is useless and the note sync engines fall back to long polling.
let isSseBufferedByProxy = typeof window !== 'undefined' && sessionStorage.getItem('mindsift_sse_buffered') === 'true';
let sseFirstEventWatchdog: any = null;
const SSE_FIRST_EVENT_TIMEOUT_MS = 5000;
// Some proxies accept the WebSocket upgrade but then forward nothing. An app-level ping detects that.
let wsHeartbeatTimer: any = null;
let wsLastMessageAt = 0;
const WS_PING_INTERVAL_MS = 15000;
const WS_DEAD_AFTER_MS = 35000;
let sseReconnectTimer: any = null; let sseReconnectTimer: any = null;
let sseReconnectDelay = 2000; let sseReconnectDelay = 2000;
const globalSubscribedRooms = new Set<string>(); const globalSubscribedRooms = new Set<string>();
@@ -1153,6 +1143,7 @@ function notifyRealtimeStatusChange(connected: boolean, transport: RealtimeTrans
function switchToSSE(reason: string) { function switchToSSE(reason: string) {
console.warn(`[MindSift Realtime] ${reason} -> Schalte sofort und dauerhaft auf Server-Sent Events (SSE) um.`); console.warn(`[MindSift Realtime] ${reason} -> Schalte sofort und dauerhaft auf Server-Sent Events (SSE) um.`);
clearInterval(wsHeartbeatTimer);
isWsBlockedByProxy = true; isWsBlockedByProxy = true;
try { try {
sessionStorage.setItem('mindsift_ws_blocked', 'true'); sessionStorage.setItem('mindsift_ws_blocked', 'true');
@@ -1177,6 +1168,12 @@ export function connectSSE(rooms?: string[]): void {
clearTimeout(sseReconnectTimer); clearTimeout(sseReconnectTimer);
clearTimeout(wsReconnectTimer); clearTimeout(wsReconnectTimer);
clearTimeout(wsConnectionWatchdog); clearTimeout(wsConnectionWatchdog);
clearTimeout(sseFirstEventWatchdog);
if (isSseBufferedByProxy) {
notifyRealtimeStatusChange(false, 'none');
return;
}
if (activeGlobalSSE) { if (activeGlobalSSE) {
try { activeGlobalSSE.close(); } catch {} try { activeGlobalSSE.close(); } catch {}
@@ -1190,18 +1187,28 @@ export function connectSSE(rooms?: string[]): void {
es.onopen = () => { es.onopen = () => {
sseReconnectDelay = 2000; sseReconnectDelay = 2000;
notifyRealtimeStatusChange(true, 'sse'); // Connected only counts once the first event really arrives (see SSE_CONNECTED below)
if (activeSSEId && globalSubscribedRooms.size > 0) { clearTimeout(sseFirstEventWatchdog);
globalSubscribedRooms.forEach((room) => { sseFirstEventWatchdog = setTimeout(() => {
api.joinSSERoom(activeSSEId!, room, 'join').catch(() => {}); console.warn('[MindSift Realtime] SSE liefert keine Ereignisse (Proxy puffert) -> Long-Polling');
}); isSseBufferedByProxy = true;
} try {
sessionStorage.setItem('mindsift_sse_buffered', 'true');
} catch {}
try {
es.close();
} catch {}
if (activeGlobalSSE === es) activeGlobalSSE = null;
notifyRealtimeStatusChange(false, 'none');
}, SSE_FIRST_EVENT_TIMEOUT_MS);
}; };
es.onmessage = (event) => { es.onmessage = (event) => {
try { try {
const msg = JSON.parse(event.data) as WSMessage; const msg = JSON.parse(event.data) as WSMessage;
if (msg.event === 'SSE_CONNECTED') { if (msg.event === 'SSE_CONNECTED') {
clearTimeout(sseFirstEventWatchdog);
notifyRealtimeStatusChange(true, 'sse');
activeSSEId = msg.payload?.sseId || null; activeSSEId = msg.payload?.sseId || null;
if (activeSSEId && globalSubscribedRooms.size > 0) { if (activeSSEId && globalSubscribedRooms.size > 0) {
globalSubscribedRooms.forEach((room) => { globalSubscribedRooms.forEach((room) => {
@@ -1223,6 +1230,7 @@ export function connectSSE(rooms?: string[]): void {
}; };
es.onerror = (e) => { es.onerror = (e) => {
clearTimeout(sseFirstEventWatchdog);
console.warn('[MindSift Realtime] SSE Verbindungsfehler:', e); console.warn('[MindSift Realtime] SSE Verbindungsfehler:', e);
if (activeTransport === 'sse') { if (activeTransport === 'sse') {
notifyRealtimeStatusChange(false, 'none'); notifyRealtimeStatusChange(false, 'none');
@@ -1279,6 +1287,17 @@ export function connectWebSocket(): void {
ws.onopen = () => { ws.onopen = () => {
clearTimeout(wsConnectionWatchdog); clearTimeout(wsConnectionWatchdog);
wsLastMessageAt = Date.now();
clearInterval(wsHeartbeatTimer);
wsHeartbeatTimer = setInterval(() => {
if (Date.now() - wsLastMessageAt > WS_DEAD_AFTER_MS) {
switchToSSE('WebSocket liefert keine Antworten mehr (Proxy leitet nichts weiter)');
return;
}
try {
ws.send(JSON.stringify({ event: 'PING' }));
} catch {}
}, WS_PING_INTERVAL_MS);
if (activeGlobalSSE) { if (activeGlobalSSE) {
try { activeGlobalSSE.close(); } catch {} try { activeGlobalSSE.close(); } catch {}
activeGlobalSSE = null; activeGlobalSSE = null;
@@ -1295,8 +1314,10 @@ export function connectWebSocket(): void {
}; };
ws.onmessage = (event) => { ws.onmessage = (event) => {
wsLastMessageAt = Date.now();
try { try {
const msg = JSON.parse(event.data) as WSMessage; const msg = JSON.parse(event.data) as WSMessage;
if (msg.event === 'PONG') return;
wsListeners.forEach((fn) => { wsListeners.forEach((fn) => {
try { try {
fn(msg); fn(msg);
@@ -1312,6 +1333,7 @@ export function connectWebSocket(): void {
ws.onclose = (evt: CloseEvent) => { ws.onclose = (evt: CloseEvent) => {
clearTimeout(wsConnectionWatchdog); clearTimeout(wsConnectionWatchdog);
clearInterval(wsHeartbeatTimer);
activeGlobalWS = null; activeGlobalWS = null;
// In corporate environments, blocked WebSockets close abruptly without clean handshake // In corporate environments, blocked WebSockets close abruptly without clean handshake
@@ -1368,6 +1390,8 @@ if (typeof window !== 'undefined') {
} }
export function disconnectWebSocket(): void { export function disconnectWebSocket(): void {
clearInterval(wsHeartbeatTimer);
clearTimeout(sseFirstEventWatchdog);
manualDisconnect = true; manualDisconnect = true;
clearTimeout(wsReconnectTimer); clearTimeout(wsReconnectTimer);
clearTimeout(wsConnectionWatchdog); clearTimeout(wsConnectionWatchdog);
@@ -0,0 +1,96 @@
import { describe, it, expect } from 'vitest';
import type { DocumentBlock } from '../../types';
import { diffFlat, toDocumentBlocks } from '../diff';
import { applyEffect, applyOp, cloneDoc, docsEqual, emptyDoc, Doc } from '../model';
let opSeq = 0;
const nextOp = () => `op${++opSeq}`;
function build(blocks: DocumentBlock[]): Doc {
const d = emptyDoc();
for (const op of diffFlat(d, blocks, nextOp)) applyOp(d, op);
return d;
}
const B = (id: string, level: number, content = id, extra: Partial<DocumentBlock> = {}): DocumentBlock => ({
id,
level,
content,
type: 'bullet',
...extra,
});
const shape = (blocks: DocumentBlock[]) => blocks.map((b) => [b.id, b.level || 0, b.content, !!b.checked, b.type]);
describe('diffFlat', () => {
it('builds the tree from levels and produces no ops for an unchanged list', () => {
const list = [B('a', 0), B('b', 1), B('c', 2), B('d', 1), B('e', 0)];
const d = build(list);
expect(shape(toDocumentBlocks(d, 'doc'))).toEqual(shape(list));
expect(diffFlat(d, toDocumentBlocks(d, 'doc'), nextOp)).toEqual([]);
});
it('turns typing into a single set_content without moves', () => {
const d = build([B('a', 0), B('b', 0), B('c', 0)]);
const ops = diffFlat(d, [B('a', 0), B('b', 0, 'b geändert'), B('c', 0)], nextOp);
expect(ops.map((o) => o.kind)).toEqual(['set_content']);
});
it('inserting in the middle does not move the neighbours', () => {
const d = build([B('a', 0), B('c', 0)]);
const ops = diffFlat(d, [B('a', 0), B('b', 0), B('c', 0)], nextOp);
expect(ops.map((o) => o.kind)).toEqual(['insert']);
});
it('reaches any target list (random edits)', () => {
let seed = 99;
const rnd = (n: number) => {
seed = (seed * 1103515245 + 12345) % 2147483648;
return Math.floor((seed / 2147483648) * n);
};
let current: DocumentBlock[] = [B('x0', 0)];
let d = build(current);
for (let step = 0; step < 300; step++) {
const next = current.map((b) => ({ ...b }));
const i = rnd(next.length);
switch (rnd(6)) {
case 0:
next.splice(i + 1, 0, B(`x${step + 1}`, next[i] ? next[i].level || 0 : 0));
break;
case 1:
if (next.length > 1) next.splice(i, 1);
break;
case 2:
next[i].content += '!';
break;
case 3:
next[i].level = (next[i].level || 0) + 1;
break;
case 4:
next[i].level = Math.max(0, (next[i].level || 0) - 1);
break;
case 5: {
const [moved] = next.splice(i, 1);
next.splice(rnd(next.length + 1), 0, moved);
break;
}
}
// Normalise like the editor would (levels can only grow by one)
let prev = -1;
for (const b of next) {
b.level = Math.min(b.level || 0, prev + 1);
prev = b.level;
}
const before = cloneDoc(d);
const replay = cloneDoc(d);
for (const op of diffFlat(d, next, nextOp)) {
const { effects } = applyOp(d, op);
effects.forEach((e) => applyEffect(replay, e));
}
expect(shape(toDocumentBlocks(d, 'doc'))).toEqual(shape(next));
expect(docsEqual(d, replay)).toBe(true);
expect(before).not.toBe(d);
current = next;
}
});
});
@@ -0,0 +1,529 @@
import 'fake-indexeddb/auto';
import { describe, it, expect, vi } from 'vitest';
import type { DocumentBlock } from '../../types';
import { DocSyncEngine, formBatch } from '../engine';
import { applyEffect, applyOp, cloneDoc, Doc, Effect, emptyDoc, Op, Result } from '../model';
import { IndexedDBStorage, MemoryStorage } from '../persist';
import { PushChannel, PushKind } from '../push';
import { LogEntry, SnapshotResponse, SyncRequest, SyncResponse, Transport, TransportError } from '../transport';
/** In-memory stand-in for the Go server: same operation rules, log and idempotency. */
class FakeServer {
doc: Doc = emptyDoc();
log: LogEntry[] = [];
results = new Map<string, Result>();
roles = new Map<string, string>(); // clientId -> role
applyCount = new Map<string, number>();
floor = 0; // log entries up to here were compacted away
private waiters: (() => void)[] = [];
/** Resolves on the next change (used by the long-poll stand-in). */
nextChange(): Promise<void> {
return new Promise((r) => this.waiters.push(r));
}
constructor(initial: { id: string; content: string }[] = [{ id: 'b1', content: 'Start' }]) {
let key = 'a0';
for (const b of initial) {
this.apply({ opId: `seed-${b.id}`, kind: 'insert', blockId: b.id, parentId: '', orderKey: key, content: b.content }, 'seed');
key = String.fromCharCode(key.charCodeAt(0)) + String.fromCharCode(key.charCodeAt(1) + 1);
}
}
apply(op: Op, clientId: string): Result {
const known = this.results.get(op.opId);
if (known) return known;
this.applyCount.set(op.opId, (this.applyCount.get(op.opId) || 0) + 1);
const { effects, result } = applyOp(this.doc, op, clientId);
for (const e of effects) this.log.push({ seq: this.log.length + 1, userId: clientId, clientId, opId: op.opId, effect: e });
if (effects.length > 0) result.seq = this.log.length;
this.results.set(op.opId, result);
if (effects.length > 0) {
const w = this.waiters;
this.waiters = [];
w.forEach((fn) => fn());
}
return result;
}
/** Log compaction: purges the given tombstones (as a log entry) and drops all older log entries. */
compact(purge: string[]) {
this.floor = this.log.length;
const effect: Effect = { kind: 'purge', ids: purge };
applyEffect(this.doc, effect);
this.log.push({ seq: this.log.length + 1, userId: '', clientId: '', opId: '', effect });
}
snapshot(): SnapshotResponse {
return {
docId: 'doc',
title: this.doc.title,
titleVersion: this.doc.titleVersion,
headSeq: this.log.length,
blocks: Array.from(cloneDoc(this.doc).blocks.values()),
role: 'owner',
shared: true,
};
}
sync(body: SyncRequest): SyncResponse {
const role = this.roles.get(body.clientId) || 'owner';
if (body.ops.length > 0 && role === 'viewer') throw new TransportError('nur Lesen', 403);
const results = body.ops.map((op) => this.apply(op, body.clientId));
const compacted = body.lastSeq < this.floor;
return {
results,
resync: compacted || undefined,
ops: compacted ? [] : this.log.filter((e) => e.seq > body.lastSeq),
headSeq: this.log.length,
hasMore: false,
role,
shared: true,
sessions: 2,
presence: [],
leases: { granted: [], denied: [], others: [] },
};
}
}
/** Network between one client and the fake server, with switchable failures. */
class FakeNet implements Transport {
mode: 'ok' | 'down' | 'lose-response' = 'ok';
longPollDown = false;
changesCalls = 0;
duringRequest: (() => void) | null = null; // runs while the request is "on the wire"
constructor(private server: FakeServer, private role = 'owner') {}
async snapshot() {
if (this.mode === 'down') throw new TransportError('offline', 0);
return { ...this.server.snapshot(), role: this.role };
}
async sync(_docId: string, body: SyncRequest) {
if (this.mode === 'down') throw new TransportError('offline', 0);
this.server.roles.set(body.clientId, this.role);
const hook = this.duringRequest;
this.duringRequest = null;
hook?.();
const res = this.server.sync(body);
if (this.mode === 'lose-response') throw new TransportError('timeout', 0);
return res;
}
async leave() {}
async changes(_docId: string, since: number, waitSec: number, signal?: AbortSignal) {
this.changesCalls++;
if (this.mode === 'down' || this.longPollDown) throw new TransportError('blocked', 0);
if (this.server.log.length === since) {
await Promise.race([
this.server.nextChange(),
new Promise((r) => setTimeout(r, waitSec * 1000)),
new Promise((_, reject) => signal?.addEventListener('abort', () => reject(new Error('aborted')))),
]);
}
return { headSeq: this.server.log.length, changed: this.server.log.length !== since };
}
}
/** Push channel controlled by the test. */
class FakePush implements PushChannel {
kind: PushKind = 'none';
private poke: (() => void) | null = null;
private status: ((k: PushKind) => void) | null = null;
subscribe(_docId: string, onPoke: () => void, onStatus: (k: PushKind) => void) {
this.poke = onPoke;
this.status = onStatus;
onStatus(this.kind);
return () => {
this.poke = null;
};
}
setKind(k: PushKind) {
this.kind = k;
this.status?.(k);
}
notify() {
this.poke?.();
}
}
function flatOf(engine: DocSyncEngine): DocumentBlock[] {
return engine.getState().blocks;
}
function edit(engine: DocSyncEngine, fn: (blocks: DocumentBlock[]) => DocumentBlock[]) {
engine.applyFlat(fn(flatOf(engine).map((b) => ({ ...b }))));
}
const contents = (engine: DocSyncEngine) => flatOf(engine).map((b) => b.content);
const serverContents = (server: FakeServer) => docContents(server.doc);
function docContents(d: Doc): string[] {
return Array.from(d.blocks.values())
.filter((b) => !b.deleted)
.sort((a, b) => (a.orderKey < b.orderKey ? -1 : a.orderKey > b.orderKey ? 1 : a.id < b.id ? -1 : 1))
.map((b) => b.content);
}
async function startEngine(net: Transport, storage = new MemoryStorage()) {
const engine = new DocSyncEngine('doc', { transport: net, storage, manual: true });
await engine.start();
return engine;
}
describe('DocSyncEngine', () => {
it('sends edits and confirms them', async () => {
const server = new FakeServer();
const engine = await startEngine(new FakeNet(server));
edit(engine, (b) => {
b[0].content = 'Hallo';
return [...b, { id: 'b2', type: 'bullet', content: 'Neu', level: 1 }];
});
expect(engine.getState().pendingCount).toBeGreaterThan(0);
await engine.syncNow();
expect(engine.getState().status).toBe('saved');
expect(engine.getState().pendingCount).toBe(0);
expect(serverContents(server)).toEqual(['Hallo', 'Neu']);
});
it('coalesces fast typing into one text operation', async () => {
const server = new FakeServer();
const engine = await startEngine(new FakeNet(server));
for (const text of ['H', 'Ha', 'Hal', 'Hall', 'Hallo']) edit(engine, (b) => [{ ...b[0], content: text }]);
await engine.syncNow();
const textOps = server.log.filter((e) => e.effect.kind === 'set_content');
expect(textOps).toHaveLength(1);
expect(textOps[0].effect.content).toBe('Hallo');
});
it('keeps edits through an offline phase and a reload, and applies them exactly once', async () => {
const server = new FakeServer();
const storage = new MemoryStorage();
const net = new FakeNet(server);
const first = await startEngine(net, storage);
net.mode = 'down';
edit(first, (b) => [{ ...b[0], content: 'Offline getippt' }, { id: 'b9', type: 'bullet', content: 'Noch einer', level: 0 }]);
await new Promise((r) => setTimeout(r, 0)); // let the queue reach storage
await first.syncNow();
expect(first.getState().status).toBe('offline');
expect(contents(first)).toEqual(['Offline getippt', 'Noch einer']);
// Tab is closed while offline; a new tab opens the note from storage, still offline
const second = await startEngine(net, storage);
expect(contents(second)).toEqual(['Offline getippt', 'Noch einer']);
net.mode = 'ok';
await second.syncNow();
await second.syncNow();
expect(serverContents(server)).toEqual(['Offline getippt', 'Noch einer']);
for (const count of server.applyCount.values()) expect(count).toBe(1);
});
it('a client offline for weeks reloads after log compaction and keeps its unsent text', async () => {
const server = new FakeServer([
{ id: 'b1', content: 'Start' },
{ id: 'b2', content: 'Wird gelöscht' },
]);
const netA = new FakeNet(server);
const netB = new FakeNet(server);
const a = await startEngine(netA);
const b = await startEngine(netB);
netB.mode = 'down';
edit(b, (blocks) => [{ ...blocks[0], content: 'Start (B)' }, { ...blocks[1], content: 'B tippt weiter' }]);
await b.syncNow();
// Meanwhile A deletes b2, and weeks later the tombstone is purged and the log compacted
edit(a, (blocks) => [blocks[0]]);
await a.syncNow();
server.compact(['b2']);
await a.syncNow();
expect(contents(a)).toEqual(['Start']);
netB.mode = 'ok';
await b.syncNow();
await b.syncNow();
// Nothing typed offline is lost: the edit of the purged block comes back as a new block
expect(serverContents(server)).toEqual(['Start (B)', 'B tippt weiter']);
expect(contents(b)).toEqual(['Start (B)', 'B tippt weiter']);
await a.syncNow();
expect(contents(a)).toEqual(['Start (B)', 'B tippt weiter']);
});
it('retries after a lost response without applying twice', async () => {
const server = new FakeServer();
const net = new FakeNet(server);
const engine = await startEngine(net);
edit(engine, (b) => [...b, { id: 'b2', type: 'bullet', content: 'Einmal', level: 0 }]);
net.mode = 'lose-response';
await engine.syncNow();
expect(engine.getState().status).toBe('offline');
net.mode = 'ok';
await engine.syncNow();
expect(serverContents(server)).toEqual(['Start', 'Einmal']);
expect(server.log.filter((e) => e.effect.kind === 'insert' && e.effect.blockId === 'b2')).toHaveLength(1);
});
it('two clients converge, and a conflict keeps both texts', async () => {
const server = new FakeServer();
const a = await startEngine(new FakeNet(server));
const b = await startEngine(new FakeNet(server));
edit(a, (blocks) => [{ ...blocks[0], content: 'Von A' }]);
edit(b, (blocks) => [{ ...blocks[0], content: 'Von B' }]);
await a.syncNow();
await b.syncNow(); // B's base version is stale -> conflict copy
await a.syncNow();
expect(contents(a)).toEqual(contents(b));
expect(contents(a)).toEqual(['Von A', 'Von B']);
expect(flatOf(a)[1].conflictOf).toBe('b1');
expect(b.getState().conflicts).toHaveLength(1);
});
it('keeps text when two clients create the same first block of an empty note', async () => {
const server = new FakeServer([]);
const a = await startEngine(new FakeNet(server));
const b = await startEngine(new FakeNet(server));
a.applyFlat([{ id: 'doc-b0', type: 'text', content: 'Text von A', level: 0 }]);
b.applyFlat([{ id: 'doc-b0', type: 'text', content: 'Text von B', level: 0 }]);
await a.syncNow();
await b.syncNow(); // insert rejected: block exists
await b.syncNow(); // text resent as set_content -> conflict copy
await a.syncNow();
expect(contents(a)).toEqual(contents(b));
expect(contents(a)).toEqual(expect.arrayContaining(['Text von A', 'Text von B']));
});
it('does not overwrite a change that arrived while the user kept typing', async () => {
const server = new FakeServer([
{ id: 'b1', content: 'Start' },
{ id: 'b2', content: 'Anderer Punkt' },
]);
const netA = new FakeNet(server);
const a = await startEngine(netA);
const b = await startEngine(new FakeNet(server));
edit(a, (blocks) => [blocks[0], { ...blocks[1], content: 'Anderer Punkt!' }]);
netA.duringRequest = () => {
// While A's request travels: B's change of b1 reaches the server first, and A starts typing into
// b1 (still seeing 'Start')
server.apply({ opId: 'b-op', kind: 'set_content', blockId: 'b1', baseVersion: 1, content: 'B war schneller' }, 'cB');
edit(a, (blocks) => [{ ...blocks[0], content: 'Start + A' }, blocks[1]]);
};
await a.syncNow(); // response brings B's change; A's text for b1 is still pending
await a.syncNow();
await b.syncNow();
expect(contents(a)).toEqual(contents(b));
expect(contents(a)).toEqual(expect.arrayContaining(['B war schneller', 'Start + A', 'Anderer Punkt!']));
});
it('a stale focused text field cannot overwrite a newer remote version', async () => {
const server = new FakeServer();
const a = await startEngine(new FakeNet(server));
const b = await startEngine(new FakeNet(server));
a.focusBlock('b1'); // A's cursor stays in the block; its field keeps showing 'Start'
edit(b, (blocks) => [{ ...blocks[0], content: 'B hat geändert' }]);
await b.syncNow();
await a.syncNow(); // A receives B's version, but the focused field still shows the old text
a.applyFlat([{ id: 'b1', type: 'bullet', content: 'Start + A', level: 0 }]);
await a.syncNow();
await b.syncNow();
expect(contents(a)).toEqual(contents(b));
expect(contents(a)).toEqual(expect.arrayContaining(['B hat geändert', 'Start + A']));
});
it('viewers cannot change anything', async () => {
const server = new FakeServer();
const viewer = await startEngine(new FakeNet(server, 'viewer'));
edit(viewer, (b) => [{ ...b[0], content: 'Leser' }]);
await viewer.syncNow();
expect(viewer.getState().pendingCount).toBe(0);
expect(serverContents(server)).toEqual(['Start']);
});
it('changes of others arrive within seconds right after reopening a shared note from local storage', async () => {
vi.useFakeTimers();
try {
const server = new FakeServer();
const net = new FakeNet(server);
net.longPollDown = true; // worst case: only plain polling works
// Opened before: the note is reopened from local storage (like after a reload)
const storage = new MemoryStorage();
const first = new DocSyncEngine('doc', { transport: net, storage, manual: true });
await first.start();
first.stop();
const engine = new DocSyncEngine('doc', { transport: net, storage, push: new FakePush() });
await engine.start();
await vi.advanceTimersByTimeAsync(5_000); // long polling gives up after three attempts
server.apply({ opId: 'remote', kind: 'set_content', blockId: 'b1', baseVersion: 1, content: 'Von anderen' }, 'other');
await vi.advanceTimersByTimeAsync(3_500);
expect(contents(engine)).toEqual(['Von anderen']); // not only after 30 s
engine.stop();
} finally {
vi.useRealTimers();
}
});
it('formBatch keeps the last text per block with the base of the first', () => {
const batch = formBatch([
{ opId: '1', kind: 'set_content', blockId: 'a', content: 'x', baseVersion: 5 },
{ opId: '2', kind: 'set_title', title: 'T1', baseVersion: 3 },
{ opId: '3', kind: 'set_content', blockId: 'a', content: 'xy', baseVersion: 6 },
{ opId: '4', kind: 'set_title', title: 'T2', baseVersion: 4 },
]);
expect(batch).toEqual([
{ opId: '3', kind: 'set_content', blockId: 'a', content: 'xy', baseVersion: 5 },
{ opId: '4', kind: 'set_title', title: 'T2', baseVersion: 3 },
]);
});
it('stores state in IndexedDB', async () => {
const storage = new IndexedDBStorage(indexedDB);
await storage.save({ docId: 'd', title: 'T', titleVersion: 1, lastSeq: 7, role: 'owner', blocks: [], pending: [], inflight: null, savedAt: 1 });
const loaded = await storage.load('d');
expect(loaded?.lastSeq).toBe(7);
expect(await storage.load('missing')).toBeNull();
});
});
describe('how changes of others arrive', () => {
async function live(push: FakePush, net: FakeNet) {
const engine = new DocSyncEngine('doc', { transport: net, storage: new MemoryStorage(), push });
await engine.start();
await vi.advanceTimersByTimeAsync(100); // let the start-up round trips settle
return engine;
}
const otherEdit = (server: FakeServer, text: string) =>
server.apply({ opId: `other-${text}`, kind: 'set_content', blockId: 'b1', baseVersion: server.doc.blocks.get('b1')!.version, content: text }, 'other');
it('push: syncs on a notification and otherwise only checks every 30 s', async () => {
vi.useFakeTimers();
try {
const server = new FakeServer();
const net = new FakeNet(server);
const push = new FakePush();
push.kind = 'ws';
const engine = await live(push, net);
expect(engine.getState().channel).toBe('ws');
otherEdit(server, 'Von anderen');
await vi.advanceTimersByTimeAsync(20_000);
expect(contents(engine)).toEqual(['Start']); // no notification, no poll yet
push.notify();
await vi.advanceTimersByTimeAsync(100);
expect(contents(engine)).toEqual(['Von anderen']);
expect(net.changesCalls).toBe(0);
engine.stop();
} finally {
vi.useRealTimers();
}
});
it('push: renews leases every 7 s while a block is held', async () => {
vi.useFakeTimers();
try {
const server = new FakeServer();
const net = new FakeNet(server);
const push = new FakePush();
push.kind = 'sse';
const engine = await live(push, net);
let syncs = 0;
const orig = net.sync.bind(net);
net.sync = async (...a: Parameters<typeof orig>) => {
syncs++;
return orig(...a);
};
engine.focusBlock('b1');
await vi.advanceTimersByTimeAsync(15_000);
expect(syncs).toBeGreaterThanOrEqual(2);
engine.stop();
} finally {
vi.useRealTimers();
}
});
it('without push: long polling delivers changes right away', async () => {
vi.useFakeTimers();
try {
const server = new FakeServer();
const net = new FakeNet(server);
const engine = await live(new FakePush(), net);
await vi.advanceTimersByTimeAsync(10);
expect(engine.getState().channel).toBe('longpoll');
otherEdit(server, 'Per Long-Polling');
await vi.advanceTimersByTimeAsync(200);
expect(contents(engine)).toEqual(['Per Long-Polling']);
engine.stop();
} finally {
vi.useRealTimers();
}
});
it('falls back to polling every 3 s when long polling is blocked too', async () => {
vi.useFakeTimers();
try {
const server = new FakeServer();
const net = new FakeNet(server);
net.longPollDown = true;
const engine = await live(new FakePush(), net);
await vi.advanceTimersByTimeAsync(5_000); // three failed attempts with back-off
expect(engine.getState().channel).toBe('interval');
otherEdit(server, 'Per Abfrage');
await vi.advanceTimersByTimeAsync(3_500);
expect(contents(engine)).toEqual(['Per Abfrage']);
engine.stop();
} finally {
vi.useRealTimers();
}
});
it('push: requests the lease right after focusing a block', async () => {
vi.useFakeTimers();
try {
const server = new FakeServer();
const net = new FakeNet(server);
const push = new FakePush();
push.kind = 'ws';
const engine = await live(push, net);
const holds: string[][] = [];
const orig = net.sync.bind(net);
net.sync = async (id: string, body: SyncRequest, o?: { keepalive?: boolean }) => {
holds.push(body.leases?.hold || []);
return orig(id, body, o);
};
engine.focusBlock('b1');
await vi.advanceTimersByTimeAsync(1_000);
expect(holds.some((h) => h.includes('b1'))).toBe(true);
engine.stop();
} finally {
vi.useRealTimers();
}
});
it('switches between push and long polling when the connection changes', async () => {
vi.useFakeTimers();
try {
const server = new FakeServer();
const net = new FakeNet(server);
const push = new FakePush();
push.kind = 'ws';
const engine = await live(push, net);
push.setKind('none');
await vi.advanceTimersByTimeAsync(10);
expect(engine.getState().channel).toBe('longpoll');
push.setKind('ws');
await vi.advanceTimersByTimeAsync(10);
expect(engine.getState().channel).toBe('ws');
otherEdit(server, 'Wieder per Push');
push.notify();
await vi.advanceTimersByTimeAsync(100);
expect(contents(engine)).toEqual(['Wieder per Push']);
engine.stop();
} finally {
vi.useRealTimers();
}
});
});
@@ -0,0 +1,46 @@
import { describe, it, expect } from 'vitest';
import { keyBetween, keysBetween, validOrderKey } from '../orderKey';
// Generated with internal/collab (Go); both implementations must agree exactly.
const VECTORS: [string, string, string][] = [
['', '', 'a0'],
['a0', '', 'a1'],
['', 'a0', 'Zz'],
['a0', 'a1', 'a0V'],
['a0', 'a0V', 'a0F'],
['a0V', 'a1', 'a0k'],
['Zz', 'a0', 'ZzV'],
['az', '', 'b00'],
['', 'Zz', 'Zy'],
['a1', 'a1V', 'a1F'],
['b00', 'b01', 'b00V'],
];
describe('order keys', () => {
it('match the Go implementation', () => {
for (const [a, b, want] of VECTORS) expect(keyBetween(a, b)).toBe(want);
expect(keysBetween('a0', 'a1', 5)).toEqual(['a07', 'a0F', 'a0V', 'a0c', 'a0k']);
let last = '';
for (let i = 0; i < 70; i++) last = keyBetween(last, '');
expect(last).toBe('b07');
});
it('rejects invalid ranges and keys', () => {
expect(() => keyBetween('zz', '')).toThrow();
expect(() => keyBetween('a1', 'a0')).toThrow();
expect(validOrderKey('a0V0')).toBe(false);
});
it('stays sorted under random inserts', () => {
const keys: string[] = [];
let seed = 7;
const rnd = () => (seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648;
for (let i = 0; i < 2000; i++) {
const pos = Math.floor(rnd() * (keys.length + 1));
keys.splice(pos, 0, keyBetween(keys[pos - 1] || '', keys[pos] || ''));
}
const sorted = [...keys].sort();
expect(keys).toEqual(sorted);
expect(new Set(keys).size).toBe(keys.length);
});
});
@@ -0,0 +1,185 @@
import { describe, it, expect } from 'vitest';
import type { DocumentBlock } from '../../types';
import { DocSyncEngine } from '../engine';
import { applyOp, Doc, emptyDoc, Op, Result } from '../model';
import { MemoryStorage } from '../persist';
import { LogEntry, SyncRequest, Transport, TransportError } from '../transport';
/** Minimal stand-in server (same rules as the Go server, plus configurable leases of others). */
class Server {
doc: Doc = emptyDoc();
log: LogEntry[] = [];
results = new Map<string, Result>();
othersLeases: { blockId: string; userId: string; name: string }[] = [];
constructor(blocks: [string, string][]) {
blocks.forEach(([id, content], i) => this.apply({ opId: `seed${i}`, kind: 'insert', blockId: id, parentId: '', orderKey: `a${i}`, content }, 'seed'));
}
apply(op: Op, by: string): Result {
const known = this.results.get(op.opId);
if (known) return known;
const { effects, result } = applyOp(this.doc, op, by);
for (const e of effects) this.log.push({ seq: this.log.length + 1, userId: by, clientId: by, opId: op.opId, effect: e });
if (effects.length) result.seq = this.log.length;
this.results.set(op.opId, result);
return result;
}
transport(): Transport {
return {
snapshot: async () => ({
docId: 'd',
title: this.doc.title,
titleVersion: this.doc.titleVersion,
headSeq: this.log.length,
blocks: Array.from(this.doc.blocks.values()).map((b) => ({ ...b, props: { ...b.props } })),
role: 'owner',
shared: true,
}),
sync: async (_id: string, body: SyncRequest) => {
if (!body) throw new TransportError('x', 0);
const results = body.ops.map((op) => this.apply(op, body.clientId));
return {
results,
ops: this.log.filter((e) => e.seq > body.lastSeq),
headSeq: this.log.length,
hasMore: false,
role: 'owner',
shared: true,
sessions: 2,
presence: [],
leases: { granted: [], denied: [], others: this.othersLeases },
};
},
leave: async () => {},
changes: async () => ({ headSeq: this.log.length, changed: false }),
};
}
}
let clock = 1_000_000;
async function engineFor(server: Server) {
const e = new DocSyncEngine('d', { transport: server.transport(), storage: new MemoryStorage(), manual: true, now: () => clock });
await e.start();
return e;
}
const flat = (e: DocSyncEngine) => e.getState().blocks;
const texts = (e: DocSyncEngine) => flat(e).map((b) => b.content);
function edit(e: DocSyncEngine, fn: (b: DocumentBlock[]) => DocumentBlock[]) {
e.applyFlat(fn(flat(e).map((b) => ({ ...b }))));
}
describe('undo/redo', () => {
it('groups typing in one block and reverts it in one step', async () => {
const server = new Server([['a', 'Hallo']]);
const e = await engineFor(server);
edit(e, (b) => [{ ...b[0], content: 'Hallo W' }]);
clock += 300;
edit(e, (b) => [{ ...b[0], content: 'Hallo Welt' }]);
expect(e.undo().ok).toBe(true);
expect(texts(e)).toEqual(['Hallo']);
expect(e.getState().canUndo).toBe(false);
expect(e.redo().ok).toBe(true);
expect(texts(e)).toEqual(['Hallo Welt']);
await e.syncNow();
expect(server.doc.blocks.get('a')!.content).toBe('Hallo Welt');
});
it('separates typing steps after a pause', async () => {
const server = new Server([['a', '']]);
const e = await engineFor(server);
edit(e, (b) => [{ ...b[0], content: 'Eins' }]);
clock += 2000;
edit(e, (b) => [{ ...b[0], content: 'Eins Zwei' }]);
e.undo();
expect(texts(e)).toEqual(['Eins']);
});
it('undoes inserting and deleting blocks', async () => {
const server = new Server([['a', 'Erster']]);
const e = await engineFor(server);
edit(e, (b) => [...b, { id: 'n', type: 'bullet', content: 'Neu', level: 0 }]);
clock += 2000;
edit(e, (b) => b.filter((x) => x.id !== 'a'));
expect(texts(e)).toEqual(['Neu']);
e.undo();
expect(texts(e)).toEqual(['Erster', 'Neu']);
e.undo();
expect(texts(e)).toEqual(['Erster']);
await e.syncNow();
expect(server.doc.blocks.get('a')!.deleted).toBe(false);
expect(server.doc.blocks.get('n')!.deleted).toBe(true);
});
it('skips undo when someone else changed the block in between', async () => {
const server = new Server([['a', 'Start']]);
const e = await engineFor(server);
edit(e, (b) => [{ ...b[0], content: 'Meins' }]);
await e.syncNow();
server.apply({ opId: 'other', kind: 'set_content', blockId: 'a', baseVersion: 2, content: 'Fremd' }, 'other');
await e.syncNow();
const res = e.undo();
expect(res.ok).toBe(false);
expect(res.message).toMatch(/jemand anderem/);
expect(texts(e)).toEqual(['Fremd']);
});
it('skips undo on a block someone else is editing', async () => {
const server = new Server([['a', 'Start']]);
const e = await engineFor(server);
edit(e, (b) => [{ ...b[0], content: 'Meins' }]);
await e.syncNow();
server.othersLeases = [{ blockId: 'a', userId: 'u2', name: 'Anna' }];
await e.syncNow();
const res = e.undo();
expect(res.ok).toBe(false);
expect(res.message).toMatch(/Anna/);
expect(texts(e)).toEqual(['Meins']);
});
});
describe('conflict copies', () => {
async function withConflict() {
const server = new Server([['a', 'Start']]);
const e = await engineFor(server);
server.apply({ opId: 'first', kind: 'set_content', blockId: 'a', baseVersion: 1, content: 'Gewinner' }, 'other');
server.apply({ opId: 'second', kind: 'set_content', blockId: 'a', baseVersion: 1, content: 'Verlierer' }, 'mine');
await e.syncNow();
const copy = flat(e).find((b) => b.conflictOf === 'a')!;
return { server, e, copy };
}
it('accepting replaces the original text and removes the copy', async () => {
const { server, e, copy } = await withConflict();
expect(e.resolveConflict(copy.id, true).ok).toBe(true);
expect(texts(e)).toEqual(['Verlierer']);
await e.syncNow();
expect(server.doc.blocks.get('a')!.content).toBe('Verlierer');
expect(server.doc.blocks.get(copy.id)!.deleted).toBe(true);
e.undo();
expect(texts(e)).toEqual(['Gewinner', 'Verlierer']);
});
it('is not started while someone else holds the copy', async () => {
const { server, e, copy } = await withConflict();
server.othersLeases = [{ blockId: copy.id, userId: 'u2', name: 'Ben' }];
await e.syncNow();
const res = e.resolveConflict(copy.id, true);
expect(res.ok).toBe(false);
expect(res.message).toMatch(/Ben/);
expect(texts(e)).toEqual(['Gewinner', 'Verlierer']);
});
it('discarding removes only the copy', async () => {
const { e, copy } = await withConflict();
e.resolveConflict(copy.id, false);
expect(texts(e)).toEqual(['Gewinner']);
});
it('a copy whose original is gone becomes a normal block', async () => {
const { server, e, copy } = await withConflict();
server.apply({ opId: 'del', kind: 'delete', blockId: 'a' }, 'other');
await e.syncNow();
e.resolveConflict(copy.id, true);
expect(flat(e).find((b) => b.id === copy.id)?.conflictOf).toBeUndefined();
});
});
+139
View File
@@ -0,0 +1,139 @@
// Turns the flat block list the editor works with (level + order) into operations against the tree.
// Same algorithm as legacyOps in internal/storage/docsync.go.
import type { DocumentBlock } from '../types';
import { keysBetween } from './orderKey';
import { Doc, Op, flatten, normProps, propsEqual } from './model';
/** Parent index (-1 = top level) for every entry of a flat list; impossible levels are clamped. */
export function parentsFromLevels(levels: number[]): number[] {
const parents: number[] = [];
const stack: number[] = [];
levels.forEach((raw, i) => {
let lvl = Math.max(0, raw || 0);
if (lvl > stack.length) lvl = stack.length;
stack.length = lvl;
parents.push(lvl > 0 ? stack[lvl - 1] : -1);
stack.push(i);
});
return parents;
}
/** Marks a longest strictly increasing subsequence of the non-empty keys. */
export function longestIncreasing(keys: string[]): boolean[] {
const keep = new Array(keys.length).fill(false);
const tails: number[] = [];
const prev = new Array(keys.length).fill(-1);
keys.forEach((k, i) => {
if (!k) return;
let lo = 0;
let hi = tails.length;
while (lo < hi) {
const mid = (lo + hi) >> 1;
if (keys[tails[mid]] < k) lo = mid + 1;
else hi = mid;
}
if (lo > 0) prev[i] = tails[lo - 1];
if (lo === tails.length) tails.push(i);
else tails[lo] = i;
});
if (tails.length > 0) {
for (let i = tails[tails.length - 1]; i >= 0; i = prev[i]) keep[i] = true;
}
return keep;
}
/**
* Computes the operations that turn the tree into the given flat list. newOpId creates unique op IDs.
* Blocks that keep their place keep their order key, so only real changes become operations.
*/
export function diffFlat(d: Doc, blocks: DocumentBlock[], newOpId: () => string): Op[] {
const list = blocks.map((b) => ({ ...b, type: b.type || 'bullet' }));
const parentIdx = parentsFromLevels(list.map((b) => b.level || 0));
const parentOf = list.map((_, i) => (parentIdx[i] >= 0 ? list[parentIdx[i]].id : ''));
const byParent = new Map<string, number[]>();
list.forEach((_, i) => {
const p = parentOf[i];
const arr = byParent.get(p);
if (arr) arr.push(i);
else byParent.set(p, [i]);
});
const keys: string[] = new Array(list.length);
byParent.forEach((idxs, parent) => {
const existing = idxs.map((i) => {
const b = d.blocks.get(list[i].id);
return b && !b.deleted && b.parentId === parent ? b.orderKey : '';
});
const keep = longestIncreasing(existing);
let j = 0;
while (j < idxs.length) {
if (keep[j]) {
keys[idxs[j]] = existing[j];
j++;
continue;
}
const start = j;
while (j < idxs.length && !keep[j]) j++;
const left = start > 0 ? keys[idxs[start - 1]] : '';
const right = j < idxs.length ? existing[j] : '';
let fresh: string[];
try {
fresh = keysBetween(left, right, j - start);
} catch {
fresh = keysBetween(left, '', j - start);
}
for (let k = start; k < j; k++) keys[idxs[k]] = fresh[k - start];
}
});
const ops: Op[] = [];
const push = (op: Omit<Op, 'opId'>) => ops.push({ opId: newOpId(), ...op } as Op);
const wanted = new Set<string>();
list.forEach((b, i) => {
wanted.add(b.id);
const parent = parentOf[i];
const props = normProps({ checked: b.checked, collapsed: b.collapsed, lang: b.lang });
const cur = d.blocks.get(b.id);
if (!cur) {
push({ kind: 'insert', blockId: b.id, parentId: parent, orderKey: keys[i], type: b.type, content: b.content || '', props });
return;
}
if (cur.deleted) push({ kind: 'restore', ids: [b.id] });
if (cur.deleted || cur.parentId !== parent || cur.orderKey !== keys[i]) {
push({ kind: 'move', blockId: b.id, parentId: parent, orderKey: keys[i] });
}
if (cur.content !== (b.content || '')) {
push({ kind: 'set_content', blockId: b.id, baseVersion: cur.version, content: b.content || '' });
}
const curProps = normProps({ checked: cur.props.checked, collapsed: cur.props.collapsed, lang: cur.props.lang });
if (cur.type !== b.type || !propsEqual(curProps, props)) {
push({ kind: 'set_props', blockId: b.id, patch: { type: b.type, checked: !!b.checked, collapsed: !!b.collapsed, lang: b.lang || '' } });
}
});
// Deletes last, after moves have taken wanted children out of deleted branches
for (const fb of flatten(d)) {
if (!wanted.has(fb.id)) push({ kind: 'delete', blockId: fb.id });
}
return ops;
}
/** Converts the tree into the flat list used by the editor components. */
export function toDocumentBlocks(d: Doc, docId: string, taskLinks?: Map<string, string>): DocumentBlock[] {
return flatten(d).map((b, i) => ({
id: b.id,
documentId: docId,
type: b.type as DocumentBlock['type'],
content: b.content,
checked: !!b.props.checked,
collapsed: !!b.props.collapsed,
lang: b.props.lang || undefined,
level: b.level,
position: (i + 1) * 1000,
taskId: taskLinks?.get(b.id),
conflictOf: b.props.conflictOf,
conflictAuthor: b.props.conflictAuthor,
}));
}
+714
View File
@@ -0,0 +1,714 @@
// DocSyncEngine keeps one note in sync with the server: confirmed state from the server log, plus the
// user's own unconfirmed operations on top. Every change is stored in IndexedDB before it is sent, sent
// with a unique op ID (safe to resend), and the server decides about conflicts. See
// Doku/plans/kollaboration-notizen.md.
import type { DocumentBlock } from '../types';
import { diffFlat, toDocumentBlocks } from './diff';
import { applyEffect, applyOp, cloneDoc, Doc, docFromSnapshot, emptyDoc, Op, Result } from './model';
import { defaultStorage, SyncStorage } from './persist';
import { httpTransport, LeaseInfo, Presence, Transport, TransportError } from './transport';
import { buildStep, failedChecks, touchedBlocks, UndoStep } from './undo';
import { PushChannel, PushKind, realtimePush } from './push';
export type EngineStatus = 'loading' | 'saved' | 'saving' | 'offline' | 'error';
/** How changes of others reach this client: push (WebSocket/SSE), long polling or plain polling. */
export type SyncChannel = 'ws' | 'sse' | 'longpoll' | 'interval';
export interface EngineState {
ready: boolean;
status: EngineStatus;
errorMessage?: string;
role: string; // owner | editor | viewer | '' (unknown yet)
title: string;
blocks: DocumentBlock[];
pendingCount: number;
shared: boolean;
sessions: number;
presence: Presence[];
lockedByOthers: LeaseInfo[];
conflicts: Result[]; // conflicts of own operations since start
canUndo: boolean;
canRedo: boolean;
channel: SyncChannel;
}
export interface ActionResult {
ok: boolean;
message?: string;
}
export interface EngineOptions {
transport?: Transport;
storage?: SyncStorage;
now?: () => number;
/** Disables timers, window listeners, push and long polling (tests drive the engine with syncNow()). */
manual?: boolean;
/** Push notifications; defaults to the app's realtime connection (not used in manual mode). */
push?: PushChannel;
}
const LEASE_GRACE_MS = 20_000; // keep a left block for this long
const IDLE_RELEASE_MS = 60_000; // release even the focused block after this long without input
const SYNC_DEBOUNCE_MS = 300;
const POLL_COLLAB_MS = 3_000; // shared note without push or long polling
const POLL_SOLO_MS = 30_000; // private note, opened once
const HEARTBEAT_MS = 7_000; // renews block leases while changes arrive by push/long polling
const SAFETY_POLL_MS = 30_000; // in case a push notification got lost
const LONG_POLL_WAIT_S = 25; // below typical proxy timeouts (nginx 60 s, Cloudflare 100 s)
const LONG_POLL_MAX_FAILURES = 3;
const BACKOFF_MAX_MS = 30_000;
const UNDO_LIMIT = 200;
const UNDO_GROUP_MS = 1_000; // keystrokes in the same block are one undo step until a pause this long
function randomHex(bytes: number): string {
const arr = new Uint8Array(bytes);
crypto.getRandomValues(arr);
return Array.from(arr, (b) => b.toString(16).padStart(2, '0')).join('');
}
/** Random block ID for new blocks (unique across clients). */
export function newBlockId(): string {
return `blk_${randomHex(10)}`;
}
/**
* Keeps only the last text change per block (and the last title change). The base version is the one of
* the first change: all later keystrokes build on what the user saw when starting to type, so a change
* from someone else in between is detected as a conflict instead of being overwritten.
*/
export function formBatch(pending: Op[]): Op[] {
const lastText = new Map<string, number>();
const firstTextBase = new Map<string, number | undefined>();
let lastTitle = -1;
let firstTitleBase: number | undefined;
pending.forEach((op, i) => {
if (op.kind === 'set_content') {
lastText.set(op.blockId!, i);
if (!firstTextBase.has(op.blockId!)) firstTextBase.set(op.blockId!, op.baseVersion);
}
if (op.kind === 'set_title') {
if (lastTitle < 0) firstTitleBase = op.baseVersion;
lastTitle = i;
}
});
const batch: Op[] = [];
pending.forEach((op, i) => {
if (op.kind === 'set_content') {
if (lastText.get(op.blockId!) !== i) return;
batch.push({ ...op, baseVersion: firstTextBase.get(op.blockId!) ?? 0 });
return;
}
if (op.kind === 'set_title') {
if (i !== lastTitle) return;
batch.push({ ...op, baseVersion: firstTitleBase ?? 0 });
return;
}
batch.push(op);
});
return batch;
}
export class DocSyncEngine {
readonly clientId = `c_${randomHex(8)}`;
private transport: Transport;
private storage: SyncStorage;
private now: () => number;
private manual: boolean;
private confirmed: Doc = emptyDoc();
private lastSeq = 0;
private pending: Op[] = [];
private inflight: Op[] | null = null;
private local: Doc = emptyDoc();
private opCounter = 0;
private ready = false;
private status: EngineStatus = 'loading';
private errorMessage?: string;
private role = '';
private shared = false;
private sessions = 0;
private presence: Presence[] = [];
private lockedByOthers: LeaseInfo[] = [];
private conflicts: Result[] = [];
private taskLinks = new Map<string, string>();
private focused: string | null = null;
private activity = new Map<string, number>(); // blockId -> last edit/focus time
// Version of each block the editor last showed in agreement with the engine. The focused text field does
// not take over remote changes while typing, so its edits are based on this version, not the newest one.
private uiSeen = new Map<string, number>();
private undoStack: UndoStep[] = [];
private redoStack: UndoStep[] = [];
private push: PushChannel | null;
private pushKind: PushKind = 'none';
private unsubscribePush: (() => void) | null = null;
private pokeTimer: ReturnType<typeof setTimeout> | null = null;
private longPollRunning = false;
private longPollBroken = false;
private longPollAbort: AbortController | null = null;
private syncing = false;
private syncAgain = false;
private stopped = false;
private timer: ReturnType<typeof setTimeout> | null = null;
private backoff = 0;
private listeners = new Set<(s: EngineState) => void>();
private cleanup: Array<() => void> = [];
constructor(readonly docId: string, opts: EngineOptions = {}) {
this.transport = opts.transport || httpTransport;
this.storage = opts.storage || defaultStorage();
this.now = opts.now || (() => Date.now());
this.manual = !!opts.manual;
this.push = opts.push || (this.manual ? null : realtimePush);
}
// --- Public API ---
/** Loads the stored state (instant, works offline), then syncs with the server. */
async start(): Promise<void> {
try {
const stored = await this.storage.load(this.docId);
if (stored) {
this.confirmed = docFromSnapshot(stored.title, stored.titleVersion, stored.blocks);
this.lastSeq = stored.lastSeq;
this.role = stored.role || '';
this.pending = stored.pending || [];
this.inflight = stored.inflight || null;
this.ready = true;
this.recompute();
}
} catch {
// Storage unavailable: continue with the server state only
}
if (!this.manual) this.attachWindowListeners();
if (this.push) {
this.unsubscribePush = this.push.subscribe(
this.docId,
() => this.onPoke(),
(kind) => this.onPushStatus(kind)
);
}
if (!this.ready) {
try {
await this.loadSnapshot();
} catch (err) {
this.handleError(err);
this.emit();
this.schedule(this.nextBackoff());
return;
}
}
this.emit();
await this.syncNow();
}
/** Flushes pending changes (keepalive) and leaves the note. */
stop(): void {
if (this.stopped) return;
this.flushOnExit();
this.stopped = true;
if (this.timer) clearTimeout(this.timer);
if (this.pokeTimer) clearTimeout(this.pokeTimer);
this.longPollAbort?.abort();
this.unsubscribePush?.();
this.unsubscribePush = null;
this.cleanup.forEach((fn) => fn());
this.cleanup = [];
this.listeners.clear();
}
subscribe(fn: (s: EngineState) => void): () => void {
this.listeners.add(fn);
fn(this.getState());
return () => this.listeners.delete(fn);
}
getState(): EngineState {
return {
ready: this.ready,
status: this.status,
errorMessage: this.errorMessage,
role: this.role,
title: this.local.title,
blocks: toDocumentBlocks(this.local, this.docId, this.taskLinks),
pendingCount: this.pending.length + (this.inflight?.length || 0),
shared: this.shared,
sessions: this.sessions,
presence: this.presence,
lockedByOthers: this.lockedByOthers,
conflicts: this.conflicts,
canUndo: this.undoStack.length > 0,
canRedo: this.redoStack.length > 0,
channel: this.channel(),
};
}
get canEdit(): boolean {
return this.role === 'owner' || this.role === 'editor';
}
/** Task IDs of blocks linked to tasks (kept outside the synced tree). */
setTaskLinks(links: Map<string, string>): void {
this.taskLinks = new Map(links);
this.emit();
}
/** Records the editor's complete block list; differences become operations. */
applyFlat(blocks: DocumentBlock[]): void {
if (!this.ready || !this.canEdit) return;
for (const b of blocks) {
if (b.taskId) this.taskLinks.set(b.id, b.taskId);
}
for (const b of blocks) {
const cur = this.local.blocks.get(b.id);
if (cur && !cur.deleted && cur.content === (b.content || '')) this.uiSeen.set(b.id, cur.version);
}
const ops = diffFlat(this.local, blocks, () => this.newOpId());
if (ops.length === 0) return;
const before = this.local;
const t = this.now();
for (const op of ops) {
if ((op.kind === 'insert' || op.kind === 'set_content') && op.blockId) this.activity.set(op.blockId, t);
if (op.kind === 'set_content' && this.uiSeen.has(op.blockId!)) op.baseVersion = this.uiSeen.get(op.blockId!);
}
this.enqueue(ops);
this.record(before, ops);
// The editor now shows exactly the local state for these blocks
for (const op of ops) {
if (op.kind === 'set_content' || op.kind === 'insert') {
const cur = this.local.blocks.get(op.blockId!);
if (cur) this.uiSeen.set(op.blockId!, cur.version);
}
}
}
setTitle(title: string): void {
if (!this.ready || !this.canEdit || title === this.local.title) return;
// The title field always shows the engine's title, so the current local version is what the user saw
const before = this.local;
const ops: Op[] = [{ opId: this.newOpId(), kind: 'set_title', title, baseVersion: this.local.titleVersion }];
this.enqueue(ops);
this.record(before, ops);
}
/** Name of the user holding the lease on a block, if it is someone else. */
lockedBy(blockId: string): string | undefined {
return this.lockedByOthers.find((l) => l.blockId === blockId)?.name;
}
/** Reverts the user's last own change (skipped if someone else changed the blocks since). */
undo(): ActionResult {
return this.replay(this.undoStack, this.redoStack, 'Rückgängig');
}
/** Re-applies the last undone change. */
redo(): ActionResult {
return this.replay(this.redoStack, this.undoStack, 'Wiederholen');
}
/** Accepts (replaces the original with the conflict copy) or discards a conflict copy. */
resolveConflict(copyId: string, accept: boolean): ActionResult {
if (!this.ready || !this.canEdit) return { ok: false };
const copy = this.local.blocks.get(copyId);
if (!copy || copy.deleted || !copy.props.conflictOf) return { ok: false };
const target = copy.props.conflictOf;
// The author of the conflicting text holds the copy for a moment; deleting it now would fail half-way
const copyHolder = this.lockedBy(copyId);
if (copyHolder) return { ok: false, message: `${copyHolder} bearbeitet diese Version gerade – bitte kurz warten` };
const ops: Op[] = [];
if (!accept) {
ops.push({ opId: this.newOpId(), kind: 'delete', blockId: copyId });
} else if (target === 'title') {
ops.push({ opId: this.newOpId(), kind: 'set_title', title: copy.content, baseVersion: this.local.titleVersion });
ops.push({ opId: this.newOpId(), kind: 'delete', blockId: copyId });
} else {
const original = this.local.blocks.get(target);
if (original && !original.deleted) {
const holder = this.lockedBy(target);
if (holder) return { ok: false, message: `Der ursprüngliche Punkt wird gerade von ${holder} bearbeitet` };
ops.push({ opId: this.newOpId(), kind: 'set_content', blockId: target, content: copy.content, baseVersion: original.version });
ops.push({ opId: this.newOpId(), kind: 'delete', blockId: copyId });
} else {
// The original is gone: the copy simply becomes a normal block
ops.push({ opId: this.newOpId(), kind: 'set_props', blockId: copyId, patch: { clearConflict: true } });
}
}
const before = this.local;
this.enqueue(ops);
this.record(before, ops);
return { ok: true };
}
/** Tells the engine which block the cursor is in (null = none); used for block leases. */
focusBlock(blockId: string | null): void {
const t = this.now();
const changed = blockId !== this.focused;
if (this.focused) this.activity.set(this.focused, t);
this.focused = blockId;
if (blockId) this.activity.set(blockId, t);
// Announce the lease right away: with push the next regular sync may be up to 30 s away
if (changed && blockId && this.canEdit && this.collaborative()) {
if (this.syncing) this.syncAgain = true;
else this.schedule(SYNC_DEBOUNCE_MS);
}
}
/** Runs one sync round trip now (or right after the running one). */
async syncNow(): Promise<void> {
if (this.stopped) return;
if (this.syncing) {
this.syncAgain = true;
return;
}
if (this.timer) {
clearTimeout(this.timer);
this.timer = null;
}
this.syncing = true;
let failed = false;
try {
await this.roundTrip();
this.backoff = 0;
} catch (err) {
this.handleError(err);
failed = true;
} finally {
this.syncing = false;
this.recompute();
if (!failed) this.ensureLongPoll();
// Decided only now: the response tells whether the note is shared and how changes arrive
const delay = failed
? this.status === 'error'
? this.pollDelay()
: this.nextBackoff()
: this.syncAgain || this.pending.length > 0
? 0
: this.pollDelay();
this.syncAgain = false;
this.emit();
this.schedule(delay);
}
}
// --- Internals ---
private newOpId(): string {
this.opCounter++;
return `${this.clientId}.${this.opCounter}`;
}
private enqueue(ops: Op[]): void {
this.pending.push(...ops);
this.recompute();
if (this.status !== 'offline' && this.status !== 'error') this.status = 'saving';
this.emit();
// Store first, then send: typed text must survive a crash between the two
void this.persist().finally(() => this.schedule(SYNC_DEBOUNCE_MS));
}
private record(before: Doc, ops: Op[]): void {
const step = buildStep(before, ops, this.now());
if (!step) return;
this.redoStack = [];
const top = this.undoStack[this.undoStack.length - 1];
if (step.textKey && top && top.textKey === step.textKey && step.at - top.at < UNDO_GROUP_MS) {
// Same block, still typing: keep the original text as undo target, update the expected result
top.checks = step.checks;
top.at = step.at;
} else {
this.undoStack.push(step);
if (this.undoStack.length > UNDO_LIMIT) this.undoStack.shift();
}
this.emit();
}
private replay(from: UndoStep[], to: UndoStep[], label: string): ActionResult {
if (!this.ready || !this.canEdit) return { ok: false };
const step = from.pop();
if (!step) return { ok: false };
if (failedChecks(this.local, step).length > 0) {
this.emit();
return { ok: false, message: `${label} übersprungen: Der Punkt wurde inzwischen von jemand anderem geändert` };
}
const holder = touchedBlocks(step).map((id) => this.lockedBy(id)).find(Boolean);
if (holder) {
this.emit();
return { ok: false, message: `${label} übersprungen: ${holder} bearbeitet den Punkt gerade` };
}
const ops: Op[] = step.inverse.map((o) => {
const op: Op = { ...o, opId: this.newOpId() } as Op;
if (op.kind === 'set_content') op.baseVersion = this.local.blocks.get(op.blockId!)?.version ?? 0;
if (op.kind === 'set_title') op.baseVersion = this.local.titleVersion;
return op;
});
const before = this.local;
this.enqueue(ops);
const back = buildStep(before, ops, this.now());
if (back) {
back.textKey = undefined;
to.push(back);
}
for (const op of ops) {
const b = op.blockId ? this.local.blocks.get(op.blockId) : undefined;
if (b) this.uiSeen.set(b.id, b.version);
}
this.emit();
return { ok: true };
}
private recompute(): void {
const v = cloneDoc(this.confirmed);
for (const op of [...(this.inflight || []), ...this.pending]) {
const local = { ...op };
if (local.kind === 'set_content') local.baseVersion = v.blocks.get(local.blockId!)?.version ?? 0;
if (local.kind === 'set_title') local.baseVersion = v.titleVersion;
applyOp(v, local);
}
this.local = v;
}
private emit(): void {
// Unfocused blocks take over whatever the engine shows; only the focused text field may lag behind
this.local.blocks.forEach((b, id) => {
if (id !== this.focused) this.uiSeen.set(id, b.version);
});
if (this.listeners.size === 0) return;
const s = this.getState();
this.listeners.forEach((fn) => fn(s));
}
private async persist(): Promise<void> {
try {
await this.storage.save({
docId: this.docId,
title: this.confirmed.title,
titleVersion: this.confirmed.titleVersion,
lastSeq: this.lastSeq,
role: this.role,
blocks: Array.from(this.confirmed.blocks.values()),
pending: this.pending,
inflight: this.inflight,
savedAt: this.now(),
});
} catch {
// Storage full or unavailable: changes stay in memory and are still sent
}
}
private async loadSnapshot(): Promise<void> {
const snap = await this.transport.snapshot(this.docId);
this.confirmed = docFromSnapshot(snap.title, snap.titleVersion, snap.blocks || []);
this.lastSeq = snap.headSeq;
this.role = snap.role;
this.shared = snap.shared;
this.ready = true;
this.recompute();
await this.persist();
}
private holds(): string[] {
const t = this.now();
const out: string[] = [];
this.activity.forEach((at, id) => {
const fresh = t - at < LEASE_GRACE_MS || (id === this.focused && t - at < IDLE_RELEASE_MS);
if (fresh && this.local.blocks.has(id)) out.push(id);
else if (!fresh) this.activity.delete(id);
});
return out.sort();
}
private collaborative(): boolean {
return this.shared || this.sessions > 1;
}
private async roundTrip(): Promise<void> {
if (!this.ready) await this.loadSnapshot();
if (!this.inflight && this.pending.length > 0 && this.canEdit) {
this.inflight = formBatch(this.pending);
this.pending = [];
await this.persist();
}
const ops = this.canEdit ? this.inflight || [] : [];
if (ops.length > 0) this.status = 'saving';
for (;;) {
const res = await this.transport.sync(this.docId, {
clientId: this.clientId,
lastSeq: this.lastSeq,
ops: this.inflight ? ops : [],
leases: this.canEdit && this.collaborative() ? { hold: this.holds(), release: [] } : undefined,
});
if (this.inflight) {
const sent = new Map(this.inflight.map((op) => [op.opId, op]));
for (const r of res.results) {
if (r.status === 'conflict') this.conflicts = [...this.conflicts, r];
// Someone else created a block with the same ID first (e.g. the first block of an empty note):
// keep the typed text by sending it as a regular text change of that block
const op = sent.get(r.opId);
if (r.status === 'rejected' && r.reason === 'exists' && op?.kind === 'insert' && op.content) {
// Base 0: the user never saw the other client's version of this block
this.pending.unshift({ opId: this.newOpId(), kind: 'set_content', blockId: op.blockId, content: op.content, baseVersion: 0 });
}
}
this.inflight = null;
}
this.role = res.role;
this.shared = res.shared;
this.sessions = res.sessions;
this.presence = res.presence || [];
this.lockedByOthers = res.leases?.others || [];
let gap = !!res.resync;
for (const entry of res.ops) {
if (entry.seq <= this.lastSeq) continue;
if (entry.seq !== this.lastSeq + 1) {
gap = true;
break;
}
applyEffect(this.confirmed, entry.effect);
this.lastSeq = entry.seq;
}
if (gap) await this.loadSnapshot();
await this.persist();
if (!res.hasMore || gap) break;
}
this.errorMessage = undefined;
this.status = this.pending.length > 0 ? 'saving' : 'saved';
}
private handleError(err: unknown): void {
if (err instanceof TransportError && (err.status === 403 || err.status === 404)) {
// Access lost: keep the queue (nothing is thrown away), but stop sending
this.status = 'error';
this.errorMessage = err.message;
if (err.status === 404) this.role = '';
else if (this.role !== 'owner') this.role = 'viewer';
return;
}
this.status = 'offline';
this.errorMessage = err instanceof Error ? err.message : String(err);
}
private channel(): SyncChannel {
if (this.pushKind !== 'none') return this.pushKind;
if (this.longPollRunning && !this.longPollBroken) return 'longpoll';
return 'interval';
}
private pollDelay(): number {
if (!this.collaborative()) return POLL_SOLO_MS;
if (this.channel() === 'interval') return POLL_COLLAB_MS;
return this.canEdit && this.holds().length > 0 ? HEARTBEAT_MS : SAFETY_POLL_MS;
}
private onPoke(): void {
if (this.pokeTimer || this.stopped) return;
// Several notifications in a row (e.g. typing of others) lead to one sync
this.pokeTimer = setTimeout(() => {
this.pokeTimer = null;
void this.syncNow();
}, 50);
}
private onPushStatus(kind: PushKind): void {
const wasLive = this.pushKind !== 'none';
this.pushKind = kind;
if (kind !== 'none') {
this.longPollAbort?.abort();
if (!wasLive) void this.syncNow(); // catch up on what was missed while disconnected
} else {
this.ensureLongPoll();
if (!this.syncing) this.schedule(this.pollDelay());
}
this.emit();
}
/** Starts long polling when there is no push channel (e.g. a proxy blocks WebSocket and SSE). */
private ensureLongPoll(): void {
if (this.manual || this.stopped || !this.ready || this.longPollRunning || this.longPollBroken) return;
if (this.pushKind !== 'none' || !this.collaborative() || this.status === 'error') return;
void this.longPollLoop();
}
private async longPollLoop(): Promise<void> {
this.longPollRunning = true;
this.emit();
let failures = 0;
try {
while (!this.stopped && this.pushKind === 'none' && this.collaborative() && this.status !== 'error') {
const ctrl = new AbortController();
this.longPollAbort = ctrl;
try {
const res = await this.transport.changes(this.docId, this.lastSeq, LONG_POLL_WAIT_S, ctrl.signal);
failures = 0;
if (res.changed || res.headSeq !== this.lastSeq) await this.syncNow();
} catch {
if (ctrl.signal.aborted || this.stopped) break;
failures++;
if (failures >= LONG_POLL_MAX_FAILURES) {
// Long polling does not get through either: plain polling every 3 s remains
this.longPollBroken = true;
break;
}
await new Promise((r) => setTimeout(r, 1000 * failures));
}
}
} finally {
this.longPollRunning = false;
this.longPollAbort = null;
this.emit();
if (!this.syncing && !this.stopped) this.schedule(this.pollDelay());
}
}
private nextBackoff(): number {
this.backoff = this.backoff ? Math.min(this.backoff * 2, BACKOFF_MAX_MS) : 1000;
return this.backoff;
}
private schedule(delay: number): void {
if (this.manual || this.stopped) return;
if (this.timer) clearTimeout(this.timer);
this.timer = setTimeout(() => {
this.timer = null;
void this.syncNow();
}, delay);
}
/** Sends what is pending with keepalive (survives page unload) and releases leases. */
private flushOnExit(): void {
if (this.canEdit && (this.inflight || this.pending.length > 0)) {
const ops = this.inflight || formBatch(this.pending);
this.transport
.sync(this.docId, { clientId: this.clientId, lastSeq: this.lastSeq, ops }, { keepalive: true })
.catch(() => {
// Not delivered: the queue is in IndexedDB and is resent on the next visit (same op IDs)
});
}
this.transport.leave(this.docId, this.clientId).catch(() => {});
}
private attachWindowListeners(): void {
const onOnline = () => {
this.longPollBroken = false; // the network changed: long polling may work again
void this.syncNow();
};
const onVisibility = () => {
if (document.visibilityState === 'hidden') this.flushOnExit();
else void this.syncNow();
};
const onPageHide = () => this.flushOnExit();
window.addEventListener('online', onOnline);
document.addEventListener('visibilitychange', onVisibility);
window.addEventListener('pagehide', onPageHide);
this.cleanup.push(
() => window.removeEventListener('online', onOnline),
() => document.removeEventListener('visibilitychange', onVisibility),
() => window.removeEventListener('pagehide', onPageHide)
);
}
}
+437
View File
@@ -0,0 +1,437 @@
// Block tree, operations and effects of collaborative note editing. Port of internal/collab (doc.go, ops.go).
// applyEffect must behave exactly like the server's Doc.ApplyEffect, because clients replay the server log.
import { keyBetween, validOrderKey } from './orderKey';
export interface Props {
checked?: boolean;
collapsed?: boolean;
lang?: string;
conflictOf?: string;
conflictAuthor?: string;
}
export interface Block {
id: string;
parentId: string; // '' = top level
orderKey: string;
type: string;
content: string;
props: Props;
version: number;
deleted?: boolean;
updatedBy?: string;
}
export interface Doc {
title: string;
titleVersion: number;
blocks: Map<string, Block>;
}
export interface PropsPatch {
type?: string;
checked?: boolean;
collapsed?: boolean;
lang?: string;
clearConflict?: boolean;
}
// 'purge' only appears as a log effect: log compaction removed old tombstones for good
export type OpKind = 'insert' | 'set_content' | 'set_props' | 'move' | 'delete' | 'restore' | 'set_title' | 'purge';
export interface Op {
opId: string;
kind: OpKind;
blockId?: string;
baseVersion?: number;
parentId?: string;
orderKey?: string;
type?: string;
content?: string;
props?: Props;
patch?: PropsPatch;
ids?: string[];
title?: string;
}
export interface Effect {
kind: OpKind;
blockId?: string;
parentId?: string;
orderKey?: string;
type?: string;
content?: string;
props?: Props;
version?: number;
ids?: string[];
title?: string;
titleVersion?: number;
by?: string;
}
export interface Result {
opId: string;
status: 'applied' | 'conflict' | 'rejected';
reason?: string;
version?: number;
conflictBlockId?: string;
seq?: number;
}
export interface FlatBlock extends Block {
level: number;
}
export function emptyDoc(): Doc {
return { title: '', titleVersion: 0, blocks: new Map() };
}
export function cloneDoc(d: Doc): Doc {
const blocks = new Map<string, Block>();
d.blocks.forEach((b, id) => blocks.set(id, { ...b, props: { ...b.props } }));
return { title: d.title, titleVersion: d.titleVersion, blocks };
}
/** Normalizes props so that missing and false/empty values compare equal (the server omits them in JSON). */
export function normProps(p?: Props): Props {
const out: Props = {};
if (p?.checked) out.checked = true;
if (p?.collapsed) out.collapsed = true;
if (p?.lang) out.lang = p.lang;
if (p?.conflictOf) out.conflictOf = p.conflictOf;
if (p?.conflictAuthor) out.conflictAuthor = p.conflictAuthor;
return out;
}
export function propsEqual(a?: Props, b?: Props): boolean {
const x = normProps(a);
const y = normProps(b);
return (
x.checked === y.checked &&
x.collapsed === y.collapsed &&
x.lang === y.lang &&
x.conflictOf === y.conflictOf &&
x.conflictAuthor === y.conflictAuthor
);
}
function siblingLess(a: Block, b: Block): number {
if (a.orderKey !== b.orderKey) return a.orderKey < b.orderKey ? -1 : 1;
return a.id < b.id ? -1 : a.id > b.id ? 1 : 0;
}
/** Live children of parentId in display order. */
export function children(d: Doc, parentId: string): Block[] {
const out: Block[] = [];
d.blocks.forEach((b) => {
if (!b.deleted && b.parentId === parentId) out.push(b);
});
return out.sort(siblingLess);
}
/** All live blocks in display order with their nesting level. */
export function flatten(d: Doc): FlatBlock[] {
const byParent = new Map<string, Block[]>();
d.blocks.forEach((b) => {
if (b.deleted) return;
const list = byParent.get(b.parentId);
if (list) list.push(b);
else byParent.set(b.parentId, [b]);
});
byParent.forEach((list) => list.sort(siblingLess));
const out: FlatBlock[] = [];
const walk = (parent: string, level: number) => {
for (const b of byParent.get(parent) || []) {
out.push({ ...b, level });
walk(b.id, level + 1);
}
};
walk('', 0);
return out;
}
function subtree(d: Doc, id: string): string[] {
const kids = new Map<string, string[]>();
d.blocks.forEach((b) => {
if (b.deleted) return;
const list = kids.get(b.parentId);
if (list) list.push(b.id);
else kids.set(b.parentId, [b.id]);
});
const out = [id];
for (let i = 0; i < out.length; i++) out.push(...(kids.get(out[i]) || []));
return [out[0], ...out.slice(1).sort()];
}
function liveAncestor(d: Doc, id: string): string {
const seen = new Set<string>();
while (id && !seen.has(id)) {
seen.add(id);
const b = d.blocks.get(id);
if (!b) return '';
if (!b.deleted) return id;
id = b.parentId;
}
return '';
}
function isAncestor(d: Doc, ancestorId: string, id: string): boolean {
const seen = new Set<string>();
while (id && !seen.has(id)) {
if (id === ancestorId) return true;
seen.add(id);
const b = d.blocks.get(id);
if (!b) return false;
id = b.parentId;
}
return false;
}
/** Mutates the document with one effect, exactly like the server. */
export function applyEffect(d: Doc, e: Effect): void {
switch (e.kind) {
case 'insert':
d.blocks.set(e.blockId!, {
id: e.blockId!,
parentId: e.parentId || '',
orderKey: e.orderKey || '',
type: e.type || '',
content: e.content || '',
props: normProps(e.props),
version: e.version || 0,
deleted: false,
updatedBy: e.by || '',
});
break;
case 'set_content': {
const b = d.blocks.get(e.blockId!);
if (b) {
b.content = e.content || '';
b.version = e.version || 0;
b.updatedBy = e.by || '';
}
break;
}
case 'set_props': {
const b = d.blocks.get(e.blockId!);
if (b) {
b.type = e.type || '';
if (e.props) b.props = normProps(e.props);
}
break;
}
case 'move': {
const b = d.blocks.get(e.blockId!);
if (b) {
b.parentId = e.parentId || '';
b.orderKey = e.orderKey || '';
}
break;
}
case 'delete':
for (const id of e.ids || []) {
const b = d.blocks.get(id);
if (b) b.deleted = true;
}
break;
case 'restore':
for (const id of e.ids || []) {
const b = d.blocks.get(id);
if (b) b.deleted = false;
}
break;
case 'set_title':
d.title = e.title || '';
d.titleVersion = e.titleVersion || 0;
break;
case 'purge':
for (const id of e.ids || []) d.blocks.delete(id);
break;
}
}
/**
* Applies an operation the way the server does (without leases). The client uses it to show its own
* unconfirmed operations on top of the confirmed state; tests use it as a stand-in server.
* conflictId derives the ID of a conflict copy (the server uses a hash of the op ID).
*/
export function applyOp(d: Doc, op: Op, by = '', conflictId: (opId: string) => string = (id) => `blk_cf_${id}`): { effects: Effect[]; result: Result } {
const effects: Effect[] = [];
const emit = (e: Effect) => {
e.by = by;
applyEffect(d, e);
effects.push(e);
};
const done = (r: Omit<Result, 'opId'>) => ({ effects, result: { opId: op.opId, ...r } as Result });
const reject = (reason: string) => done({ status: 'rejected', reason });
const restore = (ids: string[]) => {
const set = new Set(ids);
const sorted = [...ids].sort();
emit({ kind: 'restore', ids: sorted });
for (const id of sorted) {
const b = d.blocks.get(id)!;
if (b.parentId === '' || set.has(b.parentId)) continue;
const p = d.blocks.get(b.parentId);
if (p && !p.deleted) continue;
emit({ kind: 'move', blockId: b.id, parentId: liveAncestor(d, b.parentId), orderKey: b.orderKey });
}
};
const keyAfter = (b: Block): string => {
const sibs = children(d, b.parentId);
let next = '';
for (const s of sibs) {
if (s.orderKey > b.orderKey) {
next = s.orderKey;
break;
}
}
try {
return keyBetween(b.orderKey, next);
} catch {
return keyBetween(b.orderKey, '');
}
};
switch (op.kind) {
case 'insert': {
if (!op.blockId || op.blockId.length > 64) return reject('invalid_id');
if (d.blocks.has(op.blockId)) return reject('exists');
let parent = op.parentId || '';
if (parent) {
// A purged parent (compaction) puts the block at the top level, like the server
parent = d.blocks.has(parent) ? liveAncestor(d, parent) : '';
}
if (!op.orderKey || !validOrderKey(op.orderKey)) return reject('invalid_order_key');
emit({
kind: 'insert',
blockId: op.blockId,
parentId: parent,
orderKey: op.orderKey,
type: op.type || 'bullet',
content: op.content || '',
props: normProps(op.props),
version: 1,
});
return done({ status: 'applied', version: 1 });
}
case 'set_content': {
const b = d.blocks.get(op.blockId || '');
if (op.content === undefined) return reject('no_content');
if (!b) {
// The block was purged (offline for weeks): like the server, the text becomes a new block at the end
if (!op.content || !op.blockId || op.blockId.length > 64) return reject('missing');
const roots = children(d, '');
const key = keyBetween(roots.length ? roots[roots.length - 1].orderKey : '', '');
emit({ kind: 'insert', blockId: op.blockId, parentId: '', orderKey: key, type: 'bullet', content: op.content, props: {}, version: 1 });
return done({ status: 'applied', version: 1, reason: 'revived' });
}
if (b.deleted) restore([b.id]);
if (op.content === b.content) return done({ status: 'applied', version: b.version });
if ((op.baseVersion ?? 0) !== b.version) {
const id = conflictId(op.opId);
if (!d.blocks.has(id)) {
emit({
kind: 'insert',
blockId: id,
parentId: b.parentId,
orderKey: keyAfter(b),
type: b.type,
content: op.content,
props: { conflictOf: b.id },
version: 1,
});
}
return done({ status: 'conflict', version: b.version, conflictBlockId: id });
}
emit({ kind: 'set_content', blockId: b.id, content: op.content, version: b.version + 1 });
return done({ status: 'applied', version: b.version });
}
case 'set_props': {
const b = d.blocks.get(op.blockId || '');
if (!b || b.deleted) return reject('missing');
if (!op.patch) return reject('no_patch');
const type = op.patch.type ? op.patch.type : b.type;
const props = normProps({
...b.props,
...(op.patch.checked !== undefined ? { checked: op.patch.checked } : {}),
...(op.patch.collapsed !== undefined ? { collapsed: op.patch.collapsed } : {}),
...(op.patch.lang !== undefined ? { lang: op.patch.lang } : {}),
...(op.patch.clearConflict ? { conflictOf: undefined, conflictAuthor: undefined } : {}),
});
if (type !== b.type || !propsEqual(props, b.props)) emit({ kind: 'set_props', blockId: b.id, type, props });
return done({ status: 'applied' });
}
case 'move': {
const b = d.blocks.get(op.blockId || '');
if (!b || b.deleted) return reject('missing');
const parent = op.parentId || '';
if (parent) {
const p = d.blocks.get(parent);
if (!p || p.deleted) return reject('parent_missing');
if (isAncestor(d, b.id, parent)) return reject('cycle');
}
if (!op.orderKey || !validOrderKey(op.orderKey)) return reject('invalid_order_key');
if (parent !== b.parentId || op.orderKey !== b.orderKey) emit({ kind: 'move', blockId: b.id, parentId: parent, orderKey: op.orderKey });
return done({ status: 'applied' });
}
case 'delete': {
const b = d.blocks.get(op.blockId || '');
if (!b) return reject('missing');
if (!b.deleted) emit({ kind: 'delete', ids: subtree(d, b.id) });
return done({ status: 'applied' });
}
case 'restore': {
const ids = (op.ids || []).filter((id) => d.blocks.get(id)?.deleted);
if (ids.length > 0) restore(ids);
return done({ status: 'applied' });
}
case 'set_title': {
if (op.title === undefined) return reject('no_title');
if (op.title === d.title) return done({ status: 'applied', version: d.titleVersion });
if ((op.baseVersion ?? 0) !== d.titleVersion) return done({ status: 'conflict', version: d.titleVersion });
emit({ kind: 'set_title', title: op.title, titleVersion: d.titleVersion + 1 });
return done({ status: 'applied', version: d.titleVersion });
}
}
return reject('unknown_kind');
}
/** Builds a Doc from snapshot blocks (as delivered by GET /docs/{id}/snapshot). */
export function docFromSnapshot(title: string, titleVersion: number, blocks: Block[]): Doc {
const d: Doc = { title, titleVersion, blocks: new Map() };
for (const b of blocks) {
d.blocks.set(b.id, {
id: b.id,
parentId: b.parentId || '',
orderKey: b.orderKey,
type: b.type || '',
content: b.content || '',
props: normProps(b.props),
version: b.version || 0,
deleted: !!b.deleted,
updatedBy: b.updatedBy || '',
});
}
return d;
}
export function docsEqual(a: Doc, b: Doc): boolean {
if (a.title !== b.title || a.titleVersion !== b.titleVersion || a.blocks.size !== b.blocks.size) return false;
for (const [id, x] of a.blocks) {
const y = b.blocks.get(id);
if (
!y ||
x.parentId !== y.parentId ||
x.orderKey !== y.orderKey ||
x.type !== y.type ||
x.content !== y.content ||
x.version !== y.version ||
!!x.deleted !== !!y.deleted ||
!propsEqual(x.props, y.props)
) {
return false;
}
}
return true;
}
+18
View File
@@ -0,0 +1,18 @@
// Sends a few operations for a note that is not open in an editor (e.g. ticking a to-do in the overview).
// Goes through the same sync endpoint and rules as the editor, so the change is logged and pushed to
// everyone who has the note open.
import type { Op, Result } from './model';
import { httpTransport } from './transport';
export async function sendDocOps(docId: string, ops: Omit<Op, 'opId'>[]): Promise<Result[]> {
const bytes = new Uint8Array(8);
crypto.getRandomValues(bytes);
const clientId = `once_${Array.from(bytes, (b) => b.toString(16).padStart(2, '0')).join('')}`;
const res = await httpTransport.sync(docId, {
clientId,
lastSeq: 0,
ops: ops.map((op, i) => ({ ...op, opId: `${clientId}.${i}` }) as Op),
skipLog: true,
});
return res.results;
}
+148
View File
@@ -0,0 +1,148 @@
// Order keys sort siblings (ties broken by block ID). Port of internal/collab/orderkey.go; both sides must
// produce identical keys, see orderKey.test.ts for shared test vectors.
const DIGITS = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';
const SMALLEST_INTEGER = 'A00000000000000000000000000';
function digitIndex(c: string): number {
return DIGITS.indexOf(c);
}
function integerLength(head: string): number {
if (head >= 'a' && head <= 'z') return head.charCodeAt(0) - 'a'.charCodeAt(0) + 2;
if (head >= 'A' && head <= 'Z') return 'Z'.charCodeAt(0) - head.charCodeAt(0) + 2;
return 0;
}
function integerPart(key: string): string | null {
if (!key) return null;
const n = integerLength(key[0]);
if (n === 0 || n > key.length) return null;
return key.slice(0, n);
}
export function validOrderKey(k: string): boolean {
if (!k || k.length > 256 || k === SMALLEST_INTEGER) return false;
const ip = integerPart(k);
if (ip === null) return false;
for (let i = 1; i < k.length; i++) {
if (digitIndex(k[i]) < 0) return false;
}
const frac = k.slice(ip.length);
return frac === '' || frac[frac.length - 1] !== '0';
}
function incrementInteger(x: string): string | null {
const head = x[0];
const digs = x.slice(1).split('');
let carry = true;
for (let i = digs.length - 1; carry && i >= 0; i--) {
const d = digitIndex(digs[i]) + 1;
if (d === DIGITS.length) {
digs[i] = '0';
} else {
digs[i] = DIGITS[d];
carry = false;
}
}
if (!carry) return head + digs.join('');
if (head === 'Z') return 'a0';
if (head === 'z') return null;
const h = String.fromCharCode(head.charCodeAt(0) + 1);
if (h > 'a') digs.push('0');
else digs.pop();
return h + digs.join('');
}
function decrementInteger(x: string): string | null {
const head = x[0];
const digs = x.slice(1).split('');
let borrow = true;
for (let i = digs.length - 1; borrow && i >= 0; i--) {
const d = digitIndex(digs[i]) - 1;
if (d === -1) {
digs[i] = 'z';
} else {
digs[i] = DIGITS[d];
borrow = false;
}
}
if (!borrow) return head + digs.join('');
if (head === 'a') return 'Z' + DIGITS[DIGITS.length - 1];
if (head === 'A') return null;
const h = String.fromCharCode(head.charCodeAt(0) - 1);
if (h < 'Z') digs.push('z');
else digs.pop();
return h + digs.join('');
}
function midpoint(a: string, b: string): string {
if (b !== '') {
let n = 0;
while (n < b.length) {
const ca = n < a.length ? a[n] : '0';
if (ca !== b[n]) break;
n++;
}
if (n > 0) return b.slice(0, n) + midpoint(n < a.length ? a.slice(n) : '', b.slice(n));
}
const da = a !== '' ? digitIndex(a[0]) : 0;
const db = b !== '' ? digitIndex(b[0]) : DIGITS.length;
if (db - da > 1) return DIGITS[Math.floor((da + db) / 2)];
if (b !== '' && b.length > 1) return b.slice(0, 1);
return DIGITS[da] + midpoint(a.length > 1 ? a.slice(1) : '', '');
}
/** Returns a key strictly between a and b; '' means open end. Throws on an invalid range. */
export function keyBetween(a: string, b: string): string {
if ((a !== '' && !validOrderKey(a)) || (b !== '' && !validOrderKey(b)) || (a !== '' && b !== '' && a >= b)) {
throw new Error(`invalid order key range ${a}..${b}`);
}
if (a === '' && b === '') return 'a0';
if (a === '') {
const ib = integerPart(b)!;
const fb = b.slice(ib.length);
if (ib === SMALLEST_INTEGER) return ib + midpoint('', fb);
if (ib < b) return ib;
const res = decrementInteger(ib);
if (res === null) throw new Error('order key range exhausted');
return res;
}
if (b === '') {
const ia = integerPart(a)!;
const i = incrementInteger(ia);
return i !== null ? i : ia + midpoint(a.slice(ia.length), '');
}
const ia = integerPart(a)!;
const ib = integerPart(b)!;
if (ia === ib) return ia + midpoint(a.slice(ia.length), b.slice(ib.length));
const i = incrementInteger(ia);
if (i === null) throw new Error('order key range exhausted');
return i < b ? i : ia + midpoint(a.slice(ia.length), '');
}
/** Returns n increasing keys between a and b, spread out so bulk inserts stay short. */
export function keysBetween(a: string, b: string, n: number): string[] {
if (n <= 0) return [];
if (n === 1) return [keyBetween(a, b)];
if (b === '') {
const out: string[] = [];
let prev = a;
for (let i = 0; i < n; i++) {
prev = keyBetween(prev, '');
out.push(prev);
}
return out;
}
if (a === '') {
const out: string[] = new Array(n);
let next = b;
for (let i = n - 1; i >= 0; i--) {
next = keyBetween('', next);
out[i] = next;
}
return out;
}
const mid = Math.floor(n / 2);
const c = keyBetween(a, b);
return [...keysBetween(a, c, mid), c, ...keysBetween(c, b, n - mid - 1)];
}
+87
View File
@@ -0,0 +1,87 @@
// Persists the sync state of each note in IndexedDB so that typed text survives reloads, crashes and
// offline phases. Falls back to memory when IndexedDB is unavailable (private mode, old browsers).
import type { Block, Op } from './model';
export interface StoredDocState {
docId: string;
title: string;
titleVersion: number;
lastSeq: number;
role: string;
blocks: Block[];
pending: Op[];
inflight: Op[] | null;
savedAt: number;
}
export interface SyncStorage {
load(docId: string): Promise<StoredDocState | null>;
save(state: StoredDocState): Promise<void>;
}
const DB_NAME = 'mindsift-sync';
const STORE = 'docs';
export class MemoryStorage implements SyncStorage {
private data = new Map<string, StoredDocState>();
async load(docId: string) {
const s = this.data.get(docId);
return s ? JSON.parse(JSON.stringify(s)) : null;
}
async save(state: StoredDocState) {
this.data.set(state.docId, JSON.parse(JSON.stringify(state)));
}
}
export class IndexedDBStorage implements SyncStorage {
private dbPromise: Promise<IDBDatabase> | null = null;
constructor(private factory: IDBFactory = indexedDB) {}
private open(): Promise<IDBDatabase> {
if (!this.dbPromise) {
this.dbPromise = new Promise((resolve, reject) => {
const req = this.factory.open(DB_NAME, 1);
req.onupgradeneeded = () => {
if (!req.result.objectStoreNames.contains(STORE)) req.result.createObjectStore(STORE, { keyPath: 'docId' });
};
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
this.dbPromise.catch(() => {
this.dbPromise = null;
});
}
return this.dbPromise;
}
async load(docId: string): Promise<StoredDocState | null> {
const db = await this.open();
return new Promise((resolve, reject) => {
const req = db.transaction(STORE, 'readonly').objectStore(STORE).get(docId);
req.onsuccess = () => resolve((req.result as StoredDocState) || null);
req.onerror = () => reject(req.error);
});
}
async save(state: StoredDocState): Promise<void> {
const db = await this.open();
return new Promise((resolve, reject) => {
const tx = db.transaction(STORE, 'readwrite');
tx.objectStore(STORE).put(state);
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
tx.onabort = () => reject(tx.error);
});
}
}
/** IndexedDB when available, otherwise memory (state is then lost on reload). */
export function defaultStorage(): SyncStorage {
try {
if (typeof indexedDB !== 'undefined') return new IndexedDBStorage(indexedDB);
} catch {
// ignore
}
return new MemoryStorage();
}
+25
View File
@@ -0,0 +1,25 @@
// Push notifications for note sync: the server sends DOC_SEQ into the note's room whenever the note, its
// leases or its presence change. Delivery uses the app-wide realtime connection (WebSocket, else SSE).
import { createRealtimeConnection, getRealtimeTransport, isRealtimeConnected } from '../api';
export type PushKind = 'ws' | 'sse' | 'none';
export interface PushChannel {
/** Calls onPoke for changes of docId and onStatus whenever the channel goes up or down. */
subscribe(docId: string, onPoke: () => void, onStatus: (kind: PushKind) => void): () => void;
}
export const realtimePush: PushChannel = {
subscribe(docId, onPoke, onStatus) {
const current = (): PushKind => (isRealtimeConnected() ? (getRealtimeTransport() as PushKind) : 'none');
const conn = createRealtimeConnection({
docId,
onMessage: (msg: any) => {
if (msg?.event === 'DOC_SEQ' && msg.payload?.docId === docId) onPoke();
},
onStatusChange: () => onStatus(current()),
});
onStatus(current());
return () => conn.close();
},
};
+122
View File
@@ -0,0 +1,122 @@
// HTTP transport of the note sync protocol (see Doku/api/rest.md). Every data exchange is a plain request,
// so it works behind proxies that block WebSockets.
import { getToken } from '../api';
import type { Block, Effect, Op, Result } from './model';
export interface LeaseInfo {
blockId: string;
userId: string;
name: string;
}
export interface Presence {
userId: string;
name: string;
}
export interface SnapshotResponse {
docId: string;
title: string;
titleVersion: number;
headSeq: number;
blocks: Block[];
role: string;
shared: boolean;
}
export interface LogEntry {
seq: number;
opId?: string;
userId: string;
clientId: string;
effect: Effect;
}
export interface SyncRequest {
clientId: string;
lastSeq: number;
ops: Op[];
leases?: { hold: string[]; release: string[] };
skipLog?: boolean; // one-off changes do not need the log back
}
export interface SyncResponse {
results: Result[];
ops: LogEntry[];
headSeq: number;
hasMore: boolean;
resync?: boolean;
role: string;
shared: boolean;
sessions: number;
presence: Presence[];
leases: { granted: string[]; denied: string[]; others: LeaseInfo[] };
}
/** status 0 = network error or timeout; otherwise the HTTP status. */
export class TransportError extends Error {
constructor(message: string, public status: number) {
super(message);
}
}
export interface ChangesResponse {
headSeq: number;
changed: boolean;
}
export interface Transport {
snapshot(docId: string): Promise<SnapshotResponse>;
/** Long poll: resolves when the note changes, at the latest after waitSec seconds. */
changes(docId: string, since: number, waitSec: number, signal?: AbortSignal): Promise<ChangesResponse>;
sync(docId: string, body: SyncRequest, opts?: { keepalive?: boolean }): Promise<SyncResponse>;
leave(docId: string, clientId: string): Promise<void>;
}
const TIMEOUT_MS = 15000;
async function call<T>(path: string, init: RequestInit & { keepalive?: boolean } = {}, timeoutMs = TIMEOUT_MS): Promise<T> {
const headers: Record<string, string> = { 'Content-Type': 'application/json' };
const token = getToken();
if (token) headers.Authorization = `Bearer ${token}`;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeoutMs);
init.signal?.addEventListener('abort', () => controller.abort());
let res: Response;
try {
res = await fetch(path, { ...init, headers, cache: 'no-store', signal: init.keepalive ? undefined : controller.signal });
} catch (err: any) {
throw new TransportError(err?.message || 'Netzwerkfehler', 0);
} finally {
clearTimeout(timer);
}
if (res.status === 401) {
window.dispatchEvent(new CustomEvent('mindsift:unauthorized'));
}
if (!res.ok) {
let msg = `HTTP ${res.status}`;
try {
const data = await res.json();
if (data?.error) msg = data.error;
} catch {
// ignore
}
throw new TransportError(msg, res.status);
}
return res.json();
}
export const httpTransport: Transport = {
snapshot: (docId) => call(`/api/v1/docs/${encodeURIComponent(docId)}/snapshot`),
changes: (docId, since, waitSec, signal) =>
call(`/api/v1/docs/${encodeURIComponent(docId)}/changes?since=${since}&wait=${waitSec}`, { signal }, (waitSec + 10) * 1000),
sync: (docId, body, opts) =>
call(`/api/v1/docs/${encodeURIComponent(docId)}/sync`, { method: 'POST', body: JSON.stringify(body), keepalive: opts?.keepalive }),
leave: async (docId, clientId) => {
await call(`/api/v1/docs/${encodeURIComponent(docId)}/leave`, {
method: 'POST',
body: JSON.stringify({ clientId }),
keepalive: true,
});
},
};
+150
View File
@@ -0,0 +1,150 @@
// Undo/redo of the user's own changes. For every action the engine records the inverse operations plus a
// check of the state the action left behind. Undo only runs if that state is still there, i.e. nobody
// else changed the affected blocks in between; otherwise the step is skipped instead of overwriting.
import { applyOp, cloneDoc, Doc, normProps, Op, propsEqual } from './model';
export type UndoCheck =
| { kind: 'content'; blockId: string; content: string }
| { kind: 'alive'; blockId: string; content: string }
| { kind: 'deleted'; ids: string[] }
| { kind: 'place'; blockId: string; parentId: string; orderKey: string; restoreParent: string }
| { kind: 'props'; blockId: string; type: string; checked: boolean; collapsed: boolean; lang: string }
| { kind: 'title'; title: string };
export interface UndoStep {
inverse: Omit<Op, 'opId'>[]; // already in the order to apply
checks: UndoCheck[];
at: number;
textKey?: string; // set for pure typing steps (block ID or '#title'), used to group keystrokes
}
/** Applies ops to a copy of `before` and records how to revert them. */
export function buildStep(before: Doc, ops: Op[], at: number): UndoStep | null {
const v = cloneDoc(before);
const inverse: Omit<Op, 'opId'>[] = [];
const checks: UndoCheck[] = [];
for (const raw of ops) {
const op = { ...raw };
const pre = op.blockId ? v.blocks.get(op.blockId) : undefined;
const preCopy = pre ? { ...pre, props: { ...pre.props } } : undefined;
const preTitle = v.title;
if (op.kind === 'set_content') op.baseVersion = pre?.version ?? 0;
if (op.kind === 'set_title') op.baseVersion = v.titleVersion;
const { effects } = applyOp(v, op);
if (effects.length === 0) continue;
const post = op.blockId ? v.blocks.get(op.blockId) : undefined;
switch (op.kind) {
case 'insert':
if (post && !post.deleted) {
inverse.push({ kind: 'delete', blockId: post.id });
checks.push({ kind: 'alive', blockId: post.id, content: post.content });
}
break;
case 'set_content':
if (preCopy && post) {
inverse.push({ kind: 'set_content', blockId: post.id, content: preCopy.content });
checks.push({ kind: 'content', blockId: post.id, content: post.content });
}
break;
case 'set_props':
if (preCopy && post) {
const p = normProps(preCopy.props);
inverse.push({
kind: 'set_props',
blockId: post.id,
patch: { type: preCopy.type, checked: !!p.checked, collapsed: !!p.collapsed, lang: p.lang || '' },
});
const q = normProps(post.props);
checks.push({ kind: 'props', blockId: post.id, type: post.type, checked: !!q.checked, collapsed: !!q.collapsed, lang: q.lang || '' });
}
break;
case 'move':
if (preCopy && post) {
inverse.push({ kind: 'move', blockId: post.id, parentId: preCopy.parentId, orderKey: preCopy.orderKey });
checks.push({ kind: 'place', blockId: post.id, parentId: post.parentId, orderKey: post.orderKey, restoreParent: preCopy.parentId });
}
break;
case 'delete': {
const ids = effects.flatMap((e) => (e.kind === 'delete' ? e.ids || [] : []));
if (ids.length > 0) {
inverse.push({ kind: 'restore', ids });
checks.push({ kind: 'deleted', ids });
}
break;
}
case 'restore': {
const ids = effects.flatMap((e) => (e.kind === 'restore' ? e.ids || [] : []));
for (const id of ids) inverse.push({ kind: 'delete', blockId: id });
for (const id of ids) checks.push({ kind: 'alive', blockId: id, content: v.blocks.get(id)?.content || '' });
break;
}
case 'set_title':
inverse.push({ kind: 'set_title', title: preTitle });
checks.push({ kind: 'title', title: v.title });
break;
}
}
if (inverse.length === 0) return null;
inverse.reverse();
const onlyText = ops.every((o) => o.kind === 'set_content') && new Set(ops.map((o) => o.blockId)).size === 1;
const onlyTitle = ops.every((o) => o.kind === 'set_title');
return { inverse, checks, at, textKey: onlyText ? ops[0].blockId : onlyTitle ? '#title' : undefined };
}
/** Returns the IDs of blocks the step touches whose state no longer matches, or [] if undo is safe. */
export function failedChecks(d: Doc, step: UndoStep): string[] {
const failed: string[] = [];
for (const c of step.checks) {
switch (c.kind) {
case 'content': {
const b = d.blocks.get(c.blockId);
if (!b || b.deleted || b.content !== c.content) failed.push(c.blockId);
break;
}
case 'alive': {
const b = d.blocks.get(c.blockId);
if (!b || b.deleted || b.content !== c.content) failed.push(c.blockId);
break;
}
case 'deleted':
for (const id of c.ids) if (!d.blocks.get(id)?.deleted) failed.push(id);
break;
case 'place': {
const b = d.blocks.get(c.blockId);
const parent = c.restoreParent ? d.blocks.get(c.restoreParent) : null;
if (!b || b.deleted || b.parentId !== c.parentId || b.orderKey !== c.orderKey || (c.restoreParent && (!parent || parent.deleted))) {
failed.push(c.blockId);
}
break;
}
case 'props': {
const b = d.blocks.get(c.blockId);
if (
!b ||
b.deleted ||
b.type !== c.type ||
!propsEqual(normProps({ checked: b.props.checked, collapsed: b.props.collapsed, lang: b.props.lang }), normProps({ checked: c.checked, collapsed: c.collapsed, lang: c.lang }))
) {
failed.push(c.blockId);
}
break;
}
case 'title':
if (d.title !== c.title) failed.push('#title');
break;
}
}
return failed;
}
/** Block IDs an inverse step would modify (for lease checks). */
export function touchedBlocks(step: UndoStep): string[] {
const ids = new Set<string>();
for (const op of step.inverse) {
if (op.blockId) ids.add(op.blockId);
for (const id of op.ids || []) ids.add(id);
}
return [...ids];
}
+38 -6
View File
@@ -7,11 +7,20 @@ import React from 'react';
* - ~~strikethrough~~ * - ~~strikethrough~~
* - `code` * - `code`
* - [label](url) * - [label](url)
* - #hashtag (only when `options.onTagClick` is given), rendered inline as a clickable chip
*/ */
function renderInlineTokens(text: string): React.ReactNode[] { export interface InlineMarkdownOptions {
onTagClick?: (tag: string) => void;
}
function renderInlineTokens(text: string, options?: InlineMarkdownOptions): React.ReactNode[] {
if (!text) return []; if (!text) return [];
const regex = /(`[^`]+`|\*\*[^*]+\*\*|__[^_]+__|~~[^~]+~~|\*[^*]+\*|_[^_]+_|\[([^\]]+)\]\(([^)]+)\))/g; const tagAlt = options?.onTagClick ? '|#[a-zA-Z0-9äöüÄÖÜß_-]+' : '';
const regex = new RegExp(
'(`[^`]+`|\\*\\*[^*]+\\*\\*|__[^_]+__|~~[^~]+~~|\\*[^*]+\\*|_[^_]+_|\\[([^\\]]+)\\]\\(([^)]+)\\)' + tagAlt + ')',
'g'
);
const elements: React.ReactNode[] = []; const elements: React.ReactNode[] = [];
let lastIndex = 0; let lastIndex = 0;
@@ -24,7 +33,30 @@ function renderInlineTokens(text: string): React.ReactNode[] {
const token = match[0]; const token = match[0];
const key = `${match.index}-${token}`; const key = `${match.index}-${token}`;
if (token.startsWith('`') && token.endsWith('`')) { if (token.startsWith('#')) {
const prevChar = match.index > 0 ? text[match.index - 1] : '';
if (prevChar && !/[\s(]/.test(prevChar)) {
// e.g. an URL fragment like https://x/#anchor: keep as plain text
elements.push(token);
} else {
const tag = token.slice(1);
elements.push(
<span
key={key}
role="button"
tabIndex={-1}
onClick={(e) => {
e.stopPropagation();
options?.onTagClick?.(tag);
}}
className="text-accent bg-accent-dim hover:bg-accent/20 rounded-btn px-0.5 cursor-pointer transition-colors"
title={`Nach #${tag} filtern`}
>
{token}
</span>
);
}
} else if (token.startsWith('`') && token.endsWith('`')) {
elements.push( elements.push(
<code key={key} className="px-1.5 py-0.5 rounded bg-panel-elevated text-accent font-mono text-[0.9em] border border-border/50"> <code key={key} className="px-1.5 py-0.5 rounded bg-panel-elevated text-accent font-mono text-[0.9em] border border-border/50">
{token.slice(1, -1)} {token.slice(1, -1)}
@@ -92,7 +124,7 @@ function renderInlineTokens(text: string): React.ReactNode[] {
* - `code` * - `code`
* - [label](url) * - [label](url)
*/ */
export function renderInlineMarkdown(text: string): React.ReactNode[] { export function renderInlineMarkdown(text: string, options?: InlineMarkdownOptions): React.ReactNode[] {
if (!text) return []; if (!text) return [];
if (text.includes('```')) { if (text.includes('```')) {
@@ -122,14 +154,14 @@ export function renderInlineMarkdown(text: string): React.ReactNode[] {
</div> </div>
); );
} else if (part) { } else if (part) {
nodes.push(...renderInlineTokens(part)); nodes.push(...renderInlineTokens(part, options));
} }
}); });
return nodes; return nodes;
} }
return renderInlineTokens(text); return renderInlineTokens(text, options);
} }
export function MarkdownViewer({ content, className = '' }: { content: string; className?: string }) { export function MarkdownViewer({ content, className = '' }: { content: string; className?: string }) {
+2
View File
@@ -173,6 +173,8 @@ export interface DocumentBlock {
collapsed?: boolean; collapsed?: boolean;
sortOrder?: number; sortOrder?: number;
updatedAt?: string; updatedAt?: string;
conflictOf?: string; // set on conflict copies: ID of the contested block, or 'title'
conflictAuthor?: string;
} }
export interface DocBlocksPatchRequest { export interface DocBlocksPatchRequest {
+9
View File
@@ -0,0 +1,9 @@
import { afterEach } from 'vitest';
import { cleanup } from '@testing-library/react';
// jsdom has no layout engine, so scrolling is a no-op here
Element.prototype.scrollIntoView = () => {};
afterEach(() => {
cleanup();
});
+4
View File
@@ -0,0 +1,4 @@
declare const __APP_VERSION__: string;
/** Version shown in the header; set at build time in vite.config.ts. */
export const APP_VERSION = typeof __APP_VERSION__ !== 'undefined' ? __APP_VERSION__ : 'dev';
+9
View File
@@ -1,8 +1,17 @@
import { defineConfig } from 'vite'; import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react'; import react from '@vitejs/plugin-react';
import { readFileSync } from 'node:fs';
// Shown in the header. Release builds get the git tag (APP_VERSION build argument, e.g. "v0.2");
// other builds show the package version with a "-dev" suffix.
const pkg = JSON.parse(readFileSync(new URL('./package.json', import.meta.url), 'utf8'));
const appVersion = process.env.APP_VERSION || `v${pkg.version.replace(/\.0$/, '')}-dev`;
export default defineConfig({ export default defineConfig({
plugins: [react()], plugins: [react()],
define: {
__APP_VERSION__: JSON.stringify(appVersion),
},
server: { server: {
port: 5173, port: 5173,
proxy: { proxy: {
+10
View File
@@ -0,0 +1,10 @@
import { defineConfig } from 'vitest/config';
import react from '@vitejs/plugin-react';
export default defineConfig({
plugins: [react()],
test: {
environment: 'jsdom',
setupFiles: ['./src/test/setup.ts'],
},
});
+284
View File
@@ -0,0 +1,284 @@
package api
import (
"encoding/json"
"errors"
"log/slog"
"net/http"
"strconv"
"time"
"github.com/go-chi/chi/v5"
"github.com/sandrodach/mindsift/internal/auth"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/storage"
)
const (
syncMaxOps = 500
syncMaxLog = 1000
syncMaxBody = 8 << 20
longPollMax = 25 * time.Second
maxClientIDLen = 64
)
type syncLeases struct {
Granted []string `json:"granted"`
Denied []string `json:"denied"`
Others []collab.LeaseInfo `json:"others"`
}
type syncResponse struct {
Results []collab.Result `json:"results"`
Ops []storage.DocLogEntry `json:"ops"`
HeadSeq int64 `json:"headSeq"`
HasMore bool `json:"hasMore"`
Resync bool `json:"resync,omitempty"` // reload the snapshot (state ahead of the server or log compacted)
Role string `json:"role"`
Shared bool `json:"shared"`
Sessions int `json:"sessions"`
Presence []collab.Presence `json:"presence"`
Leases syncLeases `json:"leases"`
}
// docAccess resolves the caller's role; it writes 404 and returns "" without access.
func (s *Server) docAccess(w http.ResponseWriter, r *http.Request) (userID, docID, role string) {
userID = auth.GetUserID(r.Context())
docID = chi.URLParam(r, "id")
role = s.store.DocRole(userID, docID)
if role == "" {
writeError(w, http.StatusNotFound, storage.ErrDocNotFound.Error())
}
return userID, docID, role
}
func (s *Server) handleDocSnapshot(w http.ResponseWriter, r *http.Request) {
_, docID, role := s.docAccess(w, r)
if role == "" {
return
}
snap, err := s.store.GetDocSnapshot(docID)
if err != nil {
writeError(w, http.StatusNotFound, err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"docId": snap.DocID,
"title": snap.Title,
"titleVersion": snap.TitleVersion,
"headSeq": snap.HeadSeq,
"blocks": snap.Blocks,
"role": role,
"shared": s.store.IsDocShared(docID),
})
}
// handleDocSync is the single round trip of the sync protocol: send own operations, hold or release
// block leases, and receive every log entry after lastSeq.
func (s *Server) handleDocSync(w http.ResponseWriter, r *http.Request) {
userID, docID, role := s.docAccess(w, r)
if role == "" {
return
}
var req struct {
ClientID string `json:"clientId"`
LastSeq int64 `json:"lastSeq"`
Ops []collab.Op `json:"ops"`
Leases *struct {
Hold []string `json:"hold"`
Release []string `json:"release"`
} `json:"leases"`
SkipLog bool `json:"skipLog"` // one-off changes (e.g. ticking a to-do in the overview) need no log
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, syncMaxBody)).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "Ungültiges JSON")
return
}
if req.ClientID == "" || len(req.ClientID) > maxClientIDLen {
writeError(w, http.StatusBadRequest, "clientId fehlt oder ist zu lang")
return
}
if len(req.Ops) > syncMaxOps {
writeError(w, http.StatusRequestEntityTooLarge, "Zu viele Änderungen in einer Anfrage")
return
}
canEdit := storage.CanEditDoc(role)
if len(req.Ops) > 0 && !canEdit {
writeError(w, http.StatusForbidden, "Keine Schreibrechte für diese Notiz (nur Lesen)")
return
}
actor := collab.Actor{UserID: userID, ClientID: req.ClientID}
if u, err := s.store.GetUserByID(userID); err == nil {
actor.Name = u.DisplayName
}
s.leases.Touch(docID, actor)
resp := syncResponse{Results: []collab.Result{}, Role: role, Leases: syncLeases{Granted: []string{}, Denied: []string{}}}
changed := false
if req.Leases != nil && canEdit {
s.leases.ReleaseAllExcept(docID, req.ClientID, req.Leases.Hold)
s.leases.Release(docID, req.ClientID, req.Leases.Release)
resp.Leases.Granted, resp.Leases.Denied = s.leases.Hold(docID, actor, req.Leases.Hold)
changed = true
}
if len(req.Ops) > 0 {
results, _, err := s.store.ApplyDocOps(docID, actor, req.Ops, s.leases.View(docID))
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
resp.Results = results
// New blocks (and conflict copies) belong to their author right away
var own []string
for i, res := range results {
if res.Status == collab.StatusApplied && req.Ops[i].Kind == collab.OpInsert {
own = append(own, req.Ops[i].BlockID)
}
if res.ConflictBlockID != "" {
own = append(own, res.ConflictBlockID)
}
if res.Seq > 0 {
changed = true
}
}
if len(own) > 0 {
granted, _ := s.leases.Hold(docID, actor, own)
resp.Leases.Granted = append(resp.Leases.Granted, granted...)
}
}
if changed {
s.afterDocChange(docID, req.Ops, resp.Results)
}
logLimit := syncMaxLog
if req.SkipLog {
logLimit = 0
}
entries, head, err := s.store.GetDocOpsSince(docID, req.LastSeq, logLimit)
compacted := errors.Is(err, storage.ErrLogCompacted)
if err != nil && !compacted {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
// Client state is ahead of the server, or so far behind that the log entries are compacted away
if (req.LastSeq > head || compacted) && !req.SkipLog {
resp.Resync = true
entries = []storage.DocLogEntry{}
}
resp.Ops = entries
resp.HeadSeq = head
resp.HasMore = len(entries) == syncMaxLog && entries[len(entries)-1].Seq < head
resp.Shared = s.store.IsDocShared(docID)
resp.Sessions, resp.Presence = s.leases.Sessions(docID)
resp.Leases.Others = s.leases.Others(docID, req.ClientID)
if changed {
s.notifyDocChanged(docID, head)
}
writeJSON(w, http.StatusOK, resp)
}
// afterDocChange runs side effects of applied operations: linked tasks follow their blocks, and a new
// title reaches the note lists of everyone with access.
func (s *Server) afterDocChange(docID string, ops []collab.Op, results []collab.Result) {
titleChanged := false
for i, res := range results {
if i < len(ops) && ops[i].Kind == collab.OpSetTitle && res.Seq > 0 {
titleChanged = true
}
}
for _, taskID := range s.store.ReconcileDocTaskLinks(docID) {
if th, err := s.store.GetThoughtByIDUnchecked(taskID); err == nil {
s.wsHub.BroadcastToUsers(s.taskAudience(th), "TASK_UPDATED", th)
}
}
if titleChanged {
s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED")
}
}
// notifyDocChanged wakes long-poll requests and pokes WebSocket/SSE clients in the note's room.
func (s *Server) notifyDocChanged(docID string, head int64) {
s.notifier.Notify(docID)
s.wsHub.BroadcastDocEvent(docID, "DOC_SEQ", map[string]interface{}{"docId": docID, "headSeq": head})
}
// handleDocChanges is the long-poll fallback: it returns as soon as the note changes (or leases/presence
// change), at the latest after `wait` seconds. Clients then call /sync to fetch the data.
func (s *Server) handleDocChanges(w http.ResponseWriter, r *http.Request) {
_, docID, role := s.docAccess(w, r)
if role == "" {
return
}
since, _ := strconv.ParseInt(r.URL.Query().Get("since"), 10, 64)
wait := longPollMax
if sec, err := strconv.Atoi(r.URL.Query().Get("wait")); err == nil && sec >= 0 && time.Duration(sec)*time.Second < wait {
wait = time.Duration(sec) * time.Second
}
_ = http.NewResponseController(w).SetWriteDeadline(time.Now().Add(wait + 10*time.Second))
woken := s.notifier.Changed(docID) // subscribe before reading, so no change can slip through
_, head, err := s.store.GetDocOpsSince(docID, since, 0)
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
changed := head > since
if !changed {
select {
case <-woken:
changed = true
case <-time.After(wait):
case <-r.Context().Done():
return
}
_, head, _ = s.store.GetDocOpsSince(docID, since, 0)
}
writeJSON(w, http.StatusOK, map[string]interface{}{"headSeq": head, "changed": changed})
}
// handleDocLeave ends a client's session on a note and frees its leases (tab closed, app in background).
func (s *Server) handleDocLeave(w http.ResponseWriter, r *http.Request) {
_, docID, role := s.docAccess(w, r)
if role == "" {
return
}
var req struct {
ClientID string `json:"clientId"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&req); err != nil || req.ClientID == "" {
writeError(w, http.StatusBadRequest, "clientId fehlt")
return
}
s.leases.Leave(docID, req.ClientID)
_, head, _ := s.store.GetDocOpsSince(docID, 0, 0)
s.notifyDocChanged(docID, head)
writeJSON(w, http.StatusOK, map[string]bool{"success": true})
}
// compactionInterval is how often the operation log and tombstones are compacted (also once at start).
const compactionInterval = 6 * time.Hour
func (s *Server) startCompaction() {
for {
s.compactOnce()
time.Sleep(compactionInterval)
}
}
func (s *Server) compactOnce() {
s.leases.Sweep()
st, err := s.store.CompactDocs(time.Now())
if err != nil {
slog.Error("Verdichtung des Änderungsprotokolls fehlgeschlagen", "error", err)
}
// Open editors drop the purged tombstones with their next sync
for docID, head := range st.Purged {
s.notifyDocChanged(docID, head)
}
if st.Docs > 0 || st.OpResults > 0 {
slog.Info("Änderungsprotokoll verdichtet", "notes", st.Docs, "logEntries", st.LogEntries, "tombstones", st.Tombstones, "opResults", st.OpResults)
}
}
+231
View File
@@ -0,0 +1,231 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain"
)
func syncDoc(t *testing.T, ts *httptest.Server, token, docID string, body map[string]interface{}) (int, syncResponse) {
t.Helper()
resp := doJSON(t, ts, http.MethodPost, "/api/v1/docs/"+docID+"/sync", token, body)
var out syncResponse
if resp.StatusCode == http.StatusOK {
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatalf("decode sync response: %v", err)
}
}
return resp.StatusCode, out
}
func strp(s string) *string { return &s }
type collabFixture struct {
ts *httptest.Server
docID string
ownerTok, editorTok, viewerTok string
outsiderTok string
}
func newCollabFixture(t *testing.T) collabFixture {
t.Helper()
_, store, ts, owner, ownerTok := startRealtimeServer(t)
editor, editorTok := newTestUser(t, store, "editor@example.com")
viewer, viewerTok := newTestUser(t, store, "viewer@example.com")
_, outsiderTok := newTestUser(t, store, "outsider2@example.com")
doc, err := store.CreateDocument(owner.ID, "", "Gemeinsam", "📄", "", []domain.DocumentBlock{{ID: "b1", Type: "bullet", Content: "Erster Punkt"}}, nil, false)
if err != nil {
t.Fatalf("create doc: %v", err)
}
if _, err := store.CreateDocShare(owner.ID, doc.ID, editor.ID, "write"); err != nil {
t.Fatalf("share editor: %v", err)
}
if _, err := store.CreateDocShare(owner.ID, doc.ID, viewer.ID, "read"); err != nil {
t.Fatalf("share viewer: %v", err)
}
return collabFixture{ts: ts, docID: doc.ID, ownerTok: ownerTok, editorTok: editorTok, viewerTok: viewerTok, outsiderTok: outsiderTok}
}
func TestDocSyncBetweenTwoEditors(t *testing.T) {
f := newCollabFixture(t)
// Owner inserts a block and edits the existing one in one round trip
code, a := syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{
"clientId": "owner-tab", "lastSeq": 0,
"ops": []collab.Op{
{OpID: "o1", Kind: collab.OpInsert, BlockID: "b2", ParentID: strp(""), OrderKey: "a5", Content: strp("Zweiter Punkt")},
{OpID: "o2", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Erster Punkt (bearbeitet)")},
},
})
if code != http.StatusOK || a.Results[0].Status != collab.StatusApplied || a.Results[1].Status != collab.StatusApplied {
t.Fatalf("owner sync: %d %+v", code, a.Results)
}
if len(a.Leases.Granted) == 0 || a.Leases.Granted[0] != "b2" {
t.Fatalf("inserted block must be leased to its author: %+v", a.Leases)
}
// Editor pulls everything since 0 and replays it
code, b := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{"clientId": "editor-tab", "lastSeq": 0})
if code != http.StatusOK || b.HeadSeq != a.HeadSeq || b.Role != "editor" || !b.Shared {
t.Fatalf("editor pull: %d head=%d role=%s shared=%v", code, b.HeadSeq, b.Role, b.Shared)
}
replay := collab.NewDoc()
for _, e := range b.Ops {
replay.ApplyEffect(e.Effect)
}
flat := replay.Flatten()
if len(flat) != 2 || flat[0].Content != "Erster Punkt (bearbeitet)" || flat[1].Content != "Zweiter Punkt" {
t.Fatalf("replayed state wrong: %+v", flat)
}
if len(b.Leases.Others) == 0 {
t.Fatal("editor must see the owner's lease on b2")
}
// Editor writes into the leased block: owner keeps it, editor's text becomes a conflict copy
_, c := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{
"clientId": "editor-tab", "lastSeq": b.HeadSeq,
"ops": []collab.Op{{OpID: "e1", Kind: collab.OpSetContent, BlockID: "b2", BaseVersion: 1, Content: strp("Editor war hier")}},
})
if c.Results[0].Status != collab.StatusConflict || c.Results[0].ConflictBlockID == "" {
t.Fatalf("write into foreign lease must conflict: %+v", c.Results)
}
}
func TestDocSyncRetryIsIdempotent(t *testing.T) {
f := newCollabFixture(t)
body := map[string]interface{}{
"clientId": "tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "same-op", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Einmal")}},
}
_, first := syncDoc(t, f.ts, f.ownerTok, f.docID, body)
_, second := syncDoc(t, f.ts, f.ownerTok, f.docID, body) // response of the first was "lost"
if first.Results[0] != second.Results[0] || first.HeadSeq != second.HeadSeq {
t.Fatalf("retry applied twice: %+v / %+v", first, second)
}
}
func TestDocSyncPermissions(t *testing.T) {
f := newCollabFixture(t)
op := map[string]interface{}{
"clientId": "tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "v1", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Leser schreibt")}},
}
if code, _ := syncDoc(t, f.ts, f.viewerTok, f.docID, op); code != http.StatusForbidden {
t.Errorf("viewer sending ops: want 403, got %d", code)
}
code, pull := syncDoc(t, f.ts, f.viewerTok, f.docID, map[string]interface{}{"clientId": "tab", "lastSeq": 0, "leases": map[string]interface{}{"hold": []string{"b1"}}})
if code != http.StatusOK || pull.Role != "viewer" || len(pull.Leases.Granted) != 0 {
t.Errorf("viewer pull: %d role=%s granted=%v", code, pull.Role, pull.Leases.Granted)
}
if code, _ := syncDoc(t, f.ts, f.outsiderTok, f.docID, map[string]interface{}{"clientId": "tab"}); code != http.StatusNotFound {
t.Errorf("outsider: want 404, got %d", code)
}
for _, path := range []string{"/snapshot", "/changes?wait=0"} {
if resp := doJSON(t, f.ts, http.MethodGet, "/api/v1/docs/"+f.docID+path, f.outsiderTok, nil); resp.StatusCode != http.StatusNotFound {
t.Errorf("outsider %s: want 404, got %d", path, resp.StatusCode)
}
}
}
func TestDocSnapshotMatchesLog(t *testing.T) {
f := newCollabFixture(t)
syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{
"clientId": "tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "t1", Kind: collab.OpSetTitle, BaseVersion: 0, Title: strp("Neuer Titel")}},
})
resp := doJSON(t, f.ts, http.MethodGet, "/api/v1/docs/"+f.docID+"/snapshot", f.editorTok, nil)
var snap struct {
Title string `json:"title"`
HeadSeq int64 `json:"headSeq"`
Blocks []collab.Block `json:"blocks"`
Role string `json:"role"`
}
if err := json.NewDecoder(resp.Body).Decode(&snap); err != nil {
t.Fatal(err)
}
if snap.Title != "Neuer Titel" || snap.HeadSeq == 0 || len(snap.Blocks) != 1 || snap.Role != "editor" {
t.Fatalf("snapshot: %+v", snap)
}
}
func TestDocChangesLongPollWakesOnChange(t *testing.T) {
f := newCollabFixture(t)
_, base := syncDoc(t, f.ts, f.editorTok, f.docID, map[string]interface{}{"clientId": "editor-tab", "lastSeq": 0})
type pollResult struct {
Changed bool `json:"changed"`
HeadSeq int64 `json:"headSeq"`
}
done := make(chan pollResult, 1)
start := time.Now()
go func() {
req, _ := http.NewRequest(http.MethodGet, f.ts.URL+"/api/v1/docs/"+f.docID+"/changes?wait=20&since="+itoa(base.HeadSeq), nil)
req.Header.Set("Authorization", "Bearer "+f.editorTok)
resp, err := http.DefaultClient.Do(req)
if err != nil {
done <- pollResult{}
return
}
defer resp.Body.Close()
var r pollResult
_ = json.NewDecoder(resp.Body).Decode(&r)
done <- r
}()
time.Sleep(300 * time.Millisecond)
syncDoc(t, f.ts, f.ownerTok, f.docID, map[string]interface{}{
"clientId": "owner-tab", "lastSeq": 0,
"ops": []collab.Op{{OpID: "lp1", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Weckruf")}},
})
select {
case r := <-done:
if !r.Changed || r.HeadSeq <= base.HeadSeq || time.Since(start) > 5*time.Second {
t.Fatalf("long poll result: %+v after %v", r, time.Since(start))
}
case <-time.After(10 * time.Second):
t.Fatal("long poll did not wake up")
}
}
func itoa(n int64) string {
b, _ := json.Marshal(n)
return string(b)
}
func TestDocSyncAfterCompaction(t *testing.T) {
srv, store, ts, owner, ownerTok := startRealtimeServer(t)
doc, err := store.CreateDocument(owner.ID, "", "Alt", "📄", "", []domain.DocumentBlock{
{ID: "b1", Type: "bullet", Content: "Bleibt"}, {ID: "b2", Type: "bullet", Content: "Weg"},
}, nil, false)
if err != nil {
t.Fatal(err)
}
_, live := syncDoc(t, ts, ownerTok, doc.ID, map[string]interface{}{
"clientId": "live", "lastSeq": 0, "ops": []collab.Op{{OpID: "d", Kind: collab.OpDelete, BlockID: "b2"}},
})
// 40 days later: log and tombstone are past their retention
st, err := srv.store.CompactDocs(time.Now().Add(40 * 24 * time.Hour))
if err != nil || st.Tombstones != 1 {
t.Fatalf("compaction: %+v %v", st, err)
}
// A client that was up to date only gets the purge entry
_, up := syncDoc(t, ts, ownerTok, doc.ID, map[string]interface{}{"clientId": "live", "lastSeq": live.HeadSeq})
if up.Resync || len(up.Ops) != 1 || up.Ops[0].Effect.Kind != collab.OpPurge {
t.Fatalf("up-to-date client: resync=%v ops=%+v", up.Resync, up.Ops)
}
// A client behind the compacted log must reload; its own changes are still applied
code, old := syncDoc(t, ts, ownerTok, doc.ID, map[string]interface{}{
"clientId": "old", "lastSeq": 1,
"ops": []collab.Op{{OpID: "late", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: strp("Spät")}},
})
if code != http.StatusOK || !old.Resync || len(old.Ops) != 0 || old.Results[0].Status != collab.StatusApplied {
t.Fatalf("stale client: %d resync=%v results=%+v", code, old.Resync, old.Results)
}
}
+338
View File
@@ -0,0 +1,338 @@
package api
import (
"bufio"
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/gorilla/websocket"
"github.com/sandrodach/mindsift/internal/auth"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain"
"github.com/sandrodach/mindsift/internal/storage"
)
const testJWTSecret = "test-jwt-secret-key"
type sseMsg struct {
Event string `json:"event"`
Payload json.RawMessage `json:"payload"`
}
type sseStream struct {
id string
events chan sseMsg
}
// startRealtimeServer serves the router over real HTTP with a short WriteTimeout, like production but faster.
func startRealtimeServer(t *testing.T) (*Server, *storage.Store, *httptest.Server, *domain.User, string) {
t.Helper()
srv, store, token := setupTestServer(t)
claims, err := auth.ValidateToken(testJWTSecret, token)
if err != nil {
t.Fatalf("token: %v", err)
}
owner, _ := store.GetUserByID(claims.UserID)
ts := httptest.NewUnstartedServer(srv.Router())
ts.Config.WriteTimeout = time.Second
ts.Start()
t.Cleanup(ts.Close)
return srv, store, ts, owner, token
}
func newTestUser(t *testing.T, store *storage.Store, email string) (*domain.User, string) {
t.Helper()
u, err := store.CreateUser(email, "hash", strings.Split(email, "@")[0], "member")
if err != nil {
t.Fatalf("create user: %v", err)
}
tok, err := auth.GenerateToken(testJWTSecret, u.ID, u.Email, u.DisplayName, u.TokenVersion)
if err != nil {
t.Fatalf("token: %v", err)
}
return u, tok
}
func openSSE(t *testing.T, ts *httptest.Server, token string) *sseStream {
t.Helper()
ctx, cancel := context.WithCancel(context.Background())
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, ts.URL+"/api/v1/events?token="+token, nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
cancel()
t.Fatalf("open sse: %v", err)
}
if resp.StatusCode != http.StatusOK {
cancel()
t.Fatalf("open sse: status %d", resp.StatusCode)
}
st := &sseStream{events: make(chan sseMsg, 64)}
go func() {
defer close(st.events)
sc := bufio.NewScanner(resp.Body)
for sc.Scan() {
if line, ok := strings.CutPrefix(sc.Text(), "data: "); ok {
var m sseMsg
if json.Unmarshal([]byte(line), &m) == nil {
st.events <- m
}
}
}
}()
t.Cleanup(func() {
cancel()
_ = resp.Body.Close()
})
connected := st.waitFor(t, "SSE_CONNECTED", 2*time.Second)
if connected == nil {
t.Fatal("no SSE_CONNECTED event")
}
var p struct {
SSEID string `json:"sseId"`
}
_ = json.Unmarshal(connected.Payload, &p)
st.id = p.SSEID
return st
}
func (st *sseStream) waitFor(t *testing.T, event string, timeout time.Duration) *sseMsg {
t.Helper()
deadline := time.After(timeout)
for {
select {
case m, ok := <-st.events:
if !ok {
return nil
}
if m.Event == event {
return &m
}
case <-deadline:
return nil
}
}
}
func doJSON(t *testing.T, ts *httptest.Server, method, path, token string, body interface{}) *http.Response {
t.Helper()
var buf bytes.Buffer
if body != nil {
_ = json.NewEncoder(&buf).Encode(body)
}
req, _ := http.NewRequest(method, ts.URL+path, &buf)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("%s %s: %v", method, path, err)
}
t.Cleanup(func() { _ = resp.Body.Close() })
return resp
}
func TestRealtimeEndpointsRequireAuth(t *testing.T) {
_, _, ts, _, _ := startRealtimeServer(t)
for _, path := range []string{"/api/v1/events", "/api/v1/ws", "/api/v1/events/rooms"} {
method := http.MethodGet
if strings.HasSuffix(path, "/rooms") {
method = http.MethodPost
}
req, _ := http.NewRequest(method, ts.URL+path, strings.NewReader(`{}`))
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("%s: %v", path, err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusUnauthorized {
t.Errorf("%s without token: want 401, got %d", path, resp.StatusCode)
}
}
}
func TestSSEStreamOutlivesWriteTimeout(t *testing.T) {
srv, _, ts, owner, token := startRealtimeServer(t)
st := openSSE(t, ts, token)
time.Sleep(1500 * time.Millisecond) // longer than the server's WriteTimeout of 1s
srv.wsHub.BroadcastToUser(owner.ID, "LATE_EVENT", map[string]string{"ok": "1"})
if st.waitFor(t, "LATE_EVENT", 2*time.Second) == nil {
t.Fatal("SSE stream was cut by the WriteTimeout")
}
}
func TestTaskEventsOnlyReachAuthorizedUsers(t *testing.T) {
_, store, ts, owner, ownerToken := startRealtimeServer(t)
member, memberToken := newTestUser(t, store, "member@example.com")
_, outsiderToken := newTestUser(t, store, "outsider@example.com")
list, err := store.CreateTaskList(owner.ID, "", "", "Haushalt", "🏠", "#000000")
if err != nil {
t.Fatalf("create list: %v", err)
}
if _, err := store.AddTaskListShare(list.ID, "user", member.ID, "viewer", owner.ID); err != nil {
t.Fatalf("share: %v", err)
}
task, err := store.CreateThoughtWithDetails(owner.ID, "Fenster putzen", "normal", "manual", "", list.ID, "todo", "")
if err != nil {
t.Fatalf("create task: %v", err)
}
memberStream := openSSE(t, ts, memberToken)
outsiderStream := openSSE(t, ts, outsiderToken)
resp := doJSON(t, ts, http.MethodPatch, "/api/v1/thoughts/"+task.ID, ownerToken, map[string]string{"content": "Fenster putzen (geheim)"})
if resp.StatusCode != http.StatusOK {
t.Fatalf("update task: status %d", resp.StatusCode)
}
if memberStream.waitFor(t, "TASK_UPDATED", 2*time.Second) == nil {
t.Error("list member did not receive TASK_UPDATED")
}
if outsiderStream.waitFor(t, "TASK_UPDATED", 500*time.Millisecond) != nil {
t.Error("outsider received TASK_UPDATED of a foreign list")
}
resp = doJSON(t, ts, http.MethodPut, "/api/v1/task-lists/"+list.ID, ownerToken, map[string]string{"title": "Haushalt 2"})
if resp.StatusCode != http.StatusOK {
t.Fatalf("update list: status %d", resp.StatusCode)
}
msg := memberStream.waitFor(t, "TASK_LIST_UPDATED", 2*time.Second)
if msg == nil {
t.Fatal("list member did not receive TASK_LIST_UPDATED")
}
if strings.Contains(string(msg.Payload), "permission") || !strings.Contains(string(msg.Payload), list.ID) {
t.Errorf("TASK_LIST_UPDATED must only carry the list ID, got %s", msg.Payload)
}
if outsiderStream.waitFor(t, "TASK_LIST_UPDATED", 500*time.Millisecond) != nil {
t.Error("outsider received TASK_LIST_UPDATED of a foreign list")
}
}
func TestSSERoomJoinRequiresAccess(t *testing.T) {
_, store, ts, owner, _ := startRealtimeServer(t)
friend, friendToken := newTestUser(t, store, "friend@example.com")
_, strangerToken := newTestUser(t, store, "stranger@example.com")
list, err := store.CreateList(owner.ID, "Einkauf", "🛒", "shopping")
if err != nil {
t.Fatalf("create list: %v", err)
}
friendStream := openSSE(t, ts, friendToken)
strangerStream := openSSE(t, ts, strangerToken)
join := func(token, sseID string) int {
resp := doJSON(t, ts, http.MethodPost, "/api/v1/events/rooms", token, map[string]string{"sseId": sseID, "room": list.ID})
return resp.StatusCode
}
if code := join(strangerToken, strangerStream.id); code != http.StatusForbidden {
t.Errorf("stranger joining a foreign list room: want 403, got %d", code)
}
if code := join(friendToken, friendStream.id); code != http.StatusForbidden {
t.Errorf("friend joining before being shared: want 403, got %d", code)
}
if _, err := store.AddListShare(owner.ID, list.ID, "user", friend.ID, "viewer"); err != nil {
t.Fatalf("share list: %v", err)
}
if code := join(strangerToken, friendStream.id); code != http.StatusForbidden {
t.Errorf("joining with someone else's sseId: want 403, got %d", code)
}
if code := join(friendToken, friendStream.id); code != http.StatusOK {
t.Errorf("friend joining shared list room: want 200, got %d", code)
}
}
func TestReorderRejectsForeignTasks(t *testing.T) {
_, store, ts, owner, _ := startRealtimeServer(t)
_, attackerToken := newTestUser(t, store, "attacker@example.com")
task, err := store.CreateThoughtWithDetails(owner.ID, "Privat", "normal", "manual", "", "", "todo", "")
if err != nil {
t.Fatalf("create task: %v", err)
}
done := "done"
resp := doJSON(t, ts, http.MethodPost, "/api/v1/tasks/reorder", attackerToken, map[string]interface{}{
"items": []domain.ThoughtReorderItem{{ID: task.ID, SortOrder: 99, KanbanStatus: &done}},
})
if resp.StatusCode == http.StatusOK {
t.Fatal("reordering a foreign task must be rejected")
}
after, _ := store.GetThoughtByID(owner.ID, task.ID)
if after.SortOrder == 99 || after.IsCompleted {
t.Fatalf("foreign task was modified: sortOrder=%d completed=%v", after.SortOrder, after.IsCompleted)
}
}
func dialWS(t *testing.T, ts *httptest.Server, token string) *websocket.Conn {
t.Helper()
url := "ws" + strings.TrimPrefix(ts.URL, "http") + "/api/v1/ws?token=" + token
conn, _, err := websocket.DefaultDialer.Dial(url, nil)
if err != nil {
t.Fatalf("dial ws: %v", err)
}
t.Cleanup(func() { _ = conn.Close() })
return conn
}
func readWSEvent(conn *websocket.Conn, event string, timeout time.Duration) *sseMsg {
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
_ = conn.SetReadDeadline(deadline)
var m sseMsg
if err := conn.ReadJSON(&m); err != nil {
return nil
}
if m.Event == event {
return &m
}
}
return nil
}
func TestWebSocketHeartbeat(t *testing.T) {
_, _, ts, _, token := startRealtimeServer(t)
conn := dialWS(t, ts, token)
if err := conn.WriteJSON(map[string]string{"event": "PING"}); err != nil {
t.Fatal(err)
}
if readWSEvent(conn, "PONG", 2*time.Second) == nil {
t.Fatal("no PONG for an app-level PING")
}
}
func TestDocChangesArePushedToTheNoteRoom(t *testing.T) {
_, store, ts, owner, ownerToken := startRealtimeServer(t)
editor, editorToken := newTestUser(t, store, "push-editor@example.com")
doc, err := store.CreateDocument(owner.ID, "", "Push", "📄", "", []domain.DocumentBlock{{ID: "push_b1", Type: "bullet", Content: "Start"}}, nil, false)
if err != nil {
t.Fatal(err)
}
if _, err := store.CreateDocShare(owner.ID, doc.ID, editor.ID, "write"); err != nil {
t.Fatal(err)
}
conn := dialWS(t, ts, editorToken)
if err := conn.WriteJSON(map[string]string{"event": "JOIN_ROOM", "room": "doc:" + doc.ID}); err != nil {
t.Fatal(err)
}
time.Sleep(100 * time.Millisecond) // the join is processed asynchronously
text := "Geändert"
resp := doJSON(t, ts, http.MethodPost, "/api/v1/docs/"+doc.ID+"/sync", ownerToken, map[string]interface{}{
"clientId": "owner-tab",
"ops": []collab.Op{{OpID: "push-1", Kind: collab.OpSetContent, BlockID: "push_b1", BaseVersion: 1, Content: &text}},
})
if resp.StatusCode != http.StatusOK {
t.Fatalf("sync: %d", resp.StatusCode)
}
msg := readWSEvent(conn, "DOC_SEQ", 2*time.Second)
if msg == nil || !strings.Contains(string(msg.Payload), doc.ID) {
t.Fatalf("no DOC_SEQ push for the note: %+v", msg)
}
}
+115 -68
View File
@@ -18,6 +18,7 @@ import (
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/cors" "github.com/go-chi/cors"
"github.com/sandrodach/mindsift/internal/auth" "github.com/sandrodach/mindsift/internal/auth"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/config" "github.com/sandrodach/mindsift/internal/config"
"github.com/sandrodach/mindsift/internal/domain" "github.com/sandrodach/mindsift/internal/domain"
"github.com/sandrodach/mindsift/internal/storage" "github.com/sandrodach/mindsift/internal/storage"
@@ -29,6 +30,9 @@ type Server struct {
store *storage.Store store *storage.Store
wsHub *ws.Hub wsHub *ws.Hub
staticFS fs.FS staticFS fs.FS
leases *collab.LeaseManager // block leases and open sessions of notes
notifier *collab.Notifier // wakes long-poll requests on note changes
} }
func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS fs.FS) *Server { func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS fs.FS) *Server {
@@ -37,11 +41,32 @@ func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS
store: store, store: store,
wsHub: wsHub, wsHub: wsHub,
staticFS: staticFS, staticFS: staticFS,
leases: collab.NewLeaseManager(nil),
notifier: collab.NewNotifier(),
} }
wsHub.SetRoomAuthorizer(srv.canJoinRoom)
go srv.startAlarmTicker() go srv.startAlarmTicker()
go srv.startCompaction()
return srv return srv
} }
// canJoinRoom allows realtime rooms only to users with read access: "doc:<id>" for notes, otherwise a shared list ID.
func (s *Server) canJoinRoom(userID, room string) bool {
if docID, ok := strings.CutPrefix(room, "doc:"); ok {
return s.store.DocRole(userID, docID) != ""
}
_, err := s.store.GetListByID(userID, room)
return err == nil
}
// taskAudience returns everyone who can see a task: its owner plus all users of its task list.
func (s *Server) taskAudience(th *domain.Thought) []string {
if th == nil {
return nil
}
return append(s.store.GetTaskListUserIDs(th.TaskListID), th.UserID)
}
func (s *Server) Router() http.Handler { func (s *Server) Router() http.Handler {
r := chi.NewRouter() r := chi.NewRouter()
@@ -69,13 +94,13 @@ func (s *Server) Router() http.Handler {
r.Get("/api/v1/ws/spaces/{spaceId}", s.handleWebSocket) r.Get("/api/v1/ws/spaces/{spaceId}", s.handleWebSocket)
r.Get("/api/v1/ws/lists/{listId}", s.handleWebSocket) r.Get("/api/v1/ws/lists/{listId}", s.handleWebSocket)
r.Get("/api/v1/events", s.handleSSE) r.Get("/api/v1/events", s.handleSSE)
r.Post("/api/v1/events/rooms", s.handleSSERooms)
// Authenticated API Routes // Authenticated API Routes
r.Group(func(r chi.Router) { r.Group(func(r chi.Router) {
r.Use(s.authMiddleware) r.Use(s.authMiddleware)
r.Get("/api/v1/auth/me", s.handleMe) r.Get("/api/v1/auth/me", s.handleMe)
r.Post("/api/v1/events/rooms", s.handleSSERooms)
r.Post("/api/v1/auth/logout", s.handleLogout) r.Post("/api/v1/auth/logout", s.handleLogout)
r.Post("/api/v1/auth/revoke-all", s.handleRevokeAllSessions) r.Post("/api/v1/auth/revoke-all", s.handleRevokeAllSessions)
r.Patch("/api/v1/auth/settings", s.handleUpdateUserSettings) r.Patch("/api/v1/auth/settings", s.handleUpdateUserSettings)
@@ -135,11 +160,16 @@ func (s *Server) Router() http.Handler {
r.Post("/api/v1/documents", s.handleCreateDocument) r.Post("/api/v1/documents", s.handleCreateDocument)
r.Get("/api/v1/documents/{id}", s.handleGetDocument) r.Get("/api/v1/documents/{id}", s.handleGetDocument)
r.Put("/api/v1/documents/{id}", s.handleUpdateDocument) r.Put("/api/v1/documents/{id}", s.handleUpdateDocument)
r.Patch("/api/v1/documents/{id}/blocks", s.handlePatchDocumentBlocks)
r.Put("/api/v1/documents/{id}/move", s.handleMoveDocument) r.Put("/api/v1/documents/{id}/move", s.handleMoveDocument)
r.Delete("/api/v1/documents/{id}", s.handleDeleteDocument) r.Delete("/api/v1/documents/{id}", s.handleDeleteDocument)
r.Post("/api/v1/documents/{id}/blocks/{blockId}/enhanced-task", s.handleCreateEnhancedTask) r.Post("/api/v1/documents/{id}/blocks/{blockId}/enhanced-task", s.handleCreateEnhancedTask)
// Collaborative note editing (operation log)
r.Get("/api/v1/docs/{id}/snapshot", s.handleDocSnapshot)
r.Post("/api/v1/docs/{id}/sync", s.handleDocSync)
r.Get("/api/v1/docs/{id}/changes", s.handleDocChanges)
r.Post("/api/v1/docs/{id}/leave", s.handleDocLeave)
// Document Sharing & Collaboration // Document Sharing & Collaboration
r.Get("/api/v1/documents/{id}/shares", s.handleGetDocShares) r.Get("/api/v1/documents/{id}/shares", s.handleGetDocShares)
r.Post("/api/v1/documents/{id}/shares", s.handleCreateDocShare) r.Post("/api/v1/documents/{id}/shares", s.handleCreateDocShare)
@@ -699,7 +729,7 @@ func (s *Server) handleCaptureInbox(w http.ResponseWriter, r *http.Request) {
} }
} }
s.wsHub.BroadcastTaskEvent(thought.UserID, "TASK_CREATED", thought) s.wsHub.BroadcastToUsers(s.taskAudience(thought), "TASK_CREATED", thought)
writeJSON(w, http.StatusCreated, thought) writeJSON(w, http.StatusCreated, thought)
} }
@@ -785,7 +815,7 @@ func (s *Server) handleTriageAction(w http.ResponseWriter, r *http.Request) {
} }
} }
s.wsHub.BroadcastTaskEvent(updated.UserID, "TASK_UPDATED", updated) s.wsHub.BroadcastToUsers(s.taskAudience(updated), "TASK_UPDATED", updated)
writeJSON(w, http.StatusOK, updated) writeJSON(w, http.StatusOK, updated)
} }
@@ -819,11 +849,19 @@ func (s *Server) handleReorderTasks(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error()) writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error())
return return
} }
// ReorderTasks updates by ID only, so every item must be a task the caller may edit
audience := []string{userID}
for _, it := range req.Items { for _, it := range req.Items {
if th, err := s.store.GetThoughtByID(userID, it.ID); err == nil && th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) { th, err := s.store.GetThoughtByID(userID, it.ID)
if err != nil {
writeError(w, http.StatusNotFound, "Aufgabe nicht gefunden")
return
}
if th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) {
writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error()) writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error())
return return
} }
audience = append(audience, s.taskAudience(th)...)
} }
if err := s.store.ReorderTasks(taskListID, req.Items); err != nil { if err := s.store.ReorderTasks(taskListID, req.Items); err != nil {
@@ -831,7 +869,7 @@ func (s *Server) handleReorderTasks(w http.ResponseWriter, r *http.Request) {
return return
} }
s.wsHub.BroadcastToAll("TASK_REORDERED", map[string]interface{}{ s.wsHub.BroadcastToUsers(audience, "TASK_REORDERED", map[string]interface{}{
"taskListId": taskListID, "taskListId": taskListID,
"items": req.Items, "items": req.Items,
"userId": userID, "userId": userID,
@@ -859,13 +897,17 @@ func (s *Server) handleUpdateThought(w http.ResponseWriter, r *http.Request) {
return return
} }
before, _ := s.store.GetThoughtByID(userID, thoughtID)
updated, err := s.store.UpdateThoughtDetails(userID, thoughtID, req.Content, req.IsCompleted, req.DueDate, req.Priority, req.TaskListID, req.KanbanStatus, req.Notes, req.Subtasks, req.Tags) updated, err := s.store.UpdateThoughtDetails(userID, thoughtID, req.Content, req.IsCompleted, req.DueDate, req.Priority, req.TaskListID, req.KanbanStatus, req.Notes, req.Subtasks, req.Tags)
if err != nil { if err != nil {
writeStoreError(w, http.StatusBadRequest, err) writeStoreError(w, http.StatusBadRequest, err)
return return
} }
s.store.SyncDocBlockAndTask(userID, thoughtID, updated.IsCompleted) if docID := s.store.SyncDocBlockAndTask(userID, thoughtID, updated.IsCompleted); docID != "" {
s.wsHub.BroadcastToAll("TASK_UPDATED", updated) _, head, _ := s.store.GetDocOpsSince(docID, 0, 0)
s.notifyDocChanged(docID, head)
}
s.wsHub.BroadcastToUsers(append(s.taskAudience(before), s.taskAudience(updated)...), "TASK_UPDATED", updated)
writeJSON(w, http.StatusOK, updated) writeJSON(w, http.StatusOK, updated)
} }
@@ -879,16 +921,18 @@ func (s *Server) handleUnlinkEnhancedTask(w http.ResponseWriter, r *http.Request
func (s *Server) handleDeleteThought(w http.ResponseWriter, r *http.Request) { func (s *Server) handleDeleteThought(w http.ResponseWriter, r *http.Request) {
userID := auth.GetUserID(r.Context()) userID := auth.GetUserID(r.Context())
thoughtID := chi.URLParam(r, "id") thoughtID := chi.URLParam(r, "id")
if th, err := s.store.GetThoughtByID(userID, thoughtID); err == nil && th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) { th, err := s.store.GetThoughtByID(userID, thoughtID)
if err == nil && th.TaskListID != "" && !s.store.CanEditTaskList(userID, th.TaskListID) {
writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error()) writeError(w, http.StatusForbidden, storage.ErrTaskListReadOnly.Error())
return return
} }
audience := s.taskAudience(th)
s.store.UnlinkTaskFromDocBlock(userID, thoughtID) s.store.UnlinkTaskFromDocBlock(userID, thoughtID)
if err := s.store.DeleteThought(userID, thoughtID); err != nil { if err := s.store.DeleteThought(userID, thoughtID); err != nil {
writeStoreError(w, http.StatusInternalServerError, err) writeStoreError(w, http.StatusInternalServerError, err)
return return
} }
s.wsHub.BroadcastToAll("TASK_DELETED", map[string]string{"id": thoughtID}) s.wsHub.BroadcastToUsers(audience, "TASK_DELETED", map[string]string{"id": thoughtID})
writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) writeJSON(w, http.StatusOK, map[string]bool{"deleted": true})
} }
@@ -978,59 +1022,32 @@ func (s *Server) handleUpdateDocument(w http.ResponseWriter, r *http.Request) {
Icon *string `json:"icon"` Icon *string `json:"icon"`
FolderPath *string `json:"folderPath"` FolderPath *string `json:"folderPath"`
Blocks *[]domain.DocumentBlock `json:"blocks"` Blocks *[]domain.DocumentBlock `json:"blocks"`
Tags *[]string `json:"tags"`
IsPinned *bool `json:"isPinned"` IsPinned *bool `json:"isPinned"`
} }
if err := json.NewDecoder(r.Body).Decode(&req); err != nil { if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "Ungültiges JSON") writeError(w, http.StatusBadRequest, "Ungültiges JSON")
return return
} }
if req.Blocks != nil {
writeError(w, http.StatusBadRequest, "Blöcke werden über POST /api/v1/docs/{id}/sync gespeichert")
return
}
doc, err := s.store.UpdateDocument(userID, docID, req.Title, req.Icon, req.FolderPath, req.Blocks, req.Tags, req.IsPinned) doc, err := s.store.UpdateDocument(userID, docID, req.Title, req.Icon, req.FolderPath, req.IsPinned)
if err != nil { if err != nil {
writeError(w, http.StatusBadRequest, err.Error()) writeError(w, http.StatusBadRequest, err.Error())
return return
} }
if req.Title != nil || req.Icon != nil || req.FolderPath != nil || req.Blocks != nil || req.Tags != nil { if req.Title != nil {
_, head, _ := s.store.GetDocOpsSince(docID, 0, 0)
s.notifyDocChanged(docID, head)
}
if req.Title != nil || req.Icon != nil || req.FolderPath != nil {
s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED") s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED")
} }
writeJSON(w, http.StatusOK, doc) writeJSON(w, http.StatusOK, doc)
} }
func (s *Server) handlePatchDocumentBlocks(w http.ResponseWriter, r *http.Request) {
userID := auth.GetUserID(r.Context())
docID := chi.URLParam(r, "id")
var req domain.DocBlocksPatchRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "Ungültiges JSON")
return
}
doc, mergedOrder, err := s.store.PatchDocumentBlocks(userID, docID, req)
if err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
patchEvent := domain.DocBlocksPatchEvent{
DocID: docID,
UpdatedBy: userID,
UpdatedAt: doc.UpdatedAt,
Upserted: req.Upserted,
DeletedIDs: req.DeletedIDs,
BlockOrder: mergedOrder,
}
s.wsHub.BroadcastDocEvent(docID, "DOC_BLOCKS_PATCHED", patchEvent)
if users, err := s.store.GetAuthorizedDocUsers(docID); err == nil {
for _, uid := range users {
s.wsHub.BroadcastToUser(uid, "DOC_BLOCKS_PATCHED", patchEvent)
}
}
writeJSON(w, http.StatusOK, doc)
}
func (s *Server) handleDeleteDocument(w http.ResponseWriter, r *http.Request) { func (s *Server) handleDeleteDocument(w http.ResponseWriter, r *http.Request) {
userID := auth.GetUserID(r.Context()) userID := auth.GetUserID(r.Context())
docID := chi.URLParam(r, "id") docID := chi.URLParam(r, "id")
@@ -1065,6 +1082,8 @@ func (s *Server) handleCreateEnhancedTask(w http.ResponseWriter, r *http.Request
writeError(w, http.StatusBadRequest, err.Error()) writeError(w, http.StatusBadRequest, err.Error())
return return
} }
_, head, _ := s.store.GetDocOpsSince(docID, 0, 0)
s.notifyDocChanged(docID, head)
s.wsHub.BroadcastToUser(userID, "TASK_CREATED", thought) s.wsHub.BroadcastToUser(userID, "TASK_CREATED", thought)
writeJSON(w, http.StatusCreated, thought) writeJSON(w, http.StatusCreated, thought)
@@ -1096,7 +1115,7 @@ func (s *Server) handleCreateFolder(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, err.Error()) writeError(w, http.StatusBadRequest, err.Error())
return return
} }
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": cleanPath}) s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": cleanPath})
writeJSON(w, http.StatusCreated, map[string]string{"path": cleanPath}) writeJSON(w, http.StatusCreated, map[string]string{"path": cleanPath})
} }
@@ -1117,7 +1136,7 @@ func (s *Server) handleDeleteFolder(w http.ResponseWriter, r *http.Request) {
for _, docID := range deletedDocIDs { for _, docID := range deletedDocIDs {
s.wsHub.BroadcastDocEvent(docID, "DOC_DELETED", map[string]string{"id": docID}) s.wsHub.BroadcastDocEvent(docID, "DOC_DELETED", map[string]string{"id": docID})
} }
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": folderPath}) s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", map[string]string{"path": folderPath})
writeJSON(w, http.StatusOK, map[string]interface{}{ writeJSON(w, http.StatusOK, map[string]interface{}{
"deleted": true, "deleted": true,
@@ -1146,7 +1165,7 @@ func (s *Server) handleMoveFolder(w http.ResponseWriter, r *http.Request) {
s.broadcastDocToAuthorizedUsers(id, "DOC_UPDATED") s.broadcastDocToAuthorizedUsers(id, "DOC_UPDATED")
} }
} }
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{ s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", map[string]string{
"sourcePath": req.SourcePath, "sourcePath": req.SourcePath,
"newPath": newPath, "newPath": newPath,
}) })
@@ -1183,7 +1202,7 @@ func (s *Server) handleCreateSharedFolder(w http.ResponseWriter, r *http.Request
writeError(w, http.StatusBadRequest, err.Error()) writeError(w, http.StatusBadRequest, err.Error())
return return
} }
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", folder) s.wsHub.BroadcastToUser(userID, "WORKSPACE_FOLDERS_UPDATED", folder)
writeJSON(w, http.StatusCreated, folder) writeJSON(w, http.StatusCreated, folder)
} }
@@ -1201,11 +1220,12 @@ func (s *Server) handleGetSharedFolderByID(w http.ResponseWriter, r *http.Reques
func (s *Server) handleDeleteSharedFolder(w http.ResponseWriter, r *http.Request) { func (s *Server) handleDeleteSharedFolder(w http.ResponseWriter, r *http.Request) {
userID := auth.GetUserID(r.Context()) userID := auth.GetUserID(r.Context())
folderID := chi.URLParam(r, "id") folderID := chi.URLParam(r, "id")
audience := s.store.GetSharedFolderUserIDs(folderID)
if err := s.store.DeleteSharedFolder(userID, folderID); err != nil { if err := s.store.DeleteSharedFolder(userID, folderID); err != nil {
writeError(w, http.StatusInternalServerError, err.Error()) writeError(w, http.StatusInternalServerError, err.Error())
return return
} }
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID}) s.wsHub.BroadcastToUsers(append(audience, userID), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID})
writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) writeJSON(w, http.StatusOK, map[string]bool{"deleted": true})
} }
@@ -1273,7 +1293,7 @@ func (s *Server) handleCreateFolderShare(w http.ResponseWriter, r *http.Request)
writeError(w, http.StatusBadRequest, err.Error()) writeError(w, http.StatusBadRequest, err.Error())
return return
} }
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", share) s.wsHub.BroadcastToUsers(s.store.GetSharedFolderUserIDs(folderID), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID})
if docIDs, err := s.store.GetSharedFolderDocIDs(folderID); err == nil { if docIDs, err := s.store.GetSharedFolderDocIDs(folderID); err == nil {
for _, docID := range docIDs { for _, docID := range docIDs {
s.broadcastDocToAuthorizedUsers(docID, "DOC_CREATED") s.broadcastDocToAuthorizedUsers(docID, "DOC_CREATED")
@@ -1287,13 +1307,24 @@ func (s *Server) handleDeleteFolderShare(w http.ResponseWriter, r *http.Request)
folderID := chi.URLParam(r, "id") folderID := chi.URLParam(r, "id")
shareID := chi.URLParam(r, "shareId") shareID := chi.URLParam(r, "shareId")
docIDs, _ := s.store.GetSharedFolderDocIDs(folderID) docIDs, _ := s.store.GetSharedFolderDocIDs(folderID)
audienceBefore := s.store.GetSharedFolderUserIDs(folderID)
if err := s.store.DeleteFolderShare(userID, folderID, shareID); err != nil { if err := s.store.DeleteFolderShare(userID, folderID, shareID); err != nil {
writeError(w, http.StatusInternalServerError, err.Error()) writeError(w, http.StatusInternalServerError, err.Error())
return return
} }
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID, "shareId": shareID}) audienceAfter := s.store.GetSharedFolderUserIDs(folderID)
stillAllowed := make(map[string]bool, len(audienceAfter))
for _, uid := range audienceAfter {
stillAllowed[uid] = true
}
s.wsHub.BroadcastToUsers(append(audienceBefore, audienceAfter...), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"folderId": folderID})
for _, docID := range docIDs { for _, docID := range docIDs {
s.wsHub.BroadcastToUser(shareID, "DOC_DELETED", map[string]string{"id": docID}) // Users who lost access remove the notes; the rest refresh them
for _, uid := range audienceBefore {
if !stillAllowed[uid] {
s.wsHub.BroadcastToUser(uid, "DOC_DELETED", map[string]string{"id": docID})
}
}
s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED") s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED")
} }
writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) writeJSON(w, http.StatusOK, map[string]bool{"deleted": true})
@@ -1316,7 +1347,8 @@ func (s *Server) handleMoveDocument(w http.ResponseWriter, r *http.Request) {
} }
s.wsHub.BroadcastDocEvent(docID, "DOC_UPDATED", doc) s.wsHub.BroadcastDocEvent(docID, "DOC_UPDATED", doc)
s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED") s.broadcastDocToAuthorizedUsers(docID, "DOC_UPDATED")
s.wsHub.BroadcastDocEvent("", "WORKSPACE_FOLDERS_UPDATED", nil) docUsers, _ := s.store.GetAuthorizedDocUsers(docID)
s.wsHub.BroadcastToUsers(append(docUsers, userID), "WORKSPACE_FOLDERS_UPDATED", map[string]string{"docId": docID})
writeJSON(w, http.StatusOK, doc) writeJSON(w, http.StatusOK, doc)
} }
@@ -1751,7 +1783,8 @@ func (s *Server) handleWebSocket(w http.ResponseWriter, r *http.Request) {
userID := s.authenticateUserFromReq(r) userID := s.authenticateUserFromReq(r)
if userID == "" { if userID == "" {
userID = "anonymous" writeError(w, http.StatusUnauthorized, "Nicht autorisiert – bitte anmelden")
return
} }
s.wsHub.ServeWS(w, r, roomID, userID) s.wsHub.ServeWS(w, r, roomID, userID)
} }
@@ -1764,7 +1797,8 @@ func (s *Server) handleSSE(w http.ResponseWriter, r *http.Request) {
userID := s.authenticateUserFromReq(r) userID := s.authenticateUserFromReq(r)
if userID == "" { if userID == "" {
userID = "anonymous" writeError(w, http.StatusUnauthorized, "Nicht autorisiert – bitte anmelden")
return
} }
var initialRooms []string var initialRooms []string
@@ -1784,6 +1818,7 @@ func (s *Server) handleSSE(w http.ResponseWriter, r *http.Request) {
} }
func (s *Server) handleSSERooms(w http.ResponseWriter, r *http.Request) { func (s *Server) handleSSERooms(w http.ResponseWriter, r *http.Request) {
userID := auth.GetUserID(r.Context())
var req struct { var req struct {
SSEID string `json:"sseId"` SSEID string `json:"sseId"`
Room string `json:"room"` Room string `json:"room"`
@@ -1803,9 +1838,13 @@ func (s *Server) handleSSERooms(w http.ResponseWriter, r *http.Request) {
} }
if req.Action == "leave" { if req.Action == "leave" {
s.wsHub.LeaveSSERoomByID(req.SSEID, req.Room) s.wsHub.LeaveSSERoomByID(req.SSEID, userID, req.Room)
} else { writeJSON(w, http.StatusOK, map[string]bool{"success": true})
s.wsHub.JoinSSERoomByID(req.SSEID, req.Room) return
}
if !s.wsHub.JoinSSERoomByID(req.SSEID, userID, req.Room) {
writeError(w, http.StatusForbidden, "Kein Zugriff auf diesen Raum")
return
} }
writeJSON(w, http.StatusOK, map[string]bool{"success": true}) writeJSON(w, http.StatusOK, map[string]bool{"success": true})
} }
@@ -1943,7 +1982,7 @@ func (s *Server) handleWebhookIngest(w http.ResponseWriter, r *http.Request) {
return return
} }
s.wsHub.BroadcastTaskEvent(thought.UserID, "TASK_CREATED", thought) s.wsHub.BroadcastToUsers(s.taskAudience(thought), "TASK_CREATED", thought)
writeJSON(w, http.StatusCreated, map[string]interface{}{ writeJSON(w, http.StatusCreated, map[string]interface{}{
"status": "ingested", "status": "ingested",
@@ -2180,23 +2219,25 @@ func (s *Server) handleUpdateTaskList(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "Ungültiges JSON") writeError(w, http.StatusBadRequest, "Ungültiges JSON")
return return
} }
audienceBefore := s.store.GetTaskListUserIDs(listID)
tl, err := s.store.UpdateTaskList(userID, listID, req.Title, req.Icon, req.Color, req.TeamID, req.SpaceID, req.Columns) tl, err := s.store.UpdateTaskList(userID, listID, req.Title, req.Icon, req.Color, req.TeamID, req.SpaceID, req.Columns)
if err != nil { if err != nil {
writeStoreError(w, http.StatusBadRequest, err) writeStoreError(w, http.StatusBadRequest, err)
return return
} }
s.wsHub.BroadcastToAll("TASK_LIST_UPDATED", tl) s.broadcastTaskListChange(listID, audienceBefore, "TASK_LIST_UPDATED")
writeJSON(w, http.StatusOK, tl) writeJSON(w, http.StatusOK, tl)
} }
func (s *Server) handleDeleteTaskList(w http.ResponseWriter, r *http.Request) { func (s *Server) handleDeleteTaskList(w http.ResponseWriter, r *http.Request) {
userID := auth.GetUserID(r.Context()) userID := auth.GetUserID(r.Context())
listID := chi.URLParam(r, "id") listID := chi.URLParam(r, "id")
audienceBefore := s.store.GetTaskListUserIDs(listID)
if err := s.store.DeleteTaskList(userID, listID); err != nil { if err := s.store.DeleteTaskList(userID, listID); err != nil {
writeError(w, http.StatusInternalServerError, err.Error()) writeError(w, http.StatusInternalServerError, err.Error())
return return
} }
s.wsHub.BroadcastToAll("TASK_LIST_DELETED", map[string]string{"id": listID}) s.broadcastTaskListChange(listID, audienceBefore, "TASK_LIST_DELETED")
writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) writeJSON(w, http.StatusOK, map[string]bool{"deleted": true})
} }
@@ -2245,8 +2286,7 @@ func (s *Server) handleCreateTaskListShare(w http.ResponseWriter, r *http.Reques
writeError(w, http.StatusBadRequest, err.Error()) writeError(w, http.StatusBadRequest, err.Error())
return return
} }
updatedList, _ := s.store.GetTaskListByID(userID, listID) s.broadcastTaskListChange(listID, nil, "TASK_LIST_UPDATED")
s.wsHub.BroadcastToAll("TASK_LIST_UPDATED", updatedList)
writeJSON(w, http.StatusCreated, share) writeJSON(w, http.StatusCreated, share)
} }
@@ -2263,15 +2303,22 @@ func (s *Server) handleDeleteTaskListShare(w http.ResponseWriter, r *http.Reques
writeError(w, http.StatusForbidden, "Nur der Eigentümer kann Freigaben entfernen") writeError(w, http.StatusForbidden, "Nur der Eigentümer kann Freigaben entfernen")
return return
} }
audienceBefore := s.store.GetTaskListUserIDs(listID)
if err := s.store.RemoveTaskListShare(listID, shareID); err != nil { if err := s.store.RemoveTaskListShare(listID, shareID); err != nil {
writeError(w, http.StatusInternalServerError, err.Error()) writeError(w, http.StatusInternalServerError, err.Error())
return return
} }
updatedList, _ := s.store.GetTaskListByID(userID, listID) s.broadcastTaskListChange(listID, audienceBefore, "TASK_LIST_UPDATED")
s.wsHub.BroadcastToAll("TASK_LIST_UPDATED", updatedList)
writeJSON(w, http.StatusOK, map[string]bool{"deleted": true}) writeJSON(w, http.StatusOK, map[string]bool{"deleted": true})
} }
// broadcastTaskListChange notifies everyone who could see the list before or can see it now. Only the ID is sent,
// because the list payload depends on the receiver's permission; clients reload their lists.
func (s *Server) broadcastTaskListChange(listID string, audienceBefore []string, event string) {
users := append(audienceBefore, s.store.GetTaskListUserIDs(listID)...)
s.wsHub.BroadcastToUsers(users, event, map[string]string{"id": listID})
}
// --- ORGANIZATIONS & TEAMS --- // --- ORGANIZATIONS & TEAMS ---
func (s *Server) handleGetOrganizations(w http.ResponseWriter, r *http.Request) { func (s *Server) handleGetOrganizations(w http.ResponseWriter, r *http.Request) {
+20 -6
View File
@@ -13,6 +13,7 @@ import (
"time" "time"
"github.com/sandrodach/mindsift/internal/auth" "github.com/sandrodach/mindsift/internal/auth"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/config" "github.com/sandrodach/mindsift/internal/config"
"github.com/sandrodach/mindsift/internal/domain" "github.com/sandrodach/mindsift/internal/domain"
"github.com/sandrodach/mindsift/internal/storage" "github.com/sandrodach/mindsift/internal/storage"
@@ -1269,17 +1270,30 @@ func TestDocumentSharingStrictlyReadOnly(t *testing.T) {
t.Fatalf("expected error for User B updating document, got 200 OK") t.Fatalf("expected error for User B updating document, got 200 OK")
} }
// 3. User B tries to patch document blocks -> must fail (not owner) // 3. User B tries to change blocks through the sync API -> must fail (read-only)
patchBody, _ := json.Marshal(domain.DocBlocksPatchRequest{ hacked := "Hacked by User B"
Upserted: []domain.DocumentBlock{{ID: "b2", Content: "Hacked by User B"}}, syncBody, _ := json.Marshal(map[string]interface{}{
"clientId": "tab-b",
"ops": []collab.Op{{OpID: "b-hack", Kind: collab.OpSetContent, BlockID: "b2", BaseVersion: 1, Content: &hacked}},
}) })
req = httptest.NewRequest(http.MethodPatch, "/api/v1/documents/"+doc.ID+"/blocks", bytes.NewReader(patchBody)) req = httptest.NewRequest(http.MethodPost, "/api/v1/docs/"+doc.ID+"/sync", bytes.NewReader(syncBody))
req.Header.Set("Authorization", "Bearer "+tokenB) req.Header.Set("Authorization", "Bearer "+tokenB)
req.Header.Set("Content-Type", "application/json") req.Header.Set("Content-Type", "application/json")
rec = httptest.NewRecorder() rec = httptest.NewRecorder()
router.ServeHTTP(rec, req) router.ServeHTTP(rec, req)
if rec.Code == http.StatusOK { if rec.Code != http.StatusForbidden {
t.Fatalf("expected error for User B patching document blocks, got 200 OK") t.Fatalf("expected 403 for User B changing blocks, got %d", rec.Code)
}
// The old whole-document save no longer accepts blocks, not even from the owner
blocksBody, _ := json.Marshal(map[string]interface{}{"blocks": []domain.DocumentBlock{{ID: "b1", Content: "x"}}})
req = httptest.NewRequest(http.MethodPut, "/api/v1/documents/"+doc.ID, bytes.NewReader(blocksBody))
req.Header.Set("Authorization", "Bearer "+tokenA)
req.Header.Set("Content-Type", "application/json")
rec = httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected 400 for blocks via PUT, got %d", rec.Code)
} }
// 4. User A (owner) can successfully update document // 4. User A (owner) can successfully update document
+201
View File
@@ -0,0 +1,201 @@
// Package collab contains the database-independent core of collaborative note editing: the block tree,
// operations with their conflict rules, block leases and change notification. The same rules run on the
// server for every incoming operation; clients replay the resulting effects to stay in sync.
package collab
import (
"fmt"
"sort"
)
// Props are per-block settings that change independently of the text.
type Props struct {
Checked bool `json:"checked,omitempty"`
Collapsed bool `json:"collapsed,omitempty"`
Lang string `json:"lang,omitempty"`
ConflictOf string `json:"conflictOf,omitempty"` // ID of the block (or "title") this conflict copy belongs to
ConflictAuthor string `json:"conflictAuthor,omitempty"` // display name of the author whose text lost
}
type Block struct {
ID string `json:"id"`
ParentID string `json:"parentId"` // "" = top level
OrderKey string `json:"orderKey"`
Type string `json:"type"`
Content string `json:"content"`
Props Props `json:"props"`
Version int `json:"version"` // content version; props changes do not bump it
Deleted bool `json:"deleted,omitempty"`
UpdatedBy string `json:"updatedBy,omitempty"`
}
// Doc is the complete state of one note including deleted blocks (tombstones).
type Doc struct {
Title string
TitleVersion int
Blocks map[string]*Block
}
func NewDoc() *Doc {
return &Doc{Blocks: make(map[string]*Block)}
}
func (d *Doc) Clone() *Doc {
c := &Doc{Title: d.Title, TitleVersion: d.TitleVersion, Blocks: make(map[string]*Block, len(d.Blocks))}
for id, b := range d.Blocks {
cp := *b
c.Blocks[id] = &cp
}
return c
}
func siblingLess(a, b *Block) bool {
if a.OrderKey != b.OrderKey {
return a.OrderKey < b.OrderKey
}
return a.ID < b.ID
}
// Children returns the live children of parentID in display order.
func (d *Doc) Children(parentID string) []*Block {
out := make([]*Block, 0)
for _, b := range d.Blocks {
if !b.Deleted && b.ParentID == parentID {
out = append(out, b)
}
}
sort.Slice(out, func(i, j int) bool { return siblingLess(out[i], out[j]) })
return out
}
// FlatBlock is a live block with its depth, in document (depth-first) order.
type FlatBlock struct {
*Block
Level int
}
// Flatten returns all live blocks in display order with their nesting level.
func (d *Doc) Flatten() []FlatBlock {
children := make(map[string][]*Block)
for _, b := range d.Blocks {
if !b.Deleted {
children[b.ParentID] = append(children[b.ParentID], b)
}
}
for _, list := range children {
sort.Slice(list, func(i, j int) bool { return siblingLess(list[i], list[j]) })
}
out := make([]FlatBlock, 0, len(d.Blocks))
var walk func(parent string, level int)
walk = func(parent string, level int) {
for _, b := range children[parent] {
out = append(out, FlatBlock{Block: b, Level: level})
walk(b.ID, level+1)
}
}
walk("", 0)
return out
}
// subtree returns id and all descendants; with liveOnly, deleted blocks are skipped.
func (d *Doc) subtree(id string, liveOnly bool) []string {
children := make(map[string][]string)
for _, b := range d.Blocks {
if liveOnly && b.Deleted {
continue
}
children[b.ParentID] = append(children[b.ParentID], b.ID)
}
out := []string{id}
for i := 0; i < len(out); i++ {
out = append(out, children[out[i]]...)
}
sort.Strings(out[1:])
return out
}
// liveAncestor walks up from id until it finds a live block; "" means top level.
func (d *Doc) liveAncestor(id string) string {
seen := map[string]bool{}
for id != "" && !seen[id] {
seen[id] = true
b, ok := d.Blocks[id]
if !ok {
return ""
}
if !b.Deleted {
return id
}
id = b.ParentID
}
return ""
}
// isAncestor reports whether ancestorID is ancestorID itself or above id in the tree.
func (d *Doc) isAncestor(ancestorID, id string) bool {
seen := map[string]bool{}
for id != "" && !seen[id] {
if id == ancestorID {
return true
}
seen[id] = true
b, ok := d.Blocks[id]
if !ok {
return false
}
id = b.ParentID
}
return false
}
// keyAfter returns a key directly after block b among its live siblings.
func (d *Doc) keyAfter(b *Block) string {
next := ""
for _, s := range d.Children(b.ParentID) {
if siblingLess(b, s) {
next = s.OrderKey
break
}
}
if next == b.OrderKey {
// Equal keys are only ordered by ID; append after the whole run instead
next = ""
for _, s := range d.Children(b.ParentID) {
if s.OrderKey > b.OrderKey {
next = s.OrderKey
break
}
}
}
k, err := KeyBetween(b.OrderKey, next)
if err != nil {
k, _ = KeyBetween(b.OrderKey, "")
}
return k
}
// Validate checks the tree invariants: parents exist, live blocks have live parents, no cycles, valid keys.
func (d *Doc) Validate() error {
for id, b := range d.Blocks {
if b.ID != id {
return fmt.Errorf("block %s stored under key %s", b.ID, id)
}
if !ValidOrderKey(b.OrderKey) {
return fmt.Errorf("block %s has invalid order key %q", id, b.OrderKey)
}
if b.ParentID == "" {
continue
}
p, ok := d.Blocks[b.ParentID]
if !ok {
return fmt.Errorf("block %s has missing parent %s", id, b.ParentID)
}
if !b.Deleted && p.Deleted {
return fmt.Errorf("live block %s has deleted parent %s", id, p.ID)
}
if d.isAncestor(id, b.ParentID) {
return fmt.Errorf("block %s is part of a cycle", id)
}
}
return nil
}
+217
View File
@@ -0,0 +1,217 @@
package collab
import (
"sort"
"sync"
"time"
)
const (
LeaseTTL = 20 * time.Second
SessionTimeout = 30 * time.Second
)
type lease struct {
clientID string
userID string
name string
expires time.Time
}
type session struct {
userID string
name string
lastSeen time.Time
}
// LeaseInfo describes a block held by another client, for display ("Anna bearbeitet …").
type LeaseInfo struct {
BlockID string `json:"blockId"`
UserID string `json:"userId"`
Name string `json:"name"`
}
// Presence is one client that synced a document recently.
type Presence struct {
UserID string `json:"userId"`
Name string `json:"name"`
}
// LeaseManager keeps block leases and active sessions in memory. Leases are short-lived; after a server
// restart clients simply request them again with their next sync.
type LeaseManager struct {
mu sync.Mutex
now func() time.Time
leases map[string]map[string]lease // docID -> blockID -> lease
sessions map[string]map[string]session // docID -> clientID -> session
}
func NewLeaseManager(now func() time.Time) *LeaseManager {
if now == nil {
now = time.Now
}
return &LeaseManager{now: now, leases: map[string]map[string]lease{}, sessions: map[string]map[string]session{}}
}
func (m *LeaseManager) docLeases(docID string) map[string]lease {
l := m.leases[docID]
if l == nil {
l = map[string]lease{}
m.leases[docID] = l
}
return l
}
func (m *LeaseManager) prune(docID string, now time.Time) {
for id, l := range m.leases[docID] {
if !now.Before(l.expires) {
delete(m.leases[docID], id)
}
}
for cid, s := range m.sessions[docID] {
if now.Sub(s.lastSeen) > SessionTimeout {
delete(m.sessions[docID], cid)
}
}
}
// Hold acquires or renews leases for a client. A block held by another live client is denied.
func (m *LeaseManager) Hold(docID string, a Actor, blockIDs []string) (granted, denied []string) {
m.mu.Lock()
defer m.mu.Unlock()
now := m.now()
m.prune(docID, now)
l := m.docLeases(docID)
for _, id := range blockIDs {
if cur, ok := l[id]; ok && cur.clientID != a.ClientID {
denied = append(denied, id)
continue
}
l[id] = lease{clientID: a.ClientID, userID: a.UserID, name: a.Name, expires: now.Add(LeaseTTL)}
granted = append(granted, id)
}
return granted, denied
}
// Release gives up leases held by this client.
func (m *LeaseManager) Release(docID, clientID string, blockIDs []string) {
m.mu.Lock()
defer m.mu.Unlock()
l := m.leases[docID]
for _, id := range blockIDs {
if cur, ok := l[id]; ok && cur.clientID == clientID {
delete(l, id)
}
}
}
// ReleaseAllExcept drops every lease of the client in this document that is not in keep.
func (m *LeaseManager) ReleaseAllExcept(docID, clientID string, keep []string) {
m.mu.Lock()
defer m.mu.Unlock()
keepSet := make(map[string]bool, len(keep))
for _, id := range keep {
keepSet[id] = true
}
for id, cur := range m.leases[docID] {
if cur.clientID == clientID && !keepSet[id] {
delete(m.leases[docID], id)
}
}
}
// View returns a snapshot of the current leases of a document for operation validation.
func (m *LeaseManager) View(docID string) LeaseView {
m.mu.Lock()
defer m.mu.Unlock()
now := m.now()
m.prune(docID, now)
snap := make(leaseSnapshot, len(m.leases[docID]))
for id, l := range m.leases[docID] {
snap[id] = l.clientID
}
return snap
}
// Others lists blocks held by clients other than clientID.
func (m *LeaseManager) Others(docID, clientID string) []LeaseInfo {
m.mu.Lock()
defer m.mu.Unlock()
m.prune(docID, m.now())
out := make([]LeaseInfo, 0)
for id, l := range m.leases[docID] {
if l.clientID != clientID {
out = append(out, LeaseInfo{BlockID: id, UserID: l.userID, Name: l.name})
}
}
sort.Slice(out, func(i, j int) bool { return out[i].BlockID < out[j].BlockID })
return out
}
// Touch records that a client synced the document; it counts as an open session for SessionTimeout.
func (m *LeaseManager) Touch(docID string, a Actor) {
m.mu.Lock()
defer m.mu.Unlock()
s := m.sessions[docID]
if s == nil {
s = map[string]session{}
m.sessions[docID] = s
}
s[a.ClientID] = session{userID: a.UserID, name: a.Name, lastSeen: m.now()}
}
// Leave removes a client's session and all its leases (tab closed, app sent to background).
func (m *LeaseManager) Leave(docID, clientID string) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.sessions[docID], clientID)
for id, l := range m.leases[docID] {
if l.clientID == clientID {
delete(m.leases[docID], id)
}
}
}
// Sessions returns how many clients have the document open and who they are.
func (m *LeaseManager) Sessions(docID string) (int, []Presence) {
m.mu.Lock()
defer m.mu.Unlock()
m.prune(docID, m.now())
seen := map[string]bool{}
people := make([]Presence, 0)
for _, s := range m.sessions[docID] {
if !seen[s.userID] {
seen[s.userID] = true
people = append(people, Presence{UserID: s.userID, Name: s.name})
}
}
sort.Slice(people, func(i, j int) bool { return people[i].UserID < people[j].UserID })
return len(m.sessions[docID]), people
}
type leaseSnapshot map[string]string
func (s leaseSnapshot) HolderOf(blockID string) (string, bool) {
c, ok := s[blockID]
return c, ok
}
// Sweep drops expired leases and sessions of all documents and forgets documents nobody uses any more,
// so the maps do not grow with every note ever opened.
func (m *LeaseManager) Sweep() {
m.mu.Lock()
defer m.mu.Unlock()
now := m.now()
for docID := range m.leases {
m.prune(docID, now)
if len(m.leases[docID]) == 0 {
delete(m.leases, docID)
}
}
for docID := range m.sessions {
m.prune(docID, now)
if len(m.sessions[docID]) == 0 {
delete(m.sessions, docID)
}
}
}
+34
View File
@@ -0,0 +1,34 @@
package collab
import "sync"
// Notifier wakes up long-poll requests waiting for changes of a document.
type Notifier struct {
mu sync.Mutex
wait map[string]chan struct{}
}
func NewNotifier() *Notifier {
return &Notifier{wait: map[string]chan struct{}{}}
}
// Changed returns a channel that is closed on the next Notify for docID.
func (n *Notifier) Changed(docID string) <-chan struct{} {
n.mu.Lock()
defer n.mu.Unlock()
ch, ok := n.wait[docID]
if !ok {
ch = make(chan struct{})
n.wait[docID] = ch
}
return ch
}
func (n *Notifier) Notify(docID string) {
n.mu.Lock()
defer n.mu.Unlock()
if ch, ok := n.wait[docID]; ok {
close(ch)
delete(n.wait, docID)
}
}
+463
View File
@@ -0,0 +1,463 @@
package collab
import (
"crypto/sha256"
"encoding/hex"
"sort"
)
const (
OpInsert = "insert"
OpSetContent = "set_content"
OpSetProps = "set_props"
OpMove = "move"
OpDelete = "delete"
OpRestore = "restore"
OpSetTitle = "set_title"
// OpPurge is only produced by log compaction: deleted blocks older than the retention are removed for good
OpPurge = "purge"
StatusApplied = "applied"
StatusConflict = "conflict"
StatusRejected = "rejected"
maxContentLength = 100_000
maxTypeLength = 20
)
// PropsPatch changes selected block settings; nil fields stay as they are.
type PropsPatch struct {
Type *string `json:"type,omitempty"`
Checked *bool `json:"checked,omitempty"`
Collapsed *bool `json:"collapsed,omitempty"`
Lang *string `json:"lang,omitempty"`
// ClearConflict removes the conflict marking (after the user accepted or kept a conflict copy)
ClearConflict *bool `json:"clearConflict,omitempty"`
}
// Op is a change requested by a client. OpID is chosen by the client and makes retries idempotent.
type Op struct {
OpID string `json:"opId"`
Kind string `json:"kind"`
BlockID string `json:"blockId,omitempty"`
BaseVersion int `json:"baseVersion,omitempty"` // set_content, set_title
ParentID *string `json:"parentId,omitempty"` // insert, move ("" = top level)
OrderKey string `json:"orderKey,omitempty"` // insert, move
Type string `json:"type,omitempty"` // insert
Content *string `json:"content,omitempty"` // insert, set_content
Props *Props `json:"props,omitempty"` // insert
Patch *PropsPatch `json:"patch,omitempty"` // set_props
IDs []string `json:"ids,omitempty"` // restore
Title *string `json:"title,omitempty"` // set_title
}
// Effect is a normalized, already validated change. The server log stores effects; replaying them in
// order on an empty Doc reproduces the server state exactly.
type Effect struct {
Kind string `json:"kind"`
BlockID string `json:"blockId,omitempty"`
ParentID string `json:"parentId,omitempty"`
OrderKey string `json:"orderKey,omitempty"`
Type string `json:"type,omitempty"`
Content string `json:"content,omitempty"`
Props *Props `json:"props,omitempty"`
Version int `json:"version,omitempty"`
IDs []string `json:"ids,omitempty"`
Title string `json:"title,omitempty"`
TitleVersion int `json:"titleVersion,omitempty"`
By string `json:"by,omitempty"`
}
type Result struct {
OpID string `json:"opId"`
Status string `json:"status"`
Reason string `json:"reason,omitempty"`
Version int `json:"version,omitempty"`
ConflictBlockID string `json:"conflictBlockId,omitempty"`
Seq int64 `json:"seq,omitempty"`
}
// Actor identifies who sends an operation. ClientID is one browser tab / sync engine instance.
type Actor struct {
UserID string
ClientID string
Name string
}
// LeaseView answers which client currently holds the lease on a block.
type LeaseView interface {
HolderOf(blockID string) (clientID string, held bool)
}
// Env provides outside knowledge needed to validate operations.
type Env struct {
Leases LeaseView // nil = no leases (e.g. server-internal writes)
IDTaken func(id string) bool // true if the block ID exists in another document
}
// ApplyEffect mutates the document with one effect. It never fails; effects are validated by Apply.
func (d *Doc) ApplyEffect(e Effect) {
switch e.Kind {
case OpInsert:
b := &Block{ID: e.BlockID, ParentID: e.ParentID, OrderKey: e.OrderKey, Type: e.Type, Content: e.Content, Version: e.Version, UpdatedBy: e.By}
if e.Props != nil {
b.Props = *e.Props
}
d.Blocks[e.BlockID] = b
case OpSetContent:
if b, ok := d.Blocks[e.BlockID]; ok {
b.Content, b.Version, b.UpdatedBy = e.Content, e.Version, e.By
}
case OpSetProps:
if b, ok := d.Blocks[e.BlockID]; ok {
b.Type = e.Type
if e.Props != nil {
b.Props = *e.Props
}
}
case OpMove:
if b, ok := d.Blocks[e.BlockID]; ok {
b.ParentID, b.OrderKey = e.ParentID, e.OrderKey
}
case OpDelete:
for _, id := range e.IDs {
if b, ok := d.Blocks[id]; ok {
b.Deleted = true
}
}
case OpRestore:
for _, id := range e.IDs {
if b, ok := d.Blocks[id]; ok {
b.Deleted = false
}
}
case OpSetTitle:
d.Title, d.TitleVersion = e.Title, e.TitleVersion
case OpPurge:
for _, id := range e.IDs {
delete(d.Blocks, id)
}
}
}
// ConflictBlockID derives the ID of a conflict copy from the losing operation, so retries never duplicate it.
func ConflictBlockID(opID string) string {
sum := sha256.Sum256([]byte("conflict:" + opID))
return "blk_cf_" + hex.EncodeToString(sum[:8])
}
type applier struct {
d *Doc
a Actor
env Env
effects []Effect
}
func (ap *applier) emit(e Effect) {
e.By = ap.a.UserID
ap.d.ApplyEffect(e)
ap.effects = append(ap.effects, e)
}
func (ap *applier) foreignLease(blockID string) bool {
if ap.env.Leases == nil {
return false
}
holder, held := ap.env.Leases.HolderOf(blockID)
return held && holder != ap.a.ClientID
}
func rejected(op Op, reason string) Result {
return Result{OpID: op.OpID, Status: StatusRejected, Reason: reason}
}
// Apply validates op against d, mutates d and returns the produced effects. Rejected operations
// produce no effects; conflicting text is kept as a separate conflict block.
func Apply(d *Doc, op Op, a Actor, env Env) ([]Effect, Result) {
ap := &applier{d: d, a: a, env: env}
res := ap.apply(op)
res.OpID = op.OpID
return ap.effects, res
}
func (ap *applier) apply(op Op) Result {
d := ap.d
switch op.Kind {
case OpInsert:
if op.BlockID == "" || len(op.BlockID) > 64 {
return rejected(op, "invalid_id")
}
if _, exists := d.Blocks[op.BlockID]; exists || (ap.env.IDTaken != nil && ap.env.IDTaken(op.BlockID)) {
return rejected(op, "exists")
}
parent := ""
if op.ParentID != nil {
parent = *op.ParentID
}
if parent != "" {
if _, ok := d.Blocks[parent]; ok {
parent = d.liveAncestor(parent)
} else {
parent = "" // parent was purged after the retention period: keep the block at the top level
}
}
if !ValidOrderKey(op.OrderKey) {
return rejected(op, "invalid_order_key")
}
typ := op.Type
if typ == "" {
typ = "bullet"
}
if len(typ) > maxTypeLength {
return rejected(op, "invalid_type")
}
content := ""
if op.Content != nil {
content = *op.Content
}
if len(content) > maxContentLength {
return rejected(op, "too_long")
}
props := Props{}
if op.Props != nil {
props = *op.Props
}
ap.emit(Effect{Kind: OpInsert, BlockID: op.BlockID, ParentID: parent, OrderKey: op.OrderKey, Type: typ, Content: content, Props: &props, Version: 1})
return Result{Status: StatusApplied, Version: 1}
case OpSetContent:
b, ok := d.Blocks[op.BlockID]
if !ok {
return ap.reviveAsNewBlock(op)
}
if op.Content == nil {
return rejected(op, "no_content")
}
if len(*op.Content) > maxContentLength {
return rejected(op, "too_long")
}
if b.Deleted {
// A late edit on a deleted block brings it back instead of losing the text
ap.restore([]string{b.ID})
}
if *op.Content == b.Content {
return Result{Status: StatusApplied, Version: b.Version}
}
if ap.foreignLease(b.ID) || op.BaseVersion != b.Version {
return ap.conflictCopy(op, b, *op.Content)
}
ap.emit(Effect{Kind: OpSetContent, BlockID: b.ID, Content: *op.Content, Version: b.Version + 1})
return Result{Status: StatusApplied, Version: b.Version}
case OpSetProps:
b, ok := d.Blocks[op.BlockID]
if !ok || b.Deleted {
return rejected(op, "missing")
}
if op.Patch == nil {
return rejected(op, "no_patch")
}
typ, props := b.Type, b.Props
if p := op.Patch; p != nil {
if p.Type != nil && *p.Type != "" && len(*p.Type) <= maxTypeLength {
typ = *p.Type
}
if p.Checked != nil {
props.Checked = *p.Checked
}
if p.Collapsed != nil {
props.Collapsed = *p.Collapsed
}
if p.Lang != nil {
props.Lang = *p.Lang
}
if p.ClearConflict != nil && *p.ClearConflict {
props.ConflictOf, props.ConflictAuthor = "", ""
}
}
if typ != b.Type || props != b.Props {
ap.emit(Effect{Kind: OpSetProps, BlockID: b.ID, Type: typ, Props: &props})
}
return Result{Status: StatusApplied}
case OpMove:
b, ok := d.Blocks[op.BlockID]
if !ok || b.Deleted {
return rejected(op, "missing")
}
if ap.foreignLease(b.ID) {
return rejected(op, "locked")
}
parent := ""
if op.ParentID != nil {
parent = *op.ParentID
}
if parent != "" {
p, ok := d.Blocks[parent]
if !ok || p.Deleted {
return rejected(op, "parent_missing")
}
if d.isAncestor(b.ID, parent) {
return rejected(op, "cycle")
}
}
if !ValidOrderKey(op.OrderKey) {
return rejected(op, "invalid_order_key")
}
if parent != b.ParentID || op.OrderKey != b.OrderKey {
ap.emit(Effect{Kind: OpMove, BlockID: b.ID, ParentID: parent, OrderKey: op.OrderKey})
}
return Result{Status: StatusApplied}
case OpDelete:
b, ok := d.Blocks[op.BlockID]
if !ok {
return rejected(op, "missing")
}
if b.Deleted {
return Result{Status: StatusApplied}
}
ids := d.subtree(b.ID, true)
for _, id := range ids {
if ap.foreignLease(id) {
return rejected(op, "locked")
}
}
ap.emit(Effect{Kind: OpDelete, IDs: ids})
return Result{Status: StatusApplied}
case OpRestore:
ids := make([]string, 0, len(op.IDs))
for _, id := range op.IDs {
if b, ok := d.Blocks[id]; ok && b.Deleted {
ids = append(ids, id)
}
}
if len(ids) > 0 {
ap.restore(ids)
}
return Result{Status: StatusApplied}
case OpSetTitle:
if op.Title == nil {
return rejected(op, "no_title")
}
if len(*op.Title) > 1000 {
return rejected(op, "too_long")
}
if *op.Title == d.Title {
return Result{Status: StatusApplied, Version: d.TitleVersion}
}
if op.BaseVersion != d.TitleVersion {
return ap.titleConflict(op, *op.Title)
}
ap.emit(Effect{Kind: OpSetTitle, Title: *op.Title, TitleVersion: d.TitleVersion + 1})
return Result{Status: StatusApplied, Version: d.TitleVersion}
}
return rejected(op, "unknown_kind")
}
// restore brings back deleted blocks; blocks whose parent stays deleted move to the nearest live ancestor.
func (ap *applier) restore(ids []string) {
d := ap.d
set := make(map[string]bool, len(ids))
for _, id := range ids {
set[id] = true
}
sorted := append([]string(nil), ids...)
sort.Strings(sorted)
ap.emit(Effect{Kind: OpRestore, IDs: sorted})
for _, id := range sorted {
b := d.Blocks[id]
if b.ParentID == "" || set[b.ParentID] {
continue
}
if p, ok := d.Blocks[b.ParentID]; ok && !p.Deleted {
continue
}
ap.emit(Effect{Kind: OpMove, BlockID: b.ID, ParentID: d.liveAncestor(b.ParentID), OrderKey: b.OrderKey})
}
}
// reviveAsNewBlock handles text for a block that no longer exists (its tombstone was purged after the
// retention period, e.g. a device was offline for weeks): the text becomes a new block at the end.
func (ap *applier) reviveAsNewBlock(op Op) Result {
if op.Content == nil || *op.Content == "" || op.BlockID == "" || len(op.BlockID) > 64 {
return rejected(op, "missing")
}
if ap.env.IDTaken != nil && ap.env.IDTaken(op.BlockID) {
return rejected(op, "missing")
}
if len(*op.Content) > maxContentLength {
return rejected(op, "too_long")
}
last := ""
if roots := ap.d.Children(""); len(roots) > 0 {
last = roots[len(roots)-1].OrderKey
}
key, err := KeyBetween(last, "")
if err != nil {
return rejected(op, "missing")
}
ap.emit(Effect{Kind: OpInsert, BlockID: op.BlockID, OrderKey: key, Type: "bullet", Content: *op.Content, Props: &Props{}, Version: 1})
return Result{Status: StatusApplied, Version: 1, Reason: "revived"}
}
// Purge removes tombstones for good. Only blocks whose whole deleted subtree is purged too may go, so no
// remaining block refers to a missing parent. Returns the effect (nil if nothing to purge).
func Purge(d *Doc, candidates []string) *Effect {
set := make(map[string]bool, len(candidates))
for _, id := range candidates {
if b, ok := d.Blocks[id]; ok && b.Deleted {
set[id] = true
}
}
for changed := true; changed; {
changed = false
for _, b := range d.Blocks {
if !set[b.ID] && set[b.ParentID] {
delete(set, b.ParentID) // a remaining block still points to it
changed = true
}
}
}
if len(set) == 0 {
return nil
}
ids := make([]string, 0, len(set))
for id := range set {
ids = append(ids, id)
}
sort.Strings(ids)
e := Effect{Kind: OpPurge, IDs: ids}
d.ApplyEffect(e)
return &e
}
// conflictCopy keeps the losing text as a new block directly below the contested one.
func (ap *applier) conflictCopy(op Op, b *Block, content string) Result {
id := ConflictBlockID(op.OpID)
if _, exists := ap.d.Blocks[id]; !exists {
props := Props{ConflictOf: b.ID, ConflictAuthor: ap.a.Name}
ap.emit(Effect{Kind: OpInsert, BlockID: id, ParentID: b.ParentID, OrderKey: ap.d.keyAfter(b), Type: b.Type, Content: content, Props: &props, Version: 1})
}
return Result{Status: StatusConflict, Version: b.Version, ConflictBlockID: id}
}
// titleConflict keeps a losing title as a conflict block at the very top of the note.
func (ap *applier) titleConflict(op Op, title string) Result {
id := ConflictBlockID(op.OpID)
if _, exists := ap.d.Blocks[id]; !exists {
first := ""
if roots := ap.d.Children(""); len(roots) > 0 {
first = roots[0].OrderKey
}
key, err := KeyBetween("", first)
if err != nil {
key, _ = KeyBetween("", "")
}
props := Props{ConflictOf: "title", ConflictAuthor: ap.a.Name}
ap.emit(Effect{Kind: OpInsert, BlockID: id, OrderKey: key, Type: "text", Content: title, Props: &props, Version: 1})
}
return Result{Status: StatusConflict, Version: ap.d.TitleVersion, ConflictBlockID: id}
}
+304
View File
@@ -0,0 +1,304 @@
package collab
import (
"testing"
"time"
)
var (
anna = Actor{UserID: "u_anna", ClientID: "c_anna", Name: "Anna"}
ben = Actor{UserID: "u_ben", ClientID: "c_ben", Name: "Ben"}
)
func str(s string) *string { return &s }
type fixedLeases map[string]string
func (f fixedLeases) HolderOf(id string) (string, bool) { c, ok := f[id]; return c, ok }
// docWith builds a doc via real operations so versions and keys are realistic.
func docWith(t *testing.T, blocks ...[3]string) *Doc { // {id, parent, content}
t.Helper()
d := NewDoc()
keys := map[string]string{}
for i, b := range blocks {
k, _ := KeyBetween(keys[b[1]], "")
keys[b[1]] = k
_, res := Apply(d, Op{OpID: "seed" + string(rune('a'+i)), Kind: OpInsert, BlockID: b[0], ParentID: str(b[1]), OrderKey: k, Content: str(b[2])}, anna, Env{})
if res.Status != StatusApplied {
t.Fatalf("seed %v: %+v", b, res)
}
}
return d
}
func mustValid(t *testing.T, d *Doc) {
t.Helper()
if err := d.Validate(); err != nil {
t.Fatal(err)
}
}
func TestSetContentAppliesOnMatchingVersion(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Hallo"})
_, res := Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Hallo Welt")}, anna, Env{})
if res.Status != StatusApplied || d.Blocks["a"].Content != "Hallo Welt" || d.Blocks["a"].Version != 2 {
t.Fatalf("got %+v, block %+v", res, d.Blocks["a"])
}
}
func TestStaleVersionCreatesConflictCopy(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Start"}, [3]string{"b", "", "Danach"})
Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Von Anna")}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Von Ben")}, ben, Env{})
if res.Status != StatusConflict || res.ConflictBlockID == "" {
t.Fatalf("want conflict, got %+v", res)
}
if d.Blocks["a"].Content != "Von Anna" {
t.Fatal("first writer must keep the block")
}
flat := d.Flatten()
if len(flat) != 3 || flat[1].ID != res.ConflictBlockID || flat[1].Content != "Von Ben" || flat[2].ID != "b" {
t.Fatalf("conflict copy must sit directly below the block: %+v", flat)
}
if c := d.Blocks[res.ConflictBlockID].Props; c.ConflictOf != "a" || c.ConflictAuthor != "Ben" {
t.Fatalf("conflict props wrong: %+v", c)
}
mustValid(t, d)
}
func TestForeignLeaseCreatesConflictCopy(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Start"})
_, res := Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Ben tippt")}, ben, Env{Leases: fixedLeases{"a": "c_anna"}})
if res.Status != StatusConflict || d.Blocks["a"].Content != "Start" {
t.Fatalf("lease holder must win: %+v", res)
}
// The lease holder itself may write
_, res = Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Anna tippt")}, anna, Env{Leases: fixedLeases{"a": "c_anna"}})
if res.Status != StatusApplied {
t.Fatalf("lease holder rejected: %+v", res)
}
}
func TestIdenticalContentIsNoConflict(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Gleich"})
Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Neu")}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("Neu")}, ben, Env{})
if res.Status != StatusApplied || len(d.Blocks) != 1 {
t.Fatalf("same text must not produce a conflict copy: %+v", res)
}
}
func TestEditOnDeletedBlockRestoresIt(t *testing.T) {
d := docWith(t, [3]string{"p", "", "Eltern"}, [3]string{"c", "p", "Kind"})
Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "p"}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "c", BaseVersion: 1, Content: str("Spät getippt")}, ben, Env{})
if res.Status != StatusApplied {
t.Fatalf("got %+v", res)
}
c := d.Blocks["c"]
if c.Deleted || c.Content != "Spät getippt" || c.ParentID != "" {
t.Fatalf("block must be restored at the nearest live ancestor: %+v", c)
}
mustValid(t, d)
}
func TestDeleteIsBlockedByForeignLeaseInSubtree(t *testing.T) {
d := docWith(t, [3]string{"p", "", "Ast"}, [3]string{"c", "p", "Unterpunkt"})
_, res := Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "p"}, anna, Env{Leases: fixedLeases{"c": "c_ben"}})
if res.Status != StatusRejected || res.Reason != "locked" || d.Blocks["p"].Deleted {
t.Fatalf("got %+v", res)
}
_, res = Apply(d, Op{OpID: "2", Kind: OpDelete, BlockID: "p"}, anna, Env{})
if res.Status != StatusApplied || !d.Blocks["c"].Deleted {
t.Fatal("delete must tombstone the whole subtree")
}
}
func TestMoveRejectsCyclesAndLockedBlocks(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"})
if _, res := Apply(d, Op{OpID: "1", Kind: OpMove, BlockID: "a", ParentID: str("b"), OrderKey: "a0"}, anna, Env{}); res.Reason != "cycle" {
t.Fatalf("want cycle, got %+v", res)
}
if _, res := Apply(d, Op{OpID: "2", Kind: OpMove, BlockID: "b", ParentID: str(""), OrderKey: "a5"}, anna, Env{Leases: fixedLeases{"b": "c_ben"}}); res.Reason != "locked" {
t.Fatalf("want locked, got %+v", res)
}
if _, res := Apply(d, Op{OpID: "3", Kind: OpMove, BlockID: "b", ParentID: str(""), OrderKey: "a5"}, anna, Env{}); res.Status != StatusApplied {
t.Fatalf("got %+v", res)
}
mustValid(t, d)
}
func TestInsertIntoDeletedParentGoesToLiveAncestor(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"})
Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "b"}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpInsert, BlockID: "n", ParentID: str("b"), OrderKey: "a0", Content: str("neu")}, ben, Env{})
if res.Status != StatusApplied || d.Blocks["n"].ParentID != "a" {
t.Fatalf("got %+v parent=%q", res, d.Blocks["n"].ParentID)
}
}
func TestInsertRejectsTakenIDs(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"})
if _, res := Apply(d, Op{OpID: "1", Kind: OpInsert, BlockID: "a", OrderKey: "a5"}, anna, Env{}); res.Reason != "exists" {
t.Fatalf("got %+v", res)
}
env := Env{IDTaken: func(id string) bool { return id == "other_doc_block" }}
if _, res := Apply(d, Op{OpID: "2", Kind: OpInsert, BlockID: "other_doc_block", OrderKey: "a5"}, anna, env); res.Reason != "exists" {
t.Fatalf("block IDs of other documents must be rejected: %+v", res)
}
}
func TestTitleConflictBecomesTopBlock(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Erster"})
Apply(d, Op{OpID: "1", Kind: OpSetTitle, BaseVersion: 0, Title: str("Titel Anna")}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetTitle, BaseVersion: 0, Title: str("Titel Ben")}, ben, Env{})
if res.Status != StatusConflict || d.Title != "Titel Anna" {
t.Fatalf("got %+v title=%q", res, d.Title)
}
if flat := d.Flatten(); flat[0].ID != res.ConflictBlockID || flat[0].Props.ConflictOf != "title" {
t.Fatalf("title conflict must be the first block: %+v", flat[0])
}
}
func TestSetPropsDoesNotBumpContentVersion(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Todo"})
Apply(d, Op{OpID: "1", Kind: OpSetProps, BlockID: "a", Patch: &PropsPatch{Checked: new(bool)}}, anna, Env{})
yes := true
Apply(d, Op{OpID: "2", Kind: OpSetProps, BlockID: "a", Patch: &PropsPatch{Checked: &yes}}, ben, Env{Leases: fixedLeases{"a": "c_anna"}})
if !d.Blocks["a"].Props.Checked || d.Blocks["a"].Version != 1 {
t.Fatalf("props must apply independently of text and leases: %+v", d.Blocks["a"])
}
}
// Replaying the effects of every operation on an empty doc must give exactly the same state.
func TestEffectsReplayToSameState(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "a", "B"})
replay := d.Clone()
ops := []struct {
op Op
a Actor
}{
{Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("A2")}, anna},
{Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("A3")}, ben},
{Op{OpID: "3", Kind: OpDelete, BlockID: "a"}, anna},
{Op{OpID: "4", Kind: OpSetContent, BlockID: "b", BaseVersion: 1, Content: str("B2")}, ben},
{Op{OpID: "5", Kind: OpSetTitle, Title: str("T")}, anna},
}
for _, o := range ops {
effects, _ := Apply(d, o.op, o.a, Env{})
for _, e := range effects {
replay.ApplyEffect(e)
}
}
if !docsEqual(d, replay) {
t.Fatal("replayed effects differ from server state")
}
}
func docsEqual(a, b *Doc) bool {
if a.Title != b.Title || a.TitleVersion != b.TitleVersion || len(a.Blocks) != len(b.Blocks) {
return false
}
for id, x := range a.Blocks {
y, ok := b.Blocks[id]
if !ok || *x != *y {
return false
}
}
return true
}
func TestLeaseManager(t *testing.T) {
now := time.Unix(1000, 0)
m := NewLeaseManager(func() time.Time { return now })
g, den := m.Hold("d", anna, []string{"a", "b"})
if len(g) != 2 || len(den) != 0 {
t.Fatalf("anna: granted=%v denied=%v", g, den)
}
if g, den = m.Hold("d", ben, []string{"a", "c"}); len(g) != 1 || den[0] != "a" {
t.Fatalf("ben: granted=%v denied=%v", g, den)
}
if others := m.Others("d", "c_ben"); len(others) != 2 || others[0].Name != "Anna" {
t.Fatalf("others: %+v", others)
}
now = now.Add(LeaseTTL + time.Second)
if g, _ = m.Hold("d", ben, []string{"a"}); len(g) != 1 {
t.Fatal("expired lease must be available")
}
m.ReleaseAllExcept("d", "c_ben", []string{"a"})
if _, held := m.View("d").HolderOf("c"); held {
t.Fatal("lease not in keep set must be released")
}
m.Touch("d", anna)
m.Touch("d", ben)
if n, people := m.Sessions("d"); n != 2 || len(people) != 2 {
t.Fatalf("sessions: %d %+v", n, people)
}
m.Leave("d", "c_ben")
if _, held := m.View("d").HolderOf("a"); held {
t.Fatal("leaving must drop the client's leases")
}
}
func TestNotifier(t *testing.T) {
n := NewNotifier()
ch := n.Changed("d")
select {
case <-ch:
t.Fatal("must not fire before Notify")
default:
}
n.Notify("d")
select {
case <-ch:
case <-time.After(time.Second):
t.Fatal("Notify must wake waiters")
}
}
func TestClearConflictMarking(t *testing.T) {
d := docWith(t, [3]string{"a", "", "Start"})
Apply(d, Op{OpID: "1", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("A")}, anna, Env{})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "a", BaseVersion: 1, Content: str("B")}, ben, Env{})
yes := true
Apply(d, Op{OpID: "3", Kind: OpSetProps, BlockID: res.ConflictBlockID, Patch: &PropsPatch{ClearConflict: &yes}}, ben, Env{})
if p := d.Blocks[res.ConflictBlockID].Props; p.ConflictOf != "" || p.ConflictAuthor != "" {
t.Fatalf("conflict marking not cleared: %+v", p)
}
}
func TestPurgeKeepsTreeValid(t *testing.T) {
d := docWith(t, [3]string{"p", "", "Eltern"}, [3]string{"c", "p", "Kind"}, [3]string{"x", "", "Anderer"})
Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "c"}, anna, Env{})
Apply(d, Op{OpID: "2", Kind: OpDelete, BlockID: "p"}, anna, Env{})
// Only "p" is old enough: it must stay because its deleted child "c" still points to it
if e := Purge(d, []string{"p"}); e != nil {
t.Fatalf("parent of a remaining tombstone must not be purged: %+v", e)
}
e := Purge(d, []string{"p", "c"})
if e == nil || len(e.IDs) != 2 || len(d.Blocks) != 1 {
t.Fatalf("both tombstones must go: %+v, blocks %d", e, len(d.Blocks))
}
mustValid(t, d)
}
func TestTextForPurgedBlockBecomesNewBlock(t *testing.T) {
d := docWith(t, [3]string{"a", "", "A"}, [3]string{"b", "", "B"})
Apply(d, Op{OpID: "1", Kind: OpDelete, BlockID: "b"}, anna, Env{})
Purge(d, []string{"b"})
_, res := Apply(d, Op{OpID: "2", Kind: OpSetContent, BlockID: "b", BaseVersion: 1, Content: str("Wochen offline getippt")}, ben, Env{})
if res.Status != StatusApplied {
t.Fatalf("late text must not be lost: %+v", res)
}
flat := d.Flatten()
if len(flat) != 2 || flat[1].Content != "Wochen offline getippt" {
t.Fatalf("text must be revived at the end: %+v", flat)
}
mustValid(t, d)
}
+267
View File
@@ -0,0 +1,267 @@
package collab
import (
"errors"
"strings"
)
// Order keys sort siblings lexicographically (ties broken by block ID) and allow inserting between any two
// keys without renumbering. Format as in the "fractional-indexing" scheme: an integer part whose first
// character encodes its length ('a'..'z' positive, 'A'..'Z' negative) followed by base-62 digits, then an
// optional fraction without trailing '0'. Appending stays short ("a0", "a1", …, "az", "b00", …).
const keyDigits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
const smallestInteger = "A00000000000000000000000000"
var errInvalidOrderKey = errors.New("ungültiger Sortierschlüssel")
func digitIndex(c byte) int {
return strings.IndexByte(keyDigits, c)
}
func integerLength(head byte) int {
switch {
case head >= 'a' && head <= 'z':
return int(head-'a') + 2
case head >= 'A' && head <= 'Z':
return int('Z'-head) + 2
}
return 0
}
func integerPart(key string) (string, bool) {
if key == "" {
return "", false
}
n := integerLength(key[0])
if n == 0 || n > len(key) {
return "", false
}
return key[:n], true
}
// ValidOrderKey reports whether k is a well-formed key.
func ValidOrderKey(k string) bool {
if k == "" || len(k) > 256 || k == smallestInteger {
return false
}
ip, ok := integerPart(k)
if !ok {
return false
}
for i := 1; i < len(k); i++ {
if digitIndex(k[i]) < 0 {
return false
}
}
frac := k[len(ip):]
return frac == "" || frac[len(frac)-1] != '0'
}
func incrementInteger(x string) (string, bool) {
head, digs := x[0], []byte(x[1:])
carry := true
for i := len(digs) - 1; carry && i >= 0; i-- {
d := digitIndex(digs[i]) + 1
if d == len(keyDigits) {
digs[i] = '0'
} else {
digs[i] = keyDigits[d]
carry = false
}
}
if !carry {
return string(head) + string(digs), true
}
if head == 'Z' {
return "a0", true
}
if head == 'z' {
return "", false
}
h := head + 1
if h > 'a' {
digs = append(digs, '0')
} else {
digs = digs[:len(digs)-1]
}
return string(h) + string(digs), true
}
func decrementInteger(x string) (string, bool) {
head, digs := x[0], []byte(x[1:])
borrow := true
for i := len(digs) - 1; borrow && i >= 0; i-- {
d := digitIndex(digs[i]) - 1
if d == -1 {
digs[i] = 'z'
} else {
digs[i] = keyDigits[d]
borrow = false
}
}
if !borrow {
return string(head) + string(digs), true
}
if head == 'a' {
return "Z" + string(keyDigits[len(keyDigits)-1]), true
}
if head == 'A' {
return "", false
}
h := head - 1
if h < 'Z' {
digs = append(digs, 'z')
} else {
digs = digs[:len(digs)-1]
}
return string(h) + string(digs), true
}
// midpoint returns a fraction between a and b (a may be "" = 0, b may be "" = 1).
func midpoint(a, b string) string {
if b != "" {
n := 0
for n < len(b) {
ca := byte('0')
if n < len(a) {
ca = a[n]
}
if ca != b[n] {
break
}
n++
}
if n > 0 {
rest := ""
if n < len(a) {
rest = a[n:]
}
return b[:n] + midpoint(rest, b[n:])
}
}
da := 0
if a != "" {
da = digitIndex(a[0])
}
db := len(keyDigits)
if b != "" {
db = digitIndex(b[0])
}
if db-da > 1 {
return string(keyDigits[(da+db)/2])
}
if b != "" && len(b) > 1 {
return b[:1]
}
rest := ""
if len(a) > 1 {
rest = a[1:]
}
return string(keyDigits[da]) + midpoint(rest, "")
}
// KeyBetween returns a key strictly between a and b. An empty a means "before everything",
// an empty b means "after everything".
func KeyBetween(a, b string) (string, error) {
if (a != "" && !ValidOrderKey(a)) || (b != "" && !ValidOrderKey(b)) {
return "", errInvalidOrderKey
}
if a != "" && b != "" && a >= b {
return "", errInvalidOrderKey
}
switch {
case a == "" && b == "":
return "a0", nil
case a == "":
ib, _ := integerPart(b)
fb := b[len(ib):]
if ib == smallestInteger {
return ib + midpoint("", fb), nil
}
if ib < b {
return ib, nil
}
res, ok := decrementInteger(ib)
if !ok {
return "", errInvalidOrderKey
}
return res, nil
case b == "":
ia, _ := integerPart(a)
fa := a[len(ia):]
if i, ok := incrementInteger(ia); ok {
return i, nil
}
return ia + midpoint(fa, ""), nil
}
ia, _ := integerPart(a)
fa := a[len(ia):]
ib, _ := integerPart(b)
fb := b[len(ib):]
if ia == ib {
return ia + midpoint(fa, fb), nil
}
i, ok := incrementInteger(ia)
if !ok {
return "", errInvalidOrderKey
}
if i < b {
return i, nil
}
return ia + midpoint(fa, ""), nil
}
// KeysBetween returns n increasing keys between a and b, spread so that bulk inserts stay short.
func KeysBetween(a, b string, n int) ([]string, error) {
if n <= 0 {
return nil, nil
}
if n == 1 {
k, err := KeyBetween(a, b)
if err != nil {
return nil, err
}
return []string{k}, nil
}
if b == "" {
keys := make([]string, 0, n)
prev := a
for i := 0; i < n; i++ {
k, err := KeyBetween(prev, "")
if err != nil {
return nil, err
}
keys = append(keys, k)
prev = k
}
return keys, nil
}
if a == "" {
keys := make([]string, n)
next := b
for i := n - 1; i >= 0; i-- {
k, err := KeyBetween("", next)
if err != nil {
return nil, err
}
keys[i] = k
next = k
}
return keys, nil
}
mid := n / 2
c, err := KeyBetween(a, b)
if err != nil {
return nil, err
}
left, err := KeysBetween(a, c, mid)
if err != nil {
return nil, err
}
right, err := KeysBetween(c, b, n-mid-1)
if err != nil {
return nil, err
}
return append(append(left, c), right...), nil
}
+90
View File
@@ -0,0 +1,90 @@
package collab
import (
"math/rand"
"sort"
"testing"
)
func TestKeyBetweenBasics(t *testing.T) {
cases := [][2]string{{"", ""}, {"", "a0"}, {"a0", ""}, {"a0", "a1"}, {"a0", "a0V"}, {"a1", "a2"}, {"a0V", "a1"}, {"Zz", "a0"}, {"az", ""}, {"", "Zz"}}
for _, c := range cases {
k, err := KeyBetween(c[0], c[1])
if err != nil {
t.Fatalf("KeyBetween(%q,%q): %v", c[0], c[1], err)
}
if !ValidOrderKey(k) {
t.Fatalf("KeyBetween(%q,%q) = %q is not a valid key", c[0], c[1], k)
}
if (c[0] != "" && k <= c[0]) || (c[1] != "" && k >= c[1]) {
t.Fatalf("KeyBetween(%q,%q) = %q is not between", c[0], c[1], k)
}
}
if _, err := KeyBetween("a1", "a0"); err == nil {
t.Fatal("reversed range must fail")
}
if _, err := KeyBetween("a0V0", ""); err == nil {
t.Fatal("trailing zero key must be rejected")
}
}
// Random inserts between neighbours must always produce a strictly increasing sequence.
func TestKeyBetweenRandomInserts(t *testing.T) {
r := rand.New(rand.NewSource(42))
keys := []string{}
for i := 0; i < 3000; i++ {
pos := r.Intn(len(keys) + 1)
a, b := "", ""
if pos > 0 {
a = keys[pos-1]
}
if pos < len(keys) {
b = keys[pos]
}
k, err := KeyBetween(a, b)
if err != nil {
t.Fatalf("step %d: %v", i, err)
}
keys = append(keys[:pos], append([]string{k}, keys[pos:]...)...)
}
if !sort.StringsAreSorted(keys) {
t.Fatal("keys are not sorted")
}
for i := 1; i < len(keys); i++ {
if keys[i] == keys[i-1] {
t.Fatalf("duplicate key %q", keys[i])
}
}
}
// Appending at the end, the most common case, must keep keys short.
func TestKeyBetweenAppendStaysShort(t *testing.T) {
last := ""
for i := 0; i < 1000; i++ {
k, _ := KeyBetween(last, "")
last = k
}
if len(last) > 4 {
t.Fatalf("key after 1000 appends too long: %d chars", len(last))
}
}
func TestKeysBetweenBulk(t *testing.T) {
for _, r := range [][2]string{{"", ""}, {"a0", ""}, {"", "a0"}, {"a0", "a1"}} {
keys, err := KeysBetween(r[0], r[1], 500)
if err != nil {
t.Fatalf("%v: %v", r, err)
}
if len(keys) != 500 || !sort.StringsAreSorted(keys) {
t.Fatalf("%v: wrong or unsorted result", r)
}
for i, k := range keys {
if !ValidOrderKey(k) || (i > 0 && keys[i-1] == k) {
t.Fatalf("%v: bad key %q", r, k)
}
if (r[0] != "" && k <= r[0]) || (r[1] != "" && k >= r[1]) {
t.Fatalf("%v: key %q out of range", r, k)
}
}
}
}
-19
View File
@@ -309,25 +309,6 @@ func CompileBlocksToMarkdown(blocks []DocumentBlock) string {
return sb.String() return sb.String()
} }
type DocBlocksPatchRequest struct {
Upserted []DocumentBlock `json:"upserted,omitempty"`
DeletedIDs []string `json:"deletedIds,omitempty"`
BlockOrder []string `json:"blockOrder,omitempty"`
Title *string `json:"title,omitempty"`
Icon *string `json:"icon,omitempty"`
}
type DocBlocksPatchEvent struct {
DocID string `json:"docId"`
Upserted []DocumentBlock `json:"upserted,omitempty"`
DeletedIDs []string `json:"deletedIds,omitempty"`
BlockOrder []string `json:"blockOrder,omitempty"`
Title *string `json:"title,omitempty"`
Icon *string `json:"icon,omitempty"`
UpdatedBy string `json:"updatedBy"`
UpdatedAt string `json:"updatedAt"`
}
type DocShare struct { type DocShare struct {
ID string `json:"id"` ID string `json:"id"`
DocID string `json:"docId"` DocID string `json:"docId"`
+945
View File
@@ -0,0 +1,945 @@
package storage
import (
"database/sql"
"encoding/json"
"errors"
"fmt"
"log/slog"
"sort"
"strings"
"sync"
"time"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain"
)
// Collaborative note editing: every change to a note's blocks or title runs through collab.Apply under a
// per-document lock, is appended to doc_ops (the log) and recorded in doc_op_results (idempotency).
// ErrDocNotFound is returned when a note does not exist or the user has no access to it.
var ErrDocNotFound = errors.New("Notiz nicht gefunden oder kein Zugriff")
// ErrLogCompacted means the requested log entries were removed by compaction; the client reloads the snapshot.
var ErrLogCompacted = errors.New("Änderungsprotokoll verdichtet")
// Retention of the operation log and of tombstones (see CompactDocs).
const (
LogRetentionShared = 30 * 24 * time.Hour
LogRetentionPrivate = 7 * 24 * time.Hour
TombstoneRetention = 30 * 24 * time.Hour
OpResultRetention = 90 * 24 * time.Hour
)
// Roles a user can have on a note.
const (
DocRoleOwner = "owner"
DocRoleEditor = "editor"
DocRoleViewer = "viewer"
)
// DocLogEntry is one effect from the log, as sent to clients.
type DocLogEntry struct {
Seq int64 `json:"seq"`
OpID string `json:"opId,omitempty"`
UserID string `json:"userId"`
ClientID string `json:"clientId"`
Effect collab.Effect `json:"effect"`
}
// DocSnapshot is the complete state of a note including tombstones, needed to replay later effects.
type DocSnapshot struct {
DocID string `json:"docId"`
Title string `json:"title"`
TitleVersion int `json:"titleVersion"`
HeadSeq int64 `json:"headSeq"`
Blocks []collab.Block `json:"blocks"`
}
var docLocks sync.Map // docID -> *sync.Mutex
func lockDoc(docID string) func() {
m, _ := docLocks.LoadOrStore(docID, &sync.Mutex{})
mu := m.(*sync.Mutex)
mu.Lock()
return mu.Unlock
}
type docQueryer interface {
QueryRow(query string, args ...interface{}) *sql.Row
Query(query string, args ...interface{}) (*sql.Rows, error)
}
// migrateDocSync adds the columns and tables of the collaboration model and converts old flat block lists.
func (s *Store) migrateDocSync() {
stmts := []string{
"ALTER TABLE blocks ADD COLUMN parent_id TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN order_key TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN version INTEGER NOT NULL DEFAULT 1;",
"ALTER TABLE blocks ADD COLUMN conflict_of TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN conflict_author TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN updated_by TEXT NOT NULL DEFAULT '';",
"ALTER TABLE blocks ADD COLUMN deleted INTEGER NOT NULL DEFAULT 0;",
"CREATE INDEX IF NOT EXISTS idx_blocks_doc_parent ON blocks(document_id, parent_id);",
"ALTER TABLE documents ADD COLUMN head_seq INTEGER NOT NULL DEFAULT 0;",
"ALTER TABLE documents ADD COLUMN title_version INTEGER NOT NULL DEFAULT 0;",
`CREATE TABLE IF NOT EXISTS doc_ops (
doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
seq INTEGER NOT NULL,
op_id TEXT NOT NULL DEFAULT '',
user_id TEXT NOT NULL DEFAULT '',
client_id TEXT NOT NULL DEFAULT '',
kind TEXT NOT NULL,
block_id TEXT NOT NULL DEFAULT '',
payload TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (doc_id, seq)
);`,
`CREATE TABLE IF NOT EXISTS doc_op_results (
op_id TEXT PRIMARY KEY,
doc_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
user_id TEXT NOT NULL DEFAULT '',
result TEXT NOT NULL,
created_at TEXT NOT NULL
);`,
"CREATE INDEX IF NOT EXISTS idx_doc_op_results_doc ON doc_op_results(doc_id);",
"ALTER TABLE blocks ADD COLUMN deleted_at TEXT NOT NULL DEFAULT '';",
"ALTER TABLE documents ADD COLUMN log_floor INTEGER NOT NULL DEFAULT 0;",
"CREATE INDEX IF NOT EXISTS idx_doc_ops_created ON doc_ops(created_at);",
"CREATE INDEX IF NOT EXISTS idx_doc_op_results_created ON doc_op_results(created_at);",
}
for _, q := range stmts {
if _, err := s.db.Exec(q); err != nil && !strings.Contains(err.Error(), "duplicate column") {
slog.Error("Migration der Notiz-Synchronisation fehlgeschlagen", "error", err, "statement", q)
}
}
// Tombstones from before deleted_at existed start their retention now
if _, err := s.db.Exec(`UPDATE blocks SET deleted_at = ? WHERE deleted = 1 AND deleted_at = ''`, domain.NowISO()); err != nil {
slog.Error("Migration der Notiz-Synchronisation fehlgeschlagen", "error", err)
}
s.backfillBlockTree()
}
// backfillBlockTree converts blocks stored as a flat list (level + position) into parent_id + order_key.
func (s *Store) backfillBlockTree() {
rows, err := s.db.Query(`SELECT DISTINCT document_id FROM blocks WHERE order_key = ''`)
if err != nil {
return
}
var docIDs []string
for rows.Next() {
var id string
if rows.Scan(&id) == nil {
docIDs = append(docIDs, id)
}
}
_ = rows.Close()
for _, docID := range docIDs {
bRows, err := s.db.Query(`SELECT id, level FROM blocks WHERE document_id = ? ORDER BY position ASC, rowid ASC`, docID)
if err != nil {
continue
}
type flat struct {
id string
level int
}
var list []flat
for bRows.Next() {
var f flat
if bRows.Scan(&f.id, &f.level) == nil {
list = append(list, f)
}
}
_ = bRows.Close()
parents, keys := treeFromLevels(len(list), func(i int) int { return list[i].level })
tx, err := s.db.Begin()
if err != nil {
continue
}
for i, f := range list {
parent := ""
if parents[i] >= 0 {
parent = list[parents[i]].id
}
_, _ = tx.Exec(`UPDATE blocks SET parent_id = ?, order_key = ? WHERE id = ?`, parent, keys[i], f.id)
}
if err := tx.Commit(); err != nil {
slog.Error("Umwandlung der Blockstruktur fehlgeschlagen", "doc_id", docID, "error", err)
}
}
}
// treeFromLevels turns a flat list with nesting levels into parent indices (-1 = top level) and
// sequential order keys per parent. Levels deeper than possible are clamped.
func treeFromLevels(n int, level func(int) int) ([]int, []string) {
parents := make([]int, n)
keys := make([]string, n)
lastKey := map[int]string{}
var stack []int // stack[l] = index of the last block at level l
for i := 0; i < n; i++ {
lvl := level(i)
if lvl < 0 {
lvl = 0
}
if lvl > len(stack) {
lvl = len(stack)
}
stack = stack[:lvl]
parent := -1
if lvl > 0 {
parent = stack[lvl-1]
}
stack = append(stack, i)
parents[i] = parent
k, _ := collab.KeyBetween(lastKey[parent], "")
lastKey[parent] = k
keys[i] = k
}
return parents, keys
}
func loadCollabDoc(q docQueryer, docID string) (*collab.Doc, int64, error) {
d := collab.NewDoc()
var head int64
err := q.QueryRow(`SELECT title, title_version, head_seq FROM documents WHERE id = ?`, docID).Scan(&d.Title, &d.TitleVersion, &head)
if err != nil {
return nil, 0, err
}
rows, err := q.Query(
`SELECT id, parent_id, order_key, type, content, checked, collapsed, lang, conflict_of, conflict_author, version, deleted, updated_by
FROM blocks WHERE document_id = ?`, docID)
if err != nil {
return nil, 0, err
}
defer rows.Close()
for rows.Next() {
b := &collab.Block{}
var chk, col, del int
if err := rows.Scan(&b.ID, &b.ParentID, &b.OrderKey, &b.Type, &b.Content, &chk, &col, &b.Props.Lang,
&b.Props.ConflictOf, &b.Props.ConflictAuthor, &b.Version, &del, &b.UpdatedBy); err != nil {
return nil, 0, err
}
b.Props.Checked, b.Props.Collapsed, b.Deleted = chk != 0, col != 0, del != 0
d.Blocks[b.ID] = b
}
return d, head, rows.Err()
}
// flatDocumentBlocks converts the live tree into the flat list (level + position) the existing API returns.
func flatDocumentBlocks(docID string, d *collab.Doc) []domain.DocumentBlock {
flat := d.Flatten()
out := make([]domain.DocumentBlock, len(flat))
for i, f := range flat {
out[i] = domain.DocumentBlock{
ID: f.ID, DocumentID: docID, Type: f.Type, Content: f.Content, Checked: f.Props.Checked,
Lang: f.Props.Lang, Level: f.Level, Position: float64((i + 1) * 1000), Collapsed: f.Props.Collapsed,
}
}
return out
}
func boolInt(b bool) int {
if b {
return 1
}
return 0
}
// ApplyDocOps applies client operations to a note. Operations whose opId was already processed return
// the stored result without being applied again, so clients can safely resend after lost responses.
func (s *Store) ApplyDocOps(docID string, a collab.Actor, ops []collab.Op, leases collab.LeaseView) ([]collab.Result, int64, error) {
return s.applyDocChange(docID, a, leases, func(*collab.Doc) []collab.Op { return ops })
}
// applyDocChange builds operations from the current state (under the document lock) and applies them.
func (s *Store) applyDocChange(docID string, a collab.Actor, leases collab.LeaseView, build func(*collab.Doc) []collab.Op) ([]collab.Result, int64, error) {
unlock := lockDoc(docID)
defer unlock()
tx, err := s.db.Begin()
if err != nil {
return nil, 0, err
}
defer tx.Rollback()
doc, head, err := loadCollabDoc(tx, docID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, 0, ErrDocNotFound
}
return nil, 0, err
}
before := doc.Clone()
startHead := head
env := collab.Env{
Leases: leases,
IDTaken: func(id string) bool {
var owner string
return tx.QueryRow(`SELECT document_id FROM blocks WHERE id = ?`, id).Scan(&owner) == nil && owner != docID
},
}
now := domain.NowISO()
ops := build(doc)
results := make([]collab.Result, 0, len(ops))
for _, op := range ops {
if op.OpID == "" || len(op.OpID) > 100 {
results = append(results, collab.Result{OpID: op.OpID, Status: collab.StatusRejected, Reason: "invalid_op_id"})
continue
}
var stored, storedDoc string
err := tx.QueryRow(`SELECT result, doc_id FROM doc_op_results WHERE op_id = ?`, op.OpID).Scan(&stored, &storedDoc)
if err == nil {
var res collab.Result
if storedDoc != docID || json.Unmarshal([]byte(stored), &res) != nil {
res = collab.Result{OpID: op.OpID, Status: collab.StatusRejected, Reason: "op_id_reused"}
}
results = append(results, res)
continue
}
if !errors.Is(err, sql.ErrNoRows) {
return nil, 0, err
}
effects, res := collab.Apply(doc, op, a, env)
for _, e := range effects {
head++
payload, _ := json.Marshal(e)
if _, err := tx.Exec(
`INSERT INTO doc_ops (doc_id, seq, op_id, user_id, client_id, kind, block_id, payload, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
docID, head, op.OpID, a.UserID, a.ClientID, e.Kind, e.BlockID, string(payload), now,
); err != nil {
return nil, 0, err
}
}
if len(effects) > 0 {
res.Seq = head
}
resJSON, _ := json.Marshal(res)
if _, err := tx.Exec(`INSERT INTO doc_op_results (op_id, doc_id, user_id, result, created_at) VALUES (?, ?, ?, ?, ?)`,
op.OpID, docID, a.UserID, string(resJSON), now); err != nil {
return nil, 0, err
}
results = append(results, res)
}
if head != startHead {
if err := persistDocState(tx, docID, before, doc, head, now); err != nil {
return nil, 0, err
}
}
if err := tx.Commit(); err != nil {
return nil, 0, err
}
return results, head, nil
}
// persistDocState writes changed blocks and refreshes data derived from the note (markdown, search text, tags).
func persistDocState(tx *sql.Tx, docID string, before, after *collab.Doc, head int64, now string) error {
stmt, err := tx.Prepare(`
INSERT INTO blocks (id, document_id, parent_id, order_key, type, content, checked, collapsed, lang,
conflict_of, conflict_author, version, deleted, deleted_at, updated_by, level, position)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, 0)
ON CONFLICT(id) DO UPDATE SET
parent_id = excluded.parent_id, order_key = excluded.order_key, type = excluded.type,
content = excluded.content, checked = excluded.checked, collapsed = excluded.collapsed,
lang = excluded.lang, conflict_of = excluded.conflict_of, conflict_author = excluded.conflict_author,
version = excluded.version, deleted = excluded.deleted, updated_by = excluded.updated_by,
deleted_at = CASE WHEN excluded.deleted = 0 THEN '' WHEN blocks.deleted = 1 THEN blocks.deleted_at ELSE excluded.deleted_at END
WHERE blocks.document_id = excluded.document_id`)
if err != nil {
return err
}
defer stmt.Close()
tagsChanged := false
for id, b := range after.Blocks {
if old, ok := before.Blocks[id]; ok && *old == *b {
continue
}
tagsChanged = true
deletedAt := ""
if b.Deleted {
deletedAt = now
}
if _, err := stmt.Exec(b.ID, docID, b.ParentID, b.OrderKey, b.Type, b.Content, boolInt(b.Props.Checked),
boolInt(b.Props.Collapsed), b.Props.Lang, b.Props.ConflictOf, b.Props.ConflictAuthor, b.Version,
boolInt(b.Deleted), deletedAt, b.UpdatedBy); err != nil {
return err
}
}
for id := range before.Blocks {
if _, ok := after.Blocks[id]; !ok {
if _, err := tx.Exec(`DELETE FROM blocks WHERE id = ? AND document_id = ?`, id, docID); err != nil {
return err
}
}
}
flat := flatDocumentBlocks(docID, after)
var plain strings.Builder
for _, b := range flat {
if b.Content != "" {
plain.WriteString(b.Content)
plain.WriteString("\n")
}
}
blocksJSON, _ := json.Marshal(flat)
tagSet := map[string]bool{}
for _, b := range flat {
for _, tag := range domain.ExtractTags(b.Content) {
tagSet[tag] = true
}
}
tags := make([]string, 0, len(tagSet))
for tag := range tagSet {
tags = append(tags, tag)
}
sort.Strings(tags)
tagsJSON, _ := json.Marshal(tags)
if _, err := tx.Exec(
`UPDATE documents SET title = ?, title_version = ?, head_seq = ?, markdown_cache = ?, content = ?, blocks_json = ?, tags = ?, updated_at = ? WHERE id = ?`,
after.Title, after.TitleVersion, head, domain.CompileBlocksToMarkdown(flat), plain.String(), string(blocksJSON), string(tagsJSON), now, docID,
); err != nil {
return err
}
if tagsChanged {
if _, err := tx.Exec(`DELETE FROM block_tags WHERE document_id = ?`, docID); err != nil {
return err
}
for _, b := range flat {
for _, tag := range domain.ExtractTags(b.Content) {
if _, err := tx.Exec(`INSERT OR IGNORE INTO block_tags (tag, document_id, block_id) VALUES (?, ?, ?)`, tag, docID, b.ID); err != nil {
return err
}
}
}
}
return nil
}
// docTaskLinks maps block IDs of a note to the IDs of tasks created from them ("Erweitert").
func (s *Store) docTaskLinks(docID string) map[string]string {
links := map[string]string{}
rows, err := s.db.Query(
`SELECT id, converted_ref_id FROM thoughts WHERE converted_type = 'document_block' AND converted_ref_id LIKE ?`,
docID+":%")
if err != nil {
return links
}
defer rows.Close()
for rows.Next() {
var taskID, ref string
if rows.Scan(&taskID, &ref) == nil {
if _, blockID, ok := strings.Cut(ref, ":"); ok {
links[blockID] = taskID
}
}
}
return links
}
// ReconcileDocTaskLinks brings tasks linked to blocks in line with the note: a ticked block completes its
// task, a deleted block unlinks it (the task itself stays). Returns the IDs of changed tasks.
func (s *Store) ReconcileDocTaskLinks(docID string) []string {
links := s.docTaskLinks(docID)
if len(links) == 0 {
return nil
}
d, _, err := loadCollabDoc(s.db, docID)
if err != nil {
return nil
}
var changed []string
for blockID, taskID := range links {
b, ok := d.Blocks[blockID]
if !ok || b.Deleted {
if _, err := s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '' WHERE id = ?`, taskID); err == nil {
changed = append(changed, taskID)
}
continue
}
kanban := "todo"
if b.Props.Checked {
kanban = "done"
}
res, err := s.db.Exec(
`UPDATE thoughts SET is_completed = ?, kanban_status = ? WHERE id = ? AND is_completed != ?`,
boolInt(b.Props.Checked), kanban, taskID, boolInt(b.Props.Checked))
if err == nil {
if n, _ := res.RowsAffected(); n > 0 {
changed = append(changed, taskID)
}
}
}
sort.Strings(changed)
return changed
}
// GetThoughtByIDUnchecked loads a task without access check, for server-internal notifications.
func (s *Store) GetThoughtByIDUnchecked(thoughtID string) (*domain.Thought, error) {
return scanThought(s.db.QueryRow(
`SELECT id, user_id, content, tags, priority, source, status, is_completed, converted_type, converted_ref_id, due_date, task_list_id, kanban_status, notes, alarm_sent, created_at, triaged_at, subtasks, sort_order
FROM thoughts WHERE id = ?`, thoughtID))
}
// GetDocOpsSince returns up to limit log entries after seq, plus the current head. If entries after seq
// were already compacted away it returns ErrLogCompacted (with the head): the client must reload the snapshot.
func (s *Store) GetDocOpsSince(docID string, since int64, limit int) ([]DocLogEntry, int64, error) {
// One read transaction, so a compaction running in between cannot leave an unnoticed gap
tx, err := s.db.Begin()
if err != nil {
return nil, 0, err
}
defer tx.Rollback()
var head, floor int64
if err := tx.QueryRow(`SELECT head_seq, log_floor FROM documents WHERE id = ?`, docID).Scan(&head, &floor); err != nil {
return nil, 0, err
}
if limit > 0 && since < floor {
return nil, head, ErrLogCompacted
}
rows, err := tx.Query(
`SELECT seq, op_id, user_id, client_id, payload FROM doc_ops WHERE doc_id = ? AND seq > ? ORDER BY seq ASC LIMIT ?`,
docID, since, limit)
if err != nil {
return nil, 0, err
}
defer rows.Close()
out := make([]DocLogEntry, 0)
for rows.Next() {
var e DocLogEntry
var payload string
if err := rows.Scan(&e.Seq, &e.OpID, &e.UserID, &e.ClientID, &payload); err != nil {
return nil, 0, err
}
if err := json.Unmarshal([]byte(payload), &e.Effect); err != nil {
return nil, 0, fmt.Errorf("log entry %d unreadable: %w", e.Seq, err)
}
out = append(out, e)
}
return out, head, rows.Err()
}
// GetDocSnapshot returns the full block tree (including tombstones) and the log position it corresponds to.
func (s *Store) GetDocSnapshot(docID string) (*DocSnapshot, error) {
unlock := lockDoc(docID)
defer unlock()
d, head, err := loadCollabDoc(s.db, docID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrDocNotFound
}
return nil, err
}
snap := &DocSnapshot{DocID: docID, Title: d.Title, TitleVersion: d.TitleVersion, HeadSeq: head, Blocks: make([]collab.Block, 0, len(d.Blocks))}
for _, b := range d.Blocks {
snap.Blocks = append(snap.Blocks, *b)
}
sort.Slice(snap.Blocks, func(i, j int) bool { return snap.Blocks[i].ID < snap.Blocks[j].ID })
return snap, nil
}
// DocRole returns the user's role on a note ("owner", "editor", "viewer") or "" without access.
// Sources: ownership, direct shares, team shares and shares of the shared folder the note lives in.
func (s *Store) DocRole(userID, docID string) string {
if userID == "" {
return ""
}
var ownerID, sharedFolderID, folderOwner string
err := s.db.QueryRow(
`SELECT d.user_id, d.shared_folder_id, COALESCE(wsf.owner_id, '')
FROM documents d LEFT JOIN workspace_shared_folders wsf ON wsf.id = d.shared_folder_id
WHERE d.id = ?`, docID).Scan(&ownerID, &sharedFolderID, &folderOwner)
if err != nil {
return ""
}
if ownerID == userID {
return DocRoleOwner
}
if folderOwner == userID {
return DocRoleEditor
}
rows, err := s.db.Query(
`SELECT permission FROM doc_shares WHERE doc_id = ? AND user_id = ?
UNION ALL SELECT dts.permission FROM doc_team_shares dts JOIN team_members tm ON tm.team_id = dts.team_id
WHERE dts.doc_id = ? AND tm.user_id = ?
UNION ALL SELECT wfs.permission FROM workspace_folder_shares wfs
WHERE wfs.folder_id = ? AND wfs.folder_id != '' AND (
(wfs.target_type = 'user' AND wfs.target_id = ?)
OR (wfs.target_type = 'team' AND wfs.target_id IN (SELECT team_id FROM team_members WHERE user_id = ?)))`,
docID, userID, docID, userID, sharedFolderID, userID, userID)
if err != nil {
return ""
}
defer rows.Close()
role := ""
for rows.Next() {
var p string
if rows.Scan(&p) != nil {
continue
}
if p == "write" {
return DocRoleEditor
}
role = DocRoleViewer
}
return role
}
// CanEditDoc reports whether the role allows changing a note's content.
func CanEditDoc(role string) bool {
return role == DocRoleOwner || role == DocRoleEditor
}
// IsDocShared reports whether anyone besides the owner can access the note.
func (s *Store) IsDocShared(docID string) bool {
var n int
_ = s.db.QueryRow(
`SELECT (SELECT COUNT(*) FROM doc_shares WHERE doc_id = ?)
+ (SELECT COUNT(*) FROM doc_team_shares WHERE doc_id = ?)
+ (SELECT COUNT(*) FROM documents d JOIN workspace_folder_shares wfs ON wfs.folder_id = d.shared_folder_id
WHERE d.id = ? AND d.shared_folder_id != '')`,
docID, docID, docID).Scan(&n)
return n > 0
}
// SetBlockChecked ticks or unticks a block through the log (server-side change, e.g. a linked task was
// completed). Returns true if the block changed.
func (s *Store) SetBlockChecked(userID, docID, blockID string, checked bool) bool {
a := collab.Actor{UserID: userID, ClientID: "server"}
results, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op {
b, ok := d.Blocks[blockID]
if !ok || b.Deleted || b.Props.Checked == checked {
return nil
}
return []collab.Op{{OpID: domain.NewID("op"), Kind: collab.OpSetProps, BlockID: blockID, Patch: &collab.PropsPatch{Checked: &checked}}}
})
return err == nil && len(results) > 0 && results[0].Seq > 0
}
// --- Writing a complete block list (new notes, imports) ---
const legacyClientID = "legacy"
// replaceDocBlocksInDB stores a complete flat block list (initial blocks of a new note, old blocks_json data)
// by diffing it against the current tree and applying the difference as regular operations.
func (s *Store) replaceDocBlocksInDB(docID, userID string, blocks []domain.DocumentBlock, _ string) {
blocks = s.renameForeignBlockIDs(docID, blocks)
a := collab.Actor{UserID: userID, ClientID: legacyClientID}
if _, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op { return legacyOps(d, blocks) }); err != nil {
slog.Error("Speichern der Notiz fehlgeschlagen", "doc_id", docID, "error", err)
}
}
// renameForeignBlockIDs gives new IDs to blocks whose ID already belongs to another note (e.g. fixed IDs of
// starter content or copied blocks). Block IDs are global, so keeping them would make the insert fail.
func (s *Store) renameForeignBlockIDs(docID string, blocks []domain.DocumentBlock) []domain.DocumentBlock {
out := blocks
copied := false
for i, b := range blocks {
id := strings.TrimSpace(b.ID)
if id == "" {
continue
}
var owner string
if s.db.QueryRow(`SELECT document_id FROM blocks WHERE id = ?`, id).Scan(&owner) != nil || owner == docID {
continue
}
if !copied {
out = append([]domain.DocumentBlock(nil), blocks...)
copied = true
}
out[i].ID = domain.NewID("blk")
}
return out
}
// applyLegacyTitle records a title change made through PUT /documents/{id} in the log.
func (s *Store) applyLegacyTitle(docID, userID, title string) {
a := collab.Actor{UserID: userID, ClientID: legacyClientID}
if _, _, err := s.applyDocChange(docID, a, nil, func(d *collab.Doc) []collab.Op {
if d.Title == title {
return nil
}
return []collab.Op{{OpID: domain.NewID("op"), Kind: collab.OpSetTitle, BaseVersion: d.TitleVersion, Title: &title}}
}); err != nil {
slog.Error("Speichern des Notiztitels fehlgeschlagen", "doc_id", docID, "error", err)
}
}
// legacyOps computes the operations that turn the current tree into the given flat list.
// Last writer wins, exactly like the old whole-document save.
func legacyOps(d *collab.Doc, blocks []domain.DocumentBlock) []collab.Op {
// Normalize IDs (missing or duplicated IDs get new ones)
seen := make(map[string]bool, len(blocks))
list := make([]domain.DocumentBlock, len(blocks))
for i, b := range blocks {
id := strings.TrimSpace(b.ID)
if id == "" || seen[id] {
id = domain.NewID("blk")
}
seen[id] = true
b.ID = id
if b.Type == "" {
b.Type = "bullet"
}
list[i] = b
}
parentIdx, _ := treeFromLevels(len(list), func(i int) int { return list[i].Level })
parentOf := make([]string, len(list))
children := map[string][]int{}
for i := range list {
if parentIdx[i] >= 0 {
parentOf[i] = list[parentIdx[i]].ID
}
children[parentOf[i]] = append(children[parentOf[i]], i)
}
keys := make([]string, len(list))
for parent, idxs := range children {
assignKeys(d, parent, idxs, list, keys)
}
var ops []collab.Op
newOp := func(op collab.Op) {
op.OpID = domain.NewID("op")
ops = append(ops, op)
}
wanted := make(map[string]bool, len(list))
for i, b := range list {
wanted[b.ID] = true
parent := parentOf[i]
patch := &collab.PropsPatch{Type: &list[i].Type, Checked: &list[i].Checked, Collapsed: &list[i].Collapsed, Lang: &list[i].Lang}
cur, exists := d.Blocks[b.ID]
if !exists {
content := b.Content
props := collab.Props{Checked: b.Checked, Collapsed: b.Collapsed, Lang: b.Lang}
newOp(collab.Op{Kind: collab.OpInsert, BlockID: b.ID, ParentID: &parent, OrderKey: keys[i], Type: b.Type, Content: &content, Props: &props})
continue
}
if cur.Deleted {
newOp(collab.Op{Kind: collab.OpRestore, IDs: []string{b.ID}})
}
if cur.ParentID != parent || cur.OrderKey != keys[i] || cur.Deleted {
newOp(collab.Op{Kind: collab.OpMove, BlockID: b.ID, ParentID: &parent, OrderKey: keys[i]})
}
if cur.Content != b.Content {
content := b.Content
newOp(collab.Op{Kind: collab.OpSetContent, BlockID: b.ID, BaseVersion: cur.Version, Content: &content})
}
if cur.Type != b.Type || cur.Props.Checked != b.Checked || cur.Props.Collapsed != b.Collapsed || cur.Props.Lang != b.Lang {
newOp(collab.Op{Kind: collab.OpSetProps, BlockID: b.ID, Patch: patch})
}
}
// Deletes last, after moves have rescued wanted children out of deleted branches
for _, fb := range d.Flatten() {
if !wanted[fb.ID] {
newOp(collab.Op{Kind: collab.OpDelete, BlockID: fb.ID})
}
}
return ops
}
// assignKeys picks order keys for the children idxs of parent. Existing keys that are already in the right
// relative order (longest increasing subsequence) are kept, so unchanged blocks do not show up as moves.
func assignKeys(d *collab.Doc, parent string, idxs []int, list []domain.DocumentBlock, keys []string) {
n := len(idxs)
existing := make([]string, n)
for j, i := range idxs {
if b, ok := d.Blocks[list[i].ID]; ok && !b.Deleted && b.ParentID == parent {
existing[j] = b.OrderKey
}
}
keep := longestIncreasing(existing)
for j := 0; j < n; {
if keep[j] {
keys[idxs[j]] = existing[j]
j++
continue
}
start := j
for j < n && !keep[j] {
j++
}
left, right := "", ""
if start > 0 {
left = keys[idxs[start-1]]
}
if j < n {
right = existing[j]
}
fresh, err := collab.KeysBetween(left, right, j-start)
if err != nil {
fresh, _ = collab.KeysBetween(left, "", j-start)
}
for k := start; k < j; k++ {
keys[idxs[k]] = fresh[k-start]
}
}
}
// longestIncreasing marks a longest strictly increasing subsequence of the non-empty keys.
func longestIncreasing(keys []string) []bool {
n := len(keys)
keep := make([]bool, n)
tails := []int{} // indices of the smallest tail for each length
prev := make([]int, n) // predecessor in the subsequence
for i, k := range keys {
prev[i] = -1
if k == "" {
continue
}
lo, hi := 0, len(tails)
for lo < hi {
mid := (lo + hi) / 2
if keys[tails[mid]] < k {
lo = mid + 1
} else {
hi = mid
}
}
if lo > 0 {
prev[i] = tails[lo-1]
}
if lo == len(tails) {
tails = append(tails, i)
} else {
tails[lo] = i
}
}
if len(tails) > 0 {
for i := tails[len(tails)-1]; i >= 0; i = prev[i] {
keep[i] = true
}
}
return keep
}
// CompactStats counts what one compaction run removed.
type CompactStats struct {
Docs int
LogEntries int64
Tombstones int
OpResults int64
Purged map[string]int64 // docID -> new head, for notes whose tombstones were purged
}
// CompactDocs keeps the collaboration tables small:
// - log entries older than the retention (30 days for shared notes, 7 for private ones) are removed and
// the note remembers the boundary in log_floor; clients that are further behind reload the snapshot,
// their unsent changes stay queued and are sent afterwards,
// - deleted blocks older than 30 days are purged; this is itself a log entry ("purge"), so connected
// clients drop them without a reload,
// - idempotency results older than 90 days are removed.
func (s *Store) CompactDocs(now time.Time) (CompactStats, error) {
st := CompactStats{Purged: map[string]int64{}}
ts := func(d time.Duration) string { return now.Add(-d).UTC().Format(time.RFC3339) }
tombCutoff := ts(TombstoneRetention)
rows, err := s.db.Query(
`SELECT doc_id FROM doc_ops WHERE created_at < ?
UNION SELECT document_id FROM blocks WHERE deleted = 1 AND deleted_at != '' AND deleted_at < ?`,
ts(LogRetentionPrivate), tombCutoff)
if err != nil {
return st, err
}
var docIDs []string
for rows.Next() {
var id string
if rows.Scan(&id) == nil {
docIDs = append(docIDs, id)
}
}
rows.Close()
for _, docID := range docIDs {
cutoff := ts(LogRetentionPrivate)
if s.IsDocShared(docID) {
cutoff = ts(LogRetentionShared)
}
removed, purged, head, err := s.compactDoc(docID, cutoff, tombCutoff, now.UTC().Format(time.RFC3339))
if err != nil {
return st, fmt.Errorf("compact %s: %w", docID, err)
}
if purged > 0 {
st.Purged[docID] = head
}
if removed > 0 || purged > 0 {
st.Docs++
}
st.LogEntries += removed
st.Tombstones += purged
}
res, err := s.db.Exec(`DELETE FROM doc_op_results WHERE created_at < ?`, ts(OpResultRetention))
if err != nil {
return st, err
}
st.OpResults, _ = res.RowsAffected()
return st, nil
}
func (s *Store) compactDoc(docID, logCutoff, tombCutoff, now string) (removed int64, purged int, head int64, err error) {
unlock := lockDoc(docID)
defer unlock()
tx, err := s.db.Begin()
if err != nil {
return 0, 0, 0, err
}
defer tx.Rollback()
doc, head, err := loadCollabDoc(tx, docID)
if err != nil {
return 0, 0, 0, err
}
var candidates []string
rows, err := tx.Query(`SELECT id FROM blocks WHERE document_id = ? AND deleted = 1 AND deleted_at != '' AND deleted_at < ?`, docID, tombCutoff)
if err != nil {
return 0, 0, 0, err
}
for rows.Next() {
var id string
if rows.Scan(&id) == nil {
candidates = append(candidates, id)
}
}
rows.Close()
if e := collab.Purge(doc, candidates); e != nil {
head++
payload, _ := json.Marshal(e)
if _, err := tx.Exec(
`INSERT INTO doc_ops (doc_id, seq, op_id, user_id, client_id, kind, block_id, payload, created_at) VALUES (?, ?, '', '', '', ?, '', ?, ?)`,
docID, head, e.Kind, string(payload), now); err != nil {
return 0, 0, 0, err
}
for _, id := range e.IDs {
if _, err := tx.Exec(`DELETE FROM blocks WHERE id = ? AND document_id = ?`, id, docID); err != nil {
return 0, 0, 0, err
}
}
// Only the position changes; tombstones are not part of the note's text, so nothing derived changes
if _, err := tx.Exec(`UPDATE documents SET head_seq = ? WHERE id = ?`, head, docID); err != nil {
return 0, 0, 0, err
}
purged = len(e.IDs)
}
var floor int64
if err := tx.QueryRow(`SELECT COALESCE(MAX(seq), 0) FROM doc_ops WHERE doc_id = ? AND created_at < ?`, docID, logCutoff).Scan(&floor); err != nil {
return 0, 0, 0, err
}
if floor > 0 {
res, err := tx.Exec(`DELETE FROM doc_ops WHERE doc_id = ? AND seq <= ?`, docID, floor)
if err != nil {
return 0, 0, 0, err
}
removed, _ = res.RowsAffected()
if _, err := tx.Exec(`UPDATE documents SET log_floor = MAX(log_floor, ?) WHERE id = ?`, floor, docID); err != nil {
return 0, 0, 0, err
}
}
return removed, purged, head, tx.Commit()
}
+468
View File
@@ -0,0 +1,468 @@
package storage
import (
"errors"
"fmt"
"math/rand"
"sort"
"testing"
"time"
"github.com/sandrodach/mindsift/internal/collab"
)
// The simulation drives several clients against the real store over an unreliable network and checks
// that everyone converges to the server state without losing text or applying anything twice.
type simClient struct {
actor collab.Actor
confirmed *collab.Doc
lastSeq int64
pending []collab.Op // not sent yet; base versions are filled in when a batch is formed
inflight []collab.Op // sent, response not received; resent unchanged (same op IDs)
holds map[string]bool
offline bool
counter int
}
type simWorld struct {
t *testing.T
r *rand.Rand
store *Store
docID string
leases *collab.LeaseManager
now time.Time
clients []*simClient
sent map[string]collab.Op // every op ever sent, by ID
results map[string]collab.Result // first result received per op ID
fullLog []DocLogEntry // copy of the log taken before each compaction, for the final checks
logSeen int64
compactions int
}
func (w *simWorld) newOpID(c *simClient) string {
c.counter++
return fmt.Sprintf("%s-%d", c.actor.ClientID, c.counter)
}
// localView is what the client's editor shows: confirmed state plus its own unconfirmed operations.
func (c *simClient) localView() *collab.Doc {
v := c.confirmed.Clone()
for _, op := range append(append([]collab.Op{}, c.inflight...), c.pending...) {
if op.Kind == collab.OpSetContent {
if b, ok := v.Blocks[op.BlockID]; ok {
op.BaseVersion = b.Version
}
}
if op.Kind == collab.OpSetTitle {
op.BaseVersion = v.TitleVersion
}
collab.Apply(v, op, c.actor, collab.Env{})
}
return v
}
func pick[T any](r *rand.Rand, list []T) (T, bool) {
var zero T
if len(list) == 0 {
return zero, false
}
return list[r.Intn(len(list))], true
}
// edit lets a client perform one random editor action on its local view.
func (w *simWorld) edit(c *simClient) {
v := c.localView()
flat := v.Flatten()
var target *collab.Block
if fb, ok := pick(w.r, flat); ok {
target = fb.Block
}
switch n := w.r.Intn(100); {
case n < 45 && target != nil: // typing
content := target.Content + fmt.Sprintf(" %s%d", c.actor.ClientID, c.counter)
// Base = the version the user sees while typing (like the browser engine)
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetContent, BlockID: target.ID, BaseVersion: target.Version, Content: &content})
c.holds[target.ID] = true
case n < 65: // Enter: new block after target (or first block)
parent, after := "", ""
if target != nil {
parent, after = target.ParentID, target.OrderKey
}
next := ""
for _, s := range v.Children(parent) {
if s.OrderKey > after {
next = s.OrderKey
break
}
}
key, err := collab.KeyBetween(after, next)
if err != nil {
return
}
id := fmt.Sprintf("blk_%s_%d", c.actor.ClientID, c.counter)
content := "neu von " + c.actor.ClientID
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpInsert, BlockID: id, ParentID: &parent, OrderKey: key, Content: &content})
c.holds[id] = true
case n < 73 && target != nil: // indent under previous sibling
siblings := v.Children(target.ParentID)
for i, s := range siblings {
if s.ID == target.ID && i > 0 {
prev := siblings[i-1]
last := ""
if kids := v.Children(prev.ID); len(kids) > 0 {
last = kids[len(kids)-1].OrderKey
}
key, _ := collab.KeyBetween(last, "")
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpMove, BlockID: target.ID, ParentID: &prev.ID, OrderKey: key})
}
}
case n < 80 && target != nil: // outdent
if p, ok := v.Blocks[target.ParentID]; ok {
key, err := collab.KeyBetween(p.OrderKey, "")
if err == nil {
for _, s := range v.Children(p.ParentID) {
if s.OrderKey > p.OrderKey {
key, _ = collab.KeyBetween(p.OrderKey, s.OrderKey)
break
}
}
grand := p.ParentID
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpMove, BlockID: target.ID, ParentID: &grand, OrderKey: key})
}
}
case n < 87 && target != nil: // delete branch
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpDelete, BlockID: target.ID})
case n < 93 && target != nil: // tick checkbox
checked := !target.Props.Checked
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetProps, BlockID: target.ID, Patch: &collab.PropsPatch{Checked: &checked}})
case n < 96: // rename note
title := fmt.Sprintf("Titel %s%d", c.actor.ClientID, c.counter)
c.pending = append(c.pending, collab.Op{OpID: w.newOpID(c), Kind: collab.OpSetTitle, BaseVersion: v.TitleVersion, Title: &title})
default: // stop editing some block
for id := range c.holds {
delete(c.holds, id)
break
}
}
}
// formBatch takes pending operations and keeps only the last text change per block; its base version is
// the one of the first change, i.e. what the user saw when starting to type.
func (c *simClient) formBatch() []collab.Op {
lastText := map[string]int{}
firstBase := map[string]int{}
lastTitle, firstTitleBase := -1, 0
for i, op := range c.pending {
switch op.Kind {
case collab.OpSetContent:
if _, ok := lastText[op.BlockID]; !ok {
firstBase[op.BlockID] = op.BaseVersion
}
lastText[op.BlockID] = i
case collab.OpSetTitle:
if lastTitle < 0 {
firstTitleBase = op.BaseVersion
}
lastTitle = i
}
}
var batch []collab.Op
for i, op := range c.pending {
switch op.Kind {
case collab.OpSetContent:
if lastText[op.BlockID] != i {
continue
}
op.BaseVersion = firstBase[op.BlockID]
case collab.OpSetTitle:
if i != lastTitle {
continue
}
op.BaseVersion = firstTitleBase
}
batch = append(batch, op)
}
c.pending = nil
return batch
}
// serverSync is what POST /sync does, minus HTTP.
func (w *simWorld) serverSync(c *simClient, batch []collab.Op, holds []string) []collab.Result {
w.leases.Touch(w.docID, c.actor)
w.leases.ReleaseAllExcept(w.docID, c.actor.ClientID, holds)
w.leases.Hold(w.docID, c.actor, holds)
results, _, err := w.store.ApplyDocOps(w.docID, c.actor, batch, w.leases.View(w.docID))
if err != nil {
w.t.Fatalf("ApplyDocOps: %v", err)
}
var own []string
for i, res := range results {
if res.Status == collab.StatusApplied && batch[i].Kind == collab.OpInsert {
own = append(own, batch[i].BlockID)
}
if res.ConflictBlockID != "" {
own = append(own, res.ConflictBlockID)
}
}
w.leases.Hold(w.docID, c.actor, own)
return results
}
// pull fetches the log after lastSeq in small pages (to exercise paging) and replays it.
func (w *simWorld) pull(c *simClient) {
for {
entries, head, err := w.store.GetDocOpsSince(w.docID, c.lastSeq, 7)
if errors.Is(err, ErrLogCompacted) {
// Too far behind: reload the snapshot, unsent operations stay queued
w.loadSnapshot(c)
continue
}
if err != nil {
w.t.Fatalf("GetDocOpsSince: %v", err)
}
if len(entries) == 0 && c.lastSeq < head {
w.t.Fatalf("%s: stuck at seq %d, head %d, no log entries", c.actor.ClientID, c.lastSeq, head)
}
for _, e := range entries {
if e.Seq != c.lastSeq+1 {
w.t.Fatalf("%s: log gap, expected seq %d got %d", c.actor.ClientID, c.lastSeq+1, e.Seq)
}
c.confirmed.ApplyEffect(e.Effect)
c.lastSeq = e.Seq
}
if c.lastSeq >= head {
return
}
}
}
func (w *simWorld) loadSnapshot(c *simClient) {
snap, err := w.store.GetDocSnapshot(w.docID)
if err != nil {
w.t.Fatal(err)
}
c.confirmed = collab.NewDoc()
c.confirmed.Title, c.confirmed.TitleVersion = snap.Title, snap.TitleVersion
for _, b := range snap.Blocks {
b := b
c.confirmed.Blocks[b.ID] = &b
}
c.lastSeq = snap.HeadSeq
}
// captureLog copies log entries the simulation has not seen yet, before compaction removes them.
func (w *simWorld) captureLog() {
entries, _, err := w.store.GetDocOpsSince(w.docID, w.logSeen, 1_000_000)
if err != nil {
w.t.Fatalf("captureLog: %v", err)
}
for _, e := range entries {
w.fullLog = append(w.fullLog, e)
w.logSeen = e.Seq
}
}
// compact ages the whole log and every tombstone past their retention and runs the compaction job.
func (w *simWorld) compact() {
w.captureLog()
old := time.Now().Add(-40 * 24 * time.Hour).UTC().Format(time.RFC3339)
w.store.db.Exec(`UPDATE doc_ops SET created_at = ? WHERE doc_id = ?`, old, w.docID)
w.store.db.Exec(`UPDATE blocks SET deleted_at = ? WHERE document_id = ? AND deleted = 1`, old, w.docID)
if _, err := w.store.CompactDocs(time.Now()); err != nil {
w.t.Fatalf("CompactDocs: %v", err)
}
w.compactions++
}
// sync runs one round trip over the unreliable (or, with reliable=true, a perfect) network.
func (w *simWorld) sync(c *simClient, reliable bool) {
if c.inflight == nil {
c.inflight = c.formBatch()
for _, op := range c.inflight {
w.sent[op.OpID] = op
}
}
holds := make([]string, 0, len(c.holds))
for id := range c.holds {
holds = append(holds, id)
}
sort.Strings(holds)
fate := 100
if !reliable {
fate = w.r.Intn(100)
}
switch {
case fate < 10: // request lost
return
case fate < 20: // server applied it, response lost
w.serverSync(c, c.inflight, holds)
return
case fate < 25: // request duplicated by a retrying proxy
w.serverSync(c, c.inflight, holds)
}
results := w.serverSync(c, c.inflight, holds)
for _, res := range results {
if prev, ok := w.results[res.OpID]; ok && prev != res {
w.t.Fatalf("op %s answered differently on retry: %+v vs %+v", res.OpID, prev, res)
}
w.results[res.OpID] = res
}
c.inflight = nil
w.pull(c)
}
func runSimulation(t *testing.T, seed int64, steps int) {
store, u := newSyncTestStore(t)
doc, err := store.CreateDocument(u.ID, "", "Sim", "📄", "", flatBlocks([3]interface{}{"root", 0, "Start"}), nil, false)
if err != nil {
t.Fatal(err)
}
w := &simWorld{t: t, r: rand.New(rand.NewSource(seed)), store: store, docID: doc.ID,
now: time.Unix(1_700_000_000, 0), sent: map[string]collab.Op{}, results: map[string]collab.Result{}}
w.leases = collab.NewLeaseManager(func() time.Time { return w.now })
for i := 0; i < 4; i++ {
snap, err := store.GetDocSnapshot(doc.ID)
if err != nil {
t.Fatal(err)
}
c := &simClient{actor: collab.Actor{UserID: fmt.Sprintf("u%d", i%2), ClientID: fmt.Sprintf("c%d", i), Name: fmt.Sprintf("Client %d", i)},
confirmed: collab.NewDoc(), lastSeq: snap.HeadSeq, holds: map[string]bool{}}
c.confirmed.Title, c.confirmed.TitleVersion = snap.Title, snap.TitleVersion
for _, b := range snap.Blocks {
b := b
c.confirmed.Blocks[b.ID] = &b
}
w.clients = append(w.clients, c)
}
for step := 0; step < steps; step++ {
c := w.clients[w.r.Intn(len(w.clients))]
switch n := w.r.Intn(100); {
case n < 55:
w.edit(c)
case n < 90:
if !c.offline {
w.sync(c, false)
}
case n < 95:
c.offline = !c.offline
case n < 99:
w.now = w.now.Add(time.Duration(w.r.Intn(25)) * time.Second) // leases may expire
default:
w.compact()
}
}
// Everyone comes back online and syncs until nothing is left
for round := 0; round < 10; round++ {
for _, c := range w.clients {
c.offline = false
w.sync(c, true)
}
}
for _, c := range w.clients {
if len(c.pending) > 0 || len(c.inflight) > 0 {
t.Fatalf("%s still has unsent operations", c.actor.ClientID)
}
w.pull(c)
}
server, head, err := loadCollabDoc(store.db, doc.ID)
if err != nil {
t.Fatal(err)
}
if err := server.Validate(); err != nil {
t.Fatalf("seed %d: invalid tree: %v", seed, err)
}
for _, c := range w.clients {
if c.lastSeq != head || !docsEqual(c.confirmed, server) {
t.Fatalf("seed %d: %s diverged from the server (seq %d vs %d)", seed, c.actor.ClientID, c.lastSeq, head)
}
}
// Every operation got exactly one stored result, and no text was lost
var stored int
if err := store.db.QueryRow(`SELECT COUNT(*) FROM doc_op_results WHERE doc_id = ? AND user_id IN ('u0', 'u1')`, doc.ID).Scan(&stored); err != nil {
t.Fatal(err)
}
if stored != len(w.sent) {
t.Fatalf("seed %d: %d operations sent but %d results stored", seed, len(w.sent), stored)
}
w.captureLog()
log := w.fullLog
bySeq := map[int64]collab.Effect{}
inserted := map[string]string{}
for _, e := range log {
bySeq[e.Seq] = e.Effect
// First insert only: a purged block that is edited again comes back with the same ID
if _, seen := inserted[e.Effect.BlockID]; e.Effect.Kind == collab.OpInsert && !seen {
inserted[e.Effect.BlockID] = e.Effect.Content
}
}
conflicts := 0
for id, op := range w.sent {
res, ok := w.results[id]
if !ok {
continue // result only seen by the server (response lost, never retried after success) is covered by the count above
}
switch op.Kind {
case collab.OpInsert, collab.OpSetContent, collab.OpSetTitle:
// Clearing the text of a block that was purged meanwhile has nothing left to keep
emptied := op.Kind == collab.OpSetContent && *op.Content == "" && res.Reason == "missing"
if res.Status == collab.StatusRejected && !emptied {
t.Fatalf("seed %d: text-carrying op %s was rejected: %+v", seed, id, res)
}
}
if op.Kind == collab.OpSetContent && res.Status == collab.StatusApplied && res.Seq > 0 {
want := collab.OpSetContent
if res.Reason == "revived" { // block was purged: the text came back as a new block
want = collab.OpInsert
}
if e := bySeq[res.Seq]; e.Kind != want || e.Content != *op.Content {
t.Fatalf("seed %d: applied text of %s not in the log", seed, id)
}
}
if res.Status == collab.StatusConflict {
conflicts++
text := ""
if op.Content != nil {
text = *op.Content
} else if op.Title != nil {
text = *op.Title
}
if got, ok := inserted[res.ConflictBlockID]; !ok || got != text {
t.Fatalf("seed %d: conflict copy of %s missing or wrong: got %q (present %v), want %q", seed, id, got, ok, text)
}
}
}
t.Logf("seed %d: %d ops, %d log entries, %d conflict copies, %d blocks, %d compactions", seed, len(w.sent), len(log), conflicts, len(server.Blocks), w.compactions)
}
func docsEqual(a, b *collab.Doc) bool {
if a.Title != b.Title || a.TitleVersion != b.TitleVersion || len(a.Blocks) != len(b.Blocks) {
return false
}
for id, x := range a.Blocks {
y, ok := b.Blocks[id]
if !ok || *x != *y {
return false
}
}
return true
}
func TestSyncSimulationUnreliableNetwork(t *testing.T) {
seeds := 25
if testing.Short() {
seeds = 3
}
for seed := int64(1); seed <= int64(seeds); seed++ {
seed := seed
t.Run(fmt.Sprintf("seed-%d", seed), func(t *testing.T) {
runSimulation(t, seed, 600)
})
}
}
+408
View File
@@ -0,0 +1,408 @@
package storage
import (
"errors"
"path/filepath"
"testing"
"time"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain"
)
func newSyncTestStore(t *testing.T) (*Store, *domain.User) {
t.Helper()
store, err := NewStore(filepath.Join(t.TempDir(), "sync.db"))
if err != nil {
t.Fatalf("store: %v", err)
}
t.Cleanup(func() { _ = store.Close() })
u, err := store.CreateUser("owner@example.com", "hash", "Owner", "member")
if err != nil {
t.Fatalf("user: %v", err)
}
return store, u
}
func flatBlocks(spec ...[3]interface{}) []domain.DocumentBlock { // {id, level, content}
out := make([]domain.DocumentBlock, len(spec))
for i, s := range spec {
out[i] = domain.DocumentBlock{ID: s[0].(string), Level: s[1].(int), Content: s[2].(string), Type: "bullet"}
}
return out
}
func outline(blocks []domain.DocumentBlock) [][2]interface{} {
out := make([][2]interface{}, len(blocks))
for i, b := range blocks {
out[i] = [2]interface{}{b.ID, b.Level}
}
return out
}
func sameOutline(t *testing.T, got []domain.DocumentBlock, want []domain.DocumentBlock) {
t.Helper()
if len(got) != len(want) {
t.Fatalf("got %v, want %v", outline(got), outline(want))
}
for i := range got {
if got[i].ID != want[i].ID || got[i].Level != want[i].Level || got[i].Content != want[i].Content {
t.Fatalf("block %d: got %v, want %v", i, outline(got), outline(want))
}
}
}
func headSeq(t *testing.T, s *Store, docID string) int64 {
t.Helper()
_, head, err := s.GetDocOpsSince(docID, 0, 0)
if err != nil {
t.Fatal(err)
}
return head
}
func TestLegacySaveRoundTripsThroughTheLog(t *testing.T) {
s, u := newSyncTestStore(t)
initial := flatBlocks(
[3]interface{}{"a", 0, "Projekt"},
[3]interface{}{"b", 1, "Aufgabe 1"},
[3]interface{}{"c", 2, "Detail"},
[3]interface{}{"d", 1, "Aufgabe 2"},
[3]interface{}{"e", 0, "Anderes"},
)
doc, err := s.CreateDocument(u.ID, "", "Notiz", "📄", "", initial, nil, false)
if err != nil {
t.Fatal(err)
}
got, _ := s.GetDocumentByID(u.ID, doc.ID)
sameOutline(t, got.Blocks, initial)
// Saving the same content again must not produce any log entries
head := headSeq(t, s, doc.ID)
s.replaceDocBlocksInDB(doc.ID, u.ID, initial, "")
if h := headSeq(t, s, doc.ID); h != head {
t.Fatalf("unchanged save wrote %d log entries", h-head)
}
// Reorder, outdent, edit and delete in one save
changed := flatBlocks(
[3]interface{}{"e", 0, "Anderes"},
[3]interface{}{"a", 0, "Projekt (neu)"},
[3]interface{}{"d", 1, "Aufgabe 2"},
[3]interface{}{"c", 1, "Detail"},
[3]interface{}{"n", 2, "Neu"},
)
s.replaceDocBlocksInDB(doc.ID, u.ID, changed, "")
got, _ = s.GetDocumentByID(u.ID, doc.ID)
sameOutline(t, got.Blocks, changed)
// The log alone must rebuild exactly this state
entries, _, err := s.GetDocOpsSince(doc.ID, 0, 10_000)
if err != nil {
t.Fatal(err)
}
replay := collab.NewDoc()
for _, e := range entries {
replay.ApplyEffect(e.Effect)
}
sameOutline(t, flatDocumentBlocks(doc.ID, replay), changed)
}
func TestApplyDocOpsIsIdempotent(t *testing.T) {
s, u := newSyncTestStore(t)
doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", flatBlocks([3]interface{}{"a", 0, "Start"}), nil, false)
a := collab.Actor{UserID: u.ID, ClientID: "tab1", Name: "Owner"}
text := "Geändert"
op := collab.Op{OpID: "op-1", Kind: collab.OpSetContent, BlockID: "a", BaseVersion: 1, Content: &text}
first, head1, err := s.ApplyDocOps(doc.ID, a, []collab.Op{op}, nil)
if err != nil || first[0].Status != collab.StatusApplied {
t.Fatalf("first: %+v %v", first, err)
}
// Resend after a lost response: same result, nothing applied twice
again, head2, err := s.ApplyDocOps(doc.ID, a, []collab.Op{op}, nil)
if err != nil || again[0] != first[0] || head2 != head1 {
t.Fatalf("retry changed something: %+v head %d -> %d", again, head1, head2)
}
}
func TestInsertCannotReuseBlockIDOfAnotherNote(t *testing.T) {
s, u := newSyncTestStore(t)
other, _ := s.CreateDocument(u.ID, "", "Andere", "📄", "", flatBlocks([3]interface{}{"shared_id", 0, "Fremd"}), nil, false)
doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", nil, nil, false)
a := collab.Actor{UserID: u.ID, ClientID: "tab1"}
content, root := "Übernahme", ""
res, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "x", Kind: collab.OpInsert, BlockID: "shared_id", ParentID: &root, OrderKey: "a0", Content: &content}}, nil)
if err != nil || res[0].Reason != "exists" {
t.Fatalf("got %+v %v", res, err)
}
got, _ := s.GetDocumentByID(u.ID, other.ID)
if got.Blocks[0].Content != "Fremd" {
t.Fatal("block of another note was modified")
}
}
func TestDocRole(t *testing.T) {
s, owner := newSyncTestStore(t)
reader, _ := s.CreateUser("reader@example.com", "hash", "Reader", "member")
writer, _ := s.CreateUser("writer@example.com", "hash", "Writer", "member")
stranger, _ := s.CreateUser("stranger@example.com", "hash", "Stranger", "member")
doc, _ := s.CreateDocument(owner.ID, "", "Notiz", "📄", "", nil, nil, false)
now := domain.NowISO()
if _, err := s.db.Exec(`INSERT INTO doc_shares (id, doc_id, user_id, permission, created_at, created_by) VALUES ('s1', ?, ?, 'read', ?, ?), ('s2', ?, ?, 'write', ?, ?)`,
doc.ID, reader.ID, now, owner.ID, doc.ID, writer.ID, now, owner.ID); err != nil {
t.Fatal(err)
}
for user, want := range map[string]string{owner.ID: DocRoleOwner, writer.ID: DocRoleEditor, reader.ID: DocRoleViewer, stranger.ID: ""} {
if got := s.DocRole(user, doc.ID); got != want {
t.Errorf("role of %s: got %q, want %q", user, got, want)
}
}
if !s.IsDocShared(doc.ID) {
t.Error("note with shares must count as shared")
}
}
func TestBackfillConvertsFlatBlocksToTree(t *testing.T) {
s, u := newSyncTestStore(t)
doc, _ := s.CreateDocument(u.ID, "", "Alt", "📄", "", nil, nil, false)
if _, err := s.db.Exec(`DELETE FROM blocks WHERE document_id = ?`, doc.ID); err != nil {
t.Fatal(err)
}
// Rows as written by the old schema: only level and position
for i, row := range [][2]interface{}{{"x", 0}, {"y", 1}, {"z", 1}, {"w", 0}} {
if _, err := s.db.Exec(`INSERT INTO blocks (id, document_id, content, level, position, type) VALUES (?, ?, ?, ?, ?, 'bullet')`,
row[0], doc.ID, row[0], row[1], (i+1)*1000); err != nil {
t.Fatal(err)
}
}
s.backfillBlockTree()
got, _ := s.GetDocumentByID(u.ID, doc.ID)
want := flatBlocks([3]interface{}{"x", 0, "x"}, [3]interface{}{"y", 1, "y"}, [3]interface{}{"z", 1, "z"}, [3]interface{}{"w", 0, "w"})
sameOutline(t, got.Blocks, want)
}
func TestTaskLinksFollowTheirBlock(t *testing.T) {
s, u := newSyncTestStore(t)
doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", []domain.DocumentBlock{{ID: "todo1", Type: "todo", Content: "Anrufen"}}, nil, false)
task, err := s.CreateTaskFromDocBlock(u.ID, doc.ID, "todo1", "Anrufen", false)
if err != nil {
t.Fatal(err)
}
got, _ := s.GetDocumentByID(u.ID, doc.ID)
if got.Blocks[0].TaskID != task.ID {
t.Fatalf("block must carry its task ID after loading, got %q", got.Blocks[0].TaskID)
}
a := collab.Actor{UserID: u.ID, ClientID: "tab"}
yes := true
if _, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "chk", Kind: collab.OpSetProps, BlockID: "todo1", Patch: &collab.PropsPatch{Checked: &yes}}}, nil); err != nil {
t.Fatal(err)
}
if changed := s.ReconcileDocTaskLinks(doc.ID); len(changed) != 1 {
t.Fatalf("ticking the block must complete the task, changed=%v", changed)
}
if th, _ := s.GetThoughtByID(u.ID, task.ID); !th.IsCompleted {
t.Fatal("task not completed")
}
if _, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "del", Kind: collab.OpDelete, BlockID: "todo1"}}, nil); err != nil {
t.Fatal(err)
}
s.ReconcileDocTaskLinks(doc.ID)
th, err := s.GetThoughtByID(u.ID, task.ID)
if err != nil {
t.Fatal("deleting the block must not delete the task")
}
if th.ConvertedType == "document_block" {
t.Fatal("deleting the block must unlink the task")
}
}
func TestFixedBlockIDsWorkInSeveralNotes(t *testing.T) {
s, u := newSyncTestStore(t)
other, _ := s.CreateUser("second@example.com", "hash", "Second", "member")
blocks := flatBlocks([3]interface{}{"blk_1", 0, "Willkommen"}, [3]interface{}{"blk_2", 1, "Unterpunkt"})
first, _ := s.CreateDocument(u.ID, "", "Start", "📄", "", blocks, nil, false)
second, _ := s.CreateDocument(other.ID, "", "Start", "📄", "", blocks, nil, false)
a, _ := s.GetDocumentByID(u.ID, first.ID)
b, _ := s.GetDocumentByID(other.ID, second.ID)
if len(a.Blocks) != 2 || len(b.Blocks) != 2 || b.Blocks[0].Content != "Willkommen" || b.Blocks[1].Level != 1 {
t.Fatalf("second note must get its own copies of the blocks: %+v", b.Blocks)
}
if a.Blocks[0].ID == b.Blocks[0].ID {
t.Fatal("block IDs must differ between notes")
}
}
func backdate(t *testing.T, s *Store, docID string, age time.Duration) {
t.Helper()
old := time.Now().Add(-age).UTC().Format(time.RFC3339)
for _, q := range []string{
`UPDATE doc_ops SET created_at = ? WHERE doc_id = ?`,
`UPDATE doc_op_results SET created_at = ? WHERE doc_id = ?`,
`UPDATE blocks SET deleted_at = ? WHERE document_id = ? AND deleted = 1`,
} {
if _, err := s.db.Exec(q, old, docID); err != nil {
t.Fatal(err)
}
}
}
func TestCompactionKeepsStateAndForcesResync(t *testing.T) {
s, u := newSyncTestStore(t)
doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", flatBlocks(
[3]interface{}{"a", 0, "Bleibt"},
[3]interface{}{"b", 0, "Gelöscht"},
[3]interface{}{"c", 1, "Kind von b"},
), nil, false)
a := collab.Actor{UserID: u.ID, ClientID: "tab1"}
if res, _, _ := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "del-b", Kind: collab.OpDelete, BlockID: "b"}}, nil); res[0].Status != collab.StatusApplied {
t.Fatalf("delete: %+v", res)
}
backdate(t, s, doc.ID, 40*24*time.Hour)
// Idempotency results stay longer (90 days): only the very old one goes
s.db.Exec(`UPDATE doc_op_results SET created_at = ? WHERE op_id = 'del-b'`, time.Now().Add(-100*24*time.Hour).UTC().Format(time.RFC3339))
// A client that is up to date before compaction must only need the remaining log afterwards
snap, _ := s.GetDocSnapshot(doc.ID)
client := collab.NewDoc()
for i := range snap.Blocks {
b := snap.Blocks[i]
client.Blocks[b.ID] = &b
}
text := "Frisch"
if _, _, err := s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "fresh", Kind: collab.OpSetContent, BlockID: "a", BaseVersion: 1, Content: &text}}, nil); err != nil {
t.Fatal(err)
}
st, err := s.CompactDocs(time.Now())
if err != nil {
t.Fatal(err)
}
if st.Tombstones != 2 || st.LogEntries == 0 || st.OpResults != 1 || st.Purged[doc.ID] == 0 {
t.Fatalf("unexpected stats: %+v", st)
}
var rows int
_ = s.db.QueryRow(`SELECT COUNT(*) FROM blocks WHERE document_id = ? AND id IN ('b', 'c')`, doc.ID).Scan(&rows)
if rows != 0 {
t.Fatalf("tombstones must be removed from the table, %d left", rows)
}
// Behind the floor: reload needed
if _, _, err := s.GetDocOpsSince(doc.ID, 0, 100); !errors.Is(err, ErrLogCompacted) {
t.Fatalf("client behind the floor must be told to resync, got %v", err)
}
entries, head, err := s.GetDocOpsSince(doc.ID, snap.HeadSeq, 100)
if err != nil || len(entries) != 2 || entries[1].Effect.Kind != collab.OpPurge {
t.Fatalf("fresh edit and purge must remain: %+v %v", entries, err)
}
for _, e := range entries {
client.ApplyEffect(e.Effect)
}
final, _ := s.GetDocSnapshot(doc.ID)
if final.HeadSeq != head || len(final.Blocks) != len(client.Blocks) {
t.Fatalf("client after log (%d blocks) differs from snapshot (%d blocks)", len(client.Blocks), len(final.Blocks))
}
for _, b := range final.Blocks {
if cb := client.Blocks[b.ID]; cb == nil || *cb != b {
t.Fatalf("block %s differs: client %+v, server %+v", b.ID, cb, b)
}
}
got, _ := s.GetDocumentByID(u.ID, doc.ID)
sameOutline(t, got.Blocks, flatBlocks([3]interface{}{"a", 0, "Frisch"}))
// A second run has nothing left to do
if st, _ := s.CompactDocs(time.Now()); st.Docs != 0 || st.OpResults != 0 {
t.Fatalf("second run changed something: %+v", st)
}
}
func TestCompactionRetentionDependsOnSharing(t *testing.T) {
s, u := newSyncTestStore(t)
other, _ := s.CreateUser("ben@example.com", "hash", "Ben", "member")
private, _ := s.CreateDocument(u.ID, "", "Privat", "📄", "", flatBlocks([3]interface{}{"p", 0, "x"}), nil, false)
shared, _ := s.CreateDocument(u.ID, "", "Geteilt", "📄", "", flatBlocks([3]interface{}{"s", 0, "x"}), nil, false)
if _, err := s.CreateDocShare(u.ID, shared.ID, other.Email, "write"); err != nil {
t.Fatal(err)
}
backdate(t, s, private.ID, 10*24*time.Hour)
backdate(t, s, shared.ID, 10*24*time.Hour)
if _, err := s.CompactDocs(time.Now()); err != nil {
t.Fatal(err)
}
if _, _, err := s.GetDocOpsSince(private.ID, 0, 10); !errors.Is(err, ErrLogCompacted) {
t.Fatalf("private note: 10 days old log must be compacted, got %v", err)
}
if entries, _, err := s.GetDocOpsSince(shared.ID, 0, 10); err != nil || len(entries) == 0 {
t.Fatalf("shared note: log must be kept for 30 days: %d entries, %v", len(entries), err)
}
}
func TestDeletedAtFollowsDeleteAndRestore(t *testing.T) {
s, u := newSyncTestStore(t)
doc, _ := s.CreateDocument(u.ID, "", "Notiz", "📄", "", flatBlocks([3]interface{}{"a", 0, "x"}, [3]interface{}{"b", 0, "y"}), nil, false)
a := collab.Actor{UserID: u.ID, ClientID: "tab1"}
deletedAt := func() string {
var v string
_ = s.db.QueryRow(`SELECT deleted_at FROM blocks WHERE id = 'b'`).Scan(&v)
return v
}
s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "1", Kind: collab.OpDelete, BlockID: "b"}}, nil)
first := deletedAt()
if first == "" {
t.Fatal("deleting must set deleted_at")
}
// Other changes to the tombstone (none possible here) must not restart the clock; a restore clears it
text := "zurück"
s.ApplyDocOps(doc.ID, a, []collab.Op{{OpID: "2", Kind: collab.OpSetContent, BlockID: "b", BaseVersion: 1, Content: &text}}, nil)
if deletedAt() != "" {
t.Fatal("restoring must clear deleted_at")
}
}
func TestDemoAccountsGetSampleDataAndShares(t *testing.T) {
s, _ := newSyncTestStore(t)
if err := s.EnsureDemoAccounts(); err != nil {
t.Fatal(err)
}
admin, _ := s.GetUserByLogin("admin")
member, _ := s.GetUserByLogin("user")
var docID string
if err := s.db.QueryRow(`SELECT id FROM documents WHERE user_id = ? AND title LIKE 'Homelab 2.0%'`, admin.ID).Scan(&docID); err != nil {
t.Fatalf("sample note missing: %v", err)
}
doc, err := s.GetDocumentByID(admin.ID, docID)
if err != nil || len(doc.Blocks) < 20 || doc.FolderPath != "Projekte" {
t.Fatalf("sample note too small or misplaced: %d blocks, folder %q, %v", len(doc.Blocks), doc.FolderPath, err)
}
maxLevel := 0
for _, b := range doc.Blocks {
if b.Level > maxLevel {
maxLevel = b.Level
}
}
if maxLevel < 2 {
t.Fatalf("sample note must have nested bullets, max level %d", maxLevel)
}
if role := s.DocRole(member.ID, docID); role != DocRoleEditor {
t.Fatalf("demo user must be editor of the sample note, got %q", role)
}
var tasks, inbox int
_ = s.db.QueryRow(`SELECT COUNT(*) FROM thoughts WHERE user_id = ? AND converted_type = 'task'`, admin.ID).Scan(&tasks)
_ = s.db.QueryRow(`SELECT COUNT(*) FROM thoughts WHERE user_id = ? AND status = 'inbox'`, admin.ID).Scan(&inbox)
if tasks < 8 || inbox < 5 {
t.Fatalf("expected tasks and inbox thoughts, got %d tasks, %d inbox", tasks, inbox)
}
// Running it again (container restart) must not seed twice
if err := s.EnsureDemoAccounts(); err != nil {
t.Fatal(err)
}
var docs int
_ = s.db.QueryRow(`SELECT COUNT(*) FROM documents WHERE user_id = ?`, admin.ID).Scan(&docs)
if docs != 3 {
t.Fatalf("restart seeded again: %d notes", docs)
}
}
+210
View File
@@ -0,0 +1,210 @@
package storage
import (
"time"
"github.com/sandrodach/mindsift/internal/domain"
)
// seedBlock describes one outliner bullet of the sample notes: nesting level, block type and text.
type seedBlock struct {
level int
typ string
text string
checked bool
lang string
}
func seedBlocks(spec []seedBlock) []domain.DocumentBlock {
out := make([]domain.DocumentBlock, len(spec))
for i, b := range spec {
typ := b.typ
if typ == "" {
typ = "bullet"
}
out[i] = domain.DocumentBlock{ID: domain.NewID("blk"), Type: typ, Content: b.text, Checked: b.checked, Lang: b.lang, Level: b.level}
}
return out
}
type seedTask struct {
content string
priority string
due int // days from today; 0 = no due date unless dueToday
dueToday bool
status string // kanban column
notes string
subtasks []string
doneSubs int
}
// SeedStarterContent fills a new account with realistic sample data (optional at setup, always in demo mode):
// inbox thoughts, two task lists with Kanban states, shopping and packing lists and a few linked notes.
func (s *Store) SeedStarterContent(user *domain.User) error {
day := func(offset int) string { return time.Now().AddDate(0, 0, offset).Format("2006-01-02") }
inbox := []struct{ content, priority, source string }{
{"Zündkerzen und Luftfilter für den Rasenmäher bestellen #garage #wartung", "normal", "manual"},
{"Urlaubsidee: Wanderhütte in Südtirol für Oktober buchen #urlaub #familie", "high", "voice"},
{"Kellerfeuchtigkeit über 68 % gestiegen – Entfeuchter prüfen #smarthome #alarm", "urgent", "webhook"},
{"Buchtipp von Jonas: „Die Kunst des klaren Denkens“ #lesen", "low", "manual"},
{"Geburtstagsgeschenk für Oma – vielleicht ein Fotobuch vom Sommer? #familie", "normal", "manual"},
{"Idee: Backup-Report jeden Montag per Discord schicken #homelab", "normal", "voice"},
}
for _, t := range inbox {
_, _ = s.CreateThought(user.ID, t.content, t.priority, t.source, "")
}
addTasks := func(listID string, tasks []seedTask) {
for _, t := range tasks {
due := ""
if t.due != 0 || t.dueToday {
due = day(t.due)
}
th, err := s.CreateThoughtWithDetails(user.ID, t.content, t.priority, "manual", due, listID, t.status, "")
if err != nil {
continue
}
if _, err := s.UpdateThoughtTriage(user.ID, th.ID, "keep", "task", "", due, listID); err != nil {
continue
}
subs := make([]domain.SubTask, len(t.subtasks))
for i, title := range t.subtasks {
subs[i] = domain.SubTask{ID: domain.NewID("sub"), Title: title, IsCompleted: i < t.doneSubs, CreatedAt: domain.NowISO()}
}
done := t.status == "done"
status, notes := t.status, t.notes
_, _ = s.UpdateThoughtDetails(user.ID, th.ID, nil, &done, nil, nil, nil, &status, &notes, &subs, nil)
}
}
homelab, err := s.CreateTaskList(user.ID, "", "", "Homelab", "🖥️", "#2563eb")
if err == nil {
addTasks(homelab.ID, []seedTask{
{content: "MindSift-Container auf den neuen Server umziehen #docker", priority: "high", due: 2, status: "in_progress",
notes: "Volume `./data` vorher mit `sqlite3 .backup` sichern.",
subtasks: []string{"Backup der Datenbank ziehen", "docker-compose.yml anpassen", "DNS auf neue IP umstellen", "Alten Container stoppen"}, doneSubs: 2},
{content: "Reverse-Proxy: Nginx-Timeouts für Long-Polling erhöhen #nginx", priority: "normal", due: 5, status: "todo"},
{content: "USV-Batterie tauschen lassen #hardware", priority: "urgent", due: -1, status: "todo",
notes: "Händler hat das Ersatzteil seit Montag da."},
{content: "Wartet auf Lieferung: zweite SSD für das RAID #hardware", priority: "normal", due: 7, status: "waiting"},
{content: "Gitea-Runner auf Version 0.2 aktualisieren #devops", priority: "low", status: "done"},
{content: "Uptime-Monitoring für alle Dienste einrichten #monitoring", priority: "normal", due: 10, status: "todo",
subtasks: []string{"Uptime Kuma installieren", "Benachrichtigung per Discord", "Statusseite veröffentlichen"}, doneSubs: 1},
})
}
house, err := s.CreateTaskList(user.ID, "", "", "Haus & Garten", "🏡", "#16a34a")
if err == nil {
addTasks(house.ID, []seedTask{
{content: "Hecke schneiden, bevor die Vögel brüten #garten", priority: "normal", due: 3, status: "todo"},
{content: "Heizungswartung beim Installateur anfragen #wartung", priority: "high", dueToday: true, status: "in_progress"},
{content: "Steuerunterlagen für 2026 sortieren #finanzen", priority: "normal", due: 14, status: "todo",
subtasks: []string{"Handwerkerrechnungen", "Spendenquittungen", "Fahrtkosten"}},
{content: "Fahrradschloss nachkaufen", priority: "low", status: "done"},
})
}
shop, err := s.CreateList(user.ID, "Wocheneinkauf", "🛒", "shopping")
if err == nil {
items := []struct{ content, category, qty string }{
{"Hafermilch (Barista Edition)", "Kühlregal", "3x"},
{"Griechischer Joghurt", "Kühlregal", "500 g"},
{"Bergkäse", "Kühlregal", "200 g"},
{"Äpfel (Elstar)", "Obst & Gemüse", "1 kg"},
{"Cherrytomaten", "Obst & Gemüse", "2 Schalen"},
{"Avocados", "Obst & Gemüse", "3x"},
{"Bauernbrot", "Backwaren", "1x"},
{"Espressobohnen (100 % Arabica)", "Kaffee & Tee", "1 kg"},
{"Pfefferminztee", "Kaffee & Tee", "1x"},
{"Spülmaschinentabs", "Drogerie", "1 Pack"},
}
for i, it := range items {
item, err := s.CreateListItem(shop.ID, it.content, it.category, it.qty, i)
if err == nil && (i == 1 || i == 6 || i == 8) {
_, _ = s.ToggleListItem(item.ID, true, user.ID, user.DisplayName)
}
}
}
packing, err := s.CreateList(user.ID, "Packliste Südtirol", "🎒", "shopping")
if err == nil {
for i, it := range []struct{ content, category string }{
{"Wanderschuhe", "Kleidung"}, {"Regenjacke", "Kleidung"}, {"Fleecepullover", "Kleidung"},
{"Erste-Hilfe-Set", "Ausrüstung"}, {"Stirnlampe", "Ausrüstung"}, {"Trinkflaschen", "Ausrüstung"},
{"Personalausweise", "Dokumente"}, {"Hüttenschlafsack", "Ausrüstung"},
} {
_, _ = s.CreateListItem(packing.ID, it.content, it.category, "", i)
}
}
_, _ = s.CreateFolder(user.ID, "Projekte")
_, _ = s.CreateFolder(user.ID, "Privat")
_, _ = s.CreateDocument(user.ID, "", "Homelab 2.0 – Umzug auf den neuen Server", "🚀", "Projekte", seedBlocks([]seedBlock{
{typ: "callout", text: "Ziel: Bis Ende des Monats laufen alle Dienste auf dem neuen Mini-PC – leiser, sparsamer und mit echtem Backup. Aufgaben dazu stehen in der Liste #homelab."},
{typ: "h2", text: "Warum überhaupt?"},
{level: 1, text: "Der alte Server zieht 65 W im Leerlauf, der neue nur 9 W"},
{level: 2, text: "Ersparnis ca. 120 € im Jahr bei aktuellem Strompreis"},
{level: 1, text: "Backups liefen bisher nur manuell – das soll sich ändern #backup"},
{level: 1, text: "Endlich Platz für die Dienste, die seit Monaten auf der Wunschliste stehen"},
{typ: "h2", text: "Checkliste Umzug"},
{level: 1, typ: "todo", text: "Vorbereitung"},
{level: 2, typ: "todo", text: "Proxmox installieren und Updates einspielen", checked: true},
{level: 2, typ: "todo", text: "Netzwerk: feste IP und VLAN für Dienste", checked: true},
{level: 2, typ: "todo", text: "ZFS-Pool mit zwei SSDs anlegen"},
{level: 1, typ: "todo", text: "Dienste umziehen"},
{level: 2, typ: "todo", text: "MindSift (Daten liegen komplett in einer SQLite-Datei)", checked: true},
{level: 2, typ: "todo", text: "Gitea mit Runner und Container-Registry"},
{level: 2, typ: "todo", text: "Home Assistant inklusive Zigbee-Stick"},
{level: 1, typ: "todo", text: "Alten Server sicher löschen und verkaufen"},
{typ: "h2", text: "Dienste & Ports"},
{level: 1, text: "**MindSift** – Notizen, Aufgaben und Listen für die ganze Familie #mindsift"},
{level: 2, text: "läuft hinter Nginx, WebSocket-Upgrade und Long-Polling sind freigegeben"},
{level: 2, text: "Demo-Konten nur im Testsystem, nie öffentlich"},
{level: 1, text: "**Gitea** – Code, CI und Images, z. B. `git.example.com` #devops"},
{level: 1, text: "**Home Assistant** – Sensoren melden Alarme per Webhook direkt in die MindSift-Inbox #smarthome"},
{level: 2, text: "Beispiel: Kellerfeuchtigkeit über 65 % → Gedanke mit Priorität „dringend“"},
{typ: "h3", text: "Compose-Ausschnitt"},
{level: 1, typ: "code", lang: "yaml", text: "services:\n mindsift:\n image: git.example.com/sandrodach/mindsift:v0.2\n restart: unless-stopped\n volumes: [\"./data:/data\"]\n environment:\n - JWT_SECRET=${JWT_SECRET}"},
{typ: "h2", text: "Offene Fragen"},
{level: 1, text: "Reicht eine USV für Router und Server zusammen?"},
{level: 1, text: "Offsite-Backup: Hetzner Storage Box oder Festplatte bei den Eltern? Siehe [[Ideen-Sammlung]]"},
}), []string{"homelab", "projekt"}, true)
_, _ = s.CreateDocument(user.ID, "", "Urlaub Südtirol", "🏔️", "Privat", seedBlocks([]seedBlock{
{typ: "h2", text: "Reiseplan Oktober"},
{level: 1, text: "Anreise Samstag, früh los wegen Brenner-Stau"},
{level: 1, text: "Hütte bei Villnöß – Bestätigung kommt per Mail"},
{level: 2, text: "Halbpension, Hunde erlaubt"},
{typ: "h2", text: "Touren-Ideen"},
{level: 1, typ: "todo", text: "Adolf-Munkel-Weg unter den Geislerspitzen", checked: true},
{level: 1, typ: "todo", text: "Seceda mit der Seilbahn, oben Höhenweg"},
{level: 1, typ: "todo", text: "Regentag: Ötzi-Museum in Bozen #familie"},
{typ: "callout", text: "Packliste liegt unter Listen → Packliste Südtirol."},
}), []string{"urlaub"}, false)
_, _ = s.CreateDocument(user.ID, "", "Ideen-Sammlung", "💡", "", seedBlocks([]seedBlock{
{text: "Offsite-Backup verschlüsselt mit restic #backup"},
{level: 1, text: "Vorteil: Snapshots, Deduplizierung, läuft auf jedem Ziel"},
{text: "Familien-Wiki statt WhatsApp-Gruppe für Rezepte und Termine #familie"},
{text: "Wochenrückblick jeden Sonntag: Inbox leeren, Aufgaben für die Woche planen #routine"},
}), []string{"ideen"}, false)
return nil
}
// seedDemoShares shares part of the admin's sample data with the demo user, so both demo accounts can try
// out editing a note, a task list and a shopping list together.
func (s *Store) seedDemoShares(admin, member *domain.User) {
var docID, taskListID, listID string
_ = s.db.QueryRow(`SELECT id FROM documents WHERE user_id = ? AND title LIKE 'Homelab 2.0%'`, admin.ID).Scan(&docID)
_ = s.db.QueryRow(`SELECT id FROM task_lists WHERE user_id = ? AND title = 'Homelab'`, admin.ID).Scan(&taskListID)
_ = s.db.QueryRow(`SELECT id FROM lists WHERE user_id = ? AND title = 'Wocheneinkauf'`, admin.ID).Scan(&listID)
if docID != "" {
_, _ = s.CreateDocShare(admin.ID, docID, member.ID, "write")
}
if taskListID != "" {
_, _ = s.AddTaskListShare(taskListID, "user", member.ID, "editor", admin.ID)
}
if listID != "" {
_, _ = s.AddListShare(admin.ID, listID, "user", member.ID, "editor")
}
}
+108 -493
View File
@@ -415,11 +415,6 @@ func (s *Store) migrate() error {
// Backfill username for existing accounts where username is empty // Backfill username for existing accounts where username is empty
_, _ = s.db.Exec(`UPDATE users SET username = LOWER(SUBSTR(email, 1, INSTR(email, '@') - 1)) WHERE (username IS NULL OR username = '') AND INSTR(email, '@') > 0`) _, _ = s.db.Exec(`UPDATE users SET username = LOWER(SUBSTR(email, 1, INSTR(email, '@') - 1)) WHERE (username IS NULL OR username = '') AND INSTR(email, '@') > 0`)
// Ensure all existing document & folder shares are strictly read-only
_, _ = s.db.Exec(`UPDATE doc_shares SET permission = 'read' WHERE permission != 'read'`)
_, _ = s.db.Exec(`UPDATE doc_team_shares SET permission = 'read' WHERE permission != 'read'`)
_, _ = s.db.Exec(`UPDATE workspace_folder_shares SET permission = 'read' WHERE permission != 'read'`)
// Default system settings (public registration disabled by default; invite or admin creation required) // Default system settings (public registration disabled by default; invite or admin creation required)
_, _ = s.db.Exec(`INSERT OR IGNORE INTO system_settings (key, value) VALUES _, _ = s.db.Exec(`INSERT OR IGNORE INTO system_settings (key, value) VALUES
('sharing_mode', 'teams_only'), ('sharing_mode', 'teams_only'),
@@ -463,6 +458,9 @@ func (s *Store) migrate() error {
_, _ = s.db.Exec(`UPDATE team_members SET role = 'member' WHERE role NOT IN ('admin', 'member')`) _, _ = s.db.Exec(`UPDATE team_members SET role = 'member' WHERE role NOT IN ('admin', 'member')`)
_, _ = s.db.Exec(`UPDATE users SET role = 'member' WHERE role = 'team_admin'`) _, _ = s.db.Exec(`UPDATE users SET role = 'member' WHERE role = 'team_admin'`)
// Block tree, operation log and idempotency tables for collaborative editing
s.migrateDocSync()
// Migrate workspace documents, blocks, folders, and strip legacy folder:* tags into pure DB tables // Migrate workspace documents, blocks, folders, and strip legacy folder:* tags into pure DB tables
s.migrateWorkspaceToPureDB() s.migrateWorkspaceToPureDB()
@@ -476,58 +474,6 @@ func (s *Store) migrate() error {
return nil return nil
} }
// SeedStarterContent populates realistic initial content for a newly created user if requested
func (s *Store) SeedStarterContent(user *domain.User) error {
now := domain.NowISO()
starterThoughts := []struct {
content string
priority string
source string
dueDate string
}{
{
content: "Zündkerzen und Luftfilter für den Rasenmäher bestellen #garage #wartung",
priority: "normal",
source: "manual",
},
{
content: "Urlaubsidee: Wanderhütte in Südtirol für Oktober buchen #urlaub #familie",
priority: "high",
source: "voice",
},
{
content: "Kellerfeuchtigkeit über 68% gestiegen – Entfeuchter prüfen #smarthome #alarm",
priority: "urgent",
source: "webhook",
},
}
for _, st := range starterThoughts {
_, _ = s.CreateThought(user.ID, st.content, st.priority, st.source, st.dueDate)
}
keptTask, err := s.CreateThought(user.ID, "Gitea CI/CD Pipeline für MindSift Container testen #devops #docker", "high", "manual", now[:10])
if err == nil && keptTask != nil {
_, _ = s.UpdateThoughtTriage(user.ID, keptTask.ID, "keep", "task", "", now[:10], "")
}
shopList, err := s.CreateList(user.ID, "Wocheneinkauf Supermarkt", "🛒", "shopping")
if err == nil && shopList != nil {
_, _ = s.CreateListItem(shopList.ID, "Hafermilch (Barista Edition)", "Kühlregal", "3x", 0)
_, _ = s.CreateListItem(shopList.ID, "Espressobohnen (100% Arabica)", "Kaffee & Tee", "1 kg", 1)
}
starterBlocks := []domain.DocumentBlock{
{ID: "blk_1", Type: "callout", Content: "Willkommen in deinem MindSift Workspace! Hier wachsen flüchtige Gedanken aus deiner Inbox zu strukturierten Projekten heran."},
{ID: "blk_2", Type: "h2", Content: "Projekt-Checkliste"},
{ID: "blk_3", Type: "todo", Content: "MindSift als Docker-Container auf eigenem Server starten", Checked: true},
}
_, _ = s.CreateDocument(user.ID, "", "MindSift Architektur & Notizen", "🚀", "", starterBlocks, []string{"mindsift", "workflow"}, true)
return nil
}
// EnsureDefaultUser is retained strictly for unit test convenience // EnsureDefaultUser is retained strictly for unit test convenience
func (s *Store) EnsureDefaultUser() (*domain.User, error) { func (s *Store) EnsureDefaultUser() (*domain.User, error) {
s.initMu.Lock() s.initMu.Lock()
@@ -681,13 +627,14 @@ func (s *Store) EnsureDemoAccounts() error {
} }
// 1. Admin user (admin / admin) // 1. Admin user (admin / admin)
adminCreated := false
adminUser, err := s.GetUserByLogin("admin") adminUser, err := s.GetUserByLogin("admin")
if err != nil || adminUser == nil { if err != nil || adminUser == nil {
adminUser, err = s.CreateUserWithUsername("admin@mindsift.local", "admin", adminHash, "Admin", "admin") adminUser, err = s.CreateUserWithUsername("admin@mindsift.local", "admin", adminHash, "Admin", "admin")
if err != nil { if err != nil {
return fmt.Errorf("failed to create demo admin: %w", err) return fmt.Errorf("failed to create demo admin: %w", err)
} }
_ = s.SeedStarterContent(adminUser) adminCreated = true
} else { } else {
_ = s.UpdateUserPassword(adminUser.ID, adminHash) _ = s.UpdateUserPassword(adminUser.ID, adminHash)
_, _ = s.db.Exec(`UPDATE users SET is_active = 1, role = 'admin' WHERE id = ?`, adminUser.ID) _, _ = s.db.Exec(`UPDATE users SET is_active = 1, role = 'admin' WHERE id = ?`, adminUser.ID)
@@ -696,7 +643,7 @@ func (s *Store) EnsureDemoAccounts() error {
// 2. Regular user (user / user) // 2. Regular user (user / user)
normalUser, err := s.GetUserByLogin("user") normalUser, err := s.GetUserByLogin("user")
if err != nil || normalUser == nil { if err != nil || normalUser == nil {
_, err = s.CreateUserWithUsername("user@mindsift.local", "user", userHash, "User", "member") normalUser, err = s.CreateUserWithUsername("user@mindsift.local", "user", userHash, "User", "member")
if err != nil { if err != nil {
return fmt.Errorf("failed to create demo user: %w", err) return fmt.Errorf("failed to create demo user: %w", err)
} }
@@ -705,6 +652,12 @@ func (s *Store) EnsureDemoAccounts() error {
_, _ = s.db.Exec(`UPDATE users SET is_active = 1 WHERE id = ?`, normalUser.ID) _, _ = s.db.Exec(`UPDATE users SET is_active = 1 WHERE id = ?`, normalUser.ID)
} }
// 3. Sample data for the admin, partly shared with the demo user (only on first start)
if adminCreated {
_ = s.SeedStarterContent(adminUser)
s.seedDemoShares(adminUser, normalUser)
}
return nil return nil
} }
@@ -1449,316 +1402,30 @@ func (s *Store) migrateWorkspaceToPureDB() {
} }
func (s *Store) loadDocBlocks(docID string, fallbackJSON string) []domain.DocumentBlock { func (s *Store) loadDocBlocks(docID string, fallbackJSON string) []domain.DocumentBlock {
rows, err := s.db.Query(` if d, _, err := loadCollabDoc(s.db, docID); err == nil && len(d.Blocks) > 0 {
SELECT id, document_id, type, content, level, position, checked, collapsed, lang blocks := flatDocumentBlocks(docID, d)
FROM blocks if links := s.docTaskLinks(docID); len(links) > 0 {
WHERE document_id = ? for i := range blocks {
ORDER BY position ASC, rowid ASC blocks[i].TaskID = links[blocks[i].ID]
`, docID)
if err == nil {
blocks := make([]domain.DocumentBlock, 0)
for rows.Next() {
var b domain.DocumentBlock
var chk, col int
if scanErr := rows.Scan(&b.ID, &b.DocumentID, &b.Type, &b.Content, &b.Level, &b.Position, &chk, &col, &b.Lang); scanErr == nil {
b.Checked = chk != 0
b.Collapsed = col != 0
blocks = append(blocks, b)
} }
} }
_ = rows.Close()
if len(blocks) > 0 {
return blocks return blocks
} }
}
// Very old notes only stored blocks_json; convert them once
blocks := make([]domain.DocumentBlock, 0) blocks := make([]domain.DocumentBlock, 0)
if fallbackJSON != "" && fallbackJSON != "[]" { if fallbackJSON != "" && fallbackJSON != "[]" {
_ = json.Unmarshal([]byte(fallbackJSON), &blocks) _ = json.Unmarshal([]byte(fallbackJSON), &blocks)
if len(blocks) > 0 { if len(blocks) > 0 {
s.replaceDocBlocksInDB(docID, "", blocks, domain.NowISO()) s.replaceDocBlocksInDB(docID, "", blocks, domain.NowISO())
if d, _, err := loadCollabDoc(s.db, docID); err == nil {
return flatDocumentBlocks(docID, d)
}
} }
} }
return blocks return blocks
} }
func (s *Store) replaceDocBlocksInDB(docID, _ string, blocks []domain.DocumentBlock, now string) {
_, _ = s.db.Exec(`DELETE FROM blocks WHERE document_id = ?`, docID)
if len(blocks) > 0 {
stmt, err := s.db.Prepare(`
INSERT INTO blocks (id, document_id, content, level, position, type, checked, collapsed, lang)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
document_id = excluded.document_id,
content = excluded.content,
level = excluded.level,
position = excluded.position,
type = excluded.type,
checked = excluded.checked,
collapsed = excluded.collapsed,
lang = excluded.lang
`)
if err == nil {
defer stmt.Close()
for i, b := range blocks {
bID := strings.TrimSpace(b.ID)
if bID == "" {
bID = domain.NewID("blk")
}
bType := b.Type
if bType == "" {
bType = "bullet"
}
chk := 0
if b.Checked {
chk = 1
}
col := 0
if b.Collapsed {
col = 1
}
pos := b.Position
if pos <= 0 {
pos = float64((i + 1) * 1000)
}
_, _ = stmt.Exec(bID, docID, b.Content, b.Level, pos, bType, chk, col, b.Lang)
}
}
}
// Update markdown_cache on documents
mdCache := domain.CompileBlocksToMarkdown(blocks)
_, _ = s.db.Exec(`UPDATE documents SET markdown_cache = ?, updated_at = ? WHERE id = ?`, mdCache, now, docID)
}
func (s *Store) LoadDocumentState(docID string) (string, []domain.DocumentBlock, error) {
var title, blocksJSON string
err := s.db.QueryRow(`SELECT title, blocks_json FROM documents WHERE id = ?`, docID).Scan(&title, &blocksJSON)
if err != nil {
return "", nil, err
}
blocks := s.loadDocBlocks(docID, blocksJSON)
return title, blocks, nil
}
// FlushDocumentState persists the in-memory blocks into `blocks`, extracts all #tags into `block_tags`,
// and compiles the hierarchical Markdown into `documents.markdown_cache`.
func (s *Store) FlushDocumentState(docID string, title string, blocks []domain.DocumentBlock) error {
now := domain.NowISO()
// Normalize positions sequentially
normalized := make([]domain.DocumentBlock, len(blocks))
var plainBuilder strings.Builder
for i, b := range blocks {
if strings.TrimSpace(b.ID) == "" {
b.ID = domain.NewID("blk")
}
if b.Type == "" {
b.Type = "bullet"
}
b.DocumentID = docID
b.Position = float64((i + 1) * 1000)
normalized[i] = b
if b.Content != "" {
plainBuilder.WriteString(b.Content)
plainBuilder.WriteString("\n")
}
}
s.replaceDocBlocksInDB(docID, "", normalized, now)
_ = s.indexDocBlockTags(docID, "", normalized)
mdCache := domain.CompileBlocksToMarkdown(normalized)
plainContent := plainBuilder.String()
blocksJSON, _ := json.Marshal(normalized)
if strings.TrimSpace(title) != "" {
_, err := s.db.Exec(`
UPDATE documents
SET title = ?, markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ?
WHERE id = ?
`, title, mdCache, plainContent, string(blocksJSON), now, docID)
return err
}
_, err := s.db.Exec(`
UPDATE documents
SET markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ?
WHERE id = ?
`, mdCache, plainContent, string(blocksJSON), now, docID)
return err
}
// PatchDocumentBlocks applies atomic block-level upserts, deletions, and order reconciliation without overwriting concurrent edits from other sessions.
func (s *Store) PatchDocumentBlocks(userID, docID string, req domain.DocBlocksPatchRequest) (*domain.Document, []string, error) {
doc, err := s.GetDocumentByID(userID, docID)
if err != nil {
return nil, nil, err
}
if !doc.IsOwner {
return nil, nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)")
}
now := domain.NowISO()
// Ensure blocks table is seeded if previously only in blocks_json
var existingCount int
_ = s.db.QueryRow(`SELECT COUNT(*) FROM blocks WHERE document_id = ?`, docID).Scan(&existingCount)
if existingCount == 0 && len(doc.Blocks) > 0 {
s.replaceDocBlocksInDB(docID, doc.UserID, doc.Blocks, doc.UpdatedAt)
}
// Read current ordered block IDs from DB before patch
var existingOrder []string
ordRows, err := s.db.Query(`SELECT id FROM blocks WHERE document_id = ? ORDER BY position ASC, rowid ASC`, docID)
if err == nil {
for ordRows.Next() {
var bid string
if err := ordRows.Scan(&bid); err == nil {
existingOrder = append(existingOrder, bid)
}
}
_ = ordRows.Close()
}
// 1. Delete explicitly deleted block IDs
deletedSet := make(map[string]bool, len(req.DeletedIDs))
for _, delID := range req.DeletedIDs {
delID = strings.TrimSpace(delID)
if delID == "" {
continue
}
deletedSet[delID] = true
_, _ = s.db.Exec(`DELETE FROM blocks WHERE document_id = ? AND id = ?`, docID, delID)
}
// Filter existingOrder to exclude deleted blocks
survivingExisting := make([]string, 0, len(existingOrder))
survivingSet := make(map[string]bool, len(existingOrder))
for _, bid := range existingOrder {
if !deletedSet[bid] {
survivingExisting = append(survivingExisting, bid)
survivingSet[bid] = true
}
}
// 2. Upsert modified or newly created blocks
upsertedIDs := make([]string, 0, len(req.Upserted))
for idx, b := range req.Upserted {
bID := strings.TrimSpace(b.ID)
if bID == "" || deletedSet[bID] {
continue
}
bType := b.Type
if bType == "" {
bType = "bullet"
}
chk := 0
if b.Checked {
chk = 1
}
col := 0
if b.Collapsed {
col = 1
}
defaultSort := float64((len(survivingExisting) + idx + 1) * 1000)
_, _ = s.db.Exec(`
INSERT INTO blocks (id, document_id, content, level, position, type, checked, collapsed, lang)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
type = excluded.type,
content = excluded.content,
level = excluded.level,
checked = excluded.checked,
collapsed = excluded.collapsed,
lang = excluded.lang
`, bID, docID, b.Content, b.Level, defaultSort, bType, chk, col, b.Lang)
upsertedIDs = append(upsertedIDs, bID)
survivingSet[bID] = true
}
// 3. Reconcile block ordering without dropping blocks concurrently created by other users
var mergedOrder []string
inMerged := make(map[string]bool)
if len(req.BlockOrder) > 0 {
for _, bid := range req.BlockOrder {
if !deletedSet[bid] && survivingSet[bid] && !inMerged[bid] {
mergedOrder = append(mergedOrder, bid)
inMerged[bid] = true
}
}
for i, bid := range survivingExisting {
if inMerged[bid] {
continue
}
insertPos := 0
for j := i - 1; j >= 0; j-- {
predID := survivingExisting[j]
if inMerged[predID] {
for k, mID := range mergedOrder {
if mID == predID {
insertPos = k + 1
break
}
}
break
}
}
mergedOrder = append(mergedOrder[:insertPos], append([]string{bid}, mergedOrder[insertPos:]...)...)
inMerged[bid] = true
}
for _, bid := range upsertedIDs {
if !inMerged[bid] {
mergedOrder = append(mergedOrder, bid)
inMerged[bid] = true
}
}
} else {
for _, bid := range survivingExisting {
if !inMerged[bid] {
mergedOrder = append(mergedOrder, bid)
inMerged[bid] = true
}
}
for _, bid := range upsertedIDs {
if !inMerged[bid] {
mergedOrder = append(mergedOrder, bid)
inMerged[bid] = true
}
}
}
// Update position in blocks
for idx, bid := range mergedOrder {
_, _ = s.db.Exec(`UPDATE blocks SET position = ? WHERE document_id = ? AND id = ?`, float64((idx+1)*1000), docID, bid)
}
// Reload authoritative block list and update document snapshot
finalBlocks := s.loadDocBlocks(docID, "")
var plainBuilder strings.Builder
for _, b := range finalBlocks {
if b.Content != "" {
plainBuilder.WriteString(b.Content)
plainBuilder.WriteString("\n")
}
}
plainContent := plainBuilder.String()
mdCache := domain.CompileBlocksToMarkdown(finalBlocks)
blocksJSON, _ := json.Marshal(finalBlocks)
_, _ = s.db.Exec(`UPDATE documents SET markdown_cache = ?, content = ?, blocks_json = ?, updated_at = ? WHERE id = ?`,
mdCache, plainContent, string(blocksJSON), now, docID)
doc.Blocks = finalBlocks
doc.MarkdownCache = mdCache
doc.Content = plainContent
doc.UpdatedAt = now
_ = s.indexDocBlockTags(doc.ID, doc.UserID, finalBlocks)
return doc, mergedOrder, nil
}
// GetAuthorizedDocUsers returns all user IDs who have read/write/owner access to docID. // GetAuthorizedDocUsers returns all user IDs who have read/write/owner access to docID.
func (s *Store) GetAuthorizedDocUsers(docID string) ([]string, error) { func (s *Store) GetAuthorizedDocUsers(docID string) ([]string, error) {
userSet := make(map[string]bool) userSet := make(map[string]bool)
@@ -2334,16 +2001,16 @@ func (s *Store) GetDocumentByID(userID, docID string) (*domain.Document, error)
return d, nil return d, nil
} }
func (s *Store) UpdateDocument(userID, docID string, title, icon, folderPath *string, blocks *[]domain.DocumentBlock, tags *[]string, isPinned *bool) (*domain.Document, error) { func (s *Store) UpdateDocument(userID, docID string, title, icon, folderPath *string, isPinned *bool) (*domain.Document, error) {
doc, err := s.GetDocumentByID(userID, docID) doc, err := s.GetDocumentByID(userID, docID)
if err != nil { if err != nil {
return nil, err return nil, err
} }
isOnlyPinChange := isPinned != nil && title == nil && icon == nil && folderPath == nil && blocks == nil && tags == nil // Pins and folder placement are personal; title and icon need edit rights. The content (blocks) is
isOnlyFolderMove := folderPath != nil && title == nil && icon == nil && blocks == nil && tags == nil && isPinned == nil // only changed through the operation log (POST /docs/{id}/sync).
changesContent := title != nil || icon != nil
if !doc.IsOwner && !isOnlyPinChange && !isOnlyFolderMove { if changesContent && !CanEditDoc(s.DocRole(userID, docID)) {
return nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)") return nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)")
} }
@@ -2362,10 +2029,6 @@ func (s *Store) UpdateDocument(userID, docID string, title, icon, folderPath *st
} }
} }
if isOnlyPinChange {
return doc, nil
}
if folderPath != nil { if folderPath != nil {
movedDoc, moveErr := s.MoveDocument(userID, docID, *folderPath) movedDoc, moveErr := s.MoveDocument(userID, docID, *folderPath)
if moveErr != nil { if moveErr != nil {
@@ -2377,78 +2040,21 @@ func (s *Store) UpdateDocument(userID, docID string, title, icon, folderPath *st
doc.IsSharedFolder = movedDoc.IsSharedFolder doc.IsSharedFolder = movedDoc.IsSharedFolder
} }
if isOnlyFolderMove { if !changesContent {
return doc, nil return doc, nil
} }
if title != nil { if title != nil {
doc.Title = strings.TrimSpace(*title) doc.Title = strings.TrimSpace(*title)
s.applyLegacyTitle(docID, userID, doc.Title)
} }
if icon != nil { if icon != nil {
doc.Icon = *icon doc.Icon = *icon
} }
if tags != nil {
doc.Tags = stripFolderTags(*tags)
}
doc.UpdatedAt = domain.NowISO() doc.UpdatedAt = domain.NowISO()
if blocks != nil { if _, err := s.db.Exec(`UPDATE documents SET icon = ?, updated_at = ? WHERE id = ?`, doc.Icon, doc.UpdatedAt, docID); err != nil {
doc.Blocks = *blocks
s.replaceDocBlocksInDB(docID, userID, doc.Blocks, doc.UpdatedAt)
var plainBuilder strings.Builder
for _, b := range doc.Blocks {
if b.Content != "" {
plainBuilder.WriteString(b.Content)
plainBuilder.WriteString("\n")
}
}
doc.Content = plainBuilder.String()
}
blocksJSON, _ := json.Marshal(doc.Blocks)
tagsJSON, _ := json.Marshal(doc.Tags)
// IMPORTANT: Never overwrite documents.folder_path here! Folder placement is managed exclusively by MoveDocument.
_, err = s.db.Exec(
`UPDATE documents SET title = ?, icon = ?, content = ?, blocks_json = ?, tags = ?, updated_at = ?
WHERE id = ?`,
doc.Title, doc.Icon, doc.Content, string(blocksJSON), string(tagsJSON), doc.UpdatedAt, docID,
)
if err != nil {
return nil, err return nil, err
} }
if blocks != nil { return s.GetDocumentByID(userID, docID)
_ = s.indexDocBlockTags(doc.ID, doc.UserID, doc.Blocks)
for _, b := range doc.Blocks {
if b.TaskID != "" {
s.SyncTaskFromBlockCheck(doc.UserID, b.TaskID, b.Checked)
}
}
// When a block is deleted from the note, unlink the corresponding task in thoughts without deleting the task
remainingBlockIDs := make(map[string]bool)
for _, b := range doc.Blocks {
remainingBlockIDs[b.ID] = true
}
rows, err := s.db.Query(`SELECT id, converted_ref_id, block_id FROM thoughts WHERE (doc_id = ? OR converted_ref_id LIKE ?) AND converted_type = 'document_block'`, docID, docID+":%")
if err == nil {
for rows.Next() {
var tid, refID, bID string
if err := rows.Scan(&tid, &refID, &bID); err == nil {
bid := bID
if bid == "" && strings.Contains(refID, ":") {
parts := strings.SplitN(refID, ":", 2)
bid = parts[1]
}
if !remainingBlockIDs[bid] {
_, _ = s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '', doc_id = '', block_id = '' WHERE id = ?`, tid)
}
}
}
_ = rows.Close()
}
}
return doc, nil
} }
func (s *Store) DeleteDocument(userID, docID string) error { func (s *Store) DeleteDocument(userID, docID string) error {
@@ -3445,8 +3051,7 @@ func (s *Store) CreateFolderShare(currentUserID, folderID, targetType, targetIde
} }
} }
// Geteilte Ordner können ausschließlich mit Leserberechtigung geteilt werden (wie Dokumente) permission = normalizeSharePermission(permission)
permission = "read"
var targetID, targetName, email, avatarURL string var targetID, targetName, email, avatarURL string
if cleanType == "team" { if cleanType == "team" {
@@ -3671,8 +3276,7 @@ func (s *Store) CreateDocTeamShare(currentUserID, docID, teamID, permission stri
return nil, fmt.Errorf("team nicht gefunden") return nil, fmt.Errorf("team nicht gefunden")
} }
// Document sharing is strictly read-only permission = normalizeSharePermission(permission)
permission = "read"
now := domain.NowISO() now := domain.NowISO()
shareID := domain.NewID("share") shareID := domain.NewID("share")
@@ -3699,6 +3303,15 @@ func (s *Store) CreateDocTeamShare(currentUserID, docID, teamID, permission stri
}, nil }, nil
} }
// normalizeSharePermission maps share roles from the API to the stored values: "write" (editor) or "read" (viewer).
func normalizeSharePermission(p string) string {
switch strings.ToLower(strings.TrimSpace(p)) {
case "write", "editor":
return "write"
}
return "read"
}
func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permission string) (*domain.DocShare, error) { func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permission string) (*domain.DocShare, error) {
doc, err := s.GetDocumentByID(currentUserID, docID) doc, err := s.GetDocumentByID(currentUserID, docID)
if err != nil { if err != nil {
@@ -3712,11 +3325,10 @@ func (s *Store) CreateDocShare(currentUserID, docID, targetUserIdentifier, permi
// If identifier looks like a team or is prefixed with team_, route to CreateDocTeamShare // If identifier looks like a team or is prefixed with team_, route to CreateDocTeamShare
if strings.HasPrefix(cleanTarget, "team_") { if strings.HasPrefix(cleanTarget, "team_") {
return s.CreateDocTeamShare(currentUserID, docID, cleanTarget, "read") return s.CreateDocTeamShare(currentUserID, docID, cleanTarget, permission)
} }
// Document sharing is strictly read-only permission = normalizeSharePermission(permission)
permission = "read"
// Resolve target user by ID, username, or Email // Resolve target user by ID, username, or Email
var targetUser *domain.User var targetUser *domain.User
@@ -4215,9 +3827,9 @@ func (s *Store) UpdateListItem(itemID string, content *string, category *string,
} }
func (s *Store) CreateTaskFromDocBlock(userID, docID, blockID, content string, checked bool) (*domain.Thought, error) { func (s *Store) CreateTaskFromDocBlock(userID, docID, blockID, content string, checked bool) (*domain.Thought, error) {
doc, err := s.GetDocumentByID(userID, docID) // Linking changes the block (its tick follows the task), so edit rights are needed
if err != nil { if !CanEditDoc(s.DocRole(userID, docID)) {
return nil, err return nil, fmt.Errorf("keine Schreibberechtigung für dieses Dokument (nur Lesezugriff)")
} }
t, err := s.CreateThoughtWithDetails(userID, content, "normal", "manual", "", "", "todo", "") t, err := s.CreateThoughtWithDetails(userID, content, "normal", "manual", "", "", "todo", "")
if err != nil { if err != nil {
@@ -4237,77 +3849,31 @@ func (s *Store) CreateTaskFromDocBlock(userID, docID, blockID, content string, c
_, _ = s.db.Exec(`UPDATE thoughts SET converted_type = 'document_block', converted_ref_id = ?, is_completed = ?, kanban_status = ?, status = 'kept' WHERE id = ?`, _, _ = s.db.Exec(`UPDATE thoughts SET converted_type = 'document_block', converted_ref_id = ?, is_completed = ?, kanban_status = ?, status = 'kept' WHERE id = ?`,
t.ConvertedRefID, doneInt, t.KanbanStatus, t.ID) t.ConvertedRefID, doneInt, t.KanbanStatus, t.ID)
var updatedBlocks []domain.DocumentBlock s.SetBlockChecked(userID, docID, blockID, checked)
found := false
for _, b := range doc.Blocks {
if b.ID == blockID {
b.TaskID = t.ID
b.Checked = checked
found = true
}
updatedBlocks = append(updatedBlocks, b)
}
if found {
_, _ = s.UpdateDocument(userID, docID, nil, nil, nil, &updatedBlocks, nil, nil)
}
return t, nil return t, nil
} }
func (s *Store) SyncDocBlockAndTask(userID, taskID string, isCompleted bool) { // SyncDocBlockAndTask ticks the block a task was created from when the task is completed (or reopened).
// Returns the note ID if the block changed, so the caller can notify its viewers.
func (s *Store) SyncDocBlockAndTask(userID, taskID string, isCompleted bool) string {
t, err := s.GetThoughtByID(userID, taskID) t, err := s.GetThoughtByID(userID, taskID)
if err != nil || t == nil || t.ConvertedType != "document_block" || !strings.Contains(t.ConvertedRefID, ":") { if err != nil || t == nil || t.ConvertedType != "document_block" || !strings.Contains(t.ConvertedRefID, ":") {
return return ""
} }
parts := strings.SplitN(t.ConvertedRefID, ":", 2) docID, blockID, _ := strings.Cut(t.ConvertedRefID, ":")
docID, blockID := parts[0], parts[1] if s.SetBlockChecked(userID, docID, blockID, isCompleted) {
doc, err := s.GetDocumentByID(userID, docID) return docID
if err != nil || doc == nil {
return
}
var updatedBlocks []domain.DocumentBlock
found := false
for _, b := range doc.Blocks {
if b.ID == blockID {
if b.Checked != isCompleted {
b.Checked = isCompleted
found = true
}
}
updatedBlocks = append(updatedBlocks, b)
}
if found {
_, _ = s.UpdateDocument(userID, docID, nil, nil, nil, &updatedBlocks, nil, nil)
} }
return ""
} }
// UnlinkTaskFromDocBlock removes the link between a task and its note block; both stay. The block's
// task ID is derived from this link (docTaskLinks), so nothing in the note itself needs to change.
func (s *Store) UnlinkTaskFromDocBlock(userID, taskID string) { func (s *Store) UnlinkTaskFromDocBlock(userID, taskID string) {
t, err := s.GetThoughtByID(userID, taskID) if t, err := s.GetThoughtByID(userID, taskID); err != nil || t == nil {
if err != nil || t == nil {
return return
} }
docID := t.DocID _, _ = s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '' WHERE id = ?`, taskID)
blockID := t.BlockID
if (docID == "" || blockID == "") && strings.Contains(t.ConvertedRefID, ":") {
parts := strings.SplitN(t.ConvertedRefID, ":", 2)
docID, blockID = parts[0], parts[1]
}
if docID != "" {
if doc, err := s.GetDocumentByID(userID, docID); err == nil && doc != nil {
var updatedBlocks []domain.DocumentBlock
found := false
for _, b := range doc.Blocks {
if b.ID == blockID || b.TaskID == taskID {
b.TaskID = ""
found = true
}
updatedBlocks = append(updatedBlocks, b)
}
if found {
_, _ = s.UpdateDocument(userID, docID, nil, nil, nil, &updatedBlocks, nil, nil)
}
}
}
_, _ = s.db.Exec(`UPDATE thoughts SET converted_type = '', converted_ref_id = '', doc_id = '', block_id = '' WHERE id = ?`, taskID)
} }
func (s *Store) SyncTaskFromBlockCheck(userID, taskID string, checked bool) { func (s *Store) SyncTaskFromBlockCheck(userID, taskID string, checked bool) {
@@ -4747,6 +4313,55 @@ func (s *Store) GetTaskListShares(taskListID string) ([]domain.TaskListShare, er
return shares, nil return shares, nil
} }
// queryUserIDs runs a query returning a single user_id column and collects the distinct IDs.
func (s *Store) queryUserIDs(query string, args ...interface{}) []string {
rows, err := s.db.Query(query, args...)
if err != nil {
return []string{}
}
defer rows.Close()
ids := make([]string, 0)
seen := make(map[string]bool)
for rows.Next() {
var id string
if rows.Scan(&id) == nil && id != "" && !seen[id] {
seen[id] = true
ids = append(ids, id)
}
}
return ids
}
// GetTaskListUserIDs returns every user who can see the task list: owner, direct shares, members of shared teams
// and members of the legacy team_id.
func (s *Store) GetTaskListUserIDs(listID string) []string {
if listID == "" {
return []string{}
}
return s.queryUserIDs(
`SELECT user_id FROM task_lists WHERE id = ?
UNION SELECT target_id FROM task_list_shares WHERE task_list_id = ? AND target_type = 'user'
UNION SELECT tm.user_id FROM team_members tm
WHERE tm.team_id IN (SELECT target_id FROM task_list_shares WHERE task_list_id = ? AND target_type = 'team')
OR tm.team_id IN (SELECT team_id FROM task_lists WHERE id = ? AND team_id != '')`,
listID, listID, listID, listID,
)
}
// GetSharedFolderUserIDs returns the owner of a shared workspace folder and everyone it is shared with.
func (s *Store) GetSharedFolderUserIDs(folderID string) []string {
if folderID == "" {
return []string{}
}
return s.queryUserIDs(
`SELECT owner_id FROM workspace_shared_folders WHERE id = ?
UNION SELECT target_id FROM workspace_folder_shares WHERE folder_id = ? AND target_type = 'user'
UNION SELECT tm.user_id FROM team_members tm
WHERE tm.team_id IN (SELECT target_id FROM workspace_folder_shares WHERE folder_id = ? AND target_type = 'team')`,
folderID, folderID, folderID,
)
}
func (s *Store) getMembersForTaskList(tl *domain.TaskList) []domain.TaskListMember { func (s *Store) getMembersForTaskList(tl *domain.TaskList) []domain.TaskListMember {
memberMap := make(map[string]domain.TaskListMember) memberMap := make(map[string]domain.TaskListMember)
+21 -45
View File
@@ -6,6 +6,7 @@ import (
"testing" "testing"
"github.com/sandrodach/mindsift/internal/auth" "github.com/sandrodach/mindsift/internal/auth"
"github.com/sandrodach/mindsift/internal/collab"
"github.com/sandrodach/mindsift/internal/domain" "github.com/sandrodach/mindsift/internal/domain"
) )
@@ -208,13 +209,13 @@ func TestStoreLifecycle(t *testing.T) {
// user2 tries to edit read-only doc -> must fail // user2 tries to edit read-only doc -> must fail
updatedTitle := "Hacked Title" updatedTitle := "Hacked Title"
_, err = store.UpdateDocument(user2.ID, newDoc.ID, &updatedTitle, nil, nil, nil, nil, nil) _, err = store.UpdateDocument(user2.ID, newDoc.ID, &updatedTitle, nil, nil, nil)
if err == nil { if err == nil {
t.Fatalf("expected UpdateDocument to fail for read-only user, but succeeded") t.Fatalf("expected UpdateDocument to fail for read-only user, but succeeded")
} }
// user2 is a read-only collaborator -> editing must fail // user2 is a read-only collaborator -> editing must fail
_, err = store.UpdateDocument(user2.ID, newDoc.ID, &updatedTitle, nil, nil, nil, nil, nil) _, err = store.UpdateDocument(user2.ID, newDoc.ID, &updatedTitle, nil, nil, nil)
if err == nil { if err == nil {
t.Fatalf("expected UpdateDocument to fail for read-only collaborator, but succeeded") t.Fatalf("expected UpdateDocument to fail for read-only collaborator, but succeeded")
} }
@@ -271,7 +272,7 @@ func TestPerUserDocPinning(t *testing.T) {
// user2 pins the shared document (even with read-only access) // user2 pins the shared document (even with read-only access)
pinTrue := true pinTrue := true
updatedDoc2, err := store.UpdateDocument(user2.ID, doc.ID, nil, nil, nil, nil, nil, &pinTrue) updatedDoc2, err := store.UpdateDocument(user2.ID, doc.ID, nil, nil, nil, &pinTrue)
if err != nil { if err != nil {
t.Fatalf("user2 pinning read-only doc failed: %v", err) t.Fatalf("user2 pinning read-only doc failed: %v", err)
} }
@@ -290,7 +291,7 @@ func TestPerUserDocPinning(t *testing.T) {
// user1 unpins the doc // user1 unpins the doc
pinFalse := false pinFalse := false
_, err = store.UpdateDocument(user1.ID, doc.ID, nil, nil, nil, nil, nil, &pinFalse) _, err = store.UpdateDocument(user1.ID, doc.ID, nil, nil, nil, &pinFalse)
if err != nil { if err != nil {
t.Fatalf("user1 unpinning doc failed: %v", err) t.Fatalf("user1 unpinning doc failed: %v", err)
} }
@@ -518,10 +519,7 @@ func TestWorkspaceBlockHashtagsAndDetails(t *testing.T) {
updatedBlocks := []domain.DocumentBlock{ updatedBlocks := []domain.DocumentBlock{
{ID: "b1", Type: "bullet", Content: "Einkauf planen #Kühlregal #Vorrat", Level: 0}, {ID: "b1", Type: "bullet", Content: "Einkauf planen #Kühlregal #Vorrat", Level: 0},
} }
_, err = store.UpdateDocument(user.ID, doc.ID, nil, nil, nil, &updatedBlocks, nil, nil) store.replaceDocBlocksInDB(doc.ID, user.ID, updatedBlocks, "")
if err != nil {
t.Fatalf("UpdateDocument failed: %v", err)
}
tagsAfter, err := store.GetWorkspaceTags(user.ID) tagsAfter, err := store.GetWorkspaceTags(user.ID)
if err != nil { if err != nil {
@@ -574,7 +572,7 @@ func TestDatabaseFirstWorkspaceArchitecture(t *testing.T) {
} }
} }
_, err = store.CreateDocShare(userA.ID, doc.ID, userB.ID, "write") _, err = store.CreateDocShare(userA.ID, doc.ID, userB.ID, "read")
if err != nil { if err != nil {
t.Fatalf("CreateDocShare failed: %v", err) t.Fatalf("CreateDocShare failed: %v", err)
} }
@@ -585,43 +583,22 @@ func TestDatabaseFirstWorkspaceArchitecture(t *testing.T) {
t.Fatalf("MoveDocument by userB failed: %v", err) t.Fatalf("MoveDocument by userB failed: %v", err)
} }
// User B is a read-only collaborator -> PatchDocumentBlocks must fail // User A edits b1 and adds b4 through the operation log; blocks A did not touch stay
_, _, err = store.PatchDocumentBlocks(userB.ID, doc.ID, domain.DocBlocksPatchRequest{ editedB1, newB4, top := "Zeile 1 von A bearbeitet", "Zeile 4 neu von A", ""
Upserted: []domain.DocumentBlock{ res, _, err := store.ApplyDocOps(doc.ID, collab.Actor{UserID: userA.ID, ClientID: "tab-a"}, []collab.Op{
{ID: "b2", Type: "bullet", Content: "Zeile 2 von B bearbeitet", Level: 0}, {OpID: "a-b1", Kind: collab.OpSetContent, BlockID: "b1", BaseVersion: 1, Content: &editedB1},
{ID: "b3", Type: "bullet", Content: "Zeile 3 neu von B", Level: 0}, {OpID: "a-b4", Kind: collab.OpInsert, BlockID: "b4", ParentID: &top, OrderKey: "a9", Content: &newB4},
}, }, nil)
BlockOrder: []string{"b1", "b2", "b3"}, if err != nil || res[0].Status != collab.StatusApplied || res[1].Status != collab.StatusApplied {
}) t.Fatalf("ApplyDocOps by userA failed: %+v %v", res, err)
if err == nil {
t.Fatalf("expected PatchDocumentBlocks by read-only userB to fail, but succeeded")
}
// User A's client only knew about b1, b2 when sending its patch (adds b4)
patchedDocA, _, err := store.PatchDocumentBlocks(userA.ID, doc.ID, domain.DocBlocksPatchRequest{
Upserted: []domain.DocumentBlock{
{ID: "b1", Type: "bullet", Content: "Zeile 1 von A bearbeitet", Level: 0},
{ID: "b4", Type: "bullet", Content: "Zeile 4 neu von A", Level: 0},
},
BlockOrder: []string{"b1", "b2", "b4"}, // stale order without b3!
})
if err != nil {
t.Fatalf("PatchDocumentBlocks by userA failed: %v", err)
}
// Verify User A's blocks are preserved: b1, b2, b4 (3 blocks)
if len(patchedDocA.Blocks) != 3 {
t.Fatalf("expected 3 blocks after patch, got %d: %+v", len(patchedDocA.Blocks), patchedDocA.Blocks)
} }
patchedDocA, _ := store.GetDocumentByID(userA.ID, doc.ID)
blockContent := make(map[string]string) blockContent := make(map[string]string)
for _, b := range patchedDocA.Blocks { for _, b := range patchedDocA.Blocks {
blockContent[b.ID] = b.Content blockContent[b.ID] = b.Content
} }
if blockContent["b1"] != "Zeile 1 von A bearbeitet" { if blockContent["b1"] != editedB1 || blockContent["b4"] != newB4 || blockContent["b2"] == "" {
t.Fatalf("expected b1 updated by A, got %q", blockContent["b1"]) t.Fatalf("unexpected blocks after edit: %+v", blockContent)
}
if blockContent["b4"] != "Zeile 4 neu von A" {
t.Fatalf("expected b4 added by A, got %q", blockContent["b4"])
} }
// Verify folder independence: User A still sees "Privat", User B still sees "Arbeit/Eingang" // Verify folder independence: User A still sees "Privat", User B still sees "Arbeit/Eingang"
@@ -710,7 +687,7 @@ func TestDatabaseFirstWorkspaceArchitecture(t *testing.T) {
} }
} }
func TestFlushDocumentStateAndSchema(t *testing.T) { func TestBlockListAndDerivedData(t *testing.T) {
tmpDir := t.TempDir() tmpDir := t.TempDir()
dbPath := filepath.Join(tmpDir, "tabula_rasa_test.db") dbPath := filepath.Join(tmpDir, "tabula_rasa_test.db")
@@ -746,9 +723,8 @@ func TestFlushDocumentStateAndSchema(t *testing.T) {
}, },
} }
if err := store.FlushDocumentState(doc.ID, "Architektur Notiz", blocks); err != nil { store.replaceDocBlocksInDB(doc.ID, user.ID, blocks, "")
t.Fatalf("FlushDocumentState failed: %v", err) store.applyLegacyTitle(doc.ID, user.ID, "Architektur Notiz")
}
updatedDoc, err := store.GetDocumentByID(user.ID, doc.ID) updatedDoc, err := store.GetDocumentByID(user.ID, doc.ID)
if err != nil { if err != nil {
+63 -38
View File
@@ -93,6 +93,27 @@ type Hub struct {
sseUserRooms map[string]map[*SSEClient]bool sseUserRooms map[string]map[*SSEClient]bool
sseRooms map[string]map[*SSEClient]bool sseRooms map[string]map[*SSEClient]bool
closed bool closed bool
// authorizeRoom decides whether a user may receive events of a room; without it every join is denied
authorizeRoom func(userID, roomID string) bool
}
// SetRoomAuthorizer installs the permission check used for every room join (URL room, JOIN_ROOM, SSE rooms).
func (h *Hub) SetRoomAuthorizer(f func(userID, roomID string) bool) {
h.mu.Lock()
h.authorizeRoom = f
h.mu.Unlock()
}
// canJoin must be called without holding h.mu, since the authorizer queries the database.
func (h *Hub) canJoin(userID, roomID string) bool {
if roomID == "" || userID == "" || userID == "anonymous" {
return false
}
h.mu.RLock()
f := h.authorizeRoom
h.mu.RUnlock()
return f != nil && f(userID, roomID)
} }
func NewHub() *Hub { func NewHub() *Hub {
@@ -107,6 +128,9 @@ func NewHub() *Hub {
} }
func (h *Hub) Register(c *Client) { func (h *Hub) Register(c *Client) {
if c.roomID != "" && !h.canJoin(c.userID, c.roomID) {
c.roomID = ""
}
h.mu.Lock() h.mu.Lock()
h.allClients[c] = true h.allClients[c] = true
if c.roomID != "" { if c.roomID != "" {
@@ -167,6 +191,14 @@ func (h *Hub) Unregister(c *Client) {
} }
func (h *Hub) RegisterSSE(c *SSEClient) { func (h *Hub) RegisterSSE(c *SSEClient) {
if c.roomID != "" && !h.canJoin(c.userID, c.roomID) {
c.roomID = ""
}
for rID := range c.subscribedRooms {
if !h.canJoin(c.userID, rID) {
delete(c.subscribedRooms, rID)
}
}
h.mu.Lock() h.mu.Lock()
h.sseClients[c] = true h.sseClients[c] = true
if c.roomID != "" { if c.roomID != "" {
@@ -232,11 +264,15 @@ func (h *Hub) UnregisterSSE(c *SSEClient) {
h.mu.Unlock() h.mu.Unlock()
} }
func (h *Hub) JoinSSERoomByID(sseID, roomID string) bool { // JoinSSERoomByID adds the SSE connection sseID to a room; it only succeeds for the user owning that connection.
func (h *Hub) JoinSSERoomByID(sseID, userID, roomID string) bool {
if !h.canJoin(userID, roomID) {
return false
}
h.mu.Lock() h.mu.Lock()
defer h.mu.Unlock() defer h.mu.Unlock()
for client := range h.sseClients { for client := range h.sseClients {
if client.id == sseID { if client.id == sseID && client.userID == userID {
if h.sseRooms[roomID] == nil { if h.sseRooms[roomID] == nil {
h.sseRooms[roomID] = make(map[*SSEClient]bool) h.sseRooms[roomID] = make(map[*SSEClient]bool)
} }
@@ -251,11 +287,11 @@ func (h *Hub) JoinSSERoomByID(sseID, roomID string) bool {
return false return false
} }
func (h *Hub) LeaveSSERoomByID(sseID, roomID string) bool { func (h *Hub) LeaveSSERoomByID(sseID, userID, roomID string) bool {
h.mu.Lock() h.mu.Lock()
defer h.mu.Unlock() defer h.mu.Unlock()
for client := range h.sseClients { for client := range h.sseClients {
if client.id == sseID { if client.id == sseID && client.userID == userID {
if room, ok := h.sseRooms[roomID]; ok { if room, ok := h.sseRooms[roomID]; ok {
delete(room, client) delete(room, client)
if len(room) == 0 { if len(room) == 0 {
@@ -305,32 +341,6 @@ func (h *Hub) Close() {
// Broadcasting Methods // Broadcasting Methods
func (h *Hub) BroadcastToAll(event string, payload interface{}) {
msg := domain.WSMessage{
Event: event,
Payload: payload,
Timestamp: domain.NowISO(),
}
data, err := json.Marshal(msg)
if err != nil {
return
}
h.mu.RLock()
if h.closed {
h.mu.RUnlock()
return
}
defer h.mu.RUnlock()
for client := range h.allClients {
client.trySend(data)
}
for sseClient := range h.sseClients {
sseClient.trySend(data)
}
}
func (h *Hub) BroadcastToUser(userID, event string, payload interface{}) { func (h *Hub) BroadcastToUser(userID, event string, payload interface{}) {
msg := domain.WSMessage{ msg := domain.WSMessage{
Event: event, Event: event,
@@ -361,15 +371,21 @@ func (h *Hub) BroadcastToUser(userID, event string, payload interface{}) {
} }
} }
func (h *Hub) BroadcastTaskEvent(userID, event string, payload interface{}) { // BroadcastToUsers sends one event to every connection of the given users (duplicates are ignored).
h.BroadcastToUser(userID, event, payload) func (h *Hub) BroadcastToUsers(userIDs []string, event string, payload interface{}) {
seen := make(map[string]bool, len(userIDs))
for _, uid := range userIDs {
if uid == "" || seen[uid] {
continue
}
seen[uid] = true
h.BroadcastToUser(uid, event, payload)
}
} }
func (h *Hub) BroadcastDocEvent(docID, event string, payload interface{}) { func (h *Hub) BroadcastDocEvent(docID, event string, payload interface{}) {
if docID != "" { if docID != "" {
h.BroadcastRoom("doc:"+docID, event, payload, nil) h.BroadcastRoom("doc:"+docID, event, payload, nil)
} else {
h.BroadcastToAll(event, payload)
} }
} }
@@ -413,7 +429,10 @@ func (h *Hub) BroadcastListEvent(listID, event string, payload interface{}) {
h.Broadcast("", listID, event, payload) h.Broadcast("", listID, event, payload)
} }
func (h *Hub) JoinRoom(c *Client, roomID string) { func (h *Hub) JoinRoom(c *Client, roomID string) bool {
if !h.canJoin(c.userID, roomID) {
return false
}
h.mu.Lock() h.mu.Lock()
defer h.mu.Unlock() defer h.mu.Unlock()
if h.rooms[roomID] == nil { if h.rooms[roomID] == nil {
@@ -424,6 +443,7 @@ func (h *Hub) JoinRoom(c *Client, roomID string) {
c.subscribedRooms = make(map[string]bool) c.subscribedRooms = make(map[string]bool)
} }
c.subscribedRooms[roomID] = true c.subscribedRooms[roomID] = true
return true
} }
func (h *Hub) LeaveRoom(c *Client, roomID string) { func (h *Hub) LeaveRoom(c *Client, roomID string) {
@@ -512,11 +532,13 @@ func (h *Hub) ServeSSE(w http.ResponseWriter, r *http.Request, roomID, userID st
return return
} }
// The server's WriteTimeout would cut long-lived streams; lift it for this response only
_ = http.NewResponseController(w).SetWriteDeadline(time.Time{})
w.Header().Set("Content-Type", "text/event-stream") w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache, no-transform") w.Header().Set("Cache-Control", "no-cache, no-transform")
w.Header().Set("Connection", "keep-alive") w.Header().Set("Connection", "keep-alive")
w.Header().Set("X-Accel-Buffering", "no") w.Header().Set("X-Accel-Buffering", "no")
w.Header().Set("Access-Control-Allow-Origin", "*")
client := &SSEClient{ client := &SSEClient{
id: domain.NewID("sse"), id: domain.NewID("sse"),
@@ -591,7 +613,7 @@ func (c *Client) readPump() {
break break
} }
// Handle client-initiated lock, CRDT, room subscription, and presence messages // Handle client-initiated room subscription messages
var incoming struct { var incoming struct {
Event string `json:"event"` Event string `json:"event"`
DocID string `json:"docId"` DocID string `json:"docId"`
@@ -609,7 +631,10 @@ func (c *Client) readPump() {
} }
// Room subscription without requiring docID // Room subscription without requiring docID
if incoming.Event == "JOIN_ROOM" { if incoming.Event == "PING" {
// App-level heartbeat: lets clients detect proxies that accept the upgrade but forward nothing
c.SendEvent("PONG", nil)
} else if incoming.Event == "JOIN_ROOM" {
room := incoming.Room room := incoming.Room
if room == "" { if room == "" {
room = docID room = docID
+36 -2
View File
@@ -22,7 +22,7 @@ func TestHubCloseAndGracefulShutdown(t *testing.T) {
} }
// Verify broadcast works // Verify broadcast works
hub.BroadcastToAll("TEST_EVENT", map[string]string{"msg": "hello"}) hub.BroadcastToUser("user-1", "TEST_EVENT", map[string]string{"msg": "hello"})
// Concurrently invoke Close() and Broadcast() to verify no panics or deadlocks // Concurrently invoke Close() and Broadcast() to verify no panics or deadlocks
var wg sync.WaitGroup var wg sync.WaitGroup
@@ -31,7 +31,7 @@ func TestHubCloseAndGracefulShutdown(t *testing.T) {
go func() { go func() {
defer wg.Done() defer wg.Done()
for i := 0; i < 50; i++ { for i := 0; i < 50; i++ {
hub.BroadcastToAll("SPAM", map[string]int{"i": i}) hub.BroadcastToUser("user-1", "SPAM", map[string]int{"i": i})
time.Sleep(1 * time.Millisecond) time.Sleep(1 * time.Millisecond)
} }
}() }()
@@ -56,3 +56,37 @@ func TestHubCloseAndGracefulShutdown(t *testing.T) {
} }
} }
} }
func TestRoomJoinsRequireAuthorization(t *testing.T) {
hub := NewHub()
c := &Client{hub: hub, userID: "user-1", send: make(chan []byte, 10)}
if hub.JoinRoom(c, "lst_1") {
t.Fatal("join must be denied without an authorizer")
}
hub.SetRoomAuthorizer(func(userID, roomID string) bool {
return userID == "user-1" && roomID == "lst_allowed"
})
if hub.JoinRoom(c, "lst_other") {
t.Fatal("join to an unauthorized room must be denied")
}
if !hub.JoinRoom(c, "lst_allowed") {
t.Fatal("join to an authorized room must succeed")
}
anon := &Client{hub: hub, userID: "anonymous", send: make(chan []byte, 10)}
if hub.JoinRoom(anon, "lst_allowed") {
t.Fatal("anonymous clients must never join rooms")
}
sse := &SSEClient{id: "sse_1", userID: "user-1", send: make(chan []byte, 10),
subscribedRooms: map[string]bool{"lst_allowed": true, "doc:secret": true}}
hub.RegisterSSE(sse)
if sse.subscribedRooms["doc:secret"] {
t.Fatal("unauthorized initial SSE rooms must be dropped")
}
if hub.JoinSSERoomByID("sse_1", "user-2", "lst_allowed") {
t.Fatal("joining with another user's sseId must fail")
}
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 197 KiB

After

Width:  |  Height:  |  Size: 213 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

After

Width:  |  Height:  |  Size: 197 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 261 KiB

After

Width:  |  Height:  |  Size: 254 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 249 KiB

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 315 KiB

After

Width:  |  Height:  |  Size: 362 KiB