feat(voice): self-hosted speech recognition via Whisper (STT_*)
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m49s
MindSift CI & Docker Release / Build, Test in Container & Push (push) Successful in 1m49s
- STT_URL switches voice input to an OpenAI-compatible Whisper server (e.g. speaches); without it the browser's recognition stays. Configured by environment only, with limits for upload size, parallel jobs, hourly budget per user and timeout. The demo always uses the browser. - POST /voice/transcribe forwards recordings and returns the text; nothing is stored. On startup MindSift waits for the server and asks it to download the model if missing. - Admin card shows the active mode and server state, with a test recording. - docker-compose: optional "whisper" service (speaches 0.8.3 CPU, profile "whisper", models in ./whisper). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
sandrodachandClaude Opus 5.5 committed
1 parent
6a5e82431f
commit
7801cd13b9
18 files changed
+816
-6
No files matched your search
@@ -25,3 +25,19 @@ LEGAL_IMPRINT_URL=
|
||||
LEGAL_PRIVACY_URL=
|
||||
LEGAL_IMPRINT_CONTENT_FILE=
|
||||
LEGAL_PRIVACY_CONTENT_FILE=
|
||||
|
||||
# Optional: own speech recognition (Whisper). Without STT_URL the browser's recognition is used.
|
||||
# Enable: STT_URL=http://whisper:8000 and COMPOSE_PROFILES=whisper (starts the "whisper" service),
|
||||
# then: mkdir -p whisper && sudo chown 1000:1000 whisper && docker compose up -d
|
||||
# COMPOSE_PROFILES=whisper
|
||||
STT_URL=
|
||||
STT_MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
|
||||
STT_LANGUAGE=de
|
||||
STT_API_KEY=
|
||||
STT_DISABLED=false
|
||||
# Limits (server mode)
|
||||
STT_MAX_SECONDS=600
|
||||
STT_MAX_UPLOAD_MB=25
|
||||
STT_MAX_PARALLEL=1
|
||||
STT_PER_USER_PER_HOUR=60
|
||||
STT_TIMEOUT_SECONDS=120
|
||||
@@ -18,3 +18,6 @@ dev-data/
|
||||
.idea/
|
||||
.vscode/
|
||||
e2e/node_modules/
|
||||
|
||||
# Whisper models (docker compose service "whisper")
|
||||
whisper/
|
||||
@@ -6,6 +6,10 @@ Das Format folgt [Keep a Changelog](https://keepachangelog.com/de/1.1.0/), die V
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Eigene Spracherkennung (optional): Mit `STT_URL` schickt MindSift Aufnahmen an einen selbst betriebenen Whisper-Server (OpenAI-kompatibel, z. B. speaches) statt an Google oder Microsoft. Einstellung nur per Umgebungsvariablen (`STT_*`) inklusive Grenzwerten; `docker-compose.yml` bringt den Dienst `whisper` als Profil mit, das Modell lädt MindSift beim ersten Start selbst. Administration → Server & Isolation zeigt den aktiven Modus und bietet eine Testaufnahme. In der Demo bleibt es bei der Browser-Erkennung.
|
||||
- API: `POST /voice/transcribe`, `GET /admin/voice`; `/auth/status` meldet `voice.mode`.
|
||||
|
||||
## [0.6.1] - 2026-10-29
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -17,6 +17,7 @@ Basis: `/api/v1`. Antworten sind JSON; Fehler haben die Form `{"error": "Text"}`
|
||||
| POST | `/auth/quickstart` | Immer `403` (Gastzugang deaktiviert) |
|
||||
| POST | `/webhooks/ingest` | Einliefern per Webhook, Key als `X-API-Key` (empfohlen), Bearer oder `?token=` (in der URL nur API-Keys, keine Sitzungs-Tokens). Body: `title`/`content`/`text`/`message`, `priority`, `source`, `tags`, `dueDate`; `target` (oder `type`): `triage` (Standard, Inbox), `task` (optional `taskListId`), `list_item` (`listId`, `category`, `quantity`). Klartext-Body wird als Inhalt genommen. `403` ohne passenden Scope oder ohne Schreibrecht auf die Liste |
|
||||
| POST | `/auth/stream-ticket` | Einmal-Ticket (60 s) für WebSocket/SSE |
|
||||
| POST | `/voice/transcribe` | Nur im Server-Modus der Spracherkennung: `multipart/form-data` mit `file` (Aufnahme, z. B. WebM/Opus oder MP4/AAC) → `{text}`. `404` ohne `STT_URL`, `413` zu groß, `429` Stundenlimit, `503` ausgelastet, `502` Whisper-Fehler (Details im Log) |
|
||||
| GET | `/ws`, `/ws/spaces/{id}`, `/ws/lists/{id}` | WebSocket, siehe [websocket.md](websocket.md); `?ticket=` oder `Authorization`-Header |
|
||||
| GET | `/events` | SSE-Fallback; `?ticket=` oder `Authorization`-Header |
|
||||
|
||||
@@ -193,6 +194,7 @@ Wiederholte Operationen (gleiche `opId`) werden nicht erneut angewendet; die Ant
|
||||
| GET/PUT | `/admin/settings` | `instanceName`, `allowRegistration`, `sharingMode` (`private`/`organizations`/`open`), SMTP-Felder; ein Moduswechsel bereinigt Freigaben. `smtpPass` kommt nie zurück (`smtpPassSet` zeigt, ob eines gespeichert ist); leer = unverändert |
|
||||
| GET | `/admin/settings/sharing-impact?mode=` | Probelauf: wie viele Freigaben und Gruppen-Mitgliedschaften ein Wechsel entfernen würde (`affected`) |
|
||||
| POST | `/admin/smtp/test` | Testmail (`toEmail`) |
|
||||
| GET | `/admin/voice` | Aktive Spracherkennung (`mode`: `server`/`browser`/`off`), Einstellungen aus `STT_*` ohne Schlüssel, Grenzen und im Server-Modus ein frischer Zustand (`status.reachable`, `status.modelReady`) |
|
||||
| GET/POST | `/admin/backups` | Datenbank-Abzüge auflisten bzw. sofort sichern (Abzug + alle Benutzer-Backups) |
|
||||
| GET | `/admin/backups/{name}` | Datenbank-Abzug herunterladen |
|
||||
|
||||
|
||||
@@ -69,6 +69,11 @@ Das Frontend liegt per `go:embed` im Binary ([cmd/server/main.go](../cmd/server/
|
||||
| `DEMO_RESET_MINUTES` | `20` | Demo-Modus: periodischer Reset auf die Beispieldaten (`0` = aus). Feste IDs der Demo-Konten (`usr_demo_admin`, `usr_demo_user`) halten Sitzungen gültig; laufende Anfragen sperren den Reset (`internal/api/demo.go`, Streams ausgenommen); danach `DEMO_RESET` an alle Verbindungen, `/auth/status` liefert `demoNextReset`/`demoResetInSeconds`. |
|
||||
| `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASS`, `SMTP_FROM`, `SMTP_TLS` | leer / `587` / `true` | Mailversand, auch im Admin-UI einstellbar |
|
||||
| `SMTP_SKIP_VERIFY` | `false` | Zertifikat des Mailservers nicht prüfen (nur interne Server mit selbstsigniertem Zertifikat) |
|
||||
| `STT_URL` | leer | Eigene Spracherkennung: Basis-URL eines OpenAI-kompatiblen Whisper-Servers (z. B. `http://whisper:8000`, speaches). Gesetzt = Server-Modus, sonst Spracherkennung des Browsers; in der Demo immer Browser |
|
||||
| `STT_MODEL`, `STT_LANGUAGE`, `STT_API_KEY` | `deepdml/faster-whisper-large-v3-turbo-ct2` / `de` / leer | Modell, Sprache (`auto` = erkennen), Schlüssel falls der Whisper-Server einen verlangt |
|
||||
| `STT_DISABLED` | `false` | `true` blendet die Spracheingabe überall aus |
|
||||
| `STT_MAX_SECONDS`, `STT_MAX_UPLOAD_MB` | `600` / `25` | Längste Aufnahme (stoppt im Browser), größte Aufnahme |
|
||||
| `STT_MAX_PARALLEL`, `STT_PER_USER_PER_HOUR`, `STT_TIMEOUT_SECONDS` | `1` / `60` / `120` | Gleichzeitige Erkennungen (weitere warten), Erkennungen pro Person und Stunde, Höchstdauer inkl. Warten |
|
||||
| `LEGAL_IMPRINT_URL`, `LEGAL_PRIVACY_URL` | leer | Impressum/Datenschutz als externer Link (nur `http(s)`) |
|
||||
| `LEGAL_IMPRINT_CONTENT`, `LEGAL_PRIVACY_CONTENT` (oder `…_CONTENT_FILE` mit Dateipfad) | leer | Impressum/Datenschutz als Markdown, angezeigt in einem Lesefenster. Ohne URL und Text bleibt der Link unsichtbar. |
|
||||
|
||||
@@ -85,6 +90,12 @@ Hinweis: `DATABASE_URL` mit `postgres…` führt dazu, dass der Standardpfad in
|
||||
- **Passwort vergessen:** nur wenn SMTP und `BASE_URL` gesetzt sind (nicht im Demo-Modus; `passwordReset` in `/auth/status`). `POST /auth/forgot-password` antwortet immer gleich und verschickt die Mail im Hintergrund, höchstens alle 2 Minuten pro Konto. Der Link `BASE_URL/?resetToken=…` gilt eine Stunde und einmal; gespeichert wird nur der SHA-256-Hash (`password_reset_tokens`). `POST /auth/reset-password` setzt das Passwort, entwertet alle offenen Links des Kontos und erhöht `token_version` (alle Sitzungen enden). Die App entfernt den Token sofort aus der Adresszeile.
|
||||
- **Registrierung:** nur mit Einladungscode oder wenn `allow_registration` aktiv ist. Das erste Konto entsteht über `/auth/setup` und wird Administrator.
|
||||
|
||||
## Spracherkennung
|
||||
|
||||
Nur über Umgebungsvariablen (`STT_*`, siehe oben), nicht in der Oberfläche; Administration → Server & Isolation zeigt den aktiven Modus, den Zustand des Whisper-Servers und bietet eine Testaufnahme. Im Server-Modus nimmt der Browser per `MediaRecorder` auf und schickt die Aufnahme an `POST /voice/transcribe`; MindSift reicht sie an `STT_URL/v1/audio/transcriptions` weiter (`model`, `language`) und gibt nur den Text zurück. Nichts wird gespeichert oder protokolliert außer Fehlern (`internal/api/voice.go`). Beim Start wartet MindSift auf den Whisper-Server und stößt den Download des Modells an, falls es fehlt (`POST /v1/models/<id>` bei speaches).
|
||||
|
||||
Docker Compose bringt den optionalen Dienst `whisper` (speaches 0.8.3, CPU, Profil `whisper`) mit; das Modell liegt in `./whisper` (Ordner gehört UID 1000). Gemessen auf einem Desktop mit 16 Kernen (int8) für einen Satz von 4 s: `Systran/faster-whisper-small` 1,6 s, `deepdml/faster-whisper-large-v3-turbo-ct2` 8 s bei besserer Erkennung (RAM ca. 1 bzw. 2 GB). Auf geteilten vCPUs ist mit dem Mehrfachen zu rechnen.
|
||||
|
||||
## Hintergrundjob: Alarme
|
||||
|
||||
`startAlarmTicker` prüft jede Minute alle offenen, noch nicht gemeldeten Aufgaben mit Fälligkeit. Ist der Alarmzeitpunkt erreicht (Fälligkeit mit Uhrzeit bzw. 9:00 bei ganztägigen Aufgaben, minus Vorlaufzeit des Benutzers, Zeitzone `TZ`), sendet er über die Kanäle des Benutzers (`email`, `discord`, `webhook`) und setzt `alarm_sent`, sobald mindestens ein Kanal erfolgreich war. Nach 3 Durchläufen ohne Erfolg gibt er auf (`alarm_failures`). Details und Einschränkungen: [modules/tasks.md](modules/tasks.md).
|
||||
|
||||
@@ -34,7 +34,7 @@ Die meisten Notiz- und Aufgaben-Apps verlangen Struktur im falschen Moment: Wenn
|
||||
|
||||
### ⚡ Schnellerfassung & Triage
|
||||
|
||||
* Eingabeleiste im Kopf der App, jederzeit mit `C` oder `/` fokussierbar; auch per Spracheingabe (Browser-Funktion).
|
||||
* Eingabeleiste im Kopf der App, jederzeit mit `C` oder `/` fokussierbar; auch per Spracheingabe: über den Browser oder datenschutzfreundlich über einen eigenen Whisper-Container (optional in `docker-compose.yml`, siehe [Architektur](Doku/architecture.md#spracherkennung)).
|
||||
* Ziel beim Erfassen wählbar: Posteingang, Aufgabe, Listeneintrag oder Notiz.
|
||||
* **Triage-Deck:** `H`/`←` verwerfen, `L`/`→` zur Aufgabe, `T` Aufgabe mit Termin, `D` in Notiz verwandeln, Leertaste = später. Auf dem Smartphone per Wischgeste.
|
||||
* Papierkorb mit Wiederherstellen.
|
||||
|
||||
@@ -32,3 +32,37 @@ services:
|
||||
- LEGAL_PRIVACY_URL=${LEGAL_PRIVACY_URL:-}
|
||||
- LEGAL_IMPRINT_CONTENT_FILE=${LEGAL_IMPRINT_CONTENT_FILE:-}
|
||||
- LEGAL_PRIVACY_CONTENT_FILE=${LEGAL_PRIVACY_CONTENT_FILE:-}
|
||||
# Optional: eigene Spracherkennung (Whisper, siehe Dienst "whisper" unten). Ohne STT_URL nutzt
|
||||
# MindSift die Spracherkennung des Browsers. Alles nur hier einstellbar, nicht in der Oberfläche;
|
||||
# Probleme stehen im Log (docker compose logs mindsift whisper).
|
||||
- STT_URL=${STT_URL:-} # z. B. http://whisper:8000 – gesetzt = Server-Modus
|
||||
- STT_MODEL=${STT_MODEL:-deepdml/faster-whisper-large-v3-turbo-ct2} # kleiner/schneller: Systran/faster-whisper-small
|
||||
- STT_LANGUAGE=${STT_LANGUAGE:-de} # Sprachcode, "auto" = automatisch erkennen
|
||||
- STT_API_KEY=${STT_API_KEY:-} # nur falls der Whisper-Server einen verlangt (API_KEY dort)
|
||||
- STT_DISABLED=${STT_DISABLED:-false} # true = Mikrofon überall ausblenden
|
||||
# Schutzwerte für den Server-Modus
|
||||
- STT_MAX_SECONDS=${STT_MAX_SECONDS:-600} # längste Aufnahme in Sekunden (danach stoppt der Browser)
|
||||
- STT_MAX_UPLOAD_MB=${STT_MAX_UPLOAD_MB:-25} # größte Aufnahme in MB
|
||||
- STT_MAX_PARALLEL=${STT_MAX_PARALLEL:-1} # gleichzeitige Erkennungen, weitere warten in der Schlange
|
||||
- STT_PER_USER_PER_HOUR=${STT_PER_USER_PER_HOUR:-60} # Erkennungen pro Benutzer und Stunde
|
||||
- STT_TIMEOUT_SECONDS=${STT_TIMEOUT_SECONDS:-120} # Höchstdauer pro Erkennung inkl. Warten
|
||||
|
||||
# Optionale Spracherkennung auf dem eigenen Server (speaches mit faster-whisper, nur CPU).
|
||||
# Start: in .env STT_URL=http://whisper:8000 und COMPOSE_PROFILES=whisper setzen, dann
|
||||
# mkdir -p whisper && sudo chown 1000:1000 whisper (der Container läuft als UID 1000)
|
||||
# docker compose up -d
|
||||
# Das Modell (large-v3-turbo ca. 1,6 GB) lädt MindSift beim ersten Start automatisch in ./whisper.
|
||||
# RAM: large-v3-turbo ca. 2 GB, small ca. 1 GB.
|
||||
whisper:
|
||||
image: ghcr.io/speaches-ai/speaches:0.8.3-cpu
|
||||
container_name: mindsift-whisper
|
||||
restart: unless-stopped
|
||||
profiles: ["whisper"]
|
||||
volumes:
|
||||
- ./whisper:/home/ubuntu/.cache/huggingface/hub
|
||||
environment:
|
||||
- WHISPER__COMPUTE_TYPE=int8 # schneller und sparsamer auf der CPU, kaum Qualitätsverlust
|
||||
- STT_MODEL_TTL=-1 # Modell im Speicher halten (sonst nach 5 Min. entladen, nächste Erkennung lädt neu)
|
||||
- ENABLE_UI=false # keine Weboberfläche von speaches
|
||||
- LOG_LEVEL=info
|
||||
# Kein Port nach außen: nur MindSift spricht im Docker-Netz mit dem Dienst.
|
||||
@@ -29,6 +29,7 @@ import { passwordProblem } from '../../lib/password';
|
||||
import { UserAvatar } from '../../components/UserAvatar';
|
||||
import { ToggleSwitch } from '../../components/ToggleSwitch';
|
||||
import { BackupAdminCard } from './BackupAdminCard';
|
||||
import { VoiceStatusCard } from './VoiceStatusCard';
|
||||
import { CountBadge } from '../../components/CountBadge';
|
||||
|
||||
interface AdminSettingsViewProps {
|
||||
@@ -1115,6 +1116,8 @@ export const AdminSettingsView: React.FC<AdminSettingsViewProps> = ({
|
||||
|
||||
<BackupAdminCard settings={settings} onSave={(patch) => handleSaveSystemSettings(patch)} />
|
||||
|
||||
<VoiceStatusCard />
|
||||
|
||||
{/* Alarm webhooks into the local network (SSRF protection) */}
|
||||
<div className="p-4 bg-canvas border border-border rounded-card flex items-center justify-between gap-4">
|
||||
<div className="pr-4">
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
import React, { useEffect, useRef, useState } from 'react';
|
||||
import { AlertCircle, Check, Loader2, Mic, RefreshCw, Square } from 'lucide-react';
|
||||
import { api } from '../../lib/api';
|
||||
import { VoiceAdminStatus } from '../../lib/types';
|
||||
|
||||
const TEST_SECONDS = 6;
|
||||
|
||||
// Admin: which speech recognition is active. Configured only by environment (STT_*), so this card
|
||||
// just shows the state and offers a short test recording through the own Whisper server.
|
||||
export const VoiceStatusCard: React.FC = () => {
|
||||
const [info, setInfo] = useState<VoiceAdminStatus | null>(null);
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [recording, setRecording] = useState(false);
|
||||
const [testing, setTesting] = useState(false);
|
||||
const [result, setResult] = useState<{ text: string; ms: number } | null>(null);
|
||||
const recorderRef = useRef<MediaRecorder | null>(null);
|
||||
const stopTimerRef = useRef<number | null>(null);
|
||||
|
||||
const load = async () => {
|
||||
setLoading(true);
|
||||
try {
|
||||
setInfo(await api.adminVoiceStatus());
|
||||
setError(null);
|
||||
} catch (err: any) {
|
||||
setError(err.message || 'Status konnte nicht geladen werden');
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
void load();
|
||||
return () => {
|
||||
if (stopTimerRef.current !== null) window.clearTimeout(stopTimerRef.current);
|
||||
recorderRef.current?.stream.getTracks().forEach((t) => t.stop());
|
||||
};
|
||||
}, []);
|
||||
|
||||
const stopRecording = () => {
|
||||
if (stopTimerRef.current !== null) window.clearTimeout(stopTimerRef.current);
|
||||
stopTimerRef.current = null;
|
||||
if (recorderRef.current?.state === 'recording') recorderRef.current.stop();
|
||||
};
|
||||
|
||||
const startTest = async () => {
|
||||
setResult(null);
|
||||
setError(null);
|
||||
let stream: MediaStream;
|
||||
try {
|
||||
stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
||||
} catch {
|
||||
setError('Kein Zugriff auf das Mikrofon (Berechtigung, oder die Seite läuft nicht über HTTPS)');
|
||||
return;
|
||||
}
|
||||
const recorder = new MediaRecorder(stream);
|
||||
const chunks: Blob[] = [];
|
||||
recorder.ondataavailable = (e) => {
|
||||
if (e.data.size > 0) chunks.push(e.data);
|
||||
};
|
||||
recorder.onstop = async () => {
|
||||
stream.getTracks().forEach((t) => t.stop());
|
||||
setRecording(false);
|
||||
setTesting(true);
|
||||
const started = performance.now();
|
||||
try {
|
||||
const text = await api.transcribe(new Blob(chunks, { type: recorder.mimeType }));
|
||||
setResult({ text, ms: Math.round(performance.now() - started) });
|
||||
} catch (err: any) {
|
||||
setError(err.message || 'Spracherkennung fehlgeschlagen');
|
||||
} finally {
|
||||
setTesting(false);
|
||||
}
|
||||
};
|
||||
recorderRef.current = recorder;
|
||||
recorder.start();
|
||||
setRecording(true);
|
||||
stopTimerRef.current = window.setTimeout(stopRecording, TEST_SECONDS * 1000);
|
||||
};
|
||||
|
||||
const status = info?.status;
|
||||
const serverOk = info?.mode === 'server' && status?.reachable && status?.modelReady;
|
||||
|
||||
return (
|
||||
<div className="p-4 bg-canvas border border-border rounded-card space-y-3">
|
||||
<div className="flex items-start justify-between gap-3">
|
||||
<div>
|
||||
<h3 className="font-bold text-main flex items-center gap-1.5">
|
||||
<Mic className="w-4 h-4 text-accent" />
|
||||
Spracherkennung
|
||||
</h3>
|
||||
<p className="text-muted mt-0.5">
|
||||
Wird nur über Umgebungsvariablen eingestellt (<code className="font-mono">STT_URL</code>,{' '}
|
||||
<code className="font-mono">STT_MODEL</code> …); Fehler stehen im Container-Log.
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
onClick={load}
|
||||
disabled={loading}
|
||||
title="Status neu prüfen"
|
||||
className="btn-action h-11 w-11 sm:h-8 sm:w-8 flex items-center justify-center shrink-0 disabled:opacity-40"
|
||||
>
|
||||
<RefreshCw className="w-3.5 h-3.5" />
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{info && (
|
||||
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1 text-xs">
|
||||
<dt className="text-muted">Aktiv</dt>
|
||||
<dd className="text-main font-semibold">
|
||||
{info.mode === 'server'
|
||||
? 'Eigener Server (Whisper)'
|
||||
: info.mode === 'off'
|
||||
? 'Ausgeschaltet'
|
||||
: info.demo && info.url
|
||||
? 'Browser (in der Demo immer)'
|
||||
: 'Browser (Spracherkennung des Browsers)'}
|
||||
</dd>
|
||||
{info.url && (
|
||||
<>
|
||||
<dt className="text-muted">Server</dt>
|
||||
<dd className="text-main font-mono break-all">{info.url}</dd>
|
||||
<dt className="text-muted">Modell</dt>
|
||||
<dd className="text-main font-mono break-all">{info.model}</dd>
|
||||
<dt className="text-muted">Sprache</dt>
|
||||
<dd className="text-main">{info.language || 'automatisch'}</dd>
|
||||
</>
|
||||
)}
|
||||
{info.mode === 'server' && (
|
||||
<>
|
||||
<dt className="text-muted">Zustand</dt>
|
||||
<dd className="flex items-center gap-1.5">
|
||||
{serverOk ? (
|
||||
<span className="text-status-ok flex items-center gap-1">
|
||||
<Check className="w-3.5 h-3.5" /> Erreichbar, Modell geladen
|
||||
</span>
|
||||
) : (
|
||||
<span className="text-status-err flex items-center gap-1">
|
||||
<AlertCircle className="w-3.5 h-3.5" />
|
||||
{status?.reachable ? 'Modell fehlt noch (wird heruntergeladen)' : 'Nicht erreichbar'}
|
||||
{status?.error && !status.reachable ? `: ${status.error}` : ''}
|
||||
</span>
|
||||
)}
|
||||
</dd>
|
||||
<dt className="text-muted">Grenzen</dt>
|
||||
<dd className="text-main">
|
||||
{Math.round(info.maxSeconds / 60)} Min. bzw. {info.maxUploadMb} MB pro Aufnahme, {info.maxParallel}{' '}
|
||||
gleichzeitig, {info.perUserPerHour} pro Person und Stunde, {info.timeoutSeconds} s Zeitlimit
|
||||
</dd>
|
||||
</>
|
||||
)}
|
||||
</dl>
|
||||
)}
|
||||
|
||||
{info?.mode === 'server' && (
|
||||
<div className="pt-2 border-t border-border flex flex-col sm:flex-row sm:items-center gap-2">
|
||||
{recording ? (
|
||||
<button
|
||||
type="button"
|
||||
onClick={stopRecording}
|
||||
className="btn-danger h-11 sm:h-8 px-3 text-xs flex items-center justify-center gap-1.5"
|
||||
>
|
||||
<Square className="w-3.5 h-3.5" /> Aufnahme beenden
|
||||
</button>
|
||||
) : (
|
||||
<button
|
||||
type="button"
|
||||
onClick={startTest}
|
||||
disabled={testing}
|
||||
className="btn-secondary h-11 sm:h-8 px-3 text-xs flex items-center justify-center gap-1.5 disabled:opacity-40"
|
||||
>
|
||||
{testing ? <Loader2 className="w-3.5 h-3.5 animate-spin" /> : <Mic className="w-3.5 h-3.5" />}
|
||||
{testing ? 'Wird erkannt…' : `Testaufnahme (${TEST_SECONDS} s)`}
|
||||
</button>
|
||||
)}
|
||||
{recording && <span className="text-muted">Sprich jetzt …</span>}
|
||||
{result && (
|
||||
<span className="text-main">
|
||||
„{result.text || '(nichts erkannt)'}“ <span className="text-faint">· {(result.ms / 1000).toFixed(1)} s</span>
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{error && <p className="text-status-err text-xs">{error}</p>}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -35,6 +35,7 @@ import {
|
||||
TagSummary,
|
||||
User,
|
||||
UserInvite,
|
||||
VoiceAdminStatus,
|
||||
WSMessage,
|
||||
} from './types';
|
||||
import { extractHashtags } from './hashtags';
|
||||
@@ -1047,6 +1048,26 @@ export const api = {
|
||||
await request(`/api/v1/admin/invites/${id}`, { method: 'DELETE' });
|
||||
},
|
||||
|
||||
async adminVoiceStatus(): Promise<VoiceAdminStatus> {
|
||||
return request<VoiceAdminStatus>('/api/v1/admin/voice');
|
||||
},
|
||||
|
||||
// Sends a recording to the own Whisper server (voice mode "server") and returns the text
|
||||
async transcribe(audio: Blob): Promise<string> {
|
||||
const token = getToken();
|
||||
const ext = audio.type.includes('mp4') ? 'm4a' : audio.type.includes('ogg') ? 'ogg' : 'webm';
|
||||
const form = new FormData();
|
||||
form.append('file', audio, `aufnahme.${ext}`);
|
||||
const res = await fetch('/api/v1/voice/transcribe', {
|
||||
method: 'POST',
|
||||
headers: token ? { Authorization: `Bearer ${token}` } : {},
|
||||
body: form,
|
||||
});
|
||||
const data = await res.json().catch(() => ({}));
|
||||
if (!res.ok) throw new Error(data.error || `Spracherkennung fehlgeschlagen (HTTP ${res.status})`);
|
||||
return data.text || '';
|
||||
},
|
||||
|
||||
async adminGetSettings(): Promise<SystemSettings> {
|
||||
return request<SystemSettings>('/api/v1/admin/settings');
|
||||
},
|
||||
|
||||
@@ -21,6 +21,26 @@ export interface AuthStatus {
|
||||
privacyUrl?: string;
|
||||
hasPrivacyContent?: boolean;
|
||||
};
|
||||
// Voice input: own Whisper server, the browser's recognition, or switched off (STT_* environment)
|
||||
voice?: { mode: VoiceMode; maxSeconds: number };
|
||||
}
|
||||
|
||||
export type VoiceMode = 'server' | 'browser' | 'off';
|
||||
|
||||
// Admin view of the speech recognition (read-only, configured by environment)
|
||||
export interface VoiceAdminStatus {
|
||||
mode: VoiceMode;
|
||||
demo: boolean;
|
||||
url?: string;
|
||||
model?: string;
|
||||
language?: string;
|
||||
apiKeySet?: boolean;
|
||||
maxSeconds: number;
|
||||
maxUploadMb: number;
|
||||
maxParallel: number;
|
||||
perUserPerHour: number;
|
||||
timeoutSeconds: number;
|
||||
status?: { reachable: boolean; modelReady: boolean; error?: string; checkedAt?: string };
|
||||
}
|
||||
|
||||
// Who can see whom: nobody, members of the same organization, or everybody (Doku/sharing-and-permissions.md)
|
||||
|
||||
@@ -12,8 +12,10 @@ import (
|
||||
// with huge uploads. Note sync and imports need more; their handlers apply the same limits again.
|
||||
const defaultMaxBody = 2 << 20
|
||||
|
||||
func bodyLimitFor(path string) int64 {
|
||||
func (s *Server) bodyLimitFor(path string) int64 {
|
||||
switch {
|
||||
case path == "/api/v1/voice/transcribe":
|
||||
return int64(s.cfg.Speech.MaxUploadMB)<<20 + 1<<20
|
||||
case strings.HasPrefix(path, "/api/v1/import/"):
|
||||
return storage.MaxImportUploadBytes + 1<<20
|
||||
case strings.HasPrefix(path, "/api/v1/docs/") && strings.HasSuffix(path, "/sync"):
|
||||
@@ -22,10 +24,10 @@ func bodyLimitFor(path string) int64 {
|
||||
return defaultMaxBody
|
||||
}
|
||||
|
||||
func bodyLimit(next http.Handler) http.Handler {
|
||||
func (s *Server) bodyLimit(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Body != nil && r.Body != http.NoBody {
|
||||
r.Body = http.MaxBytesReader(w, r.Body, bodyLimitFor(r.URL.Path))
|
||||
r.Body = http.MaxBytesReader(w, r.Body, s.bodyLimitFor(r.URL.Path))
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
|
||||
@@ -16,7 +16,7 @@ func TestRequestBodyLimited(t *testing.T) {
|
||||
if rec := sessionCall(router, "POST", "/api/v1/inbox", token, `{"content":"klein"}`, ""); rec.Code >= 300 {
|
||||
t.Fatalf("normal body: got %d %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if bodyLimitFor("/api/v1/docs/doc_1/sync") != syncMaxBody || bodyLimitFor("/api/v1/import/obsidian") <= defaultMaxBody {
|
||||
if srv.bodyLimitFor("/api/v1/docs/doc_1/sync") != syncMaxBody || srv.bodyLimitFor("/api/v1/import/obsidian") <= defaultMaxBody {
|
||||
t.Fatal("sync and import keep their larger limits")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -44,6 +44,8 @@ func (s *Server) handleAuthStatus(w http.ResponseWriter, r *http.Request) {
|
||||
"demoMode": s.cfg.DemoMode,
|
||||
"sharingMode": s.store.SharingMode(),
|
||||
"passwordReset": s.passwordResetAvailable(),
|
||||
// How voice input works: "server" (own Whisper), "browser" or "off"; set by STT_* only
|
||||
"voice": map[string]interface{}{"mode": s.voiceMode(), "maxSeconds": s.cfg.Speech.MaxSeconds},
|
||||
// Public so the login screen can link Impressum and Datenschutz
|
||||
"legal": map[string]interface{}{
|
||||
"imprintUrl": s.cfg.Legal.ImprintURL,
|
||||
|
||||
@@ -33,6 +33,7 @@ type Server struct {
|
||||
demo demoState // periodic reset in demo mode (demo.go)
|
||||
streamTickets streamTickets // one-time tickets for WebSocket/SSE (streamticket.go)
|
||||
loginGuard *loginGuard // brute-force protection (loginguard.go)
|
||||
voice *voiceService // self-hosted speech recognition (voice.go)
|
||||
}
|
||||
|
||||
func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS fs.FS) *Server {
|
||||
@@ -45,6 +46,10 @@ func NewServer(cfg *config.Config, store *storage.Store, wsHub *ws.Hub, staticFS
|
||||
notifier: collab.NewNotifier(),
|
||||
}
|
||||
srv.loginGuard = newLoginGuard(!cfg.DisableLoginProtection)
|
||||
srv.voice = newVoiceService(cfg.Speech.MaxParallel)
|
||||
if srv.voiceMode() == "server" {
|
||||
go srv.startVoiceCheck()
|
||||
}
|
||||
wsHub.SetRoomAuthorizer(srv.canJoinRoom)
|
||||
wsHub.SetOriginCheck(srv.streamOriginAllowed)
|
||||
// Next occurrences of repeating tasks appear live on every connected client
|
||||
@@ -84,7 +89,7 @@ func (s *Server) Router() http.Handler {
|
||||
|
||||
r.Use(s.clientIPMiddleware)
|
||||
r.Use(s.securityHeaders)
|
||||
r.Use(bodyLimit)
|
||||
r.Use(s.bodyLimit)
|
||||
r.Use(s.demoResetGate)
|
||||
r.Use(middleware.Recoverer)
|
||||
r.Use(s.apiLoggingMiddleware)
|
||||
@@ -126,6 +131,7 @@ func (s *Server) Router() http.Handler {
|
||||
r.Get("/api/v1/auth/me", s.handleMe)
|
||||
r.Post("/api/v1/events/rooms", s.handleSSERooms)
|
||||
r.Post("/api/v1/auth/stream-ticket", s.handleStreamTicket)
|
||||
r.Post("/api/v1/voice/transcribe", s.handleVoiceTranscribe)
|
||||
r.Get("/api/v1/auth/sessions", s.handleListSessions)
|
||||
r.Delete("/api/v1/auth/sessions/{id}", s.notInDemo(s.handleEndSession))
|
||||
r.Post("/api/v1/auth/logout", s.handleLogout)
|
||||
@@ -271,6 +277,7 @@ func (s *Server) Router() http.Handler {
|
||||
r.Get("/api/v1/admin/backups/{name}", s.handleAdminDownloadBackup)
|
||||
r.Put("/api/v1/admin/settings", s.notInDemo(s.handleAdminUpdateSettings))
|
||||
r.Get("/api/v1/admin/settings/sharing-impact", s.handleAdminSharingImpact)
|
||||
r.Get("/api/v1/admin/voice", s.handleAdminVoiceStatus)
|
||||
|
||||
// Organizations decide who can see whom; only server admins manage them
|
||||
r.Get("/api/v1/admin/organizations", s.handleAdminGetOrganizations)
|
||||
|
||||
@@ -0,0 +1,306 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/textproto"
|
||||
"path"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/sandrodach/mindsift/internal/auth"
|
||||
)
|
||||
|
||||
// Self-hosted speech recognition: the browser records audio and sends it here, MindSift passes it on
|
||||
// to an OpenAI-compatible Whisper server (STT_URL, e.g. speaches) and returns the text. Nothing is
|
||||
// stored. A limited number of transcriptions run at the same time (STT_MAX_PARALLEL), the others
|
||||
// wait in line; each user has an hourly budget (STT_PER_USER_PER_HOUR). Problems are logged, the
|
||||
// browser only gets a short message.
|
||||
|
||||
type voiceService struct {
|
||||
slots chan struct{}
|
||||
|
||||
mu sync.Mutex
|
||||
usage map[string][]time.Time // userID -> transcriptions within the last hour
|
||||
status voiceStatus
|
||||
checked time.Time
|
||||
}
|
||||
|
||||
// voiceStatus is what the admin page shows about the Whisper server.
|
||||
type voiceStatus struct {
|
||||
Reachable bool `json:"reachable"`
|
||||
ModelReady bool `json:"modelReady"`
|
||||
Error string `json:"error,omitempty"`
|
||||
CheckedAt string `json:"checkedAt,omitempty"`
|
||||
}
|
||||
|
||||
func newVoiceService(parallel int) *voiceService {
|
||||
if parallel < 1 {
|
||||
parallel = 1
|
||||
}
|
||||
return &voiceService{slots: make(chan struct{}, parallel), usage: map[string][]time.Time{}}
|
||||
}
|
||||
|
||||
func (s *Server) voiceMode() string {
|
||||
return s.cfg.Speech.Mode(s.cfg.DemoMode)
|
||||
}
|
||||
|
||||
// allowVoice counts one transcription for the user; false when the hourly budget is used up.
|
||||
func (v *voiceService) allowVoice(userID string, perHour int, now time.Time) bool {
|
||||
v.mu.Lock()
|
||||
defer v.mu.Unlock()
|
||||
recent := v.usage[userID][:0]
|
||||
for _, t := range v.usage[userID] {
|
||||
if now.Sub(t) < time.Hour {
|
||||
recent = append(recent, t)
|
||||
}
|
||||
}
|
||||
if len(recent) >= perHour {
|
||||
v.usage[userID] = recent
|
||||
return false
|
||||
}
|
||||
v.usage[userID] = append(recent, now)
|
||||
return true
|
||||
}
|
||||
|
||||
// sttEndpoint joins the base URL with an API path; a base URL ending in /v1 works as well.
|
||||
func (s *Server) sttEndpoint(p string) string {
|
||||
base := strings.TrimSuffix(s.cfg.Speech.URL, "/v1")
|
||||
return base + p
|
||||
}
|
||||
|
||||
func (s *Server) sttRequest(ctx context.Context, method, p string, body io.Reader, contentType string) (*http.Response, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, method, s.sttEndpoint(p), body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if contentType != "" {
|
||||
req.Header.Set("Content-Type", contentType)
|
||||
}
|
||||
if s.cfg.Speech.APIKey != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+s.cfg.Speech.APIKey)
|
||||
}
|
||||
return http.DefaultClient.Do(req)
|
||||
}
|
||||
|
||||
func (s *Server) handleVoiceTranscribe(w http.ResponseWriter, r *http.Request) {
|
||||
if s.voiceMode() != "server" {
|
||||
writeError(w, http.StatusNotFound, "Serverseitige Spracherkennung ist nicht eingerichtet")
|
||||
return
|
||||
}
|
||||
cfg := s.cfg.Speech
|
||||
userID := auth.GetUserID(r.Context())
|
||||
if !s.voice.allowVoice(userID, cfg.PerUserPerHour, time.Now()) {
|
||||
writeError(w, http.StatusTooManyRequests, "Stundenlimit für die Spracherkennung erreicht. Bitte später erneut versuchen.")
|
||||
return
|
||||
}
|
||||
allowLongTransfer(w)
|
||||
file, header, err := r.FormFile("file")
|
||||
if err != nil {
|
||||
var tooBig *http.MaxBytesError
|
||||
if errors.As(err, &tooBig) {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("Aufnahme zu groß (höchstens %d MB)", cfg.MaxUploadMB))
|
||||
return
|
||||
}
|
||||
writeError(w, http.StatusBadRequest, "Keine Aufnahme im Feld \"file\"")
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
audio, err := io.ReadAll(file)
|
||||
if err != nil || len(audio) == 0 {
|
||||
writeError(w, http.StatusBadRequest, "Aufnahme konnte nicht gelesen werden")
|
||||
return
|
||||
}
|
||||
|
||||
// Wait for a free slot; the request may be cancelled meanwhile (user stopped waiting)
|
||||
ctx, cancel := context.WithTimeout(r.Context(), time.Duration(cfg.TimeoutSeconds)*time.Second)
|
||||
defer cancel()
|
||||
select {
|
||||
case s.voice.slots <- struct{}{}:
|
||||
defer func() { <-s.voice.slots }()
|
||||
case <-ctx.Done():
|
||||
slog.Warn("Spracherkennung: keine freie Kapazität innerhalb der Zeitgrenze", "user_id", userID, "parallel", cfg.MaxParallel)
|
||||
writeError(w, http.StatusServiceUnavailable, "Die Spracherkennung ist gerade ausgelastet. Bitte gleich noch einmal versuchen.")
|
||||
return
|
||||
}
|
||||
|
||||
started := time.Now()
|
||||
text, err := s.transcribe(ctx, audio, header.Filename, header.Header.Get("Content-Type"))
|
||||
if err != nil {
|
||||
slog.Warn("Spracherkennung fehlgeschlagen", "url", s.sttEndpoint("/v1/audio/transcriptions"), "model", cfg.Model,
|
||||
"bytes", len(audio), "duration_ms", time.Since(started).Milliseconds(), "error", err)
|
||||
msg := "Spracherkennung fehlgeschlagen"
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
msg = "Spracherkennung hat zu lange gedauert"
|
||||
}
|
||||
writeError(w, http.StatusBadGateway, msg)
|
||||
return
|
||||
}
|
||||
slog.Debug("Spracherkennung", "user_id", userID, "bytes", len(audio), "duration_ms", time.Since(started).Milliseconds())
|
||||
writeJSON(w, http.StatusOK, map[string]string{"text": text})
|
||||
}
|
||||
|
||||
// transcribe sends the audio to the Whisper server (OpenAI API: POST /v1/audio/transcriptions).
|
||||
func (s *Server) transcribe(ctx context.Context, audio []byte, filename, contentType string) (string, error) {
|
||||
var body bytes.Buffer
|
||||
mw := multipart.NewWriter(&body)
|
||||
if filename == "" {
|
||||
filename = "aufnahme.webm"
|
||||
}
|
||||
part := textproto.MIMEHeader{}
|
||||
part.Set("Content-Disposition", fmt.Sprintf(`form-data; name="file"; filename="%s"`, path.Base(strings.ReplaceAll(filename, `"`, ""))))
|
||||
if contentType == "" {
|
||||
contentType = "application/octet-stream"
|
||||
}
|
||||
part.Set("Content-Type", contentType)
|
||||
fw, err := mw.CreatePart(part)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, err := fw.Write(audio); err != nil {
|
||||
return "", err
|
||||
}
|
||||
_ = mw.WriteField("model", s.cfg.Speech.Model)
|
||||
if lang := s.cfg.Speech.Language; lang != "" && lang != "auto" {
|
||||
_ = mw.WriteField("language", lang)
|
||||
}
|
||||
_ = mw.WriteField("response_format", "json")
|
||||
if err := mw.Close(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
resp, err := s.sttRequest(ctx, http.MethodPost, "/v1/audio/transcriptions", &body, mw.FormDataContentType())
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return "", fmt.Errorf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(raw)))
|
||||
}
|
||||
var out struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return "", fmt.Errorf("unerwartete Antwort: %s", strings.TrimSpace(string(raw)))
|
||||
}
|
||||
return strings.TrimSpace(out.Text), nil
|
||||
}
|
||||
|
||||
// checkVoice asks the Whisper server whether it is up and has the model.
|
||||
func (s *Server) checkVoice(ctx context.Context) voiceStatus {
|
||||
st := voiceStatus{CheckedAt: time.Now().UTC().Format(time.RFC3339)}
|
||||
resp, err := s.sttRequest(ctx, http.MethodGet, "/v1/models", nil, "")
|
||||
if err != nil {
|
||||
st.Error = err.Error()
|
||||
return st
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
st.Error = fmt.Sprintf("GET /v1/models: HTTP %d", resp.StatusCode)
|
||||
return st
|
||||
}
|
||||
st.Reachable = true
|
||||
var list struct {
|
||||
Data []struct {
|
||||
ID string `json:"id"`
|
||||
} `json:"data"`
|
||||
}
|
||||
_ = json.NewDecoder(io.LimitReader(resp.Body, 4<<20)).Decode(&list)
|
||||
for _, m := range list.Data {
|
||||
if m.ID == s.cfg.Speech.Model {
|
||||
st.ModelReady = true
|
||||
}
|
||||
}
|
||||
if !st.ModelReady {
|
||||
st.Error = "Modell ist noch nicht heruntergeladen"
|
||||
}
|
||||
return st
|
||||
}
|
||||
|
||||
func (s *Server) setVoiceStatus(st voiceStatus) {
|
||||
s.voice.mu.Lock()
|
||||
s.voice.status = st
|
||||
s.voice.checked = time.Now()
|
||||
s.voice.mu.Unlock()
|
||||
}
|
||||
|
||||
// startVoiceCheck waits for the Whisper server after startup (it may start later than MindSift) and
|
||||
// asks it to download the model if it does not have it yet, so the first recording works.
|
||||
func (s *Server) startVoiceCheck() {
|
||||
cfg := s.cfg.Speech
|
||||
slog.Info("Spracherkennung: Server-Modus", "url", cfg.URL, "model", cfg.Model, "language", cfg.Language,
|
||||
"max_parallel", cfg.MaxParallel, "per_user_per_hour", cfg.PerUserPerHour, "max_upload_mb", cfg.MaxUploadMB)
|
||||
downloadRequested := false
|
||||
for attempt := 0; ; attempt++ {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
st := s.checkVoice(ctx)
|
||||
cancel()
|
||||
s.setVoiceStatus(st)
|
||||
switch {
|
||||
case st.ModelReady:
|
||||
slog.Info("Spracherkennung bereit", "url", cfg.URL, "model", cfg.Model)
|
||||
return
|
||||
case st.Reachable && !downloadRequested:
|
||||
downloadRequested = true
|
||||
slog.Info("Spracherkennung: Modell wird heruntergeladen (kann einige Minuten dauern)", "model", cfg.Model)
|
||||
go s.requestModelDownload()
|
||||
case !st.Reachable && attempt%6 == 0:
|
||||
slog.Warn("Spracherkennung: Whisper-Server nicht erreichbar, neuer Versuch folgt", "url", cfg.URL, "error", st.Error)
|
||||
}
|
||||
time.Sleep(10 * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) requestModelDownload() {
|
||||
// Model IDs contain a slash (owner/name); speaches takes them as a path
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Hour)
|
||||
defer cancel()
|
||||
resp, err := s.sttRequest(ctx, http.MethodPost, "/v1/models/"+s.cfg.Speech.Model, nil, "")
|
||||
if err != nil {
|
||||
slog.Warn("Spracherkennung: Modell-Download fehlgeschlagen", "model", s.cfg.Speech.Model, "error", err)
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
|
||||
if resp.StatusCode >= 300 {
|
||||
slog.Warn("Spracherkennung: Modell-Download abgelehnt", "model", s.cfg.Speech.Model, "status", resp.StatusCode, "body", strings.TrimSpace(string(raw)))
|
||||
}
|
||||
}
|
||||
|
||||
// handleAdminVoiceStatus shows the admin which speech recognition is active (configured by
|
||||
// environment only) and checks the Whisper server right now.
|
||||
func (s *Server) handleAdminVoiceStatus(w http.ResponseWriter, r *http.Request) {
|
||||
cfg := s.cfg.Speech
|
||||
out := map[string]interface{}{
|
||||
"mode": s.voiceMode(),
|
||||
"demo": s.cfg.DemoMode,
|
||||
"maxSeconds": cfg.MaxSeconds,
|
||||
"maxUploadMb": cfg.MaxUploadMB,
|
||||
"maxParallel": cfg.MaxParallel,
|
||||
"perUserPerHour": cfg.PerUserPerHour,
|
||||
"timeoutSeconds": cfg.TimeoutSeconds,
|
||||
}
|
||||
if cfg.URL != "" {
|
||||
out["url"] = cfg.URL
|
||||
out["model"] = cfg.Model
|
||||
out["language"] = cfg.Language
|
||||
out["apiKeySet"] = cfg.APIKey != ""
|
||||
}
|
||||
if s.voiceMode() == "server" {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
|
||||
st := s.checkVoice(ctx)
|
||||
cancel()
|
||||
s.setVoiceStatus(st)
|
||||
out["status"] = st
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"github.com/sandrodach/mindsift/internal/config"
|
||||
)
|
||||
|
||||
// fakeWhisper answers like speaches: model list and OpenAI-style transcriptions.
|
||||
func fakeWhisper(t *testing.T, models []string, got *map[string]string, downloads *int32) *httptest.Server {
|
||||
t.Helper()
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
switch {
|
||||
case r.Method == http.MethodGet && r.URL.Path == "/v1/models":
|
||||
var data []map[string]string
|
||||
for _, m := range models {
|
||||
data = append(data, map[string]string{"id": m})
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(map[string]interface{}{"data": data})
|
||||
case r.Method == http.MethodPost && strings.HasPrefix(r.URL.Path, "/v1/models/"):
|
||||
atomic.AddInt32(downloads, 1)
|
||||
w.WriteHeader(http.StatusCreated)
|
||||
case r.Method == http.MethodPost && r.URL.Path == "/v1/audio/transcriptions":
|
||||
if r.Header.Get("Authorization") != "Bearer geheim" {
|
||||
http.Error(w, "unauthorized", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
_ = r.ParseMultipartForm(10 << 20)
|
||||
f, _, _ := r.FormFile("file")
|
||||
audio, _ := io.ReadAll(f)
|
||||
*got = map[string]string{"model": r.FormValue("model"), "language": r.FormValue("language"), "audio": string(audio)}
|
||||
_ = json.NewEncoder(w).Encode(map[string]string{"text": " Hallo Welt, wie geht's? "})
|
||||
default:
|
||||
http.NotFound(w, r)
|
||||
}
|
||||
}))
|
||||
t.Cleanup(ts.Close)
|
||||
return ts
|
||||
}
|
||||
|
||||
func voiceUpload(t *testing.T, router http.Handler, token string, audio []byte) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
var body bytes.Buffer
|
||||
mw := multipart.NewWriter(&body)
|
||||
fw, _ := mw.CreateFormFile("file", "aufnahme.webm")
|
||||
_, _ = fw.Write(audio)
|
||||
_ = mw.Close()
|
||||
req := httptest.NewRequest("POST", "/api/v1/voice/transcribe", &body)
|
||||
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
rec := httptest.NewRecorder()
|
||||
router.ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
func voiceServer(t *testing.T, speech config.SpeechConfig) (*Server, string) {
|
||||
t.Helper()
|
||||
srv, _, token := setupTestServer(t)
|
||||
srv.cfg.Speech = speech
|
||||
srv.voice = newVoiceService(speech.MaxParallel)
|
||||
return srv, token
|
||||
}
|
||||
|
||||
func TestVoiceTranscribeForwardsToWhisper(t *testing.T) {
|
||||
var got map[string]string
|
||||
var downloads int32
|
||||
whisper := fakeWhisper(t, []string{"Systran/faster-whisper-small"}, &got, &downloads)
|
||||
srv, token := voiceServer(t, config.SpeechConfig{URL: whisper.URL + "/v1", Model: "Systran/faster-whisper-small", Language: "de",
|
||||
APIKey: "geheim", MaxSeconds: 600, MaxUploadMB: 1, MaxParallel: 1, PerUserPerHour: 2, TimeoutSeconds: 10})
|
||||
router := srv.Router()
|
||||
|
||||
rec := voiceUpload(t, router, token, []byte("RIFF-audio"))
|
||||
if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"text":"Hallo Welt, wie geht's?"`) {
|
||||
t.Fatalf("transcribe: %d %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if got["model"] != "Systran/faster-whisper-small" || got["language"] != "de" || got["audio"] != "RIFF-audio" {
|
||||
t.Fatalf("forwarded wrong data: %+v", got)
|
||||
}
|
||||
// Hourly budget per user
|
||||
_ = voiceUpload(t, router, token, []byte("x"))
|
||||
if rec := voiceUpload(t, router, token, []byte("x")); rec.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("over the hourly budget: want 429, got %d", rec.Code)
|
||||
}
|
||||
// Model already there: the startup check reports ready and requests no download
|
||||
st := srv.checkVoice(context.Background())
|
||||
if !st.Reachable || !st.ModelReady || downloads != 0 {
|
||||
t.Fatalf("status %+v, downloads %d", st, downloads)
|
||||
}
|
||||
// Status for the login screen and the admin card, without the API key
|
||||
if body := sessionCall(router, "GET", "/api/v1/auth/status", "", "", "").Body.String(); !strings.Contains(body, `"mode":"server"`) {
|
||||
t.Fatalf("auth status: %s", body)
|
||||
}
|
||||
body := sessionCall(router, "GET", "/api/v1/admin/voice", token, "", "").Body.String()
|
||||
if !strings.Contains(body, `"modelReady":true`) || strings.Contains(body, "geheim") {
|
||||
t.Fatalf("admin voice status: %s", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVoiceUploadLimitAndModes(t *testing.T) {
|
||||
var got map[string]string
|
||||
var downloads int32
|
||||
whisper := fakeWhisper(t, nil, &got, &downloads)
|
||||
srv, token := voiceServer(t, config.SpeechConfig{URL: whisper.URL, Model: "m", MaxUploadMB: 1, MaxParallel: 1, PerUserPerHour: 10, TimeoutSeconds: 10})
|
||||
if rec := voiceUpload(t, srv.Router(), token, bytes.Repeat([]byte("x"), 3<<20)); rec.Code != http.StatusRequestEntityTooLarge && rec.Code != http.StatusBadRequest {
|
||||
t.Fatalf("too large: want 413/400, got %d", rec.Code)
|
||||
}
|
||||
if st := srv.checkVoice(context.Background()); st.ModelReady || !st.Reachable {
|
||||
t.Fatalf("missing model must be reported: %+v", st)
|
||||
}
|
||||
|
||||
if m := (config.SpeechConfig{URL: "http://whisper:8000"}).Mode(true); m != "browser" {
|
||||
t.Fatalf("demo must use the browser, got %s", m)
|
||||
}
|
||||
if m := (config.SpeechConfig{}).Mode(false); m != "browser" {
|
||||
t.Fatalf("without URL: browser, got %s", m)
|
||||
}
|
||||
if m := (config.SpeechConfig{URL: "x", Disabled: true}).Mode(false); m != "off" {
|
||||
t.Fatalf("disabled: off, got %s", m)
|
||||
}
|
||||
browserSrv, browserToken := voiceServer(t, config.SpeechConfig{MaxParallel: 1, PerUserPerHour: 10})
|
||||
if rec := voiceUpload(t, browserSrv.Router(), browserToken, []byte("x")); rec.Code != http.StatusNotFound {
|
||||
t.Fatalf("browser mode: want 404, got %d", rec.Code)
|
||||
}
|
||||
}
|
||||
@@ -43,6 +43,37 @@ type Config struct {
|
||||
|
||||
// Legal notices (Impressum, Datenschutz): an external URL and/or Markdown text, shown even before login
|
||||
Legal LegalTexts
|
||||
|
||||
// Speech recognition, configured only by environment (STT_*)
|
||||
Speech SpeechConfig
|
||||
}
|
||||
|
||||
// SpeechConfig: with STT_URL set, voice input is transcribed by a self-hosted, OpenAI-compatible
|
||||
// Whisper server (e.g. speaches); otherwise the browser's own speech recognition is used.
|
||||
type SpeechConfig struct {
|
||||
URL string // base URL, e.g. http://whisper:8000
|
||||
Model string
|
||||
Language string
|
||||
APIKey string
|
||||
Disabled bool // STT_DISABLED=true hides voice input everywhere
|
||||
MaxSeconds int // recordings stop after this long (enforced in the browser)
|
||||
MaxUploadMB int
|
||||
MaxParallel int // transcriptions running at the same time; more wait in line
|
||||
PerUserPerHour int
|
||||
TimeoutSeconds int
|
||||
}
|
||||
|
||||
// Mode is "server", "browser" or "off". The demo never uses the server, so visitors cannot keep
|
||||
// its CPU busy.
|
||||
func (s SpeechConfig) Mode(demo bool) string {
|
||||
switch {
|
||||
case s.Disabled:
|
||||
return "off"
|
||||
case s.URL != "" && !demo:
|
||||
return "server"
|
||||
default:
|
||||
return "browser"
|
||||
}
|
||||
}
|
||||
|
||||
type LegalTexts struct {
|
||||
@@ -179,6 +210,18 @@ func Load() *Config {
|
||||
SMTPFrom: smtpFrom,
|
||||
SMTPTLS: smtpTLS,
|
||||
SMTPSkipVerify: smtpSkipVerify,
|
||||
Speech: SpeechConfig{
|
||||
URL: strings.TrimRight(getEnv("STT_URL", ""), "/"),
|
||||
Model: getEnv("STT_MODEL", "deepdml/faster-whisper-large-v3-turbo-ct2"),
|
||||
Language: getEnv("STT_LANGUAGE", "de"),
|
||||
APIKey: getEnv("STT_API_KEY", ""),
|
||||
Disabled: strings.ToLower(getEnv("STT_DISABLED", "false")) == "true",
|
||||
MaxSeconds: envInt("STT_MAX_SECONDS", 600, 5, 3600),
|
||||
MaxUploadMB: envInt("STT_MAX_UPLOAD_MB", 25, 1, 500),
|
||||
MaxParallel: envInt("STT_MAX_PARALLEL", 1, 1, 32),
|
||||
PerUserPerHour: envInt("STT_PER_USER_PER_HOUR", 60, 1, 100000),
|
||||
TimeoutSeconds: envInt("STT_TIMEOUT_SECONDS", 120, 5, 3600),
|
||||
},
|
||||
Legal: LegalTexts{
|
||||
ImprintURL: legalURL("LEGAL_IMPRINT_URL"),
|
||||
ImprintContent: legalContent("LEGAL_IMPRINT_CONTENT"),
|
||||
@@ -209,6 +252,20 @@ func getEnv(key, fallback string) string {
|
||||
return fallback
|
||||
}
|
||||
|
||||
// envInt reads a whole number within [min, max]; anything else falls back to def (with a warning).
|
||||
func envInt(key string, def, min, max int) int {
|
||||
raw := getEnv(key, "")
|
||||
if raw == "" {
|
||||
return def
|
||||
}
|
||||
n, err := strconv.Atoi(raw)
|
||||
if err != nil || n < min || n > max {
|
||||
slog.Warn("Ungültiger Wert, Standard wird verwendet", "variable", key, "value", raw, "default", def)
|
||||
return def
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func demoResetMinutes() int {
|
||||
n, err := strconv.Atoi(strings.TrimSpace(getEnv("DEMO_RESET_MINUTES", "20")))
|
||||
if err != nil || n < 0 {
|
||||
|
||||
Reference in new issue
Block a user