Add a native Slint front-end for the music player

The React UI is sluggish on the kiosk and the reasons are browser-shaped: a compositor
deciding what gets its own layer, a requestAnimationFrame loop repainting the viewport,
and a JPEG decode per album card per cold start. The last three commits chased that
through the service worker and the cover pipeline. This is the other direction - the
same backend, the same layout, the same German copy, without a browser.

Browse and play only. The typing game, the room-lights page, parent mode and the
IR-remote assignment stay web-only, and nothing here touches python-backend.

Four things carry the performance claim, in rough order of how much they should matter:

Covers are downscaled once, ever. The backend serves one size - 640px on the long edge,
no ?size= - and a grid card is 180. src/covers.rs fetches each cover once on a worker
thread, resizes it, and writes a JPEG thumbnail to ~/.cache/musicmouse-slint/covers/.
Two tiers and only two, which is what makes the cache worth keeping: a per-widget pixel
size would give each album a dozen near-identical files and a fresh decode for each. The
cache is dropped when the websocket announces a rescan, because that is the one moment
the backend rewrites the art behind an unchanged cover URL - the trap sw.js had to learn
about the hard way.

The grid is virtualized. Rust hands the UI the album list pre-chunked into rows and the
view puts those in a ListView, which instantiates only what is on screen. A flat list of
660 cards gives it no rows to skip, hence the chunking. Same purpose as
content-visibility: auto on .grid > .card.

The progress bar animates between the 2 Hz pushes rather than running a clock, so
interpolation costs a property evaluation per frame on the render side and there is no
equivalent of usePlaybackClock. It is suppressed for the frame a track changes on, so a
new track jumps instead of sliding across two unrelated positions.

Nothing on screen animates by itself. The ambient canvas is not ported, for the reason
lib/lowPower.ts already gives: a loop repainting the viewport is a floor you cannot get
under while it runs at all.

The device must not use FemtoVG. On Mesa V3D it draws every runtime-loaded Image as
solid black (slint-ui/slint#11785, open), which here means every album cover; Skia and
the software renderer are unaffected. So `kiosk` is linuxkms + Skia and FemtoVG stays
the default only for desktop development. Both profiles are verified to build; the kiosk
binary links libinput/libgbm/libdrm and no X11 or Wayland at all.

lib/search.ts, lib/keyboard.ts and lib/format.ts were already pure functions with their
own tests, so they port across as pure Rust with theirs: 43 tests, no window required.
The .slint files are layout only - nothing in them formats a number or picks a word.

Verified against the real 660-album library rather than a fixture. tools/headless-shots.sh
renders the UI inside a nested headless compositor and grabs a frame per screen, which
is how that was checked on a machine whose session was locked; a Slint window needs a
real compositor, and under bare Xvfb nothing maps at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-20 22:26:09 +02:00
parent 83d16e0058
commit 6ccb3e458f
21 changed files with 11270 additions and 0 deletions

168
slint-frontend/README.md Normal file
View File

@@ -0,0 +1,168 @@
# Musik Delfin — the Slint front-end
A native front-end for the music player, next to `../web/`. Same backend, same layout,
same German copy; no browser. It exists because the React UI is sluggish on the Pi 5
kiosk, and the things making it sluggish are browser-shaped: a compositor deciding what
gets its own layer, a `requestAnimationFrame` loop that repaints the viewport, and a
JPEG decode per album card per cold start.
Only the music player is here. The typing game (`Tippen`), the room-lights page, parent
mode and the IR-remote assignment are all still web-only.
## Running it
Start the backend, then:
```sh
nix-shell # cargo, the GL/X11/Wayland libs, and Nunito
cargo run # windowed, FemtoVG
cargo run -- --server http://musicdolphin:8080
```
Checks:
```sh
cargo test && cargo clippy --all-targets && cargo fmt --check
```
`tools/headless-shots.sh` renders the UI and screenshots it inside a nested headless
compositor, for checking a layout change on a machine with no display (or a locked one).
A Slint window needs a real compositor - under bare Xvfb nothing maps at all.
## Build profiles
| | Backend | Renderer | For |
|---|---|---|---|
| `cargo run` | winit | FemtoVG | development here |
| `cargo run --features skia` | winit | Skia | checking what the device will show |
| `cargo build --release --no-default-features --features kiosk` | linuxkms + libinput | Skia | the Pi |
**The device must not use FemtoVG.** On a Pi 5 (Mesa V3D), FemtoVG draws every
runtime-loaded `Image` as solid black - [slint#11785][bug], open as of this writing -
which for this app means every album cover. Skia and the software renderer are both
unaffected. FemtoVG stays the default *here* because it builds in seconds and is fine on
desktop Mesa; `kiosk` is the profile that ships.
`--no-default-features` is not optional on that last one: the features are additive, so
leaving `desktop` on would link winit *and* linuxkms into the same binary.
`kiosk` takes the framebuffer directly through KMS/DRM: no X, no Wayland, no compositor.
(`backend-linuxkms-noseat` is deprecated upstream in favour of the libinput variant used
here.) `SLINT_BACKEND=winit-software` forces the software renderer without a rebuild.
[bug]: https://github.com/slint-ui/slint/issues/11785
## Where the speed comes from
Four things, in rough order of how much they matter on the device:
**Covers are downscaled once, ever.** The backend serves one size - 640 px on the long
edge - and has no `?size=`. A grid card is 180 px. `src/covers.rs` fetches each cover
once on a worker thread, resizes it to the tier that was asked for, and writes a JPEG
thumbnail to `~/.cache/musicmouse-slint/covers/<album-id>@<px>.jpg`. Every later start
reads that instead. There are two tiers and only two, which is what makes the cache
worth having: a per-widget pixel size would give each album a dozen near-identical files
and a fresh decode for each.
The cache is dropped wholesale when the websocket announces a rescan
(`{"type": "library"}`), because that is the one moment the backend rewrites the art
behind an unchanged cover URL. That is the bug the web client had to learn about the
hard way - see the comment at `python-backend/musicmouse/services/web/api.py:106`.
**The grid is virtualized.** Rust hands the UI a `[GridRow]` - the album list
pre-chunked into rows - and the view puts those in a `ListView`, which only instantiates
the rows on screen. A flat list of 660 cards gives it no rows to skip, hence the
chunking. This is the counterpart of `content-visibility: auto` on `.grid > .card` in
the web UI, which exists for exactly the same measurement: 340 live cards saturated the
Pi's renderer so completely that the main thread stopped scheduling anything.
**The progress bar animates instead of ticking.** Position arrives twice a second. The
web client interpolates it onto animation frames with a JS clock and a render budget;
here the fill is a Slint `animate width` between the pushed values, so interpolation
costs a property evaluation per frame on the render side and there is no clock at all.
It is suppressed for the one frame a track changes on, so a new track jumps rather than
sliding across two unrelated positions.
**Nothing on screen animates by itself.** The web UI's per-track ambient canvas is not
here. Halving its resolution and its frame rate took it from 53.5% of a core to 25.0%
and 25% was still too much: a loop repainting the viewport is a floor you cannot get
under while it runs at all. The stage keeps its gradient, which is what `?pi=1` already
does on the device this replaces.
## What is deliberately different from the web UI
**No glass blur.** Slint has no `backdrop-filter`, so the panels use a flat translucent
fill - exactly the web app's own `data-blur="off"` fallback. Some depth is genuinely
lost. Faking it by compositing a pre-blurred copy would reintroduce the per-frame cost
that dropping the blur was meant to avoid.
Worth knowing if you ever port the blur back: on the Pi, `backdrop-filter` measured five
times *better* than no blur (25% of a core against 118%), because it is what promotes
each panel to its own compositing layer. That is a browser-compositor artifact and does
not transfer here - but it is why the web UI keeps the panel blur under `?pi=1`.
**No dolphin.** The mascot and its swim/bob loops are not ported.
**Layout** is Slint's, not a transcription of the CSS: fixed row heights instead of
`clamp()` and `vh`, a 1080p-shaped fixed measure for the track list rather than
`max-width`. The colours are the same - `ui/theme.slint` carries the web UI's oklch
tokens converted to sRGB, with the oklch source kept beside each one, because oklch is
the form the design is actually maintained in.
## How it is put together
The split that matters is the web UI's, unchanged: **what is playing** comes from the
backend, **what you are looking at** lives here. The buttons on the mouse, a figurine on
the reader and Home Assistant all move playback too, so this follows it rather than
assuming it is in charge.
Three worker threads and no async runtime - the whole client makes one library request,
one websocket connection and a trickle of covers.
| Path | What it is |
|---|---|
| `src/api.rs` | The wire types and the HTTP calls. Mirrors `services/web/schemas.py`. |
| `src/ws.rs` | The state websocket, with reconnect-and-reseed. |
| `src/library.rs` | The search index and the browse hierarchy. Port of `web/src/lib/search.ts`. |
| `src/keymap.rs` | The keyboard model, as a pure function. Port of `web/src/lib/keyboard.ts`. |
| `src/covers.rs` | The thumbnail cache, and the generated stand-in art. |
| `src/oklch.rs`, `src/format.rs` | Colour conversion; durations. |
| `src/main.rs` | Wiring: worker channels in, Slint models out. |
| `ui/*.slint` | Layout only. Nothing here formats a number or picks a word. |
`src/keymap.rs` and `src/library.rs` are ports of modules that were already pure
functions with their own tests, and they keep them: 43 tests, no window required.
## Keyboard
The same map as the web UI.
| Key | |
|---|---|
| `SPACE` | play / pause (types a space mid-query) |
| `SHIFT+H` / `SHIFT+L` | previous / next |
| `SHIFT+J` / `SHIFT+K` | quieter / louder |
| `SHIFT+M` | mute |
| `SHIFT+←` / `SHIFT+→` | seek 15 s |
| `A-Z`, `0-9` | search albums as you type |
| `?` | search individual tracks |
| `TAB` | cycle Musik / Hörbücher / Podcasts |
| `← ↑ ↓ →` | move the selection while browsing; transport and volume while playing |
| `CTRL+H/J/K/L` | the same, without leaving the home row |
| `CTRL+D` / `CTRL+U` | half a page |
| `ENTER` | play the selection |
| `SHIFT+ENTER` | open the track list |
| `ESC` | back out one layer at a time |
| `/` | back to browsing |
Everything is clickable too.
The special keys are named in `ui/app.slint`, where the `Key.*` constants live, and
reach Rust as `"escape"`, `"left"`, `"enter"` and so on - so `src/keymap.rs` is testable
without a window and without hardcoding keysym values.
## Fonts
Nunito, via `SLINT_DEFAULT_FONT`, which `shell.nix` sets. Slint cannot read the woff2
the web app self-hosts, so this wants the TTF; a deployment can drop one at
`fonts/Nunito.ttf` instead. With neither, everything still renders in the system sans.