Add a native Slint front-end for the music player
The React UI is sluggish on the kiosk and the reasons are browser-shaped: a compositor deciding what gets its own layer, a requestAnimationFrame loop repainting the viewport, and a JPEG decode per album card per cold start. The last three commits chased that through the service worker and the cover pipeline. This is the other direction - the same backend, the same layout, the same German copy, without a browser. Browse and play only. The typing game, the room-lights page, parent mode and the IR-remote assignment stay web-only, and nothing here touches python-backend. Four things carry the performance claim, in rough order of how much they should matter: Covers are downscaled once, ever. The backend serves one size - 640px on the long edge, no ?size= - and a grid card is 180. src/covers.rs fetches each cover once on a worker thread, resizes it, and writes a JPEG thumbnail to ~/.cache/musicmouse-slint/covers/. Two tiers and only two, which is what makes the cache worth keeping: a per-widget pixel size would give each album a dozen near-identical files and a fresh decode for each. The cache is dropped when the websocket announces a rescan, because that is the one moment the backend rewrites the art behind an unchanged cover URL - the trap sw.js had to learn about the hard way. The grid is virtualized. Rust hands the UI the album list pre-chunked into rows and the view puts those in a ListView, which instantiates only what is on screen. A flat list of 660 cards gives it no rows to skip, hence the chunking. Same purpose as content-visibility: auto on .grid > .card. The progress bar animates between the 2 Hz pushes rather than running a clock, so interpolation costs a property evaluation per frame on the render side and there is no equivalent of usePlaybackClock. It is suppressed for the frame a track changes on, so a new track jumps instead of sliding across two unrelated positions. Nothing on screen animates by itself. The ambient canvas is not ported, for the reason lib/lowPower.ts already gives: a loop repainting the viewport is a floor you cannot get under while it runs at all. The device must not use FemtoVG. On Mesa V3D it draws every runtime-loaded Image as solid black (slint-ui/slint#11785, open), which here means every album cover; Skia and the software renderer are unaffected. So `kiosk` is linuxkms + Skia and FemtoVG stays the default only for desktop development. Both profiles are verified to build; the kiosk binary links libinput/libgbm/libdrm and no X11 or Wayland at all. lib/search.ts, lib/keyboard.ts and lib/format.ts were already pure functions with their own tests, so they port across as pure Rust with theirs: 43 tests, no window required. The .slint files are layout only - nothing in them formats a number or picks a word. Verified against the real 660-album library rather than a fixture. tools/headless-shots.sh renders the UI inside a nested headless compositor and grabs a frame per screen, which is how that was checked on a machine whose session was locked; a Slint window needs a real compositor, and under bare Xvfb nothing maps at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
168
slint-frontend/README.md
Normal file
168
slint-frontend/README.md
Normal file
@@ -0,0 +1,168 @@
|
||||
# Musik Delfin — the Slint front-end
|
||||
|
||||
A native front-end for the music player, next to `../web/`. Same backend, same layout,
|
||||
same German copy; no browser. It exists because the React UI is sluggish on the Pi 5
|
||||
kiosk, and the things making it sluggish are browser-shaped: a compositor deciding what
|
||||
gets its own layer, a `requestAnimationFrame` loop that repaints the viewport, and a
|
||||
JPEG decode per album card per cold start.
|
||||
|
||||
Only the music player is here. The typing game (`Tippen`), the room-lights page, parent
|
||||
mode and the IR-remote assignment are all still web-only.
|
||||
|
||||
## Running it
|
||||
|
||||
Start the backend, then:
|
||||
|
||||
```sh
|
||||
nix-shell # cargo, the GL/X11/Wayland libs, and Nunito
|
||||
cargo run # windowed, FemtoVG
|
||||
cargo run -- --server http://musicdolphin:8080
|
||||
```
|
||||
|
||||
Checks:
|
||||
|
||||
```sh
|
||||
cargo test && cargo clippy --all-targets && cargo fmt --check
|
||||
```
|
||||
|
||||
`tools/headless-shots.sh` renders the UI and screenshots it inside a nested headless
|
||||
compositor, for checking a layout change on a machine with no display (or a locked one).
|
||||
A Slint window needs a real compositor - under bare Xvfb nothing maps at all.
|
||||
|
||||
## Build profiles
|
||||
|
||||
| | Backend | Renderer | For |
|
||||
|---|---|---|---|
|
||||
| `cargo run` | winit | FemtoVG | development here |
|
||||
| `cargo run --features skia` | winit | Skia | checking what the device will show |
|
||||
| `cargo build --release --no-default-features --features kiosk` | linuxkms + libinput | Skia | the Pi |
|
||||
|
||||
**The device must not use FemtoVG.** On a Pi 5 (Mesa V3D), FemtoVG draws every
|
||||
runtime-loaded `Image` as solid black - [slint#11785][bug], open as of this writing -
|
||||
which for this app means every album cover. Skia and the software renderer are both
|
||||
unaffected. FemtoVG stays the default *here* because it builds in seconds and is fine on
|
||||
desktop Mesa; `kiosk` is the profile that ships.
|
||||
|
||||
`--no-default-features` is not optional on that last one: the features are additive, so
|
||||
leaving `desktop` on would link winit *and* linuxkms into the same binary.
|
||||
|
||||
`kiosk` takes the framebuffer directly through KMS/DRM: no X, no Wayland, no compositor.
|
||||
(`backend-linuxkms-noseat` is deprecated upstream in favour of the libinput variant used
|
||||
here.) `SLINT_BACKEND=winit-software` forces the software renderer without a rebuild.
|
||||
|
||||
[bug]: https://github.com/slint-ui/slint/issues/11785
|
||||
|
||||
## Where the speed comes from
|
||||
|
||||
Four things, in rough order of how much they matter on the device:
|
||||
|
||||
**Covers are downscaled once, ever.** The backend serves one size - 640 px on the long
|
||||
edge - and has no `?size=`. A grid card is 180 px. `src/covers.rs` fetches each cover
|
||||
once on a worker thread, resizes it to the tier that was asked for, and writes a JPEG
|
||||
thumbnail to `~/.cache/musicmouse-slint/covers/<album-id>@<px>.jpg`. Every later start
|
||||
reads that instead. There are two tiers and only two, which is what makes the cache
|
||||
worth having: a per-widget pixel size would give each album a dozen near-identical files
|
||||
and a fresh decode for each.
|
||||
|
||||
The cache is dropped wholesale when the websocket announces a rescan
|
||||
(`{"type": "library"}`), because that is the one moment the backend rewrites the art
|
||||
behind an unchanged cover URL. That is the bug the web client had to learn about the
|
||||
hard way - see the comment at `python-backend/musicmouse/services/web/api.py:106`.
|
||||
|
||||
**The grid is virtualized.** Rust hands the UI a `[GridRow]` - the album list
|
||||
pre-chunked into rows - and the view puts those in a `ListView`, which only instantiates
|
||||
the rows on screen. A flat list of 660 cards gives it no rows to skip, hence the
|
||||
chunking. This is the counterpart of `content-visibility: auto` on `.grid > .card` in
|
||||
the web UI, which exists for exactly the same measurement: 340 live cards saturated the
|
||||
Pi's renderer so completely that the main thread stopped scheduling anything.
|
||||
|
||||
**The progress bar animates instead of ticking.** Position arrives twice a second. The
|
||||
web client interpolates it onto animation frames with a JS clock and a render budget;
|
||||
here the fill is a Slint `animate width` between the pushed values, so interpolation
|
||||
costs a property evaluation per frame on the render side and there is no clock at all.
|
||||
It is suppressed for the one frame a track changes on, so a new track jumps rather than
|
||||
sliding across two unrelated positions.
|
||||
|
||||
**Nothing on screen animates by itself.** The web UI's per-track ambient canvas is not
|
||||
here. Halving its resolution and its frame rate took it from 53.5% of a core to 25.0%
|
||||
and 25% was still too much: a loop repainting the viewport is a floor you cannot get
|
||||
under while it runs at all. The stage keeps its gradient, which is what `?pi=1` already
|
||||
does on the device this replaces.
|
||||
|
||||
## What is deliberately different from the web UI
|
||||
|
||||
**No glass blur.** Slint has no `backdrop-filter`, so the panels use a flat translucent
|
||||
fill - exactly the web app's own `data-blur="off"` fallback. Some depth is genuinely
|
||||
lost. Faking it by compositing a pre-blurred copy would reintroduce the per-frame cost
|
||||
that dropping the blur was meant to avoid.
|
||||
|
||||
Worth knowing if you ever port the blur back: on the Pi, `backdrop-filter` measured five
|
||||
times *better* than no blur (25% of a core against 118%), because it is what promotes
|
||||
each panel to its own compositing layer. That is a browser-compositor artifact and does
|
||||
not transfer here - but it is why the web UI keeps the panel blur under `?pi=1`.
|
||||
|
||||
**No dolphin.** The mascot and its swim/bob loops are not ported.
|
||||
|
||||
**Layout** is Slint's, not a transcription of the CSS: fixed row heights instead of
|
||||
`clamp()` and `vh`, a 1080p-shaped fixed measure for the track list rather than
|
||||
`max-width`. The colours are the same - `ui/theme.slint` carries the web UI's oklch
|
||||
tokens converted to sRGB, with the oklch source kept beside each one, because oklch is
|
||||
the form the design is actually maintained in.
|
||||
|
||||
## How it is put together
|
||||
|
||||
The split that matters is the web UI's, unchanged: **what is playing** comes from the
|
||||
backend, **what you are looking at** lives here. The buttons on the mouse, a figurine on
|
||||
the reader and Home Assistant all move playback too, so this follows it rather than
|
||||
assuming it is in charge.
|
||||
|
||||
Three worker threads and no async runtime - the whole client makes one library request,
|
||||
one websocket connection and a trickle of covers.
|
||||
|
||||
| Path | What it is |
|
||||
|---|---|
|
||||
| `src/api.rs` | The wire types and the HTTP calls. Mirrors `services/web/schemas.py`. |
|
||||
| `src/ws.rs` | The state websocket, with reconnect-and-reseed. |
|
||||
| `src/library.rs` | The search index and the browse hierarchy. Port of `web/src/lib/search.ts`. |
|
||||
| `src/keymap.rs` | The keyboard model, as a pure function. Port of `web/src/lib/keyboard.ts`. |
|
||||
| `src/covers.rs` | The thumbnail cache, and the generated stand-in art. |
|
||||
| `src/oklch.rs`, `src/format.rs` | Colour conversion; durations. |
|
||||
| `src/main.rs` | Wiring: worker channels in, Slint models out. |
|
||||
| `ui/*.slint` | Layout only. Nothing here formats a number or picks a word. |
|
||||
|
||||
`src/keymap.rs` and `src/library.rs` are ports of modules that were already pure
|
||||
functions with their own tests, and they keep them: 43 tests, no window required.
|
||||
|
||||
## Keyboard
|
||||
|
||||
The same map as the web UI.
|
||||
|
||||
| Key | |
|
||||
|---|---|
|
||||
| `SPACE` | play / pause (types a space mid-query) |
|
||||
| `SHIFT+H` / `SHIFT+L` | previous / next |
|
||||
| `SHIFT+J` / `SHIFT+K` | quieter / louder |
|
||||
| `SHIFT+M` | mute |
|
||||
| `SHIFT+←` / `SHIFT+→` | seek 15 s |
|
||||
| `A-Z`, `0-9` | search albums as you type |
|
||||
| `?` | search individual tracks |
|
||||
| `TAB` | cycle Musik / Hörbücher / Podcasts |
|
||||
| `← ↑ ↓ →` | move the selection while browsing; transport and volume while playing |
|
||||
| `CTRL+H/J/K/L` | the same, without leaving the home row |
|
||||
| `CTRL+D` / `CTRL+U` | half a page |
|
||||
| `ENTER` | play the selection |
|
||||
| `SHIFT+ENTER` | open the track list |
|
||||
| `ESC` | back out one layer at a time |
|
||||
| `/` | back to browsing |
|
||||
|
||||
Everything is clickable too.
|
||||
|
||||
The special keys are named in `ui/app.slint`, where the `Key.*` constants live, and
|
||||
reach Rust as `"escape"`, `"left"`, `"enter"` and so on - so `src/keymap.rs` is testable
|
||||
without a window and without hardcoding keysym values.
|
||||
|
||||
## Fonts
|
||||
|
||||
Nunito, via `SLINT_DEFAULT_FONT`, which `shell.nix` sets. Slint cannot read the woff2
|
||||
the web app self-hosts, so this wants the TTF; a deployment can drop one at
|
||||
`fonts/Nunito.ttf` instead. With neither, everything still renders in the system sans.
|
||||
Reference in New Issue
Block a user