Martin Bauer
6ccb3e458f
The React UI is sluggish on the kiosk and the reasons are browser-shaped: a compositor deciding what gets its own layer, a requestAnimationFrame loop repainting the viewport, and a JPEG decode per album card per cold start. The last three commits chased that through the service worker and the cover pipeline. This is the other direction - the same backend, the same layout, the same German copy, without a browser. Browse and play only. The typing game, the room-lights page, parent mode and the IR-remote assignment stay web-only, and nothing here touches python-backend. Four things carry the performance claim, in rough order of how much they should matter: Covers are downscaled once, ever. The backend serves one size - 640px on the long edge, no ?size= - and a grid card is 180. src/covers.rs fetches each cover once on a worker thread, resizes it, and writes a JPEG thumbnail to ~/.cache/musicmouse-slint/covers/. Two tiers and only two, which is what makes the cache worth keeping: a per-widget pixel size would give each album a dozen near-identical files and a fresh decode for each. The cache is dropped when the websocket announces a rescan, because that is the one moment the backend rewrites the art behind an unchanged cover URL - the trap sw.js had to learn about the hard way. The grid is virtualized. Rust hands the UI the album list pre-chunked into rows and the view puts those in a ListView, which instantiates only what is on screen. A flat list of 660 cards gives it no rows to skip, hence the chunking. Same purpose as content-visibility: auto on .grid > .card. The progress bar animates between the 2 Hz pushes rather than running a clock, so interpolation costs a property evaluation per frame on the render side and there is no equivalent of usePlaybackClock. It is suppressed for the frame a track changes on, so a new track jumps instead of sliding across two unrelated positions. Nothing on screen animates by itself. The ambient canvas is not ported, for the reason lib/lowPower.ts already gives: a loop repainting the viewport is a floor you cannot get under while it runs at all. The device must not use FemtoVG. On Mesa V3D it draws every runtime-loaded Image as solid black (slint-ui/slint#11785, open), which here means every album cover; Skia and the software renderer are unaffected. So `kiosk` is linuxkms + Skia and FemtoVG stays the default only for desktop development. Both profiles are verified to build; the kiosk binary links libinput/libgbm/libdrm and no X11 or Wayland at all. lib/search.ts, lib/keyboard.ts and lib/format.ts were already pure functions with their own tests, so they port across as pure Rust with theirs: 43 tests, no window required. The .slint files are layout only - nothing in them formats a number or picks a word. Verified against the real 660-album library rather than a fixture. tools/headless-shots.sh renders the UI inside a nested headless compositor and grabs a frame per screen, which is how that was checked on a machine whose session was locked; a Slint window needs a real compositor, and under bare Xvfb nothing maps at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
169 lines
8.1 KiB
Markdown
169 lines
8.1 KiB
Markdown
# Musik Delfin — the Slint front-end
|
|
|
|
A native front-end for the music player, next to `../web/`. Same backend, same layout,
|
|
same German copy; no browser. It exists because the React UI is sluggish on the Pi 5
|
|
kiosk, and the things making it sluggish are browser-shaped: a compositor deciding what
|
|
gets its own layer, a `requestAnimationFrame` loop that repaints the viewport, and a
|
|
JPEG decode per album card per cold start.
|
|
|
|
Only the music player is here. The typing game (`Tippen`), the room-lights page, parent
|
|
mode and the IR-remote assignment are all still web-only.
|
|
|
|
## Running it
|
|
|
|
Start the backend, then:
|
|
|
|
```sh
|
|
nix-shell # cargo, the GL/X11/Wayland libs, and Nunito
|
|
cargo run # windowed, FemtoVG
|
|
cargo run -- --server http://musicdolphin:8080
|
|
```
|
|
|
|
Checks:
|
|
|
|
```sh
|
|
cargo test && cargo clippy --all-targets && cargo fmt --check
|
|
```
|
|
|
|
`tools/headless-shots.sh` renders the UI and screenshots it inside a nested headless
|
|
compositor, for checking a layout change on a machine with no display (or a locked one).
|
|
A Slint window needs a real compositor - under bare Xvfb nothing maps at all.
|
|
|
|
## Build profiles
|
|
|
|
| | Backend | Renderer | For |
|
|
|---|---|---|---|
|
|
| `cargo run` | winit | FemtoVG | development here |
|
|
| `cargo run --features skia` | winit | Skia | checking what the device will show |
|
|
| `cargo build --release --no-default-features --features kiosk` | linuxkms + libinput | Skia | the Pi |
|
|
|
|
**The device must not use FemtoVG.** On a Pi 5 (Mesa V3D), FemtoVG draws every
|
|
runtime-loaded `Image` as solid black - [slint#11785][bug], open as of this writing -
|
|
which for this app means every album cover. Skia and the software renderer are both
|
|
unaffected. FemtoVG stays the default *here* because it builds in seconds and is fine on
|
|
desktop Mesa; `kiosk` is the profile that ships.
|
|
|
|
`--no-default-features` is not optional on that last one: the features are additive, so
|
|
leaving `desktop` on would link winit *and* linuxkms into the same binary.
|
|
|
|
`kiosk` takes the framebuffer directly through KMS/DRM: no X, no Wayland, no compositor.
|
|
(`backend-linuxkms-noseat` is deprecated upstream in favour of the libinput variant used
|
|
here.) `SLINT_BACKEND=winit-software` forces the software renderer without a rebuild.
|
|
|
|
[bug]: https://github.com/slint-ui/slint/issues/11785
|
|
|
|
## Where the speed comes from
|
|
|
|
Four things, in rough order of how much they matter on the device:
|
|
|
|
**Covers are downscaled once, ever.** The backend serves one size - 640 px on the long
|
|
edge - and has no `?size=`. A grid card is 180 px. `src/covers.rs` fetches each cover
|
|
once on a worker thread, resizes it to the tier that was asked for, and writes a JPEG
|
|
thumbnail to `~/.cache/musicmouse-slint/covers/<album-id>@<px>.jpg`. Every later start
|
|
reads that instead. There are two tiers and only two, which is what makes the cache
|
|
worth having: a per-widget pixel size would give each album a dozen near-identical files
|
|
and a fresh decode for each.
|
|
|
|
The cache is dropped wholesale when the websocket announces a rescan
|
|
(`{"type": "library"}`), because that is the one moment the backend rewrites the art
|
|
behind an unchanged cover URL. That is the bug the web client had to learn about the
|
|
hard way - see the comment at `python-backend/musicmouse/services/web/api.py:106`.
|
|
|
|
**The grid is virtualized.** Rust hands the UI a `[GridRow]` - the album list
|
|
pre-chunked into rows - and the view puts those in a `ListView`, which only instantiates
|
|
the rows on screen. A flat list of 660 cards gives it no rows to skip, hence the
|
|
chunking. This is the counterpart of `content-visibility: auto` on `.grid > .card` in
|
|
the web UI, which exists for exactly the same measurement: 340 live cards saturated the
|
|
Pi's renderer so completely that the main thread stopped scheduling anything.
|
|
|
|
**The progress bar animates instead of ticking.** Position arrives twice a second. The
|
|
web client interpolates it onto animation frames with a JS clock and a render budget;
|
|
here the fill is a Slint `animate width` between the pushed values, so interpolation
|
|
costs a property evaluation per frame on the render side and there is no clock at all.
|
|
It is suppressed for the one frame a track changes on, so a new track jumps rather than
|
|
sliding across two unrelated positions.
|
|
|
|
**Nothing on screen animates by itself.** The web UI's per-track ambient canvas is not
|
|
here. Halving its resolution and its frame rate took it from 53.5% of a core to 25.0%
|
|
and 25% was still too much: a loop repainting the viewport is a floor you cannot get
|
|
under while it runs at all. The stage keeps its gradient, which is what `?pi=1` already
|
|
does on the device this replaces.
|
|
|
|
## What is deliberately different from the web UI
|
|
|
|
**No glass blur.** Slint has no `backdrop-filter`, so the panels use a flat translucent
|
|
fill - exactly the web app's own `data-blur="off"` fallback. Some depth is genuinely
|
|
lost. Faking it by compositing a pre-blurred copy would reintroduce the per-frame cost
|
|
that dropping the blur was meant to avoid.
|
|
|
|
Worth knowing if you ever port the blur back: on the Pi, `backdrop-filter` measured five
|
|
times *better* than no blur (25% of a core against 118%), because it is what promotes
|
|
each panel to its own compositing layer. That is a browser-compositor artifact and does
|
|
not transfer here - but it is why the web UI keeps the panel blur under `?pi=1`.
|
|
|
|
**No dolphin.** The mascot and its swim/bob loops are not ported.
|
|
|
|
**Layout** is Slint's, not a transcription of the CSS: fixed row heights instead of
|
|
`clamp()` and `vh`, a 1080p-shaped fixed measure for the track list rather than
|
|
`max-width`. The colours are the same - `ui/theme.slint` carries the web UI's oklch
|
|
tokens converted to sRGB, with the oklch source kept beside each one, because oklch is
|
|
the form the design is actually maintained in.
|
|
|
|
## How it is put together
|
|
|
|
The split that matters is the web UI's, unchanged: **what is playing** comes from the
|
|
backend, **what you are looking at** lives here. The buttons on the mouse, a figurine on
|
|
the reader and Home Assistant all move playback too, so this follows it rather than
|
|
assuming it is in charge.
|
|
|
|
Three worker threads and no async runtime - the whole client makes one library request,
|
|
one websocket connection and a trickle of covers.
|
|
|
|
| Path | What it is |
|
|
|---|---|
|
|
| `src/api.rs` | The wire types and the HTTP calls. Mirrors `services/web/schemas.py`. |
|
|
| `src/ws.rs` | The state websocket, with reconnect-and-reseed. |
|
|
| `src/library.rs` | The search index and the browse hierarchy. Port of `web/src/lib/search.ts`. |
|
|
| `src/keymap.rs` | The keyboard model, as a pure function. Port of `web/src/lib/keyboard.ts`. |
|
|
| `src/covers.rs` | The thumbnail cache, and the generated stand-in art. |
|
|
| `src/oklch.rs`, `src/format.rs` | Colour conversion; durations. |
|
|
| `src/main.rs` | Wiring: worker channels in, Slint models out. |
|
|
| `ui/*.slint` | Layout only. Nothing here formats a number or picks a word. |
|
|
|
|
`src/keymap.rs` and `src/library.rs` are ports of modules that were already pure
|
|
functions with their own tests, and they keep them: 43 tests, no window required.
|
|
|
|
## Keyboard
|
|
|
|
The same map as the web UI.
|
|
|
|
| Key | |
|
|
|---|---|
|
|
| `SPACE` | play / pause (types a space mid-query) |
|
|
| `SHIFT+H` / `SHIFT+L` | previous / next |
|
|
| `SHIFT+J` / `SHIFT+K` | quieter / louder |
|
|
| `SHIFT+M` | mute |
|
|
| `SHIFT+←` / `SHIFT+→` | seek 15 s |
|
|
| `A-Z`, `0-9` | search albums as you type |
|
|
| `?` | search individual tracks |
|
|
| `TAB` | cycle Musik / Hörbücher / Podcasts |
|
|
| `← ↑ ↓ →` | move the selection while browsing; transport and volume while playing |
|
|
| `CTRL+H/J/K/L` | the same, without leaving the home row |
|
|
| `CTRL+D` / `CTRL+U` | half a page |
|
|
| `ENTER` | play the selection |
|
|
| `SHIFT+ENTER` | open the track list |
|
|
| `ESC` | back out one layer at a time |
|
|
| `/` | back to browsing |
|
|
|
|
Everything is clickable too.
|
|
|
|
The special keys are named in `ui/app.slint`, where the `Key.*` constants live, and
|
|
reach Rust as `"escape"`, `"left"`, `"enter"` and so on - so `src/keymap.rs` is testable
|
|
without a window and without hardcoding keysym values.
|
|
|
|
## Fonts
|
|
|
|
Nunito, via `SLINT_DEFAULT_FONT`, which `shell.nix` sets. Slint cannot read the woff2
|
|
the web app self-hosts, so this wants the TTF; a deployment can drop one at
|
|
`fonts/Nunito.ttf` instead. With neither, everything still renders in the system sans.
|