musicmouse/slint-frontend/README.md

# Musik Delfin — the Slint front-end

A native front-end for the music player, next to `../web/`. Same backend, same layout,
same German copy; no browser. It exists because the React UI is sluggish on the Pi 5
kiosk, and the things making it sluggish are browser-shaped: a compositor deciding what
gets its own layer, a `requestAnimationFrame` loop that repaints the viewport, and a
JPEG decode per album card per cold start.

The music player and the room panel ("Mein Zimmer"). The typing game (`Tippen`), parent
mode and the IR-remote assignment are still web-only.

## Running it

Start the backend, then:

```sh
nix-shell                 # cargo, the GL/X11/Wayland libs, and Nunito
cargo run                 # windowed, FemtoVG
cargo run -- --server http://musicdolphin:8080
cargo run -- --page room  # open on the room panel rather than the player
```

Checks:

```sh
cargo test && cargo clippy --all-targets && cargo fmt --check
```

`tools/headless-shots.sh` renders the UI and screenshots it inside a nested headless
compositor, for checking a layout change on a machine with no display (or a locked one).
A Slint window needs a real compositor - under bare Xvfb nothing maps at all.

## Build profiles

| | Backend | Renderer | For |
|---|---|---|---|
| `cargo run` | winit | FemtoVG | development here |
| `cargo run --features skia` | winit | Skia | checking what the device will show |
| `cargo build --release --no-default-features --features kiosk` | linuxkms + libinput | Skia | the Pi |

**The device must not use FemtoVG.** On a Pi 5 (Mesa V3D), FemtoVG draws every
runtime-loaded `Image` as solid black - [slint#11785][bug], open as of this writing -
which for this app means every album cover. Skia and the software renderer are both
unaffected. FemtoVG stays the default *here* because it builds in seconds and is fine on
desktop Mesa; `kiosk` is the profile that ships.

`--no-default-features` is not optional on that last one: the features are additive, so
leaving `desktop` on would link winit *and* linuxkms into the same binary.

`kiosk` takes the framebuffer directly through KMS/DRM: no X, no Wayland, no compositor.
(`backend-linuxkms-noseat` is deprecated upstream in favour of the libinput variant used
here.) `SLINT_BACKEND=winit-software` forces the software renderer without a rebuild.

[bug]: https://github.com/slint-ui/slint/issues/11785

## Where the speed comes from

Four things, in rough order of how much they matter on the device:

**Covers are downscaled once, ever.** The backend serves one size - 640 px on the long
edge - and has no `?size=`. A grid card is 180 px. `src/covers.rs` fetches each cover
once on a worker thread, resizes it to the tier that was asked for, and writes a JPEG
thumbnail to `~/.cache/musicmouse-slint/covers/<album-id>@<px>.jpg`. Every later start
reads that instead. There are two tiers and only two, which is what makes the cache
worth having: a per-widget pixel size would give each album a dozen near-identical files
and a fresh decode for each.

The cache is dropped wholesale when the websocket announces a rescan
(`{"type": "library"}`), because that is the one moment the backend rewrites the art
behind an unchanged cover URL. That is the bug the web client had to learn about the
hard way - see the comment at `python-backend/musicmouse/services/web/api.py:106`.

**The grid is virtualized.** Rust hands the UI a `[GridRow]` - the album list
pre-chunked into rows - and the view puts those in a `ListView`, which only instantiates
the rows on screen. A flat list of 660 cards gives it no rows to skip, hence the
chunking. This is the counterpart of `content-visibility: auto` on `.grid > .card` in
the web UI, which exists for exactly the same measurement: 340 live cards saturated the
Pi's renderer so completely that the main thread stopped scheduling anything.

**The progress bar animates instead of ticking.** Position arrives twice a second. The
web client interpolates it onto animation frames with a JS clock and a render budget;
here the fill is a Slint `animate width` between the pushed values, so interpolation
costs a property evaluation per frame on the render side and there is no clock at all.
It is suppressed for the one frame a track changes on, so a new track jumps rather than
sliding across two unrelated positions.

**Nothing on screen animates by itself.** The web UI's per-track ambient canvas is not
here. Halving its resolution and its frame rate took it from 53.5% of a core to 25.0%
and 25% was still too much: a loop repainting the viewport is a floor you cannot get
under while it runs at all. The stage keeps its gradient, which is what `?pi=1` already
does on the device this replaces.

## What is deliberately different from the web UI

**No glass blur.** Slint has no `backdrop-filter`, so the panels use a flat translucent
fill - exactly the web app's own `data-blur="off"` fallback. Some depth is genuinely
lost. Faking it by compositing a pre-blurred copy would reintroduce the per-frame cost
that dropping the blur was meant to avoid.

Worth knowing if you ever port the blur back: on the Pi, `backdrop-filter` measured five
times *better* than no blur (25% of a core against 118%), because it is what promotes
each panel to its own compositing layer. That is a browser-compositor artifact and does
not transfer here - but it is why the web UI keeps the panel blur under `?pi=1`.

**No dolphin.** The mascot and its swim/bob loops are not ported.

**Layout** is Slint's, not a transcription of the CSS: fixed row heights instead of
`clamp()` and `vh`, a 1080p-shaped fixed measure for the track list rather than
`max-width`. The colours are the same - `ui/theme.slint` carries the web UI's oklch
tokens converted to sRGB, with the oklch source kept beside each one, because oklch is
the form the design is actually maintained in.

## How it is put together

The split that matters is the web UI's, unchanged: **what is playing** comes from the
backend, **what you are looking at** lives here. The buttons on the mouse, a figurine on
the reader and Home Assistant all move playback too, so this follows it rather than
assuming it is in charge.

Three worker threads and no async runtime - the whole client makes one library request,
one websocket connection and a trickle of covers.

| Path | What it is |
|---|---|
| `src/api.rs` | The wire types and the HTTP calls. Mirrors `services/web/schemas.py`. |
| `src/ws.rs` | The state websocket, with reconnect-and-reseed. |
| `src/library.rs` | The search index and the browse hierarchy. Port of `web/src/lib/search.ts`. |
| `src/keymap.rs` | The keyboard model, as a pure function. Port of `web/src/lib/keyboard.ts`. |
| `src/covers.rs` | The thumbnail cache, and the generated stand-in art. |
| `src/ha.rs` | Home Assistant through the backend's proxy, and the shutter/colour conversions. Port of `web/src/hooks/useHomeAssistant.ts` and `lib/shutter.ts`. |
| `src/oklch.rs`, `src/format.rs` | Colour conversion; durations. |
| `src/main.rs` | Wiring: worker channels in, Slint models out. |
| `ui/*.slint` | Layout only. Nothing here formats a number or picks a word. |

`src/keymap.rs` and `src/library.rs` are ports of modules that were already pure
functions with their own tests, and they keep them: 43 tests, no window required.

## Keyboard

The same map as the web UI.

| Key | |
|---|---|
| `SPACE` | play / pause (types a space mid-query) |
| `SHIFT+H` / `SHIFT+L` | previous / next |
| `SHIFT+J` / `SHIFT+K` | quieter / louder |
| `SHIFT+M` | mute |
| `SHIFT+←` / `SHIFT+→` | seek 15 s |
| `A-Z`, `0-9` | search albums as you type |
| `?` | search individual tracks |
| `TAB` | cycle Musik / Hörbücher / Podcasts |
| `← ↑ ↓ →` | move the selection while browsing; transport and volume while playing |
| `CTRL+H/J/K/L` | the same, without leaving the home row |
| `CTRL+D` / `CTRL+U` | half a page |
| `ENTER` | play the selection |
| `SHIFT+ENTER` | open the track list |
| `ESC` | back out one layer at a time |
| `/` | back to browsing |
| `CTRL+TAB` | cycle Musik / Mein Zimmer / Tippen |

Everything is clickable too.

`CTRL+TAB` is the one addition to the web front-end's map. Its tab rail is reachable
only with a pointer, which sits badly with a device whose stated design is that every
screen can be reached from the keyboard. Plain `TAB` was already the group cycle.

## Mein Zimmer

The Home Assistant room panel, in its own violet rather than the player's teal so the
two read as siblings. Lights get a toggle, five brightness steps and six colours;
shutters get four presets, up/stop/down and a window that reflects the reported
position. Nothing is animated locally - a real cover reports its own movement, and
guessing where it will end up is what would fight with that.

The whole page is a poll, not a subscription, and only while it is on screen: Home
Assistant's own auth-and-subscribe websocket is more machinery than this needs, and a
lamp is not worth a request every 2.5 s when nobody is looking. A tap patches the
entity locally and remembers when, so the poll already in flight cannot undo it - the
same reconciliation `useHomeAssistant.ts` documents, and for the same reason.

The tab appears only when the backend answers `GET /api/ha` with a config; a 404 there
means Home Assistant is not set up, which is a normal answer rather than an error.

The special keys are named in `ui/app.slint`, where the `Key.*` constants live, and
reach Rust as `"escape"`, `"left"`, `"enter"` and so on - so `src/keymap.rs` is testable
without a window and without hardcoding keysym values.

## Fonts

Nunito, via `SLINT_DEFAULT_FONT`, which `shell.nix` sets. Slint cannot read the woff2
the web app self-hosts, so this wants the TTF; a deployment can drop one at
`fonts/Nunito.ttf` instead. With neither, everything still renders in the system sans.