Add a native Slint front-end for the music player

The React UI is sluggish on the kiosk and the reasons are browser-shaped: a compositor
deciding what gets its own layer, a requestAnimationFrame loop repainting the viewport,
and a JPEG decode per album card per cold start. The last three commits chased that
through the service worker and the cover pipeline. This is the other direction - the
same backend, the same layout, the same German copy, without a browser.

Browse and play only. The typing game, the room-lights page, parent mode and the
IR-remote assignment stay web-only, and nothing here touches python-backend.

Four things carry the performance claim, in rough order of how much they should matter:

Covers are downscaled once, ever. The backend serves one size - 640px on the long edge,
no ?size= - and a grid card is 180. src/covers.rs fetches each cover once on a worker
thread, resizes it, and writes a JPEG thumbnail to ~/.cache/musicmouse-slint/covers/.
Two tiers and only two, which is what makes the cache worth keeping: a per-widget pixel
size would give each album a dozen near-identical files and a fresh decode for each. The
cache is dropped when the websocket announces a rescan, because that is the one moment
the backend rewrites the art behind an unchanged cover URL - the trap sw.js had to learn
about the hard way.

The grid is virtualized. Rust hands the UI the album list pre-chunked into rows and the
view puts those in a ListView, which instantiates only what is on screen. A flat list of
660 cards gives it no rows to skip, hence the chunking. Same purpose as
content-visibility: auto on .grid > .card.

The progress bar animates between the 2 Hz pushes rather than running a clock, so
interpolation costs a property evaluation per frame on the render side and there is no
equivalent of usePlaybackClock. It is suppressed for the frame a track changes on, so a
new track jumps instead of sliding across two unrelated positions.

Nothing on screen animates by itself. The ambient canvas is not ported, for the reason
lib/lowPower.ts already gives: a loop repainting the viewport is a floor you cannot get
under while it runs at all.

The device must not use FemtoVG. On Mesa V3D it draws every runtime-loaded Image as
solid black (slint-ui/slint#11785, open), which here means every album cover; Skia and
the software renderer are unaffected. So `kiosk` is linuxkms + Skia and FemtoVG stays
the default only for desktop development. Both profiles are verified to build; the kiosk
binary links libinput/libgbm/libdrm and no X11 or Wayland at all.

lib/search.ts, lib/keyboard.ts and lib/format.ts were already pure functions with their
own tests, so they port across as pure Rust with theirs: 43 tests, no window required.
The .slint files are layout only - nothing in them formats a number or picks a word.

Verified against the real 660-album library rather than a fixture. tools/headless-shots.sh
renders the UI inside a nested headless compositor and grabs a frame per screen, which
is how that was checked on a machine whose session was locked; a Slint window needs a
real compositor, and under bare Xvfb nothing maps at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-20 22:26:09 +02:00
parent 83d16e0058
commit 6ccb3e458f
21 changed files with 11270 additions and 0 deletions

312
slint-frontend/src/api.rs Normal file
View File

@@ -0,0 +1,312 @@
//! The shapes the backend serves, and the calls that fetch them.
//!
//! Mirrors `python-backend/musicmouse/services/web/schemas.py`, by way of
//! `web/src/api/types.ts` - the three are hand-kept in step. Only the music player's
//! share of the API is here; settings, the IR remote, Home Assistant and the typing
//! game are all reachable and all deliberately absent.
//!
//! Every field the backend can omit is modelled with `#[serde(default)]` rather than a
//! required `Option`, so a backend that grows a field does not break this client and
//! one that loses a field fails at the one call site that cared.
use serde::Deserialize;
/// Default for a development run; `--server` overrides it.
pub const DEFAULT_BASE_URL: &str = "http://127.0.0.1:8080";
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum AlbumKind {
#[default]
Music,
Book,
}
impl AlbumKind {
/// Shape is how you tell the two apart without reading anything: albums square,
/// audiobooks taller than wide, everywhere they appear. Nothing else in this app
/// may set a cover's aspect ratio. Ported from `web/src/lib/covers.ts`.
pub fn aspect(self) -> f32 {
match self {
Self::Music => 1.0,
Self::Book => 0.82,
}
}
}
#[derive(Debug, Clone, Deserialize)]
pub struct Track {
pub title: String,
/// Seconds, read from the file's tags at scan time. `0.0` when it carries none.
#[serde(default)]
pub duration: f64,
/// A typing-reward track not yet earned. This front-end does not host the typing
/// game, but it still has to not show what the game is withholding.
#[serde(default)]
pub locked: bool,
}
// `series`, and the parts of `PlayerState` the UI does not read, are kept because these
// structs are the record of what the backend serves - dropping a field would make this
// file a list of what this front-end happens to use today, which is a much less useful
// thing to read next to `schemas.py`.
#[allow(dead_code)]
#[derive(Debug, Clone, Deserialize)]
pub struct Album {
pub id: String,
/// One of the four fixed shelf folder names: `Figuren`, `Musik`, `Hörbücher`,
/// `Kinderpodcasts`. What `Group::of` reads to spot a podcast.
pub section: String,
#[serde(default)]
pub kind: AlbumKind,
pub title: String,
#[serde(default)]
pub artist: String,
#[serde(default)]
pub series: Option<String>,
/// The figurine whose folder this is, when it has one.
#[serde(default)]
pub figure: Option<String>,
/// Series for audiobooks and podcasts, artist for music. What browsing groups by.
#[serde(default)]
pub category: String,
/// Exactly three `#rrggbb`: primary, secondary, accent, pulled out of the cover art
/// or synthesised from the id. The generated stand-in cover is painted from these.
#[serde(default)]
pub colors: Vec<String>,
#[serde(default)]
pub has_cover: bool,
#[serde(default)]
pub duration: f64,
#[serde(default)]
pub tracks: Vec<Track>,
#[serde(default)]
pub locked: bool,
}
impl Album {
pub fn is_book(&self) -> bool {
self.kind == AlbumKind::Book
}
/// `"Album · Artist"`, unless a podcast makes those the same string.
pub fn line(&self) -> String {
let prefix = if self.is_book() { "📖 " } else { "" };
if !self.artist.is_empty() && self.artist != self.title {
format!("{prefix}{} · {}", self.title, self.artist)
} else {
format!("{prefix}{}", self.title)
}
}
/// `colors` padded out to the three the painter wants, with the same fallbacks the
/// web client uses, so a library entry missing them looks the same in both.
pub fn palette(&self) -> [&str; 3] {
[
self.colors.first().map_or("#4a6fa5", String::as_str),
self.colors.get(1).map_or("#6a8fc5", String::as_str),
self.colors.get(2).map_or("#a5804a", String::as_str),
]
}
}
#[derive(Debug, Clone, Deserialize)]
struct LibraryOut {
albums: Vec<Album>,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Default, Deserialize)]
pub struct Connection {
#[serde(default)]
pub firmware: bool,
#[serde(default)]
pub mqtt: bool,
#[serde(default)]
pub lirc: bool,
}
/// What is playing. Owned entirely by the backend: the buttons on the mouse, a figurine
/// on the reader and Home Assistant all move it too, so this client follows it rather
/// than assuming it is in charge.
#[allow(dead_code)] // see the note above `Album`
#[derive(Debug, Clone, Default, Deserialize)]
pub struct PlayerState {
#[serde(default)]
pub playing: bool,
#[serde(default)]
pub album_id: Option<String>,
#[serde(default)]
pub album_title: Option<String>,
#[serde(default)]
pub artist: Option<String>,
#[serde(default)]
pub kind: Option<AlbumKind>,
#[serde(default)]
pub track_index: i32,
#[serde(default)]
pub track_title: Option<String>,
#[serde(default)]
pub track_count: i32,
#[serde(default)]
pub position: f64,
#[serde(default)]
pub duration: f64,
/// Percent, 0..100. The device's configured range never leaves the backend.
#[serde(default)]
pub volume: i32,
#[serde(default)]
pub active_figure: Option<String>,
#[serde(default)]
pub connected: Connection,
}
/// The push-only websocket's frames. `library` carries no payload - it is a bare "your
/// index is stale, fetch it again" after a rescan.
#[derive(Debug, Clone, Deserialize)]
#[serde(tag = "type", rename_all = "lowercase")]
pub enum ServerMessage {
State {
state: PlayerState,
},
Position {
position: f64,
#[serde(default)]
duration: f64,
},
Library,
/// A frame this build does not know. Ignored rather than fatal, so the backend can
/// grow a message type without this client falling over on it.
#[serde(other)]
Unknown,
}
// ------------------------------------------------------------------------- client --
#[derive(Debug, Clone)]
pub struct Client {
base: String,
agent: ureq::Agent,
}
/// Enough of an error type to put a cause on screen. Nothing here is recoverable in a
/// way the caller could branch on - the UI either has a library or says it cannot reach
/// the mouse - so the variants are not worth splitting.
pub type Error = String;
impl Client {
pub fn new(base_url: &str) -> Self {
Self {
base: base_url.trim_end_matches('/').to_string(),
agent: ureq::AgentBuilder::new()
.timeout_connect(std::time::Duration::from_secs(4))
.timeout_read(std::time::Duration::from_secs(30))
.build(),
}
}
/// `http://host:port` -> `ws://host:port/api/ws`. The device serves plain HTTP, but
/// deriving the scheme rather than hardcoding it keeps a TLS proxy in front working.
pub fn websocket_url(&self) -> String {
let ws = if let Some(rest) = self.base.strip_prefix("https://") {
format!("wss://{rest}")
} else if let Some(rest) = self.base.strip_prefix("http://") {
format!("ws://{rest}")
} else {
format!("ws://{}", self.base)
};
format!("{ws}/api/ws")
}
pub fn cover_url(&self, album_id: &str) -> String {
format!("{}/api/albums/{album_id}/cover", self.base)
}
pub fn library(&self) -> Result<Vec<Album>, Error> {
let out: LibraryOut = self
.agent
.get(&format!("{}/api/library", self.base))
.call()
.map_err(|e| format!("GET /api/library: {e}"))?
.into_json()
.map_err(|e| format!("GET /api/library: bad JSON: {e}"))?;
Ok(out.albums)
}
pub fn state(&self) -> Result<PlayerState, Error> {
self.agent
.get(&format!("{}/api/state", self.base))
.call()
.map_err(|e| format!("GET /api/state: {e}"))?
.into_json()
.map_err(|e| format!("GET /api/state: bad JSON: {e}"))
}
/// Raw cover bytes, or `Ok(None)` for a 404 - which is a normal answer, not a
/// failure: `has_cover` says so in advance and the client paints the album's own
/// colours instead.
pub fn cover(&self, album_id: &str) -> Result<Option<Vec<u8>>, Error> {
let response = match self.agent.get(&self.cover_url(album_id)).call() {
Ok(response) => response,
Err(ureq::Error::Status(404, _)) => return Ok(None),
Err(e) => return Err(format!("GET cover {album_id}: {e}")),
};
let mut bytes = Vec::new();
std::io::Read::read_to_end(&mut response.into_reader(), &mut bytes)
.map_err(|e| format!("GET cover {album_id}: {e}"))?;
Ok(Some(bytes))
}
// -------------------------------------------------------------------- commands --
//
// Every one of these emits the same *intent* the physical buttons emit, so this UI
// has no privileged path and no way to get out of step with a figure someone puts
// on the reader. They all answer 204 with no body - nothing to parse.
fn command(&self, path: &str, body: serde_json::Value) -> Result<(), Error> {
let request = self.agent.post(&format!("{}/api{path}", self.base));
let result = if body.is_null() {
request.call()
} else {
request.send_json(body)
};
result.map(|_| ()).map_err(|e| format!("POST {path}: {e}"))
}
pub fn play(&self, album_id: &str, track_index: i32) -> Result<(), Error> {
self.command(
"/play",
serde_json::json!({ "album_id": album_id, "track_index": track_index.max(0) }),
)
}
pub fn resume(&self) -> Result<(), Error> {
self.command("/resume", serde_json::Value::Null)
}
pub fn pause(&self) -> Result<(), Error> {
self.command("/pause", serde_json::Value::Null)
}
pub fn next(&self) -> Result<(), Error> {
self.command("/next", serde_json::Value::Null)
}
pub fn previous(&self) -> Result<(), Error> {
self.command("/previous", serde_json::Value::Null)
}
pub fn seek(&self, position: f64) -> Result<(), Error> {
self.command(
"/seek",
serde_json::json!({ "position": position.max(0.0) }),
)
}
pub fn volume(&self, percent: i32) -> Result<(), Error> {
self.command(
"/volume",
serde_json::json!({ "percent": percent.clamp(0, 100) }),
)
}
}