11 Commits

Author SHA1 Message Date
4cb2d085ca Add agent task files for the review's remaining fixes
Authentication (01), one input policy for skip/mute/volume steps (04), the
firmware's interleaved serial frames (05), podcast as an album kind (06), and eight
smaller cleanups (07). Each says why, what was decided, the design, the tests and
when it is done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:51:27 +02:00
a737ccf108 Bring the docs in line with the code
- python-backend/README: track analysis is implemented (2e0e6ad); the cache layout,
  the module table (lirc, screen, timers, podcasts, tippen) and the route table
  (track analysis, HA proxy, typing game) were missing pieces.
- docs/REPO_OVERVIEW: Mein Zimmer and the clock are implemented; Home Assistant is
  also reached over HTTP through the room page's proxy, not MQTT only; the IR remote,
  scripts/, the optional config sections and figure kind were not mentioned.
- web/README: the key table was the mockup's (Ctrl transport keys, TAB over 'all');
  ?pi=1 is now a preset with per-browser overrides, not a read-once flag; the room
  page and the typing game exist; eslint is part of the checks.
- Docstrings that described earlier designs: HaConfig (written as browser-direct in
  the same commit that built the proxy, a8ed350), services/__init__, events,
  playback, media, colors, tippen progress and AppHeader, lowPower.ts, and the
  'Hoerbuecher' spelling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:51:27 +02:00
a0ac48ccbf Remove bloat: the design bundle zip, duplicate images, course notebooks
- claude-design/Dolphin Beats Music Player.zip (32 MB): the design tool's export of
  the mockups already unpacked next to it. Still in history (30065bb) if needed.
- claude-design/pasted-*.png: byte-identical copies of dolphin-mascot.png and
  dolphin-remote.png that nothing references.
- python-backend/notebooks: the chord-recognition course material with stale paths.
  effect_debug.ipynb stays.

The hardware models are untouched, and the history keeps its size: removing files
from the tree does not shrink the pack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:51:27 +02:00
751637d2f1 Remove dead code
- wire.with_figure: the device uses dataclasses.replace directly.
- DeviceConnected/DeviceDisconnected: emitted, never subscribed to, and a duplicate
  of ConnectionChanged(target="firmware").
- ScreenService.toggle(): never called.
- GET /api/lirc (LircConfigOut, api.lircConfig): no front-end reads it.
- POST /api/volume's delta_percent (api.nudgeVolume): no front-end sends it.
- LibraryCache.shrink_stored_covers: a one-off migration that re-read every cover
  header on every scan. _INDEX_VERSION 7 rebuilds any cache that could still need it.
- Firmware: the MFRC522 library and its unused globals (lib/nfc/rc522.c is the
  driver in use), and mfrc_version.h, a reader test sketch nothing included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:51:27 +02:00
7a7ad9f0a7 Push a state snapshot only for events that change it
The hub sent every open tab a full snapshot on every StateEvent, LED writes
included - placing and removing a figure was 13 snapshots, 9 of them for LED
changes the snapshot does not even contain. List the events that matter, and make
a test fail when a new StateEvent is sorted into neither list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:51:27 +02:00
b791beed9c Make mypy run again, fix what it found, and add one check script
a7fb56c pinned mypy to Python 3.11 for the Pi. In a dev venv that also has the
analysis extra, numpy's stubs use 3.12 syntax, so mypy aborted on them before
checking a single file - and from the next day on four real errors went in
unnoticed (ebab03b, 0ef5a04, 9fd106b). Skip following numpy/librosa instead; only
librosa_analyzer.py touches them. Fix the four errors (variables reused with a
narrower type).

There is no CI, so scripts/check.sh is the gate: pytest, ruff, mypy, vitest, tsc
and eslint in one command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:49:23 +02:00
14b5133543 Resume a figure after a library rescan
play_figure resumed only when the player's playlist *was* the figure's playlist
object. rescan_library rebuilds every Playlist, so after the refresh button - or
the podcast poller landing a new episode, every few hours on its own - putting a
figure back started it over. Playlist is a frozen dataclass: compare by value.

The album picker's "keep what is already playing" check had the same identity
comparison and restarted the album after a rescan; it compares by value too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:49:23 +02:00
ecb35dd5b5 Remove the Slint front-end
It has been untouched since 2026-09-21 while the web front-end got every feature and
the Pi performance work since (thumbnails, ?pi=1, number keys, clock, volume steps).
Keeping it meant a second, already stale copy of the search, keymap, Home Assistant
client and API types. Nothing outside slint-frontend/ referenced it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:48:17 +02:00
4428357e13 Phone layout: opaque status bar and one safe-area rule for every edge
Each fix to the home-screen app's layout on iOS broke something else, and they
all traced back to black-translucent: it runs the page under the clock, so any
top-anchored control without its own inset sat behind it (the play view's back
button), and since iOS 26 it sizes the app one status bar short with a strip at
the bottom that CSS cannot paint (WebKit 301108). The status bar is opaque now
("black"), the page starts below it, and lib/fullHeight.ts - the JS workaround
for the short viewport - is gone with the CSS that depended on it.

viewport-fit=cover stays, so everything against a screen edge now adds that
edge's inset from shared --safe-* variables: the corner back button, the
remote-assign button, the assign toast, the tool buttons and the tab rail did
not before. The rules are written down under "Layout on phones" in the README.

On a phone held sideways the mini player and the tabs share one row instead of
taking half the screen.

Home-screen installs read the status-bar tag once: remove and re-add the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 10:46:06 +02:00
34f2f016f9 Firmware: port to Arduino core 3 / ESP-IDF 5.5 and pin every dependency
The unpinned platform had moved to Arduino core 3, which the firmware no longer
built against. Pinned now: pioarduino's platform 55.03.312-1 (Arduino core 3.3.12,
ESP-IDF 5.5.5) - PlatformIO's own espressif32 platform still ships core 2 - plus
MFRC522 1.4.12, AceButton 1.10.1 and native 1.2.1.

- LED strips: the legacy RMT driver is replaced by the new rmt_tx API with a bytes
  encoder, at the same 50 ns tick, bit timings and byte order. ESP-IDF 5 aborts at
  boot when both drivers are linked, and the driver now picks the channel itself,
  so Esp32DriverRGBW::begin() takes only the pin.
- Button backlights: ledcAttach instead of ledcSetup/ledcAttachPin, and ledcWrite
  is addressed by pin rather than channel.
- Rotary encoder: gpio_reset_pin instead of the removed gpio_pad_select_gpio.

The touch pads stay on the legacy touch driver: on the plain ESP32 it only warns,
its conflict check passes as long as the new driver is not linked (it is not), and
moving would change the raw counts the thresholds were tuned against.

src_filter is build_src_filter now. Not yet tried on the device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 00:29:26 +02:00
d465a74cde Figures: several albums per figure, picked on the LED ring
A figure folder that holds album folders instead of audio can play any of them,
up to six, in folder order (depth-first, alphabetical). Each is its own album,
titled "Fuchs - <folder>" and of the figure's kind. Putting the figure down plays
the album it played last, or the first, straight away; another figure placed in
between forgets the position in that album, not the album itself.

Meanwhile the ring offers all of them: one section per album from a fixed palette,
edges shifting towards the neighbours' hue, the selected one bright and wobbling.
The knob moves the selection one section per detent and wraps; pressing plays the
selected album, and left alone for 8 s the picker fades into the figure's colours
and the music carries on. While open, the device hands the knob to the picker
(`capture_knob`): turns do not change the volume and the press is not published to
Home Assistant. A tick per step and a confirm "pling" go with it.

The firmware only draws: one new ring effect, LED_WHEEL_EFFECT_SEGMENT_SELECT (23),
re-sent on every detent with the section to glide from. The look is tunable through
a temporary `general.figure_selection` config section until the right values
become constants.

Every UI sound - the screen's blubb, the parents' timer, the picker - now follows
the music volume with a floor, times a new parent-mode "Effekt-Lautstärke"
(`general.effects_volume`).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 00:29:18 +02:00
110 changed files with 2678 additions and 15248 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 697 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 788 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 788 KiB

View File

@@ -20,7 +20,9 @@ Assistant integration.
| `web/` | React + TypeScript front-end (Vite). Browse the whole library and play any of it from a browser. Built output is served by `python-backend` itself. |
| `esp-firmware/` | ESP32 firmware (C++, Arduino framework via PlatformIO). Reads the RFID reader and buttons, drives the LED strips, talks to `python-backend` over serial. |
| `hardware/` | 3D-print models for the figurines and enclosure (FreeCAD/Blender/OBJ/STL), a Fritzing electronics sketch, datasheets, and `pinout.md` (RFID reader + button-board wiring). |
| `claude-design/` | The interaction and visual spec the web front-end was built from ("Dolphin Beats"), as a standalone HTML mockup with hardcoded data. `web/` is the real implementation; the mockup is kept as the reference for the keyboard model and the styling. It also contains a second page, "Mein Zimmer" (room lights), which is **not** implemented. |
| `claude-design/` | The interaction and visual specs the web front-end was built from, as standalone HTML mockups with hardcoded data: "Dolphin Beats" (the player), "Mein Zimmer" (the room page) and "Delphin Uhr" (the clock overlay). `web/` is the real implementation of all three; the mockups are kept as the reference for the keyboard model and the styling. |
| `scripts/` | `check.sh` (every check, see below) and `sync-library.sh` (seed a device's library and analysis cache). |
| `docs/` | This file. |
## How the pieces talk to each other
@@ -41,12 +43,18 @@ Assistant integration.
out of step with a figure someone puts on the reader. See
`python-backend/README.md` for the endpoint list.
- **`python-backend` ↔ Home Assistant**: MQTT only. The backend publishes three
discoverable lights, a player sensor, a volume number, transport buttons, device
triggers for every button and touch area, and a tag scanner for the RFID reader. It
does *not* call Home Assistant services directly any more; behaviour like "the left
ear turns the room light pink" is an HA automation. See `python-backend/README.md`
for the trigger topics and the old colour mapping.
- **`python-backend` ↔ Home Assistant**: two separate paths.
- MQTT, for the mouse itself. The backend publishes three discoverable lights, a
player sensor, a volume number, transport buttons, device triggers for every button
and touch area, and a tag scanner for the RFID reader. Behaviour like "the left ear
turns the room light pink" is an HA automation, not backend code. See
`python-backend/README.md` for the trigger topics and the old colour mapping.
- HTTP, for the web front-end's room page only: `/api/ha/...` proxies state reads and
service calls to Home Assistant's REST API with the token from `general.ha`, so the
token never reaches the browser.
- **`python-backend` ↔ IR remote**: a TCP client for lircd (`general.lirc`), turned into
the same intents as every other front-end.
## Running it
@@ -89,16 +97,20 @@ production so one process serves both the UI and the API.
- `general.{library.*, serial_port, baudrate, reconnect_interval, alsa_device,
min_volume, max_volume, initial_volume, volume_increment, button_leds_brightness,
audio_extensions, mqtt.*, web.*}`
- `figures.<name>.{id, colors}` — `id` is a 5-byte hex RFID tag, unique per figure;
`colors` is exactly four (`primary, secondary, bg, accent`), each `"#rrggbb"` or
`"wNN"`.
audio_extensions, podcast_episode_limit}`, plus optional sections `mqtt`, `web`, `ha`
(room page), `lirc` (IR remote), `screen` (monitor on/off), `tippen` (typing game)
and `clock` (school bus countdown).
- `figures.<name>.{id, colors, kind}` — `id` is a 5-byte hex RFID tag, unique per
figure; `colors` is exactly four (`primary, secondary, bg, accent`), each `"#rrggbb"`
or `"wNN"`; `kind` is `music` (default) or `book`.
- `remote.<digit>` — what each IR-remote number key plays. Written by the web UI.
`general.library.root` is the one path to the music. The shelves under it are fixed
names, not settings, because each has quirks the scanner knows about:
```
<root>/Figuren/<figure name>/ one folder per figurine
<root>/Figuren/<figure name>/ one folder per figurine (or album subfolders
in it, up to six, picked on the LED ring)
<root>/Musik/<Artist> - <Album>/ albums, grouped by artist
<root>/Hörbücher/<Artist> - <Album>/ audiobooks, grouped by character
<root>/Kinderpodcasts/<Show>/ shows, newest episode first
@@ -109,16 +121,16 @@ reported at once.
## Notes for agents
- The backend has `pytest`, `ruff` and `mypy --strict` configured in
`python-backend/pyproject.toml`, and no CI. Run all three before proposing changes.
- There is no CI. `scripts/check.sh` runs every check the repo has - backend
`pytest`, `ruff` and `mypy --strict`, web `vitest`, `tsc` and `eslint` - and is the
gate: run it before proposing changes. A checker that cannot run (mypy once aborted
on numpy's stubs for two weeks without anyone noticing) is a failure, not a skip.
- Prefer adding a scenario in `python-backend/scenarios/` over a hand-rolled test when
the behaviour is end-to-end — those files are executed by the test suite.
- `python-backend/notebooks/` is university course material on chord recognition, not
part of the app.
- `python-backend/notebooks/` is LED-effect scratch work, not part of the app.
- The firmware has no automated tests beyond a PlatformIO `native` env for LED effects.
- The front-end has `vitest` over the pure modules (`web/src/lib/`) and `tsc --noEmit`;
there is no component-level test harness. `npm run test` and `npx tsc --noEmit` are
the two checks.
- The front-end has `vitest` over the pure modules (`web/src/lib/`), `tsc` and `eslint`;
there is no component-level test harness.
- The library index is cached under `general.library.cache`. It is keyed by file mtime
and size, so **a change to how the scanner derives a title, artist or series is
invisible until the cache is invalidated** — bump `_INDEX_VERSION` in

View File

@@ -0,0 +1,121 @@
# Task: password-protect the backend's HTTP API
## Why
The backend's HTTP API has no authentication and listens on `0.0.0.0`. The worst part
is the room page's Home Assistant proxy
(`python-backend/musicmouse/services/web/api.py`, `_proxy` and
`POST /api/ha/services/{domain}/{service}`): it forwards *any* service call on *any*
entity with the stored long-lived HA token. Anything on the LAN can unlock a lock,
disarm an alarm or restart Home Assistant through it. Parent mode (`/api/settings`,
`/api/remote/mapping`, `/api/timer`) is only hidden, not protected.
## What the owner decided
A browser authenticates **once** with a password, and then stays logged in for about
**a year**. The config stores only a **hash** of the password, never the password
itself. Use this design; do not substitute another one (no entity allow-list instead,
no per-endpoint tokens, no OAuth).
## Design
### Config
- Add `general.web.password_hash` (in `WebConfig`, `musicmouse/config.py`). It is
**required** whenever the `web` section is present. Follow the repo's existing
convention for opting out of something important (see `SIMULATE` for `serial_port`
and `alsa_device`): the literal value `"disabled"` turns authentication off, logs a
warning on **every** boot, and is the only way to run without it. A missing key is a
config error. Config errors already say which key is wrong.
- Hash format: `scrypt$<n>$<r>$<p>$<salt b64>$<hash b64>` using `hashlib.scrypt` from
the standard library. No new dependency. Choose parameters that verify in well under
a second on a Raspberry Pi 4 (for example n=2**14, r=8, p=1), and measure on the
desktop to sanity-check. Verify with `hmac.compare_digest`.
- Add a CLI subcommand to set it. For example,
`python -m musicmouse --config config.yml --set-password` prompts twice with
`getpass`, hashes, and writes the key back into `config.yml` using the existing
round-trip helpers in `musicmouse/services/web/settings.py` (`load_document`,
`atomic_write`), so comments survive. Exit afterwards and start nothing else.
- Document the key and the command in `python-backend/config.yml.example`.
### Sessions
- `POST /api/auth/login` with `{"password": "..."}`. On success it sets a cookie and
returns 204; on failure it returns 401 after a fixed delay of about 1 s (cheap brute-force
damping; no lockout state needed).
- The cookie (`mm_session`): `HttpOnly`, `SameSite=Strict`, `Path=/`,
`Max-Age` = 365 days. **Not** `Secure`: the device is served over plain HTTP on the
LAN, and a `Secure` cookie would never be sent back.
- The cookie value is a stateless signed token, `<expiry unix seconds>.<HMAC-SHA256>`.
The HMAC key is derived from a persistent random secret *and* the current
`password_hash`, so **changing the password logs every browser out** without extra
bookkeeping. Keep the secret in a file under `general.library.cache` (for example
`session-secret`, 32 random bytes, created on first start with mode 0600). Deleting
the cache logs everyone out, which is acceptable; say so in the docs.
- Optional, cheap, and nice: re-issue the cookie with a fresh year on any authenticated
request when fewer than 30 days remain, so a device in daily use never expires.
- `POST /api/auth/logout` clears the cookie. `GET /api/auth` returns 204 or 401, so the
front-end can check without side effects.
### What is protected
- **Every** route under `/api`, including the websocket `/api/ws`, except
`/api/auth/*`. Implement it once, as a FastAPI dependency on the router in
`build_router` (or a small middleware), not route by route. The websocket checks the
cookie *before* `accept()` and closes with code 4401 when it is missing or invalid.
- **Not** the static front-end files: the login screen has to load.
- **No localhost exemption.** The kiosk on the device logs in once like everyone else.
An exemption by peer address breaks silently the day a reverse proxy is put in front
of it (every request then looks local).
- MQTT, the IR remote and the serial link are not HTTP and are out of scope.
- Keep the room page's proxy as it is otherwise. Do not add an entity allow-list in
this task.
### Front-end (`web/`)
- `src/api/client.ts`: a 401 from any call raises a dedicated `Unauthorized` error.
`fetchOrNullOn404` / `fetchOrNullOnError` must not swallow a 401 as "not configured".
- A login screen: one password field, German copy ("Passwort", "Anmelden", "Falsches
Passwort"), styled like the existing sheets (`.overlay` / `.sheet`). It must work with
only a keyboard (the kiosk has no pointer). Show it when the first `/api/state` or
`/api/library` call, or the websocket, reports unauthorized. On success, reload the
library, the state and the websocket. The simplest correct way is `location.reload()`.
- `src/hooks/usePlayerState.ts`: on a 4401 close, stop the reconnect loop and show the
login screen instead of retrying every 1.5 s.
- The service worker (`public/sw.js`) must not cache `/api/auth/*`. It already avoids
`/api` state; check it.
## Tests (backend)
- Every existing web test goes through the API. Give the test client fixture in
`tests/test_web.py` (and `test_remote_api.py`, `test_tippen_api.py`) a logged-in
cookie, so the existing tests stay as they are. Add a separate unauthenticated client
fixture for the new tests.
- New tests:
- Each `/api` route returns 401 without the cookie. Generate the list from the router
so a new route cannot slip through unprotected.
- The websocket is refused without the cookie.
- Login succeeds with the right password and fails with a wrong one.
- A token past its expiry is rejected.
- Changing `password_hash` invalidates existing cookies.
- `"disabled"` lets everything through and logs the warning.
- A missing key is a config error.
- The `--set-password` command writes a hash that verifies and keeps the file's
comments.
- The fixed failure delay must not slow the suite down: inject it, as the codebase
injects `Clock`.
## Docs to update
- `python-backend/README.md`: the "Parent mode" paragraph that says there is no
authentication; the route table; a short "Authentication" section.
- `web/README.md`: the "Parent mode" paragraph.
- `docs/REPO_OVERVIEW.md`: the backend ↔ web paragraph and the config summary.
- The comment block above the room-control routes in `api.py`, which justifies the open
proxy with "trusted LAN device".
## Done when
`scripts/check.sh` passes, and a manual run against `--simulate` with a hash set shows
the login screen, logs in, survives a backend restart without asking again, and asks
again after the password is changed.

View File

@@ -0,0 +1,121 @@
# Task: one behaviour per button, whatever it is pressed on
## Why
The backend's README promises that front-ends only translate input into intents and that
`musicmouse/reactions/` is the one place that decides what happens. Three behaviours
break that promise today, so the same request does different things depending on where
it came from:
| Behaviour | Web front-end | IR remote | Mouse buttons / knob | MQTT |
|---|---|---|---|---|
| Next/previous on a podcast | seeks ±30 s (`web/src/App.tsx`, `skipPodcast`) | skips a track | skips a track | skips a track |
| Unmute | jumps to 60 % (`UNMUTE_PERCENT` in `App.tsx`) | restores the previous volume (`LircService._toggle_mute`) | - | - |
| Volume step | J/K: 10 % (`VOLUME_STEP` in `web/src/lib/keyboard.ts`); +/- buttons: `volume_increment` | `volume_increment` | `volume_increment` × detents | absolute only |
There is also a confirmed bug. With `min_volume > 0`, the remote's mute never unmutes:
"mute" asks for volume 0, the player clamps it to `min_volume`, the next press sees
`volume > 0` and mutes again.
## What the owner decided
1. **Next/previous on a podcast episode always seeks 30 s** forward/back, from every
input: web (buttons and keys), IR remote, the mouse's buttons, MQTT.
2. **Unmute always restores the volume from before muting**, from every input.
3. **Parent mode gets three separate volume-step settings**: the web keyboard's J/K,
the mouse's rotary knob, and the IR remote. Each is configured independently.
## Design
### 1. Podcast skip lives in the backend
- In `musicmouse/reactions/playback.py`, `next_track` / `previous_track` look at the
current album (`app.album_for(app.player.playlist)`). For a podcast episode they seek
instead: `position ± PODCAST_SKIP_SECONDS` (30, a module constant), clamped to
`[0, duration]`. Take the duration from the library track when the player does not
know it yet, the way `services/web/state.py` does.
- "Is this a podcast?" is decided by **one** helper. If `docs/agent-tasks/06-podcast-kind.md`
has landed, that is `album.kind == "podcast"`. If not, add `is_podcast(album)` next to
`Album` (`musicmouse/library/models.py`) that checks the section's
`album_unit == "episode"`, and let task 06 change its body.
- `web/src/App.tsx`: the `next` / `previous` actions always call `api.next()` /
`api.previous()`. Delete `skipPodcast` and `PODCAST_SKIP_SECONDS`. Drop the optimistic
`position: 0` patch too: the client no longer knows what the backend will do. If the
≤0.5 s until the next position frame is visibly laggy, let `StateHub` push a
`position` frame right after a next/previous/seek intent has been handled. Don't
re-add client-side policy.
- The mouse's left/right buttons only act while playing (`button_pressed`); keep that.
### 2. Mute is a player state, not a volume
- Add `muted: bool` to the `Player` protocol, plus `set_muted(muted, *, source)`,
implemented in `PlayerBase`. `VlcPlayer` applies it with libVLC's
`audio_set_mute`; `FakePlayer` just records it. Muting does **not** touch `volume`,
so unmuting restores it by construction, and `min_volume` cannot interfere.
- Announce changes with a new `StateEvent` (for example `MuteChanged`) and add it to
`SNAPSHOT_EVENTS` in `services/web/hub.py`. A test fails until every `StateEvent` is
sorted into that list or `IGNORED_EVENTS`.
- A new intent, `ToggleMuteRequested`, handled by a reaction. Any volume change
(`SetVolumeRequested`, `VolumeChangeRequested`, the step intent below) also unmutes;
that is what every TV remote does.
- `POST /api/mute/toggle`. The web's Shift+M and the mute button call it. Delete
`UNMUTE_PERCENT` and `onMute`'s volume arithmetic.
- `LircService`: `KEY_MUTE` emits `ToggleMuteRequested`. Delete `_pre_mute_volume` and
`_toggle_mute`.
- The state snapshot (`PlayerStateOut`, `web/src/api/types.ts`) gets `muted`. The volume
bars show muted distinctly instead of showing volume 0.
- MQTT is out of scope. Its volume number stays absolute, and setting it unmutes via
the rule above.
### 3. Three volume steps
- Config: replace `general.volume_increment` with
`general.volume_steps: {keyboard: int, knob: int, remote: int}`, all in **device
units** like `min_volume` / `max_volume`, each `ge=1, le=100`. Defaults: `keyboard: 10`,
`knob: 5`, `remote: 5` (today's behaviour for knob and remote).
- `volume_increment` in an existing `config.yml` must fail with a message naming the
replacement ("replaced by volume_steps.knob / .remote / .keyboard"). Config
validation rejects unknown keys anyway; make the message specific rather than the
generic "unknown option".
- One intent: `VolumeStepRequested(direction: Literal[-1, 1], count: int = 1,
input: Literal["keyboard", "knob", "remote"])`. One reaction multiplies by the
configured step and calls `change_volume`.
- `rotary_turned` emits it with `input="knob"`, `count=abs(event.increment)`.
- `LircService` volume keys emit it with `input="remote"`.
- `POST /api/volume` takes `{"steps": n, "input": "keyboard" | "remote"}` instead of
bare `steps`. J/K send `input: "keyboard"`. The on-screen −/+ buttons keep taking
the remote's step, as 1254790 deliberately chose, so they send `input: "remote"`.
- `{"percent": n}` stays for the volume bars.
- Parent mode: `SettingsIn` / `SettingsOut` (`services/web/schemas.py`),
`read_settings` / `write_settings` (`services/web/settings.py`, writing the nested
mapping), and `web/src/components/ParentPanel.tsx` get three fields, labelled in
German (for example "Lautstärke-Schritt: Tastatur (J/K)", "… Drehknopf",
"… Fernbedienung"). They replace the single "rotary step". Saving applies live, like
the other settings.
- Update `python-backend/config.yml.example`, the route table in
`python-backend/README.md`, `web/README.md`'s key table, and `HelpOverlay.tsx`.
## Tests
- Reactions (`tests/test_reactions.py`, through the simulator):
- Next/previous on a podcast episode seeks ±30 s and clamps at both ends. Use a
`Kinderpodcasts` folder in the test library (see `tests/conftest.py`).
- Next/previous on music still changes track.
- Mute then unmute restores the exact volume, including with `min_volume > 0` (the
regression).
- A volume change while muted unmutes.
- Each of the three inputs moves by its own configured step.
- `tests/test_lirc.py`: the mute key and the volume keys emit the new intents.
- `tests/test_web.py`: `/api/mute/toggle`; `/api/volume` with `input`; the settings
round-trip for `volume_steps`, which keeps the file's comments.
- `tests/test_config.py`: the defaults, and the specific error for `volume_increment`.
- `web/src/lib/__tests__/keyboard.test.ts`: J/K produce the step action, not a
percentage.
## Done when
`scripts/check.sh` passes, `grep -rn "PODCAST_SKIP\|UNMUTE_PERCENT\|volume_increment"`
finds nothing in `web/src` and `python-backend/musicmouse` apart from the config error
that names the old key, and in `--simulate` a
podcast skips 30 s from the web, from `press right` in the REPL, and from a simulated
remote key.

View File

@@ -0,0 +1,75 @@
# Task: stop the firmware garbling its own messages to the host
## Why
`sendMessageToHost` (`esp-firmware/src/Messages.h`) writes one frame as four separate
`Serial.write` calls: magic, type, size, payload. It runs on two FreeRTOS tasks at once:
- the Arduino `loop()` task, for touch, rotary encoder and button events
(`esp-firmware/src/main.cpp`);
- the RC522 reader's own task (`rc522_task` in `esp-firmware/lib/nfc/rc522.c`, priority
4), which calls `tagHandler` → `sendMessageToHost(MsgRfidTokenRead{...})` directly.
When a tag read coincides with a button or touch event, the two frames interleave on
the wire. The host's decoder (`python-backend/musicmouse/devices/wire.py`,
`FrameDecoder`) resynchronises on the magic token and drops what it cannot parse. So
the symptom is an occasionally **lost figure placement/removal or button press**, with a
"Discarding bad frame from firmware" warning in the backend log.
## What the owner decided
Fix it in whatever way keeps the firmware simplest. A mutex was the first idea. **Use
the queue below instead, unless it turns out not to work**, because it is simpler:
- With a queue, every frame and every log line is written from `loop()` alone, so no
lock is needed anywhere, including around the `Serial.println` log lines that could
otherwise land in the middle of a frame.
- It is exactly the pattern `main.cpp` already uses for the rotary encoder
(`eventQueueRotaryEncoder`, drained in `handleRotaryEncoder()` with
`xQueueReceive(..., 0)`).
A mutex would need wrapping every `Serial.write` / `Serial.print` in the firmware to be
complete. That is more code and easier to get wrong later.
## Design
In `esp-firmware/src/main.cpp`:
1. Create a FreeRTOS queue for tag reads in `setupRfidReader()`, **before**
`rc522_start`, so the callback can never run without it. Item type: the 5-byte tag id
(`MsgRfidTokenRead` itself is fine, it is a packed 5-byte struct). Depth: a handful
(4) is plenty; the reader polls every 125 ms.
2. `tagHandler` (runs on the RC522 task) only fills a `MsgRfidTokenRead`, all zeros
for "removed" exactly as now, and `xQueueSend(queue, &msg, 0)`. If the queue is full,
drop the read; the reader reports the state again on the next change. Nothing in
`tagHandler` touches `Serial` any more.
3. Add `handleRfid()`, called from `loop()` next to `handleRotaryEncoder()`. It drains
the queue with `xQueueReceive(..., 0)` and calls `sendMessageToHost` for each item.
4. Leave `Messages.h` alone. With a single writer, the four `Serial.write` calls are
fine as they are.
Do not change the wire format, message ids or the host. `python-backend/tests/test_wire.py`
parses `Messages.h` and pins the ids, and it must keep passing.
Out of scope: the touch thresholds, debouncing, anything in `lib/`.
## Checking it
There are no firmware unit tests that reach `main.cpp` (the `native` env only builds the
LED effects), and this machine may not have PlatformIO. So:
- It must build: `pio run -e esp32` in `esp-firmware/`. If PlatformIO is not
available, say so in the hand-off instead of claiming it builds.
- `pio run -e native` still builds `src/host_test.cpp`. That env excludes `main.cpp`, so
this only guards against accidental changes to shared headers.
- `scripts/check.sh backend` still passes (`test_wire.py`).
- Manual test on the device, with the backend running at `--log-level DEBUG`. Hold a
hand on an ear touch area while repeatedly placing and removing a figure for a
minute, and turn the knob while doing it. Expect every placement and removal to show
up as `RfidTokenRead`, and **no** "Discarding bad frame from firmware" warnings.
Before the fix, the same exercise should produce some.
## Done when
The firmware builds, the host tests pass, `tagHandler` contains no `Serial` call, and the
manual test shows no discarded frames.

View File

@@ -0,0 +1,89 @@
# Task: make "podcast" a kind of album
## Why
`AlbumKind` is `Literal["music", "book"]` (`python-backend/musicmouse/library/sections.py`),
and the `Kinderpodcasts` shelf is declared `kind="book"`. So "is this a podcast?" is
answered differently in each layer:
- web: `album.section === "Kinderpodcasts"` (`web/src/lib/search.ts`, `groupOf`), and
everything else goes through `groupOf(album) === "podcasts"` (App, PlayView,
BrowseView, MobileBrowse, keyboard.ts, remote.ts, ambience.ts, useTrackDetail);
- backend: `SECTIONS[album.section].album_unit == "episode"` (`MusicLibrary.latest_episode`,
`tippen/rewards.py`);
- typing game: reward kinds `"episode"` and `"podcast_episode"`
(`services/web/tippen_api.py`).
The shelf's folder name is therefore a de-facto API string the front-end depends on.
`todo.md` already asks for a cleaner podcast data model.
## What the owner decided
Add `"podcast"` to the album kind. Podcast-ness is then a property of the album, carried
in the index and the API, and nobody compares section names.
## Design
### Backend
- `AlbumKind = Literal["music", "book", "podcast"]`. `SECTIONS["Kinderpodcasts"]` gets
`kind="podcast"`.
- `Section.album_unit` stays. It is the scanner's structural fact (one album per file),
not a classification. Keep using it in the scanner (`scan_library`) and in
`podcast_feeds.find_feed_shows`. Every place that asks "is this album a podcast?"
switches to `album.kind == "podcast"`:
- `MusicLibrary.latest_episode`;
- `tippen/rewards.py` (`_chronological_episodes`, and the `album_unit == "folder"`
branches that really mean "not a podcast");
- `tippen_api._animation_kind`;
- the `is_podcast` helper, if task 04 added one: change its body, keep its callers.
- `scanner.scan_album` derives `series` only for `kind == "book"`. Check that episode
albums still get `series = folder.name` (they do, in `scan_episodes`) and that
nothing else branches on `"book"` meaning "book or podcast".
`grep -rn '"book"' python-backend/musicmouse` and decide per hit.
- `FigureConfig.kind` stays `Literal["music", "book"]`. A figurine holding a podcast is
not a thing. Type `Config.figure_kinds` accordingly.
- `cache._album_from_json` validates the kind: accept `"podcast"`. Bump
`_INDEX_VERSION` (with a comment line, like the existing ones), so every cached
podcast album is re-scanned with the new kind instead of being read back as
`"book"`.
- `_ANALYZED_KINDS` stays `("music",)`.
- `services/web/schemas.py`: `AlbumOut.kind` and `PlayerStateOut.kind` become the
`AlbumKind` literal instead of `str`, so the OpenAPI schema says what the values are.
### Web
- `src/api/types.ts`: `AlbumKind = "music" | "book" | "podcast"`.
- `src/lib/search.ts`, `groupOf`: map from `kind` alone (`podcast` → `"podcasts"`,
`book` → `"audiobooks"`, else `"music"`). No `section` comparison may remain
anywhere in `web/src`.
- `src/lib/covers.ts`: `isBook` and `aspectOfKind` treat `"book"` specially. Podcasts
are drawn like books today *because* they were `kind: "book"`. Keep them looking
exactly as they do now: decide explicitly whether `isBook` / `aspectOfKind` include
`"podcast"`, and check BrowseView's `mostlyBooks` / `allPodcasts` tiles and the play
view. The result must be visually unchanged; compare screenshots of the podcast
shelf, a podcast category and the podcast play view before and after.
- Anything else that reads `state.kind` or `album.kind`: grep and decide per hit.
### Leave alone
The `section` field stays in the API; it is still useful for display and debugging.
Don't rename the shelves, the folders or the `Group` names.
## Tests
- `tests/test_library.py`: a `Kinderpodcasts` episode scans as `kind == "podcast"`; a
cache written at the previous `_INDEX_VERSION` is discarded; `latest_episode` still
works.
- `tests/test_tippen_rewards.py` / `test_tippen_api.py`: episode rewards still resolve
and still animate as `podcast_episode`.
- `tests/test_web.py`: `/api/library` reports `"podcast"`.
- `web/src/lib/__tests__/search.test.ts`: `groupOf` uses `kind`; an album whose section
says `Kinderpodcasts` but whose kind is `music` is music. That proves nothing compares
the section any more.
## Done when
`scripts/check.sh` passes, `grep -rn "Kinderpodcasts" web/src` finds nothing, the
backend's "is it a podcast" checks all read `album.kind`, and the podcast UI looks the
same as before.

View File

@@ -0,0 +1,222 @@
# Task: eight independent cleanups
Each section below is a separate change: do them as separate commits, in any order, and
run `scripts/check.sh` after each. Each comes from a code review of commit `a55cc4f`;
the "why" says what is wrong and, where known, which commit introduced it. Do not
change behaviour beyond what a section asks for.
---
## A. Races in library updates and concurrent rescans (review item 7)
**Why.** `MusicLibrary.refresh()` and `MusicLibrary._publish_analysis()`
(`python-backend/musicmouse/library/__init__.py`) both do
`self._entries = await asyncio.to_thread(...)` from a snapshot taken before the await.
Whichever finishes last wins, so an analysis batch that started before a rescan
finished can overwrite the freshly scanned index (a new podcast episode disappears
until the next rescan). Both also write `index.json` from worker threads at the same
time. Rescans are not serialised either: `POST /api/library/refresh` fires an
unretained `asyncio.create_task` (`# noqa: RUF006` in `services/web/api.py`), and the
podcast poller rescans on its own (`App.rescan_library`, introduced with the poller in
57afc32).
Separately, `VlcPlayer` (`devices/player.py`) mutates `_playing`, `_index` and
`_playlist_changed` both from libVLC's callback thread (`_on_playing`, `_on_stopped`,
`_on_playlist_end`, `_on_next_item`) and from the event loop (`_poll`, the actions).
**Do.**
- One `asyncio.Lock` in `MusicLibrary` held around every read-modify-write of
`_entries` plus its `store_index`: the whole of `refresh()`, and the fold-swap-store
in `_publish_analysis()`. The analysis workers keep running while a scan holds the
lock; only publishing waits.
- `App.rescan_library` coalesces. While one rescan runs, further calls wait for the
running one plus at most one follow-up, not N parallel scans. Keep a reference to the
task the refresh endpoint starts, and drop the `noqa`.
- `VlcPlayer`'s libVLC callbacks do nothing but hop onto the loop (the bus already
knows the loop; give `PlayerBase` a small `_on_loop(fn)` using
`loop.call_soon_threadsafe`). All state mutation then happens on the loop thread.
Update the comment above the callbacks.
**Test.** A library test with a scan made artificially slow (an injected scanner or a
patched `scan_library`) and an analysis publish racing it. The new album must survive.
Two concurrent `rescan_library` calls run the scanner at most twice.
---
## B. Library cache invalidation gaps (review item 8)
**Why.** `scan_library` (`library/scanner.py`) reuses a cached folder album whenever
its *audio files'* names, sizes and mtimes are unchanged (`Fingerprint`). So:
1. changing a figure's `kind:` in `config.yml` has no effect until the figure's files
change: the cached album keeps the old kind;
2. a `cover.jpg` (or any of `_COVER_NAMES`) added to an album folder later is never
picked up. Podcast episodes include their sidecar covers in the fingerprint; folder
albums do not;
3. covers of albums that no longer exist (deleted folders, episodes pruned by the
podcast poller) stay in `cache/covers/` forever.
**Do.**
- Folder albums: include the existing `_COVER_NAMES` files in the fingerprint, like
`scan_episodes` does for sidecars.
- Figure albums: reuse a cached entry only if its `kind` equals the configured one.
- After a scan, delete `covers/<id>.jpg` and `covers/<id>.thumb.jpg` for ids not in
the new index. Do **not** garbage-collect `analysis/`: it is expensive, content-keyed
and documented as safe to keep.
- Bump `_INDEX_VERSION` with a comment line, because the fingerprint shape changes.
**Test.** One test per gap in `tests/test_library.py`.
---
## C. The LED "off" animation is sent twice when a figure is removed (review item 11)
**Why.** Removing a figure makes `reactions/lighting.figure_placed_or_removed` run
`off_animation`. The player pausing then emits `PlaybackChanged(playing=False)`, and
`web_playback`, whose guard is "no figure on the reader", runs `off_animation` again.
That makes 6 LED writes instead of 3 and restarts the reverse swipe halfway through.
`web_playback` came in with the web front-end (edb6e5e).
**Do.** Make `off_animation` idempotent: skip a zone whose current effect
(`app.mouse.effect(zone)`) is already the off effect it would send, and skip the button
brightness when it is already 0. That keeps both reactions simple and covers any future
third caller.
**Test.** In `tests/test_reactions.py`: removing a figure while playing produces
exactly three `LedEffectChanged` events, one per zone.
---
## D. `config.yml` has no single owner (review item 13)
**Why.**
- `PUT /api/settings` (`services/web/api.py`) applies values with `setattr` on the
pydantic config objects. That skips validation, so the handler re-implements the
min/max/initial volume checks by hand.
- `PUT /api/remote/mapping` replaces `app.config.remote`.
- Both endpoints read, modify and write `config.yml` from worker threads
(`write_settings`, `write_mapping`) with no lock, so two saves at once can lose one.
- Memory is updated *before* the file is written, so a failed write leaves the running
config and the file disagreeing.
**Do.** One small `ConfigStore` (for example `musicmouse/config_store.py`) owns the path,
the in-memory `Config` and a lock. An update:
1. validates the merged values through the real pydantic models, so the
`_check_volumes` validator runs; no duplicated checks in the endpoint;
2. writes the file (round-trip, comments kept, via the existing `load_document` /
`atomic_write`);
3. only then applies the values to the live objects.
Objects that read the config live (`ScreenService` reads `auto_off_minutes`, the player's
volume limits) must keep seeing changes: either keep mutating the same objects after
validation, or have them read through the store. Pick one and say which in the
docstring. The settings and remote-mapping endpoints both go through the store. If
`docs/agent-tasks/01-web-authentication.md` has landed, its `--set-password` command
should use it too.
**Test.** Invalid settings are rejected by the model's own validator, with an
unchanged file and unchanged memory. Two concurrent saves (settings and mapping) both
end up in the file.
---
## E. Generate the front-end's API types (review item 15)
**Why.** `web/src/api/types.ts` (~300 lines) hand-mirrors
`python-backend/musicmouse/services/web/schemas.py`. FastAPI already describes every
route and model at `/api/openapi.json`. (A third copy lived in the Slint front-end,
which has been removed.)
**Do.**
- A backend test (for example `tests/test_openapi.py`) builds the app the way
`tests/test_web.py` does and compares `api.openapi()` with a committed
`web/src/api/openapi.json`. With `UPDATE_OPENAPI=1` it rewrites the file instead. The
test fails whenever the API changes and the snapshot was not refreshed.
- `web`: add `openapi-typescript` as a dev dependency and an `npm run gen:api` script
that generates `src/api/schema.d.ts` from that JSON. Make `npm run check` fail when
the generated file is stale: use the tool's own check mode if it has one (verify),
otherwise regenerate to a temp file and diff.
- `types.ts` keeps its exported names (`Album`, `PlayerState`, ...) as aliases of
`components["schemas"][...]`, so no component changes. Move the useful doc comments
onto the pydantic fields (`Field(description=...)`), where they reach both sides.
- The websocket messages (`ServerMessage`) are not in OpenAPI. Keep them hand-written
in `types.ts` with a comment saying so.
---
## F. Copy-pasted helpers (review item 16)
**Why.**
- `ScreenService.blubb` (`services/screen.py`) is a line-for-line copy of
`sound.play_wav`. `sound.py` was extracted in a55cc4f, but the screen service was
never moved onto it.
- The "did a person do this?" filter (an `IntentEvent`, not `LedEffectRequested`, not
`source == "system"`) is duplicated in `ScreenService._on_event` and
`TimerService._on_event`.
- There are five atomic-write helpers: `library/cache._write_atomic`,
`services/web/settings.atomic_write`, `tippen/progress.save_progress`,
`school_days.save_calendar` and `library/podcast_feeds._save_failed`. Three of them
name the temp file `<name>.tmp<pid>`, which collides when two threads of the same
process write the same file. That really happens: settings and remote-mapping saves
both run in `asyncio.to_thread`.
**Do.**
- `ScreenService.blubb` calls `play_wav(self.alsa_device, wav, timeout=self.config.timeout)`.
- `is_person(event: Event) -> bool` next to the event classes in `events.py`, used by
both services (keep each service's own extra exclusions).
- One `write_atomic(path, text)` (for example `musicmouse/fileio.py`) using
`tempfile.NamedTemporaryFile(dir=path.parent, delete=False)` and `os.replace`, used by
all five. Keep the temp file a dotfile where it sits in the music library, so the
scanner never picks it up. The YAML writer dumps to a string first.
**Test.** The existing tests cover behaviour. Add one for `write_atomic`, two threads
writing the same path at once, which today can fail.
---
## G. Inject time everywhere (review item 17)
**Why.** The backend's rule (see `musicmouse/clock.py`) is that anything that waits takes a
`Clock`, and `TimerService` takes a `wall` callable for the date. Exceptions:
- `PodcastFeedService.run` and `StateHub.run` call `asyncio.sleep` directly;
- `services/web/state.clock_state` uses `date.today()`;
- `POST /api/timer` (`services/web/api.py`) uses `datetime.now()`.
**Do.** Give `PodcastFeedService` and `StateHub` a `clock`, as the other services have,
wired in `__main__._build_services` / `WebService`. Put one `wall: Callable[[],
datetime]` on `App` (default `datetime.now`) and use it in `TimerService`,
`clock_state` and the timer endpoint, so a test can pin "today" in one place.
**Test.** The podcast service test runs on `FakeClock` instead of real sleeps; a
clock-state test pins the date through `App.wall`.
---
## H. Break up the largest files (review item 19)
**Why.** Every feature since the rearchitecture was added to the same few files.
- `web/src/App.tsx` is 909 lines with ~40 hooks, routes pages via flags, and
defines `ToolButton`, `CornerButton`, `ConnectionDot`, `Splash` and the assign toast
inline.
- `BrowseView.tsx` is 932 lines.
- About 300 static `style={{...}}` objects sit next to five CSS files.
- `services/web/api.py` is one router for seven features: library, state, playback,
parent mode, typing game, remote, Home Assistant.
- One sign of insertion rather than placement: the comment
`/** The one place a keyboard action, a click or a tap all end up. */` in `App.tsx`
sits above `saveMapping`, but describes `run`.
**Do** (mechanical moves only, no behaviour change; one commit per bullet):
- `api.py`: one router module per feature under `services/web/routes/`, assembled in
`build_router`. URLs unchanged.
- `App.tsx`: move the four small components into `components/`, the remote-mapping
state into `hooks/useRemoteMapping.ts`, and the kiosk listeners (screen activity,
silencing a ringing timer) into `hooks/useKioskActivity.ts`. Put the misplaced
comment back above `run`.
- `BrowseView.tsx`: split by screen (root shelves, category, results), whichever seams
the file already has.
- Inline styles: move **static** style objects in `App.tsx`, `BrowseView.tsx` and
`ParentPanel.tsx` into classes in the existing CSS files. Leave dynamic ones (colours
from album data, computed sizes) inline.
Check visually in the dev server, at desktop, phone (`usePhoneLayout`) and `?pi=1`
sizes, that nothing moved.

View File

@@ -0,0 +1,20 @@
# Agent tasks
Self-contained instructions for fixes that came out of a code review of commit `a55cc4f`.
Each file says why, what the owner decided, the design, the tests, and when it is done.
The numbers are the review's item numbers.
| File | What |
|---|---|
| `01-web-authentication.md` | Password login, once per browser for a year, in front of the whole `/api` (including the Home Assistant proxy). |
| `04-one-input-policy.md` | Podcast next/previous seeks 30 s from every input; mute restores the previous volume; three separate volume steps in parent mode. |
| `05-firmware-serial-frames.md` | Stop RFID frames interleaving with button frames on the serial link. |
| `06-podcast-kind.md` | `"podcast"` as an album kind instead of comparing the `Kinderpodcasts` folder name. |
| `07-backend-and-frontend-cleanups.md` | Eight independent cleanups: library races, cache invalidation, double LED off-animation, a config owner, generated API types, shared helpers, injected time, splitting the largest files. |
Order: `06` before `04` (04's podcast check uses 06's kind; it says how to proceed if
06 has not landed). `07 D` is easier after `01`. The rest are independent.
For every task: one commit per logical change, `scripts/check.sh` passing before each
commit, no pushing. When a task is done, delete its file in the same commit, so this
folder only ever lists open work.

View File

@@ -2,93 +2,72 @@
#include "drivers/Esp32DriverRGBW.h"
#include <driver/gpio.h>
#include <driver/rmt.h>
#include <driver/rmt_encoder.h>
#include <esp_check.h>
// Timing constants
static constexpr uint16_t DIVIDER = 4;
static constexpr double RMT_DURATION_NS = 12.5; // Minimum time of a single RMT duration based on clock ns
// Timing constants, in ns
static constexpr uint32_t RESOLUTION_HZ = 20 * 1000 * 1000; // one RMT tick = 50 ns
static constexpr uint32_t TICK_NS = 1000 * 1000 * 1000 / RESOLUTION_HZ;
static constexpr uint32_t T0H = 300;
static constexpr uint32_t T0L = 900;
static constexpr uint32_t T1H = 600;
static constexpr uint32_t T1L = 600;
static constexpr uint32_t TRS = 80000;
static constexpr rmt_item32_t bit0Data = {uint32_t(T0H / (RMT_DURATION_NS * DIVIDER)), 1, uint32_t(T0L / (RMT_DURATION_NS * DIVIDER)), 0};
static constexpr rmt_item32_t bit1Data = {uint32_t(T1H / (RMT_DURATION_NS * DIVIDER)), 1, uint32_t(T1L / (RMT_DURATION_NS * DIVIDER)), 0};
// Function registered at the RMT driver that converts regular uint8_t values containing r,g,b,w values
// to rmt_item32_t for each bit. (8 bit -> 32 * 8 bit)
static void IRAM_ATTR uint8ToRmtAdaptor(const void *src, rmt_item32_t *dest, size_t srcSize,
size_t wantedNum, size_t *translatedSize, size_t *itemNum)
static rmt_symbol_word_t symbol(uint32_t highNs, uint32_t lowNs)
{
if (src == NULL || dest == NULL)
{
*translatedSize = 0;
*itemNum = 0;
return;
}
size_t size = 0;
size_t num = 0;
uint8_t *psrc = (uint8_t *)src;
rmt_item32_t *pdest = dest;
while (size < srcSize && num < wantedNum)
{
for (int i = 0; i < 8; i++)
{
// MSB first
const bool isBitSet = *psrc & (1 << (7 - i));
pdest->val = isBitSet ? bit1Data.val : bit0Data.val;
num++;
pdest++;
}
size++;
psrc++;
}
*translatedSize = size;
*itemNum = num;
rmt_symbol_word_t s = {};
s.duration0 = highNs / TICK_NS;
s.level0 = 1;
s.duration1 = lowNs / TICK_NS;
s.level1 = 0;
return s;
}
void Esp32DriverRGBW::begin(int gpio, int rmtChannel)
void Esp32DriverRGBW::begin(int gpio)
{
rmtChannel_ = rmtChannel;
transmitting_ = false;
rmt_config_t rmt_tx;
rmt_tx.channel = static_cast<rmt_channel_t>(rmtChannel);
rmt_tx.gpio_num = static_cast<gpio_num_t>(gpio);
rmt_tx.rmt_mode = RMT_MODE_TX;
rmt_tx.mem_block_num = 1;
rmt_tx.clk_div = DIVIDER;
rmt_tx.tx_config.loop_en = false;
rmt_tx.tx_config.carrier_level = RMT_CARRIER_LEVEL_LOW;
rmt_tx.tx_config.carrier_en = false;
rmt_tx.tx_config.idle_level = RMT_IDLE_LEVEL_LOW;
rmt_tx.tx_config.idle_output_en = true;
rmt_tx_channel_config_t channelConfig = {};
channelConfig.gpio_num = static_cast<gpio_num_t>(gpio);
channelConfig.clk_src = RMT_CLK_SRC_DEFAULT;
channelConfig.resolution_hz = RESOLUTION_HZ;
channelConfig.mem_block_symbols = 64; // one block, refilled from the interrupt
channelConfig.trans_queue_depth = 4;
ESP_ERROR_CHECK(rmt_new_tx_channel(&channelConfig, &channel_));
ESP_ERROR_CHECK(rmt_config(&rmt_tx));
ESP_ERROR_CHECK(rmt_driver_install(rmt_tx.channel, 0, 0));
// Every byte goes out MSB first, one RMT symbol per bit - the strip's own byte order
// (g, r, b, w) is already how LedStripRGBW lays the data out in memory.
rmt_bytes_encoder_config_t encoderConfig = {};
encoderConfig.bit0 = symbol(T0H, T0L);
encoderConfig.bit1 = symbol(T1H, T1L);
encoderConfig.flags.msb_first = 1;
ESP_ERROR_CHECK(rmt_new_bytes_encoder(&encoderConfig, &encoder_));
rmt_translator_init((rmt_channel_t)rmtChannel, uint8ToRmtAdaptor);
ESP_ERROR_CHECK(rmt_enable(channel_));
}
void Esp32DriverRGBW::end()
{
ESP_ERROR_CHECK(rmt_driver_uninstall((rmt_channel_t)rmtChannel_));
waitForTransmissionToFinish();
ESP_ERROR_CHECK(rmt_disable(channel_));
ESP_ERROR_CHECK(rmt_del_encoder(encoder_));
ESP_ERROR_CHECK(rmt_del_channel(channel_));
encoder_ = nullptr;
channel_ = nullptr;
}
void Esp32DriverRGBW::writeSync(const uint32_t *rgbwData, int numLeds)
{
writeAsync(rgbwData, numLeds);
waitForTransmissionToFinish();
auto data = reinterpret_cast<const uint8_t *>(rgbwData);
ESP_ERROR_CHECK(rmt_write_sample((rmt_channel_t)rmtChannel_, data, numLeds * 4, true));
}
void Esp32DriverRGBW::writeAsync(const uint32_t *rgbwData, int numLeds)
{
waitForTransmissionToFinish();
auto data = reinterpret_cast<const uint8_t *>(rgbwData);
ESP_ERROR_CHECK(rmt_write_sample((rmt_channel_t)rmtChannel_, data, numLeds * 4, false));
rmt_transmit_config_t transmitConfig = {};
transmitConfig.loop_count = 0;
ESP_ERROR_CHECK(rmt_transmit(channel_, encoder_, rgbwData, numLeds * 4, &transmitConfig));
transmitting_ = true;
}
@@ -96,8 +75,7 @@ bool Esp32DriverRGBW::waitForTransmissionToFinish(int waitMs)
{
if (!transmitting_)
return true;
auto ret = rmt_wait_tx_done((rmt_channel_t)rmtChannel_, waitMs / portTICK_PERIOD_MS);
if (ret == ESP_OK)
if (rmt_tx_wait_all_done(channel_, waitMs) == ESP_OK)
{
transmitting_ = false;
return true;

View File

@@ -4,10 +4,13 @@
#include "containers/LedStripRGBW.h"
#include <driver/rmt_tx.h>
class Esp32DriverRGBW
{
public:
void begin(int gpio, int rmtChannel);
// The RMT channel is picked by the driver; only the pin is ours to choose.
void begin(int gpio);
void end();
void writeSync(const uint32_t *rgbwData, int numLeds);
@@ -16,8 +19,9 @@ public:
bool waitForTransmissionToFinish(int waitMs = 1000);
private:
int rmtChannel_;
bool transmitting_;
rmt_channel_handle_t channel_ = nullptr;
rmt_encoder_handle_t encoder_ = nullptr;
bool transmitting_ = false;
};
#endif

View File

@@ -9,6 +9,7 @@ enum class EffectId
SWIPE_AND_CHANGE, // combination of ALEXA_SWIPE and RANDOM_TWO_COLOR_INTERPOLATION
REVERSE_SWIPE,
STATIC_DETAILED,
SEGMENT_SELECT,
};
template <EffectId id>

View File

@@ -0,0 +1,212 @@
#pragma once
#include "effects/Common.h"
#include "helpers/ColorRGBW.h"
#include "helpers/ColorHSV.h"
#include "helpers/ColorConversions.h"
#include <cmath>
#include <cstdint>
static constexpr int SEGMENT_SELECT_MAX_SEGMENTS = 6;
// The ring as a picker: one section per choice, the selected one bright and wobbling.
//
// The host re-sends this on every rotary detent. Each message builds a fresh effect, so
// `previous` is what makes the highlight glide over from where it was instead of
// jumping; previous == selected means no glide.
#pragma pack(push, 1)
struct EffectSegmentSelectConfig
{
uint8_t count; // number of sections, 1..SEGMENT_SELECT_MAX_SEGMENTS
uint8_t selected; // index of the highlighted section
uint8_t previous; // index the highlight glides from
uint8_t gapLeds; // dark LEDs between neighbouring sections
ColorRGBW colors[SEGMENT_SELECT_MAX_SEGMENTS];
float offsetDegrees; // where section 0 starts
int8_t direction; // +1 / -1: which way round the sections go
float dimFactor; // brightness of the sections that are not selected, 0..1
float wobbleAmplitude; // in LEDs
float wobblePeriodMs;
float slideMs; // time for the highlight to glide from `previous` to `selected`
float edgeBlend; // 0..0.5: how far a section's edges shift towards its neighbours' hue
};
#pragma pack(pop)
template <typename TLedStrip>
class EffectSegmentSelect
{
public:
static constexpr auto NUM_LEDS = numLeds<TLedStrip>();
static constexpr int DELAY_MS = 10;
using ConfigType = EffectSegmentSelectConfig;
EffectSegmentSelect(const EffectSegmentSelectConfig &cfg, TLedStrip &ledStrip)
: config_(cfg),
ledStrip_(ledStrip)
{
if (config_.count > SEGMENT_SELECT_MAX_SEGMENTS)
config_.count = SEGMENT_SELECT_MAX_SEGMENTS;
if (config_.count > 0)
{
config_.selected %= config_.count;
config_.previous %= config_.count;
}
if (config_.direction == 0)
config_.direction = 1;
pitch_ = float(NUM_LEDS) / float(config_.count > 0 ? config_.count : 1);
halfWidth_ = std::max(0.5f, (pitch_ - float(config_.gapLeds)) / 2.0f);
offsetLeds_ = config_.offsetDegrees / 360.0f * float(NUM_LEDS);
// The glide takes the short way round, so going from the last section to the first
// moves one step forward rather than all the way back.
int delta = int(config_.selected) - int(config_.previous);
if (delta > config_.count / 2)
delta -= config_.count;
else if (delta < -(config_.count / 2))
delta += config_.count;
glideFrom_ = float(int(config_.selected) - delta);
// The dimmed background never changes, so it is computed once.
for (int led = 0; led < NUM_LEDS; ++led)
{
const float u = ringPosition(led);
const int section = std::min(int(u / pitch_), int(config_.count) - 1);
ColorRGBW color{0, 0, 0, 0};
if (config_.count > 0)
{
const float center = (float(section) + 0.5f) * pitch_;
const float coverage = clamp01(halfWidth_ + 0.5f - std::fabs(u - center));
color = sectionColor(section, (u - center) / pitch_) * (coverage * config_.dimFactor);
}
background_[led] = color;
}
}
int operator()()
{
if (config_.count == 0)
{
clear(ledStrip_);
return 10000;
}
const float glide = config_.slideMs > 0 ? clamp01(elapsedMs_ / config_.slideMs) : 1.0f;
const float eased = glide * glide * (3.0f - 2.0f * glide); // smoothstep
const float index = glideFrom_ + (float(config_.selected) - glideFrom_) * eased;
float wobble = 0;
if (config_.wobblePeriodMs > 0)
wobble = config_.wobbleAmplitude * std::sin(2.0f * float(M_PI) * elapsedMs_ / config_.wobblePeriodMs);
const float center = (index + 0.5f) * pitch_ + wobble;
for (int led = 0; led < NUM_LEDS; ++led)
{
const float distance = wrappedDistance(ringPosition(led) - center);
const float coverage = clamp01(halfWidth_ + 0.5f - std::fabs(distance));
if (coverage <= 0)
{
setLedRGBW(ledStrip_, led, background_[led]);
continue;
}
// The highlight carries the selected section's own gradient along with it, and
// changes colour from the previous section's as it glides.
const float local = distance / pitch_;
const ColorRGBW from = sectionColor(config_.previous, local);
const ColorRGBW to = sectionColor(config_.selected, local);
const ColorRGBW highlight = ColorRGBW::interpolate(from, to, eased);
setLedRGBW(ledStrip_, led, ColorRGBW::interpolate(background_[led], highlight, coverage));
}
elapsedMs_ += DELAY_MS;
return DELAY_MS;
}
private:
static float clamp01(float x) { return x < 0 ? 0 : (x > 1 ? 1 : x); }
// An LED's position along the sections, 0..NUM_LEDS, section 0 starting at 0.
float ringPosition(int led) const
{
float u = float(config_.direction) * (float(led) + 0.5f - offsetLeds_);
u = std::fmod(u, float(NUM_LEDS));
return u < 0 ? u + float(NUM_LEDS) : u;
}
// Shortest signed distance on the ring.
static float wrappedDistance(float d)
{
const float n = float(NUM_LEDS);
d = std::fmod(d, n);
if (d > n / 2)
d -= n;
else if (d < -n / 2)
d += n;
return d;
}
// Colour of a section at `local`, -0.5 (its start) .. +0.5 (its end): its palette
// colour in the middle, shifting in hue towards each neighbour by up to edgeBlend.
ColorRGBW sectionColor(int section, float local) const
{
const int count = config_.count;
const int neighbour = local < 0 ? (section + count - 1) % count : (section + 1) % count;
const float amount = config_.edgeBlend * clamp01(std::fabs(local) * 2.0f);
return blendHue(config_.colors[section], config_.colors[neighbour], amount);
}
static ColorRGBW blendHue(const ColorRGBW &c1, const ColorRGBW &c2, float f)
{
if (f <= 0)
return c1;
ColorHSV h1 = rgb2hsv(c1);
ColorHSV h2 = rgb2hsv(c2);
// Greys have no hue of their own; borrow the other's so only saturation moves.
if (h1.s <= 0)
h1.h = h2.h;
if (h2.s <= 0)
h2.h = h1.h;
float dh = h2.h - h1.h;
if (dh > 180)
dh -= 360;
else if (dh < -180)
dh += 360;
float hue = h1.h + f * dh;
if (hue < 0)
hue += 360;
else if (hue >= 360)
hue -= 360;
ColorRGBW out = hsv2rgb(ColorHSV{hue, h1.s + f * (h2.s - h1.s), h1.v + f * (h2.v - h1.v)});
out.w = uint8_t(float(c1.w) + f * (float(c2.w) - float(c1.w)));
return out;
}
EffectSegmentSelectConfig config_;
TLedStrip &ledStrip_;
float pitch_ = 0;
float halfWidth_ = 0;
float offsetLeds_ = 0;
float glideFrom_ = 0;
float elapsedMs_ = 0;
ColorRGBW background_[NUM_LEDS];
};
// Traits
template <>
struct EffectIdToConfig<EffectId::SEGMENT_SELECT>
{
using type = EffectSegmentSelectConfig;
};
template <>
struct EffectConfigToId<EffectSegmentSelectConfig>
{
static constexpr auto id = EffectId::SEGMENT_SELECT;
};
template <typename TLedStrip>
struct EffectIdToClass<EffectId::SEGMENT_SELECT, TLedStrip>
{
using type = EffectSegmentSelect<TLedStrip>;
};

View File

@@ -220,12 +220,12 @@ esp_err_t rotary_encoder_init(rotary_encoder_info_t * info, gpio_num_t pin_a, gp
info->state.direction = ROTARY_ENCODER_DIRECTION_NOT_SET;
// configure GPIOs
gpio_pad_select_gpio(info->pin_a);
gpio_reset_pin(info->pin_a);
gpio_set_pull_mode(info->pin_a, GPIO_PULLUP_ONLY);
gpio_set_direction(info->pin_a, GPIO_MODE_INPUT);
gpio_set_intr_type(info->pin_a, GPIO_INTR_ANYEDGE);
gpio_pad_select_gpio(info->pin_b);
gpio_reset_pin(info->pin_b);
gpio_set_pull_mode(info->pin_b, GPIO_PULLUP_ONLY);
gpio_set_direction(info->pin_b, GPIO_MODE_INPUT);
gpio_set_intr_type(info->pin_b, GPIO_INTR_ANYEDGE);

View File

@@ -13,7 +13,9 @@ data_dir = data
default_envs = esp32
[env:esp32]
platform = espressif32
; pioarduino's fork rather than PlatformIO's own espressif32 platform, which stays on
; Arduino core 2.x. Pinned to a release: 55.03.312-1 is Arduino core 3.3.12 on ESP-IDF 5.5.
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.312-1/platform-espressif32.zip
board = esp-wrover-kit
board_upload.flash_size = "4MB"
build_flags = -DPLATFORM_ESP32
@@ -21,14 +23,13 @@ framework = arduino
monitor_port = /dev/ttyUSB0
upload_port = /dev/ttyUSB0
monitor_speed = 115200
src_filter = +<*> -<host_test.cpp>
build_src_filter = +<*> -<host_test.cpp>
lib_deps =
miguelbalboa/MFRC522
bxparks/AceButton@^1.9.1
bxparks/AceButton@1.10.1
[env:native]
platform = native
src_filter = +<*> -<main.cpp>
platform = native@1.2.1
build_src_filter = +<*> -<main.cpp>
build_flags = -Ilib/ledtl -g -DPLATFORM_NATIVE
lib_deps =
bxparks/AceButton@^1.9.1
bxparks/AceButton@1.10.1

View File

@@ -4,6 +4,7 @@
#include "effects/ReverseSwipe.h"
#include "effects/RandomTwoColorInterpolation.h"
#include "effects/SwipeAndChange.h"
#include "effects/SegmentSelect.h"
#include "Arduino.h"
#include <cstdint>
@@ -120,6 +121,8 @@ enum class MessageHostToFw : uint8_t
PREV_BUTTON_LED = 21,
NEXT_BUTTON_LED = 22,
LED_WHEEL_EFFECT_SEGMENT_SELECT = 23,
};
template <>
@@ -180,7 +183,7 @@ inline bool handleLedEffect(TLedTask *ledTask, MessageHostToFw msgType, MessageH
}
template <typename LedTask1, typename LedTask2, typename LedTaskShelf>
inline void handleIncomingMessagesFromHost(LedTask1 *ledTaskCircle, LedTask2 *ledTaskMouse, LedTaskShelf *ledTaskShelf, uint8_t ledChannelLeft, uint8_t ledChannelRight)
inline void handleIncomingMessagesFromHost(LedTask1 *ledTaskCircle, LedTask2 *ledTaskMouse, LedTaskShelf *ledTaskShelf, uint8_t ledPinLeft, uint8_t ledPinRight)
{
if (Serial.available() < sizeof(MAGIC_TOKEN_FW_TO_HOST) + sizeof(MessageHostToFw) + sizeof(uint16_t))
return;
@@ -214,6 +217,7 @@ inline void handleIncomingMessagesFromHost(LedTask1 *ledTaskCircle, LedTask2 *le
else if(handleLedEffect<EffectRandomTwoColorInterpolationConfig>(ledTaskCircle, MessageHostToFw::LED_WHEEL_EFFECT_RANDOM_TWO_COLOR_INTERPOLATION, msgType, msgBuffer)) {}
else if(handleLedEffect<EffectSwipeAndChangeConfig >(ledTaskCircle, MessageHostToFw::LED_WHEEL_EFFECT_SWIPE_AND_CHANGE, msgType, msgBuffer)) {}
else if(handleLedEffect<EffectReverseSwipeConfig >(ledTaskCircle, MessageHostToFw::LED_WHEEL_EFFECT_REVERSE_SWIPE, msgType, msgBuffer)) {}
else if(handleLedEffect<EffectSegmentSelectConfig >(ledTaskCircle, MessageHostToFw::LED_WHEEL_EFFECT_SEGMENT_SELECT, msgType, msgBuffer)) {}
// Mouse LEDs
else if(handleLedEffect<EffectStaticConfig >(ledTaskMouse, MessageHostToFw::MOUSE_LED_EFFECT_STATIC, msgType, msgBuffer)) {}
@@ -234,12 +238,12 @@ inline void handleIncomingMessagesFromHost(LedTask1 *ledTaskCircle, LedTask2 *le
else if (msgType == MessageHostToFw::PREV_BUTTON_LED)
{
float *val = reinterpret_cast<float *>(msgBuffer);
ledcWrite(ledChannelLeft, uint32_t(255 * (*val)));
ledcWrite(ledPinLeft, uint32_t(255 * (*val)));
}
else if (msgType == MessageHostToFw::NEXT_BUTTON_LED)
{
float *val = reinterpret_cast<float *>(msgBuffer);
ledcWrite(ledChannelRight, uint32_t(255 * (*val)));
ledcWrite(ledPinRight, uint32_t(255 * (*val)));
}
else
Serial.println("Unknown message type");

View File

@@ -2,6 +2,7 @@
#include "effects/Circular.h"
#include "effects/Common.h"
#include "effects/SegmentSelect.h"
#include "drivers/Esp32DriverRGBW.h"
#include "freertos/FreeRTOS.h"
@@ -82,6 +83,7 @@ void _led_task_func(void *params)
else if (dispatchEffectId<EffectId::RANDOM_TWO_COLOR_INTERPOLATION>(id, effectFunction, ledStrip, msgBuffer, effectStorage)) {}
else if (dispatchEffectId<EffectId::SWIPE_AND_CHANGE >(id, effectFunction, ledStrip, msgBuffer, effectStorage)) {}
else if (dispatchEffectId<EffectId::REVERSE_SWIPE >(id, effectFunction, ledStrip, msgBuffer, effectStorage)) {}
else if (dispatchEffectId<EffectId::SEGMENT_SELECT >(id, effectFunction, ledStrip, msgBuffer, effectStorage)) {}
// clang-format on
timeoutMsForEffect = 0;

View File

@@ -3,6 +3,7 @@
#include "effects/AlexaSwipe.h"
#include "effects/ReverseSwipe.h"
#include "effects/SegmentSelect.h"
#include "helpers/ColorConversions.h"
#include <iostream>
@@ -100,6 +101,29 @@ int main(int argc, char **argv)
effectToFile("reverse_swipe.py", effect, strip, 200);
}
{
EffectSegmentSelectConfig cfg{};
cfg.count = 4;
cfg.selected = 0;
cfg.previous = 3;
cfg.gapLeds = 1;
cfg.colors[0] = ColorRGBW{255, 0, 0, 0};
cfg.colors[1] = ColorRGBW{255, 160, 0, 0};
cfg.colors[2] = ColorRGBW{0, 200, 0, 0};
cfg.colors[3] = ColorRGBW{0, 60, 255, 0};
cfg.offsetDegrees = 180;
cfg.direction = 1;
cfg.dimFactor = 0.15f;
cfg.wobbleAmplitude = 1.5f;
cfg.wobblePeriodMs = 1000;
cfg.slideMs = 200;
cfg.edgeBlend = 0.25f;
LedStripRGBW<51> strip;
EffectSegmentSelect<decltype(strip)> effect(cfg, strip);
effectToFile("segment_select.py", effect, strip, 200);
}
/*
effect.currentPosition_ = 150;

View File

@@ -1,7 +1,6 @@
#include <Arduino.h>
#include "rc522.h"
#include "SPI.h"
#include <MFRC522.h>
#include "rotary_encoder.h"
#include "containers/LedStripRGBW.h"
@@ -21,9 +20,6 @@
#include "TaskLed.h"
// -------------------------------------------------- RFID Reader ----------------------------------------
MFRC522 rfid;
MFRC522::MIFARE_Key key;
void tagHandler(uint8_t *sn)
{
if (sn != nullptr)
@@ -117,11 +113,9 @@ void setupButtons()
buttonRight.setEventHandler(handleButtonEvent);
buttonRotary.setEventHandler(handleButtonEvent);
ledcSetup(0, PWM_FREQ, PWM_RESOLUTION);
ledcAttachPin(BUTTON_LEFT_LED_PIN, 0);
ledcSetup(1, PWM_FREQ, PWM_RESOLUTION);
ledcAttachPin(BUTTON_RIGHT_LED_PIN, 1);
// Arduino core 3 picks the LEDC channel itself; from here on the pin is the handle.
ledcAttach(BUTTON_LEFT_LED_PIN, PWM_FREQ, PWM_RESOLUTION);
ledcAttach(BUTTON_RIGHT_LED_PIN, PWM_FREQ, PWM_RESOLUTION);
}
void handleButtons()
@@ -139,7 +133,7 @@ LedTask<decltype(ledStripCircle)> ledTaskCircle;
void setupLedCircle()
{
ledDriverCircle.begin(22, 0);
ledDriverCircle.begin(22);
ledTaskCircle.begin(ledStripCircle, ledDriverCircle);
ledTaskCircle.startEffect(EffectStaticConfig(ColorRGBW{0, 0, 0, 0}));
}
@@ -152,7 +146,7 @@ LedTask<decltype(ledStripMouse)> ledTaskMouse;
void setupMouseLeds()
{
ledDriverMouse.begin(16, 1);
ledDriverMouse.begin(16);
ledTaskMouse.begin(ledStripMouse, ledDriverMouse);
ledTaskMouse.startEffect(EffectStaticConfig{ColorRGBW{0, 0, 0, 0}, 0, 0});
}
@@ -165,7 +159,7 @@ LedTask<decltype(ledStripShelf)> ledTaskShelf;
void setupShelfLeds()
{
ledDriverShelf.begin(17, 2);
ledDriverShelf.begin(17);
ledTaskShelf.begin(ledStripShelf, ledDriverShelf);
ledTaskShelf.startEffect(EffectStaticConfig{ColorRGBW{0, 0, 0, 0}, 0, 0});
}
@@ -234,7 +228,7 @@ void setup()
void loop()
{
handleIncomingMessagesFromHost(&ledTaskCircle, &ledTaskMouse, &ledTaskShelf, 0, 1);
handleIncomingMessagesFromHost(&ledTaskCircle, &ledTaskMouse, &ledTaskShelf, BUTTON_LEFT_LED_PIN, BUTTON_RIGHT_LED_PIN);
handleTouchInputs();
handleRotaryEncoder();
handleButtons();

View File

@@ -1,64 +0,0 @@
#include <SPI.h>
#include <MFRC522.h>
#define RST_PIN 5 // Configurable, see typical pin layout above
#define SS_PIN 23 // Configurable, see typical pin layout above
MFRC522 mfrc522(SS_PIN, RST_PIN); // Create MFRC522 instance
//*****************************************************************************************//
void setup() {
Serial.begin(115200);
SPI.begin(22, 19, 18, 23); // Init SPI bus
mfrc522.PCD_Init(); // Init MFRC522 card
}
/**
* Helper routine to dump a byte array as hex values to Serial.
*/
void PrintHex(byte *buffer, byte bufferSize) {
for (byte i = 0; i < bufferSize; i++) {
Serial.print(buffer[i] < 0x10 ? " 0" : " ");
Serial.print(buffer[i], HEX);
}
}
//*****************************************************************************************//
void loop() {
// Look for new cards
if ( !mfrc522.PICC_IsNewCardPresent()) {
return;
}
if ( !mfrc522.PICC_ReadCardSerial()) {
return;
}
//mfrc522.PICC_DumpDetailsToSerial(&(mfrc522.uid)); //dump some details about the card
Serial.print("NewCard");
PrintHex(mfrc522.uid.uidByte, mfrc522.uid.size);
Serial.println("");
// Check if Card was removed
bool cardRemoved = false;
int counter = 0;
bool current, previous;
previous = !mfrc522.PICC_IsNewCardPresent();
while(!cardRemoved){
current =!mfrc522.PICC_IsNewCardPresent();
if (current && previous) counter++;
previous = current;
cardRemoved = (counter>2);
delay(50);
}
Serial.println("Card was removed");
delay(500); //change value if you want to read cards faster
mfrc522.PICC_HaltA();
mfrc522.PCD_StopCrypto1();
}

View File

@@ -94,7 +94,10 @@ the prompt becomes a regression test by pasting it into a `.txt` file.
| `musicmouse/reactions/` | The policy. `@on(SomeEvent)` functions that get the app and act. |
| `musicmouse/library/` | The music collection: scanning, tags, cover art and its colours, the cache, and the seams for future track analysis. |
| `musicmouse/services/mqtt/` | Home Assistant entities: three lights, a player sensor, a volume number, transport buttons, device triggers, a tag scanner. |
| `musicmouse/services/web/` | The browser front-end's API: the library, a state websocket, command endpoints, and parent-mode settings. |
| `musicmouse/services/web/` | The browser front-end's API: the library, a state websocket, command endpoints, parent-mode settings, the typing game and the Home Assistant proxy. |
| `musicmouse/services/lirc/` | The IR remote, as a lircd TCP client. |
| `musicmouse/services/` (`screen.py`, `timers.py`, `podcasts.py`) | The monitor's on/off and idle auto-off; the parents' timer and school-day calendar; the podcast feed poller. |
| `musicmouse/tippen/` | The typing game's curriculum, progress and track unlocks. |
| `musicmouse/simulator/` | Fake transport and player, the driver vocabulary, the REPL and the script runner. |
| `musicmouse/config.py` | Pydantic schema, validation, and human-readable error formatting. |
@@ -106,6 +109,8 @@ ruff check .
mypy # --strict, configured in pyproject.toml
```
`../scripts/check.sh` runs these three plus the web front-end's checks in one go.
`tests/test_wire.py` parses `../esp-firmware/src/Messages.h` and fails if the Python
message ids drift from the firmware's - the contract is hand-duplicated in two
languages, and it had already drifted once (`BUTTON_EVENT` was missing on the Python
@@ -133,7 +138,9 @@ tag scanner. Topics are under `musicmouse/trigger/…` and `musicmouse/tag`.
Two behaviours used to be hard-coded in `main.py` and now need automations:
**Rotary press toggled the room light.** Trigger on the `rotary_pressed` device
trigger, action `light.toggle` on `light.kinderzimmer_fluter`.
trigger, action `light.toggle` on `light.kinderzimmer_fluter`. While the LED ring's album
picker is open (a figure with several albums was just put down) the knob is the picker's:
its turns do not change the volume and its press is not published.
**Touching a body part set a colour** on `light.kinderzimmer_fluter` and
`light.music_mouse_regal_licht`:
@@ -156,7 +163,7 @@ the scanner has to know about:
| Folder | Shown as | Why it is special |
|---|---|---|
| `Figuren/<figure>/` | an album per figurine | the folder name *is* the figure name from the config |
| `Figuren/<figure>/` | an album per figurine - or, for one whose folder holds album folders instead of audio, up to six, picked on the LED ring | the folder name *is* the figure name from the config; a sub-album is titled `Fuchs - <folder>` |
| `Musik/<Artist> - <Album>/` | music, grouped by artist | plain ID3 |
| `Hörbücher/<Artist> - <Album>/` | audiobooks, grouped by character | `album_artist` is a credit list; only the name before the first comma groups usefully |
| `Kinderpodcasts/<Show>/` | audiobooks, newest episode first | the tags are useless here - `artist` is the presenter list and `album` is the feed name, so the *folder* is the show |
@@ -182,8 +189,9 @@ different amounts to produce:
```
index.json cheap: tags and structure. Rebuilt freely.
covers/<album_id>.jpg medium: art extracted from an ID3 APIC frame
analysis/<track_key>.json expensive: reserved for offline audio analysis
covers/<album_id>.jpg medium: the album's art, downscaled to 384 px
covers/<album_id>.thumb.jpg medium: the same at 256 px, for cards and rows
analysis/<track_key>.json expensive: track analysis (plus .beats.json, .curve.json)
```
Deleting the whole directory is safe; deleting it throws away analysis that is minutes
@@ -195,13 +203,16 @@ change to how the scanner derives a title, artist or series is invisible until t
is invalidated: bump `_INDEX_VERSION` in `musicmouse/library/cache.py` when you touch
that logic.
**Track analysis is not implemented.** `musicmouse/library/analysis.py` fixes the shape
of the results - scalars (`tempo`, `energy`, `valence`, `brightness`) travel inline with
the index, and a beat grid lives in its own file and is fetched per track - so an
analyzer can be added later without touching the scanner, the API or the frontend. It
will go behind an optional dependency group, and because results are content-keyed
files, they can equally well be computed on a workstation and the `analysis/` folder
copied to the device.
**Track analysis** drives the web front-end's ambient background. It is optional: install
the `analysis` extra (`pip install -e '.[analysis]'`, librosa and numpy) and
`musicmouse/library/librosa_analyzer.py` works through the music shelf in the
background, in a pool of niced worker processes (`musicmouse/library/workers.py`), and
pauses while anything is playing. Without the extra, `build_analyzer()` returns a no-op
and the background stays static. Scalars (`tempo`, `energy`, `valence`, `brightness`,
`pulse`) travel inline with the index; the beat grid and per-second curves live in their
own files and are fetched only for the playing track. Results are content-keyed, so they
can be computed on a workstation and the `analysis/` folder copied to the device
(`scripts/sync-library.sh` does that).
## The web front-end
@@ -216,12 +227,15 @@ copied to the device.
| `WS /api/ws` | Push only. A snapshot on connect, then a frame per change, plus the position at 2 Hz while playing, and `clock_toggle` when the remote's Guide key asks for the clock. |
| `POST /api/play` | `{album_id, track_index?}` |
| `POST /api/resume` `/pause` `/next` `/previous` `/seek` | transport |
| `POST /api/volume` | `{percent}`, `{delta_percent}`, or `{steps}` - the IR remote's step (`volume_increment` device units) per step |
| `POST /api/volume` | `{percent}` or `{steps}` - the IR remote's step (`volume_increment` device units) per step |
| `POST /api/remote/press/{digit}` | what the IR remote's number key does - the keyboard's digits |
| `GET` `PUT /api/remote/mapping` | number key -> album (optionally from one `track_index`) or podcast show |
| `POST /api/screen/toggle` `/screen/activity` | the kiosk's Del key; its key presses and taps, holding off the idle screen-off |
| `GET` `PUT /api/settings` | parent mode |
| `POST` `DELETE /api/timer` | parent mode's countdown: `{minutes}` or `{until: "HH:MM"}`; stop or silence it |
| `GET /api/tracks/{album_id}/{index}/analysis` | beat grid and mood curves of one track, or 404 |
| `GET /api/ha`, `GET /api/ha/states/{entity}`, `POST /api/ha/services/{domain}/{service}` | the room page's Home Assistant proxy; 404 without an `ha` section |
| `GET /api/tippen/curriculum` `/progress`, `PUT /api/tippen/settings`, `POST /api/tippen/runs` | the typing game; 404 without a `tippen` section |
Two decisions worth knowing:

View File

@@ -10,7 +10,8 @@ general:
# The music collection. One path; the shelves underneath it are fixed names, not
# settings, because each one has its own quirks the code already knows about:
#
# <root>/Figuren/<figure name>/ one folder per figurine
# <root>/Figuren/<figure name>/ one folder per figurine - or, for a figure
# with several albums, one subfolder each
# <root>/Musik/<Artist> - <Album>/ albums, grouped by artist
# <root>/Hörbücher/<Artist> - <Album>/ audiobooks, grouped by character
# <root>/Kinderpodcasts/<Show>/ shows, newest episode first
@@ -57,6 +58,27 @@ general:
# Backlight of the prev/next buttons while a figure is playing, 0..1.
button_leds_brightness: 0.5
# Loudness of the UI sounds - the screen's on/off blubb, the parents' timer, the
# album picker's clicks - in percent of how loud they are made to sit under the
# music. They follow the music's volume, but never drop below what they are at 30 %
# of it. Editable in parent mode.
effects_volume: 100
# The LED ring's album picker, for a figure with several albums (see `figures:`).
# Temporary: for tuning on the device. Every value defaults to a constant in
# musicmouse/reactions/selection.py, which is where the right ones belong once found.
# figure_selection:
# timeout: 8.0 # seconds without a turn before the picker closes
# offset_degrees: 180 # where the first album's section starts on the ring
# ring_direction: 1 # 1 or -1: which way round the sections go
# knob_direction: 1 # 1 or -1: which way the knob turns to the next album
# gap_leds: 1 # dark LEDs between sections
# dim_factor: 0.15 # brightness of the sections not selected, 0..1
# wobble_amplitude: 1.5 # how far the selected section swings, in LEDs
# wobble_period_ms: 1000 # one swing back and forth
# slide_ms: 200 # glide from one section to the next
# edge_blend: 0.25 # 0..0.5: how far section edges shift to their neighbours' hue
# Which files count as music. Anything else - a podcast downloader's archive.json,
# a half-finished .tmp - is ignored.
audio_extensions: [".mp3", ".ogg", ".oga", ".opus", ".flac", ".wav", ".m4a", ".aac"]
@@ -175,6 +197,12 @@ general:
progress_file: tippen-progress.json
# One entry per figurine. The key is the figure name and the subfolder name.
#
# A figure folder that holds audio itself plays that. One that holds album folders
# instead can play any of them, up to six: putting the figure down plays the one it
# played last (or the first, alphabetically), and the LED ring shows one coloured
# section per album for a few seconds. Turn the knob to move between them and press it
# to play the selected one; leave it alone and the music just carries on.
figures:
fuchs:
# RFID tag id, 5 bytes as hex. Must be unique across figures.
@@ -185,7 +213,8 @@ figures:
# "music" (default) or "book". Every other shelf is named after what is on it, so
# its type is obvious; a figure folder is named after the figurine, so this is the
# one thing that has to be said out loud. The web UI draws albums square and
# audiobooks taller than wide, so getting it wrong is visible at a glance.
# audiobooks taller than wide, so getting it wrong is visible at a glance. It holds
# for every album of a figure that has several.
kind: music
eule:

View File

@@ -42,6 +42,7 @@ from musicmouse.services.podcasts import PodcastFeedService
from musicmouse.services.screen import ScreenService
from musicmouse.services.timers import TimerService
from musicmouse.services.web import WebService
from musicmouse.sound import ui_level
from musicmouse.tippen.curriculum import CurriculumError
from musicmouse.tippen.runtime import TippenRuntime, build_tippen_runtime
@@ -315,9 +316,6 @@ def _build_app(
mouse=mouse,
player=player,
library=library,
# One source of truth: the figure path and the web path must hand the player the
# same Playlist object for the same folder, because `play_figure` resumes on an
# identity check.
playlists=library.figure_playlists(),
clock=clock,
tippen=tippen,
@@ -381,7 +379,7 @@ def _build_services(
app.bus,
screen_config,
alsa_device=None if general.audio_simulated else general.alsa_device,
volume=lambda: app.player.volume / 100,
volume=lambda: ui_level(app.player.volume / 100, general.effects_volume / 100),
is_playing=lambda: app.player.is_playing,
clock=clock,
)
@@ -398,7 +396,7 @@ def _build_services(
cache_file=general.library.cache / "holidays.json",
client=httpx2.AsyncClient(timeout=30.0),
alsa_device=None if general.audio_simulated else general.alsa_device,
volume=lambda: app.player.volume / 100,
volume=lambda: ui_level(app.player.volume / 100, general.effects_volume / 100),
clock=clock,
)
)

View File

@@ -2,7 +2,9 @@
from __future__ import annotations
import asyncio
import logging
import math
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
@@ -18,7 +20,7 @@ from musicmouse.tippen.runtime import TippenRuntime
_log = logging.getLogger(__name__)
__all__ = ["App", "AppState", "ParentTimer"]
__all__ = ["App", "AppState", "FigureSelector", "ParentTimer"]
@dataclass(frozen=True, slots=True)
@@ -32,6 +34,22 @@ class ParentTimer:
ringing: bool = False
@dataclass
class FigureSelector:
"""The LED ring offering a figure's albums to pick from. Owned by
:mod:`musicmouse.reactions.selection`."""
figure: str
choices: tuple[Playlist, ...]
#: The highlighted choice - not necessarily the one playing until it is confirmed.
selected: int
#: Clock time at which the ring gives up waiting; pushed back by every turn.
deadline: float
#: Clock time of the last tick sound, so turning fast does not stack them up.
last_tick: float = -math.inf
expiry: asyncio.Task[None] | None = None
@dataclass
class AppState:
"""State that belongs to no single device but is shared between reactions."""
@@ -40,6 +58,14 @@ class AppState:
#: instead of starting over. Cleared once its playlist runs out.
last_partially_played_figure: str | None = None
#: Figure -> album id it last played, for figures with more than one album: putting
#: it back picks that one again. Kept when another figure is placed in between -
#: only the position within the album is forgotten then.
figure_albums: dict[str, str] = field(default_factory=dict)
#: The ring's album picker, while it is open.
selector: FigureSelector | None = None
#: Whether the MQTT broker is currently reachable. The firmware's equivalent is
#: readable off the transport; a broker's is not, so it is remembered here.
mqtt_connected: bool = False
@@ -63,7 +89,8 @@ class App:
mouse: MusicMouseDevice
player: Player
library: MusicLibrary
playlists: dict[str, Playlist]
#: Figure -> the albums it can play, in the order the ring offers them.
playlists: dict[str, tuple[Playlist, ...]]
clock: Clock = field(default_factory=RealClock)
state: AppState = field(default_factory=AppState)
#: `None` when `general.tippen` is absent - the typing game is off.
@@ -72,11 +99,21 @@ class App:
def colors(self, figure: str) -> FigureColors:
return self.config.figures[figure].colors
def playlist(self, figure: str) -> Playlist | None:
playlist = self.playlists.get(figure)
if playlist is None:
def choices(self, figure: str) -> tuple[Playlist, ...]:
"""The albums ``figure`` can play; empty, with a warning, if it has none."""
choices = self.playlists.get(figure, ())
if not choices:
_log.warning("No playlist for figure %r", figure)
return playlist
return choices
def default_choice(self, figure: str) -> int:
"""Which of the figure's albums plays when it is put down: the one it played
last, or else the first."""
remembered = self.state.figure_albums.get(figure)
for index, playlist in enumerate(self.playlists.get(figure, ())):
if playlist.album_id == remembered:
return index
return 0
def album_for(self, playlist: Playlist | None) -> Album | None:
"""The library album a playlist came from, if any."""

View File

@@ -9,17 +9,21 @@ it reaches the surface). Two of them make the "blubb"; switching on the second o
higher, switching off it is lower, so the two directions are told apart by ear.
The parents' timer has its own sound here too: a soft "ding-dong" bell
(:func:`chime_wav`), made the same way.
(:func:`chime_wav`), made the same way. So do the LED ring's album picker: a short
tick for every step (:func:`tick_wav`) and a rising two-note "pling" when an album is
picked (:func:`confirm_wav`).
"""
from __future__ import annotations
import functools
import io
import math
import wave
from array import array
from collections.abc import Sequence
__all__ = ["RATE", "blubb_wav", "chime_wav"]
__all__ = ["RATE", "blubb_wav", "chime_wav", "confirm_wav", "tick_wav"]
RATE = 44100
@@ -60,7 +64,58 @@ def chime_wav(*, gain: float) -> bytes:
return _wav(samples, scale)
def _wav(samples: list[float], scale: float) -> bytes:
#: (Hz, start s) of the tick's partials: a high, woody click rather than a beep.
_TICK = ((2100.0, 0.0), (3350.0, 0.0))
_TICK_LENGTH = 0.03
_TICK_DECAY = 160.0
#: (Hz, start s) of the confirm sound: a rising fifth, quick, "pling-pling".
_CONFIRM = ((880.0, 0.0), (1318.5, 0.09))
_CONFIRM_LENGTH = 0.22
_CONFIRM_DECAY = 18.0
def tick_wav(*, gain: float) -> bytes:
"""A click of a few tens of milliseconds, for every step of the album picker. Same
format as :func:`blubb_wav`."""
return _scaled(_pings(_TICK, _TICK_LENGTH, _TICK_DECAY), gain)
def confirm_wav(*, gain: float) -> bytes:
"""Two short rising notes for an album picked on the ring. Same format as
:func:`blubb_wav`."""
return _scaled(_pings(_CONFIRM, _CONFIRM_LENGTH, _CONFIRM_DECAY), gain)
@functools.cache
def _pings(
notes: tuple[tuple[float, float], ...], length: float, decay: float
) -> tuple[float, ...]:
total = max(start for _, start in notes) + length
samples = [0.0] * (int(total * RATE) + 1)
attack = int(0.002 * RATE)
count = int(length * RATE)
for frequency, start in notes:
offset = int(start * RATE)
for i in range(count):
t = i / RATE
index = offset + i
if index < len(samples):
samples[index] += (
math.sin(2 * math.pi * frequency * t)
* math.exp(-decay * t)
* min(1.0, i / attack)
)
# Cached: a tick plays on every step of the knob, from inside the event loop.
return tuple(samples)
def _scaled(samples: tuple[float, ...], gain: float) -> bytes:
peak = max(abs(s) for s in samples) or 1.0
return _wav(samples, 32767 * max(0.0, min(1.0, gain)) / peak)
def _wav(samples: Sequence[float], scale: float) -> bytes:
pcm = array("h", (round(s * scale) for s in samples))
buffer = io.BytesIO()
with wave.open(buffer, "wb") as out:

View File

@@ -38,6 +38,7 @@ __all__ = [
"Digit",
"FigureColors",
"FigureConfig",
"FigureSelectionConfig",
"GeneralConfig",
"HaConfig",
"HaDeviceConfig",
@@ -204,7 +205,7 @@ class ClockConfig(_Strict):
class LibraryConfig(_Strict):
"""Where the music lives.
One path. The shelves underneath it - ``Figuren``, ``Musik``, ``Hoerbuecher``,
One path. The shelves underneath it - ``Figuren``, ``Musik``, ``Hörbücher``,
``Kinderpodcasts`` - are fixed names, not settings; see
:mod:`musicmouse.library.sections`.
"""
@@ -258,10 +259,10 @@ class HaDeviceConfig(_Strict):
class HaConfig(_Strict):
"""Home Assistant integration for the room-control page ("Mein Zimmer").
The backend never calls Home Assistant itself - it only hands the browser the
server URL, the token, and these two ordered lists. Control happens directly from
the browser to Home Assistant's own REST API, so this token grants full HA control
to anything on the LAN that can reach musicmouse. See config.yml.example.
The browser only gets the two ordered lists. Every state read and service call goes
through the backend's ``/api/ha/...`` proxy, which attaches the token - so the token
never leaves the backend. The proxy itself is unauthenticated, though: anything on
the LAN that can reach musicmouse can use it. See config.yml.example.
"""
url: str
@@ -301,6 +302,36 @@ class TippenConfig(_Strict):
return _resolve_folder(path, info, must_exist=False, kind="file")
class FigureSelectionConfig(_Strict):
"""Overrides for the LED ring's album picker, for tuning it on the real device.
Temporary by design: every value defaults to a constant in
:mod:`musicmouse.reactions.selection`, which is where the ones that look right
belong once they have been found. ``None`` means "use that constant".
"""
#: Seconds without a turn before the picker closes by itself.
timeout: float | None = Field(default=None, gt=0)
#: Where on the ring the first album's section starts, in degrees from LED 0.
offset_degrees: float | None = None
#: 1 or -1: which way round the ring the sections follow each other.
ring_direction: Literal[1, -1] | None = None
#: 1 or -1: which way the knob has to turn to move to the next album.
knob_direction: Literal[1, -1] | None = None
#: Dark LEDs between neighbouring sections.
gap_leds: int | None = Field(default=None, ge=0, le=10)
#: Brightness of the sections that are not selected, 0..1.
dim_factor: float | None = Field(default=None, ge=0, le=1)
#: How far the selected section swings back and forth, in LEDs.
wobble_amplitude: float | None = Field(default=None, ge=0)
#: How long one swing back and forth takes.
wobble_period_ms: float | None = Field(default=None, gt=0)
#: How long the highlight takes to glide to the next section.
slide_ms: float | None = Field(default=None, ge=0)
#: How far a section's edges shift towards its neighbours' colour, 0..0.5.
edge_blend: float | None = Field(default=None, ge=0, le=0.5)
class GeneralConfig(_Strict):
library: LibraryConfig
@@ -327,6 +358,11 @@ class GeneralConfig(_Strict):
initial_volume: int = Field(default=50, ge=0, le=200)
volume_increment: int = Field(default=5, ge=1, le=100)
button_leds_brightness: float = Field(default=0.5, ge=0, le=1)
#: Loudness of the UI sounds - the screen's blubb, the parents' timer, the album
#: picker's clicks - relative to the music, 0..100 %. Editable in parent mode.
effects_volume: int = Field(default=100, ge=0, le=100)
figure_selection: FigureSelectionConfig = Field(default_factory=FigureSelectionConfig)
audio_extensions: tuple[str, ...] = DEFAULT_AUDIO_EXTENSIONS

View File

@@ -27,15 +27,17 @@ from musicmouse.devices.wire import (
from musicmouse.effects import OFF, LedEffect
from musicmouse.events import (
ActiveFigureChanged,
ButtonEvent,
ConnectionChanged,
DeviceConnected,
DeviceDisconnected,
EventSource,
InputEvent,
LedEffectChanged,
RfidTokenRead,
RotaryTurned,
SelectorPressed,
SelectorTurned,
)
from musicmouse.hardware import NO_FIGURE_TAG, Button, LedZone
from musicmouse.hardware import NO_FIGURE_TAG, Button, ButtonAction, LedZone, RotaryDirection
_log = logging.getLogger(__name__)
@@ -62,6 +64,7 @@ class MusicMouseDevice:
self._active_figure: str | None = None
self._button_brightness: float = 0.0
self._effects: dict[LedZone, LedEffect] = {}
self._knob_captured = False
# -------------------------------------------------------------------- state
@@ -81,6 +84,10 @@ class MusicMouseDevice:
def effect(self, zone: LedZone) -> LedEffect | None:
return self._effects.get(zone)
@property
def knob_captured(self) -> bool:
return self._knob_captured
# ------------------------------------------------------------------ actions
def set_effect(
@@ -107,6 +114,17 @@ class MusicMouseDevice:
self._transport.write(encode_button_brightness(button, brightness))
_log.debug("Button backlights -> %.2f (%s)", brightness, origin)
def capture_knob(self, captured: bool) -> None:
"""Hand the rotary encoder to the ring's album picker, or give it back.
While captured, turns and presses come out as :class:`SelectorTurned` and
:class:`SelectorPressed` instead of :class:`RotaryTurned` and
:class:`ButtonEvent`, so the volume and Home Assistant never see them. Done here,
as the frames are decoded, rather than by each listener checking a flag: the
press that closes the picker would otherwise race whoever handles it after.
"""
self._knob_captured = captured
def all_leds_off(self, *, origin: EventSource = "system") -> None:
for zone in LedZone:
self.set_effect(zone, OFF(), origin=origin)
@@ -116,7 +134,6 @@ class MusicMouseDevice:
def on_connected(self) -> None:
"""Re-apply memorized state, so a reconnect is invisible from the outside."""
self._bus.emit(DeviceConnected(port=self.port, source="device"))
self._bus.emit(ConnectionChanged(target="firmware", connected=True, source="device"))
for zone, effect in self._effects.items():
@@ -126,8 +143,8 @@ class MusicMouseDevice:
if self._effects:
_log.info("Restored %d LED zone(s) after reconnect", len(self._effects))
def on_disconnected(self, reason: str = "") -> None:
self._bus.emit(DeviceDisconnected(port=self.port, reason=reason or None, source="device"))
def on_disconnected(self, _reason: str = "") -> None:
# SerialLink already logs the reason; the event only says the link is down.
self._bus.emit(ConnectionChanged(target="firmware", connected=False, source="device"))
def feed(self, data: bytes) -> None:
@@ -153,6 +170,20 @@ class MusicMouseDevice:
def _publish(self, event: InputEvent) -> None:
if isinstance(event, RfidTokenRead):
self._publish_tag_read(event)
elif self._knob_captured and isinstance(event, RotaryTurned):
if event.direction is not RotaryDirection.NONE:
# The first turn after boot reports no increment, but it was a detent.
steps = abs(event.increment) or 1
if event.direction is RotaryDirection.DOWN:
steps = -steps
self._bus.emit(SelectorTurned(steps=steps, source="device"))
elif (
self._knob_captured
and isinstance(event, ButtonEvent)
and event.button is Button.ROTARY
):
if event.action is ButtonAction.PRESSED:
self._bus.emit(SelectorPressed(source="device"))
else:
self._bus.emit(event)

View File

@@ -13,7 +13,7 @@ newlines and on the magic token.
from __future__ import annotations
import struct
from dataclasses import dataclass, replace
from dataclasses import dataclass
from enum import IntEnum
from typing import TypeAlias
@@ -22,6 +22,7 @@ from musicmouse.effects import (
EffectCircularConfig,
EffectRandomTwoColorInterpolationConfig,
EffectReverseSwipe,
EffectSegmentSelect,
EffectStaticConfig,
EffectStaticDetailedConfig,
EffectSwipeAndChange,
@@ -114,9 +115,11 @@ class MessageHostToFw(IntEnum):
PREV_BUTTON_LED = 21
NEXT_BUTTON_LED = 22
LED_WHEEL_EFFECT_SEGMENT_SELECT = 23
#: Which message id carries which effect, per zone. Note the asymmetry: only the ring
#: accepts AlexaSwipe on its own, and only the shelf accepts StaticDetailed.
#: accepts AlexaSwipe and SegmentSelect, and only the shelf accepts StaticDetailed.
_EFFECT_IDS: dict[LedZone, dict[type[LedEffect], MessageHostToFw]] = {
LedZone.RING: {
EffectStaticConfig: MessageHostToFw.LED_WHEEL_EFFECT_STATIC,
@@ -127,6 +130,7 @@ _EFFECT_IDS: dict[LedZone, dict[type[LedEffect], MessageHostToFw]] = {
),
EffectSwipeAndChange: MessageHostToFw.LED_WHEEL_EFFECT_SWIPE_AND_CHANGE,
EffectReverseSwipe: MessageHostToFw.LED_WHEEL_EFFECT_REVERSE_SWIPE,
EffectSegmentSelect: MessageHostToFw.LED_WHEEL_EFFECT_SEGMENT_SELECT,
},
LedZone.MOUSE: {
EffectStaticConfig: MessageHostToFw.MOUSE_LED_EFFECT_STATIC,
@@ -341,10 +345,6 @@ class FrameDecoder:
return None
def with_figure(event: RfidTokenRead, figure: str | None, *, known: bool) -> RfidTokenRead:
"""Attach a resolved figure name to a decoded tag read."""
return replace(event, figure=figure, known=known)
# ------------------------------------------------- host -> firmware, read back
#

View File

@@ -22,10 +22,12 @@ from musicmouse.color import ColorHSV, ColorRGBW
__all__ = [
"OFF",
"SEGMENT_SELECT_MAX_SEGMENTS",
"EffectAlexaSwipeConfig",
"EffectCircularConfig",
"EffectRandomTwoColorInterpolationConfig",
"EffectReverseSwipe",
"EffectSegmentSelect",
"EffectStaticConfig",
"EffectStaticDetailedConfig",
"EffectSwipeAndChange",
@@ -41,6 +43,12 @@ _ALEXA_SWIPE = struct.Struct("<fffff?")
_TWO_COLOR = struct.Struct("<i?i??")
_CIRCULAR = struct.Struct("<ff")
_REVERSE_SWIPE = struct.Struct("<fff")
_SEGMENT_SELECT_HEAD = struct.Struct("<BBBB")
_SEGMENT_SELECT_TAIL = struct.Struct("<fbfffff")
#: ``SEGMENT_SELECT_MAX_SEGMENTS`` in ``SegmentSelect.h``: the colour array is this long
#: on the wire whatever ``count`` is.
SEGMENT_SELECT_MAX_SEGMENTS = 6
class LedEffect(Protocol):
@@ -274,6 +282,86 @@ class EffectReverseSwipe:
)
@dataclass
class EffectSegmentSelect:
"""The ring as a picker: one section per choice, the selected one bright.
Re-sent on every rotary detent. ``previous`` is where the highlight glides from, so
the fresh effect the firmware builds per message still moves smoothly.
"""
colors: tuple[ColorRGBW, ...]
selected: int = 0
previous: int | None = None # None: same as selected, no glide
gap_leds: int = 1
offset_degrees: float = 180
direction: int = 1
dim_factor: float = 0.15
wobble_amplitude: float = 1.5 # LEDs
wobble_period_ms: float = 1000
slide_ms: float = 200
edge_blend: float = 0.25
SIZE: ClassVar[int] = (
_SEGMENT_SELECT_HEAD.size
+ SEGMENT_SELECT_MAX_SEGMENTS * _RGBW_SIZE
+ _SEGMENT_SELECT_TAIL.size
)
def __post_init__(self) -> None:
if not 1 <= len(self.colors) <= SEGMENT_SELECT_MAX_SEGMENTS:
raise ValueError(
f"between 1 and {SEGMENT_SELECT_MAX_SEGMENTS} sections, got {len(self.colors)}"
)
def as_bytes(self) -> bytes:
count = len(self.colors)
previous = self.selected if self.previous is None else self.previous
padding = (ColorRGBW(0, 0, 0, 0),) * (SEGMENT_SELECT_MAX_SEGMENTS - count)
return (
_SEGMENT_SELECT_HEAD.pack(count, self.selected, previous, self.gap_leds)
+ b"".join(color.as_bytes() for color in (*self.colors, *padding))
+ _SEGMENT_SELECT_TAIL.pack(
self.offset_degrees,
self.direction,
self.dim_factor,
self.wobble_amplitude,
self.wobble_period_ms,
self.slide_ms,
self.edge_blend,
)
)
@classmethod
def from_bytes(cls, data: bytes) -> EffectSegmentSelect:
_check_size(data, cls.SIZE, cls.__name__)
count, selected, previous, gap = _SEGMENT_SELECT_HEAD.unpack_from(data)
start = _SEGMENT_SELECT_HEAD.size
colors = tuple(
ColorRGBW.from_bytes(data[start + i * _RGBW_SIZE : start + (i + 1) * _RGBW_SIZE])
for i in range(count)
)
offset, direction, dim, amplitude, period, slide, blend = _SEGMENT_SELECT_TAIL.unpack_from(
data, start + SEGMENT_SELECT_MAX_SEGMENTS * _RGBW_SIZE
)
return cls(
colors=colors,
selected=selected,
previous=None if previous == selected else previous,
gap_leds=gap,
offset_degrees=offset,
direction=direction,
dim_factor=dim,
wobble_amplitude=amplitude,
wobble_period_ms=period,
slide_ms=slide,
edge_blend=blend,
)
def __repr__(self) -> str:
return f"SegmentSelect({self.previous} -> {self.selected} of {len(self.colors)})"
def OFF() -> EffectStaticConfig: # noqa: N802 - reads as a constant at call sites
"""A fresh "all LEDs off" effect."""
return EffectStaticConfig(ColorRGBW(0, 0, 0, 0))

View File

@@ -7,8 +7,8 @@ Three flavours, distinguished by base class:
* :class:`StateEvent` - something changed.
The intent layer is what lets several front-ends drive the same behaviour: a button
press, an MQTT command and a future web request all emit ``NextTrackRequested`` and a
single reaction acts on it.
press, an MQTT command, a web request and the IR remote all emit ``NextTrackRequested``
and a single reaction acts on it.
"""
from __future__ import annotations
@@ -30,10 +30,10 @@ __all__ = [
"CancelTimerRequested",
"ClockStateChanged",
"ConnectionChanged",
"DeviceConnected",
"DeviceDisconnected",
"Event",
"EventSource",
"FigureSelectionChanged",
"FigureSelectorExpired",
"InputEvent",
"IntentEvent",
"LedEffectChanged",
@@ -50,6 +50,8 @@ __all__ = [
"PreviousTrackRequested",
"RfidTokenRead",
"RotaryTurned",
"SelectorPressed",
"SelectorTurned",
"SetTimerRequested",
"SetVolumeRequested",
"StateEvent",
@@ -141,22 +143,36 @@ class RotaryTurned(InputEvent):
return f"RotaryTurned(pos={self.position}, incr={self.increment}, {self.direction.name})"
@dataclass(frozen=True, slots=True, kw_only=True)
class SelectorTurned(InputEvent):
"""The rotary encoder turned while the ring's album picker had it.
Not a :class:`RotaryTurned`: while the picker is open the knob is not a volume
knob, and nothing that listens for one should hear it.
"""
#: Detents, positive for the direction that turns the volume up.
steps: int
@dataclass(frozen=True, slots=True, kw_only=True)
class SelectorPressed(InputEvent):
"""The rotary encoder was pressed while the ring's album picker had it. Instead of,
not as well as, a :class:`ButtonEvent` - so it is not published to Home Assistant."""
@dataclass(frozen=True, slots=True, kw_only=True)
class FigureSelectorExpired(InputEvent):
"""The ring's album picker went untouched for long enough to close by itself."""
figure: str
@dataclass(frozen=True, slots=True, kw_only=True)
class PlaylistFinished(InputEvent):
"""The player reached the end of the playlist."""
@dataclass(frozen=True, slots=True, kw_only=True)
class DeviceConnected(InputEvent):
port: str
@dataclass(frozen=True, slots=True, kw_only=True)
class DeviceDisconnected(InputEvent):
port: str
reason: str | None = None
# -------------------------------------------------------------------------- intents
@@ -300,6 +316,17 @@ class ActiveFigureChanged(StateEvent):
previous: str | None = None
@dataclass(frozen=True, slots=True, kw_only=True)
class FigureSelectionChanged(StateEvent):
"""The ring's album picker opened, moved or closed."""
figure: str
#: Index into the figure's albums of the highlighted one.
selected: int
count: int
open: bool
@dataclass(frozen=True, slots=True, kw_only=True)
class LedEffectChanged(StateEvent):
"""Emitted on *every* write to an LED zone, whatever caused it.

View File

@@ -36,6 +36,7 @@ _log = logging.getLogger(__name__)
__all__ = [
"ANALYZER_VERSION",
"MAX_FIGURE_ALBUMS",
"SECTIONS",
"Album",
"Analyzer",
@@ -50,6 +51,10 @@ __all__ = [
"track_key",
]
#: How many albums one figure can offer. Each is a section of the LED ring, and past six
#: they get too narrow to tell apart; the firmware's colour array is no longer either.
MAX_FIGURE_ALBUMS: Final = 6
#: The only kind worth spending DSP on: an audiobook chapter or a podcast episode is
#: tens of minutes of narration with no musical mood to extract, and there are far more
#: of them in a typical library than there are songs.
@@ -120,17 +125,32 @@ class MusicLibrary:
entry = self._entries.get(identifier) if identifier else None
return entry[0] if entry else None
def figure_playlists(self) -> dict[str, Playlist]:
"""One playlist per figure folder, keyed by figure name.
def figure_playlists(self) -> dict[str, tuple[Playlist, ...]]:
"""The albums a figure can play, keyed by figure name, in folder order.
The figure path and the web path must hand the player the *same* object for the
same album: ``reactions.playback.play_figure`` resumes on an identity check.
Usually one. A figure whose folder holds album subfolders rather than audio has
one per subfolder, depth-first and alphabetical, which is the order the LED ring
offers them in - and the ring has room for no more than
:data:`MAX_FIGURE_ALBUMS`.
"""
return {
album.figure: album.to_playlist()
for album, _ in self._entries.values()
if album.figure is not None
}
by_figure: dict[str, list[Album]] = {}
for album, _ in self._entries.values():
if album.figure is not None:
by_figure.setdefault(album.figure, []).append(album)
out: dict[str, tuple[Playlist, ...]] = {}
for figure, albums in by_figure.items():
albums.sort(key=lambda album: album.folder.parts)
if len(albums) > MAX_FIGURE_ALBUMS:
_log.warning(
"Figure %r has %d albums; only the first %d can be picked on the ring: %s",
figure,
len(albums),
MAX_FIGURE_ALBUMS,
", ".join(album.folder.name for album in albums[:MAX_FIGURE_ALBUMS]),
)
out[figure] = tuple(album.to_playlist() for album in albums[:MAX_FIGURE_ALBUMS])
return out
def latest_episode(self, series: str) -> Album | None:
"""The newest episode-unit album of a podcast show, by filename.

View File

@@ -54,9 +54,14 @@ __all__ = ["Fingerprint", "LibraryCache"]
# smaller - which means a cache holding 640 px files cannot be brought to 384 px by
# re-reading them, only by going back to the original art. Discarding the index does
# exactly that: every album is scanned again and `store_cover` runs on the real art.
# 7: a figure's albums may sit in subfolders. `figure` is now the folder directly under
# Figuren rather than the album folder's own name, and such an album is titled
# "Fuchs - <folder>" - neither of which an entry cached before can know.
# The same rescan retired `shrink_stored_covers`, which re-read every cover header on
# every scan to catch pre-384 px covers and covers stored before thumbnails existed.
#
# Bumping this is cheap by design - see the module docstring.
_INDEX_VERSION = 6
_INDEX_VERSION = 7
@dataclass(frozen=True, slots=True)
@@ -116,51 +121,6 @@ class LibraryCache:
)
return path
def _refresh_thumb(self, cover: Path) -> bool:
"""Make ``<id>.thumb.jpg`` exist and be newer than the cover it came from."""
thumb = cover.with_name(f"{cover.stem}.thumb.jpg")
try:
if thumb.exists() and thumb.stat().st_mtime_ns >= cover.stat().st_mtime_ns:
return False
thumb.write_bytes(
shrink_cover(cover.read_bytes(), max_px=THUMB_COVER_PX, quality=_THUMB_JPEG_QUALITY)
)
except OSError: # pragma: no cover - a cache we cannot write is not fatal
return False
return True
def shrink_stored_covers(self) -> int:
"""Rewrite any already-stored cover that predates :data:`MAX_COVER_PX`.
Needed because :func:`~musicmouse.library.scanner.scan_library` reuses an album
whose fingerprint is unchanged *without* re-reading its tags, so a cover written
by an older version would otherwise never be touched again. Reading a JPEG's
dimensions only parses its header, so once every file is within the limit this
costs one small read per album and nothing else.
Also backfills any missing (or stale) thumbnail. Returns the number of full-size
files actually rewritten.
"""
rewritten = 0
for path in sorted(self.covers.glob("*.jpg")):
if path.name.endswith(".thumb.jpg"):
continue
try:
with Image.open(path) as image:
oversized = max(image.size) > MAX_COVER_PX
except (OSError, UnidentifiedImageError):
continue
if oversized:
try:
shrunk = shrink_cover(path.read_bytes())
path.write_bytes(shrunk)
except OSError: # pragma: no cover - a cache we cannot write is not fatal
continue
rewritten += 1
# Covers stored before thumbnails existed get theirs here, without a rescan.
self._refresh_thumb(path)
return rewritten
# ------------------------------------------------------------------ analysis
def load_analysis(self, key: str) -> TrackAnalysis | None:
@@ -340,12 +300,12 @@ def shrink_cover(
for a browser to decode.
"""
try:
with Image.open(io.BytesIO(data)) as image:
if max(image.size) <= max_px:
with Image.open(io.BytesIO(data)) as source:
if max(source.size) <= max_px:
return data
# JPEG can decode straight to 1/2, 1/4 or 1/8 size; a no-op for other formats.
image.draft("RGB", (max_px, max_px))
image = image.convert("RGB")
source.draft("RGB", (max_px, max_px))
image = source.convert("RGB")
# `thumbnail` keeps the aspect ratio and never scales up.
image.thumbnail((max_px, max_px), Image.Resampling.LANCZOS)
buffer = io.BytesIO()

View File

@@ -2,7 +2,7 @@
The frontend paints an album card with these and the LED strips run an effect in them,
so a web-started album looks the same on the shelf as it does on the screen. Extraction
happens once per album ever - the result is cached next to the cover.
happens when an album is scanned; the result is stored in the cached index.
"""
from __future__ import annotations

View File

@@ -135,9 +135,9 @@ def _cover_for(
return cache.store_cover(identifier, art), art
for path in paths[:3]:
# Podcast feeds sometimes art only some episodes; a couple of tries is enough.
art = _embedded_art(path)
if art is not None:
return cache.store_cover(identifier, art), art
embedded = _embedded_art(path)
if embedded is not None:
return cache.store_cover(identifier, embedded), embedded
return None, None
@@ -155,9 +155,9 @@ def _cover_for_episode(
if candidate.is_file():
art = candidate.read_bytes()
return cache.store_cover(identifier, art), art
art = _embedded_art(path)
if art is not None:
return cache.store_cover(identifier, art), art
embedded = _embedded_art(path)
if embedded is not None:
return cache.store_cover(identifier, embedded), embedded
for name in _COVER_NAMES:
candidate = folder / name
if candidate.is_file():
@@ -197,7 +197,9 @@ def scan_album(
albums.append(tags.get("album", ""))
artists.append(tags.get("albumartist") or tags.get("artist", ""))
figure = folder.name if section.figures else None
# A figure with several albums keeps them in subfolders of its own folder, so the
# figure is the folder directly under the section, however deep the album sits.
figure = folder.relative_to(root / section_name).parts[0] if section.figures else None
# Every other shelf is named after what is on it. A figure folder is named after the
# figurine, so its media type has to be declared in the config.
kind = (figure_kinds or {}).get(figure, "music") if figure else section.kind
@@ -205,9 +207,13 @@ def scan_album(
folder_artist, folder_title = _split_folder_name(folder.name)
if section.title_from == "folder":
# A figure folder is named in lowercase ("fuchs"); it sits next to real album
# titles in the browse grid, so give it a capital.
title = folder.name[:1].upper() + folder.name[1:] if section.figures else folder.name
title = folder.name
if figure is not None:
# A figure folder is named in lowercase ("fuchs"); it sits next to real
# album titles in the browse grid, so give it a capital.
title = figure[:1].upper() + figure[1:]
if folder.name != figure:
title = f"{title} - {folder.name}"
else:
title = _most_common(albums) or folder_title or folder.name
@@ -311,13 +317,6 @@ def scan_library(
figure name to what it holds, which is the one thing the folders cannot say.
"""
cache.prepare()
# Covers written before MAX_COVER_PX existed are still whatever size the tag held,
# and the reuse path below means an unchanged album never rewrites its own. One
# pass here catches them; after the first run every file is already small and this
# is 300-odd header reads.
shrunk = cache.shrink_stored_covers()
if shrunk:
_log.info("Downscaled %d oversized cover(s) in the cache", shrunk)
known = known or {}
out: dict[str, tuple[Album, Fingerprint]] = {}
last_report = time.monotonic()

View File

@@ -1,8 +1,8 @@
"""Playlist model.
Deliberately a real type rather than a bare ``list[str]``: it is what the browse API
serves, and it keeps track metadata in one place. Building one from a folder is
:mod:`musicmouse.library`'s job.
What the player plays: a name and the files, in order. The browse API serves
:class:`~musicmouse.library.models.Album` instead, which carries the tag metadata;
``Album.to_playlist()`` builds one of these from it.
"""
from __future__ import annotations

View File

@@ -14,6 +14,7 @@ service subscribes to those events directly.
from musicmouse.reactions import ( # noqa: F401 (import = register)
lighting,
playback,
selection,
status,
)
from musicmouse.reactions.registry import Reaction, on, register_all, registered

View File

@@ -54,7 +54,10 @@ def figure_placed_or_removed(event: ActiveFigureChanged, app: App) -> None:
if event.figure is None:
off_animation(app)
else:
start_animation(app, app.colors(event.figure))
# A figure with several albums gets the album picker on the ring instead
# (see ``reactions.selection``), which fades into this ring effect when done.
ring = len(app.playlists.get(event.figure, ())) < 2
start_animation(app, app.colors(event.figure), ring=ring)
app.mouse.set_button_brightness(
app.config.general.button_leds_brightness, origin="device"
)
@@ -106,36 +109,42 @@ def touch_released(event: TouchButtonReleased, app: App) -> None:
)
if colors is None:
return
app.mouse.set_effect(
LedZone.MOUSE,
EffectRandomTwoColorInterpolationConfig(
color1=colors.primary, color2=colors.secondary, start_with_existing=True
),
origin="device",
)
fade_to_figure(app, LedZone.MOUSE, colors)
# ------------------------------------------------------------------- animations
def start_animation(app: App, colors: FigureColors) -> None:
ring = EffectSwipeAndChange()
ring.swipe.primary_color = colors.primary
ring.swipe.secondary_color = colors.secondary
ring.swipe.swipe_speed = SWIPE_SPEED
ring.change.color1 = colors.primary
ring.change.color2 = colors.secondary
def start_animation(app: App, colors: FigureColors, *, ring: bool = True) -> None:
swipe = EffectSwipeAndChange()
swipe.swipe.primary_color = colors.primary
swipe.swipe.secondary_color = colors.secondary
swipe.swipe.swipe_speed = SWIPE_SPEED
swipe.change.color1 = colors.primary
swipe.change.color2 = colors.secondary
app.mouse.set_effect(LedZone.RING, ring, origin="device")
app.mouse.set_effect(LedZone.SHELF, deepcopy(ring), origin="device")
if ring:
app.mouse.set_effect(LedZone.RING, swipe, origin="device")
app.mouse.set_effect(LedZone.SHELF, deepcopy(swipe), origin="device")
mouse = deepcopy(ring)
mouse = deepcopy(swipe)
mouse.swipe.start_position = MOUSE_SWIPE_START_DEGREES
mouse.swipe.bell_curve_width_in_leds = MOUSE_BELL_CURVE_WIDTH
app.mouse.set_effect(LedZone.MOUSE, mouse, origin="device")
def fade_to_figure(app: App, zone: LedZone, colors: FigureColors) -> None:
"""Fade ``zone`` from whatever it shows into the figure's resting colours - where
the start animation ends up after its swipe."""
app.mouse.set_effect(
zone,
EffectRandomTwoColorInterpolationConfig(
color1=colors.primary, color2=colors.secondary, start_with_existing=True
),
origin="device",
)
def web_animation(app: App, color: ColorRGBW) -> None:
for zone in LedZone:
app.mouse.set_effect(

View File

@@ -1,8 +1,8 @@
"""What the mouse plays, and when.
Physical inputs are turned into *intents*, and the intents are what actually drive the
player. That indirection is the point: an MQTT command or a future web request emits
the same intent and lands in the same handler, so there is one place per behaviour.
player. That indirection is the point: an MQTT command, a web request or an IR remote key
emits the same intent and lands in the same handler, so there is one place per behaviour.
"""
from __future__ import annotations
@@ -29,6 +29,7 @@ from musicmouse.events import (
)
from musicmouse.hardware import Button, ButtonAction, RotaryDirection
from musicmouse.library.models import Album
from musicmouse.media import Playlist
from musicmouse.reactions.registry import on
_log = logging.getLogger(__name__)
@@ -63,24 +64,46 @@ def _figure_removed(previous: str | None, app: App) -> None:
@on(PlayFigureRequested)
def play_figure(event: PlayFigureRequested, app: App) -> None:
playlist = app.playlist(event.figure)
if playlist is None:
return
"""Play the figure's album - the one it played last, if it has several.
if not event.restart and app.player.playlist is playlist:
Another figure placed in between forgets only *where* in that album it was
(``restart``), not which album it was.
"""
choices = app.choices(event.figure)
if not choices:
return
playlist = choices[app.default_choice(event.figure)]
remember_figure_album(app, event.figure, playlist)
# Equality, not identity: a library rescan (the refresh button, or the podcast
# poller landing an episode) rebuilds every Playlist object, and an `is` check made
# the figure start over after that. Equal means the same album with the same files;
# if the files changed, starting over is the honest thing to do.
if not event.restart and app.player.playlist == playlist:
_log.info("Resuming %r", event.figure)
app.player.play()
return
_log.info("Starting %r from the beginning", event.figure)
_log.info("Starting %r (%s) from the beginning", event.figure, playlist.album_id)
app.player.set_playlist(playlist)
app.player.play_from_start()
def remember_figure_album(app: App, figure: str, playlist: Playlist) -> None:
"""Make ``playlist`` the album ``figure`` picks next time it is put down."""
if playlist.album_id is not None:
app.state.figure_albums[figure] = playlist.album_id
def _start_album(app: App, album: Album, track_index: int) -> None:
# A figure album keeps the figure's own resume bookkeeping honest: it is the same
# Playlist object either way, because both come from the library index.
playlist = app.playlists.get(album.figure) if album.figure else album.to_playlist()
# A figure album started from anywhere becomes the album that figure picks next.
playlist = None
if album.figure is not None:
playlist = next(
(p for p in app.playlists.get(album.figure, ()) if p.album_id == album.id), None
)
if playlist is not None:
remember_figure_album(app, album.figure, playlist)
app.player.set_playlist(playlist or album.to_playlist())
app.player.play_track(track_index)
@@ -144,7 +167,9 @@ def button_pressed(event: ButtonEvent, app: App) -> None:
elif event.button is Button.RIGHT and app.player.is_playing:
app.bus.emit(NextTrackRequested(source="device"))
# The rotary press is published to Home Assistant by the MQTT service; what it
# controls is an automation over there, not something this backend decides.
# controls is an automation over there, not something this backend decides. While
# the ring's album picker is open the press is the picker's instead, and does not
# arrive here at all (see ``MusicMouseDevice.capture_knob``).
@on(RotaryTurned)

View File

@@ -0,0 +1,235 @@
"""Picking one of a figure's albums on the LED ring.
A figure whose folder holds several albums starts playing its default one at once, as
any figure does, and meanwhile offers all of them on the ring: one coloured section
each, the selected one bright and wobbling. The knob moves the selection one section
per detent, and pressing it plays the selected album. Left alone, the picker closes by
itself and whatever is playing carries on.
While it is open the picker has the knob to itself (``MusicMouseDevice.capture_knob``):
turning does not change the volume and pressing is not sent to Home Assistant.
The firmware only draws the ring. Which section is selected, when to give up, and what
to play are decided here, and the ring is simply sent again on every detent.
"""
from __future__ import annotations
import asyncio
import logging
from collections.abc import Callable
from dataclasses import dataclass, replace
from musicmouse.app import App, FigureSelector
from musicmouse.blubb import confirm_wav, tick_wav
from musicmouse.color import ColorRGBW
from musicmouse.effects import EffectSegmentSelect
from musicmouse.events import (
ActiveFigureChanged,
FigureSelectionChanged,
FigureSelectorExpired,
PlayAlbumRequested,
PlaylistFinished,
SelectorPressed,
SelectorTurned,
)
from musicmouse.hardware import LedZone
from musicmouse.media import Playlist
from musicmouse.reactions.lighting import fade_to_figure
from musicmouse.reactions.registry import on
from musicmouse.sound import play_wav_soon, ui_level
_log = logging.getLogger(__name__)
#: One colour per album, by position. Neighbours are far apart in hue, so the sections
#: stay easy to tell apart even with the edge blend shifting them towards each other.
PALETTE: tuple[ColorRGBW, ...] = (
ColorRGBW(1.0, 0.0, 0.0, 0.0), # red
ColorRGBW(1.0, 0.35, 0.0, 0.0), # orange
ColorRGBW(1.0, 0.8, 0.0, 0.0), # yellow
ColorRGBW(0.0, 0.9, 0.1, 0.0), # green
ColorRGBW(0.0, 0.3, 1.0, 0.0), # blue
ColorRGBW(0.6, 0.0, 1.0, 0.0), # violet
)
#: Turning faster than this plays one tick, not a pile of overlapping ones.
TICK_INTERVAL = 0.06
#: Peak of each sound at full UI loudness, as a fraction of full scale.
TICK_GAIN = 0.25
CONFIRM_GAIN = 0.3
@dataclass(frozen=True, slots=True)
class Tuning:
"""How the picker looks and behaves. Field names match
:class:`~musicmouse.config.FigureSelectionConfig`, whose values override these."""
timeout: float = 8.0 # seconds without a turn before the picker closes
#: Where the first section starts. The figure's swipe starts at LED 0 and meets
#: itself opposite, which is the front of the ring.
offset_degrees: float = 180.0
ring_direction: int = 1
knob_direction: int = 1
gap_leds: int = 1
dim_factor: float = 0.15
wobble_amplitude: float = 1.5 # LEDs
wobble_period_ms: float = 1000.0
slide_ms: float = 200.0
edge_blend: float = 0.25
DEFAULTS = Tuning()
def tuning(app: App) -> Tuning:
""":data:`DEFAULTS`, with whatever the config's ``figure_selection`` overrides."""
overrides = app.config.general.figure_selection.model_dump(exclude_none=True)
return replace(DEFAULTS, **overrides)
# ----------------------------------------------------------------------- reactions
@on(ActiveFigureChanged)
def figure_placed_or_removed(event: ActiveFigureChanged, app: App) -> None:
# Whatever replaces the figure - nothing, or another one - sets the ring itself.
if app.state.selector is not None:
_close(app, fade=False)
if event.figure is None:
return
choices = app.playlists.get(event.figure, ())
if len(choices) > 1:
_open(app, event.figure, choices)
@on(SelectorTurned)
def selector_turned(event: SelectorTurned, app: App) -> None:
selector = app.state.selector
if selector is None:
return # a turn still in the queue when the picker closed
settings = tuning(app)
now = app.clock.now()
selector.deadline = now + settings.timeout
previous = selector.selected
selector.selected = (previous + event.steps * settings.knob_direction) % len(selector.choices)
if selector.selected == previous:
return
app.mouse.set_effect(LedZone.RING, _ring(app, selector, previous), origin="device")
if now - selector.last_tick >= TICK_INTERVAL:
selector.last_tick = now
_play_sound(app, tick_wav, TICK_GAIN)
_announce(app, selector, open_=True)
@on(SelectorPressed)
def selector_pressed(_event: SelectorPressed, app: App) -> None:
selector = app.state.selector
if selector is None:
return
playlist = selector.choices[selector.selected]
_close(app, fade=True)
_play_sound(app, confirm_wav, CONFIRM_GAIN)
# By value: a rescan rebuilds the Playlist objects (see play_figure).
if app.player.playlist == playlist:
_log.info("Album picker: keeping %s", playlist.album_id)
return
_log.info("Album picker: %r -> %s", selector.figure, playlist.album_id)
if playlist.album_id is not None:
# The same way in as the web front-end, which also remembers the album as the
# figure's for next time.
app.bus.emit(PlayAlbumRequested(album_id=playlist.album_id, source="device"))
@on(FigureSelectorExpired)
def selector_expired(event: FigureSelectorExpired, app: App) -> None:
selector = app.state.selector
# Stale if the picker it was for has closed, or a newer one is still in time.
if selector is None or selector.figure != event.figure:
return
if app.clock.now() < selector.deadline:
return
_close(app, fade=True)
@on(PlaylistFinished)
def playlist_finished(_event: PlaylistFinished, app: App) -> None:
# The off animation takes the ring; the knob goes back to being a volume knob.
if app.state.selector is not None:
_close(app, fade=False)
# ----------------------------------------------------------------------- internals
def _open(app: App, figure: str, choices: tuple[Playlist, ...]) -> None:
selector = FigureSelector(
figure=figure,
choices=choices,
selected=app.default_choice(figure),
deadline=app.clock.now() + tuning(app).timeout,
)
app.state.selector = selector
app.mouse.capture_knob(True)
app.mouse.set_effect(LedZone.RING, _ring(app, selector, None), origin="device")
selector.expiry = asyncio.create_task(_expire(app, selector), name="album-picker")
_log.info("Album picker for %r: %d albums", figure, len(choices))
_announce(app, selector, open_=True)
def _close(app: App, *, fade: bool) -> None:
selector = app.state.selector
if selector is None:
return
app.state.selector = None
app.mouse.capture_knob(False)
if selector.expiry is not None:
selector.expiry.cancel()
if fade:
fade_to_figure(app, LedZone.RING, app.colors(selector.figure))
_announce(app, selector, open_=False)
async def _expire(app: App, selector: FigureSelector) -> None:
# Every turn pushes the deadline back, so sleep until it stops moving.
while (remaining := selector.deadline - app.clock.now()) > 0:
await app.clock.sleep(remaining)
app.bus.emit(FigureSelectorExpired(figure=selector.figure))
def _ring(app: App, selector: FigureSelector, previous: int | None) -> EffectSegmentSelect:
settings = tuning(app)
return EffectSegmentSelect(
colors=PALETTE[: len(selector.choices)],
selected=selector.selected,
previous=previous,
gap_leds=settings.gap_leds,
offset_degrees=settings.offset_degrees,
direction=settings.ring_direction,
dim_factor=settings.dim_factor,
wobble_amplitude=settings.wobble_amplitude,
wobble_period_ms=settings.wobble_period_ms,
slide_ms=settings.slide_ms,
edge_blend=settings.edge_blend,
)
def _announce(app: App, selector: FigureSelector, *, open_: bool) -> None:
app.bus.emit(
FigureSelectionChanged(
figure=selector.figure,
selected=selector.selected,
count=len(selector.choices),
open=open_,
source="device",
)
)
def _play_sound(app: App, make: Callable[..., bytes], base_gain: float) -> None:
general = app.config.general
if general.audio_simulated:
return
gain = base_gain * ui_level(app.player.volume / 100, general.effects_volume / 100)
if gain > 0:
play_wav_soon(general.alsa_device, make(gain=gain))

View File

@@ -1,7 +1,8 @@
"""Front-ends: things that mirror state outwards and turn requests into intents.
``MqttService`` is the reference implementation. A web service would be another file
here plus one line in the app - devices and reactions would not change.
MQTT (Home Assistant), web (the browser), lirc (the IR remote) and the screen, timer and
podcast services each live here; adding another is a file here plus a line in
``__main__._build_services`` - devices and reactions do not change.
"""
from musicmouse.services.base import Service

View File

@@ -6,8 +6,9 @@ async handlers, so running it there would hold every other event up behind ``xse
Here each press gets its own task, and a hung or failing command is only logged.
Each switch is answered with a "blubb" (:mod:`musicmouse.blubb`) on the music's own
ALSA device, at the player's volume. The command's last word of output says which way
it went - ``on`` or ``off``, as ``kiosk-screen`` prints it; anything else gets no sound.
ALSA device, as loud as UI sounds are (:func:`musicmouse.sound.ui_level`). The
command's last word of output says which way it went - ``on`` or ``off``, as
``kiosk-screen`` prints it; anything else gets no sound.
With an ``off_command`` it also switches the monitor off by itself once nothing has
played and nobody has done anything for ``auto_off_minutes`` - silently, since whoever
@@ -68,7 +69,8 @@ class ScreenService:
self.config = config
#: Where the blubb plays, or ``None`` for no sound (simulated audio).
self.alsa_device = alsa_device
#: The player's volume as a fraction, read at the moment the sound plays.
#: How loud UI sounds are right now, 0..1 (see :func:`musicmouse.sound.ui_level`),
#: read at the moment the sound plays.
self.volume = volume
self.is_playing = is_playing
self._clock = clock or RealClock()
@@ -139,9 +141,6 @@ class ScreenService:
self._running = asyncio.create_task(self._run(command, sound=sound), name="screen")
return True
async def toggle(self) -> None:
await self._run(self.config.toggle_command, sound=True)
async def _run(self, command: str, *, sound: bool) -> None:
argv = shlex.split(command)
try:

View File

@@ -49,8 +49,6 @@ RING_SECONDS = 60.0
#: The chime's peak at full player volume, as a fraction of full scale - soft.
_CHIME_GAIN = 0.3
#: Never quieter than at this player volume: a muted player must not mute the timer.
_CHIME_MIN_VOLUME = 0.3
#: After a failed holiday fetch, try again this much later rather than at midnight.
_RETRY_SECONDS = 3600.0
@@ -94,7 +92,8 @@ class TimerService:
self.client = client
#: Where the chime plays, or ``None`` for no sound (simulated audio).
self.alsa_device = alsa_device
#: The player's volume as a fraction, read when the chime plays.
#: How loud UI sounds are right now, 0..1 (see :func:`musicmouse.sound.ui_level`),
#: read when the chime plays.
self.volume = volume
self._clock = clock or RealClock()
self._wall = wall
@@ -211,7 +210,7 @@ class TimerService:
async def _chime(self) -> None:
if self.alsa_device is None:
return
gain = _CHIME_GAIN * max(_CHIME_MIN_VOLUME, min(1.0, self.volume()))
gain = _CHIME_GAIN * max(0.0, min(1.0, self.volume()))
wav = await asyncio.to_thread(chime_wav, gain=gain)
await play_wav(self.alsa_device, wav)

View File

@@ -46,7 +46,6 @@ from musicmouse.services.web.schemas import (
HaConfigOut,
HaDeviceOut,
LibraryOut,
LircConfigOut,
PlayerStateOut,
PlayIn,
RemoteMappingIn,
@@ -68,7 +67,6 @@ from musicmouse.services.web.schemas import (
from musicmouse.services.web.settings import (
read_settings,
to_device_volume,
to_percent,
write_settings,
)
from musicmouse.services.web.state import snapshot
@@ -234,18 +232,10 @@ def build_router(
return emit(
VolumeChangeRequested(delta=general.volume_increment * body.steps, source="web")
)
if body.percent is not None:
target = body.percent
elif body.delta_percent is not None:
target = to_percent(app.player.volume, general) + body.delta_percent
else:
raise HTTPException(
status_code=422, detail="percent, delta_percent or steps required"
)
if body.percent is None:
raise HTTPException(status_code=422, detail="percent or steps required")
return emit(
SetVolumeRequested(
volume=to_device_volume(max(0, min(100, target)), general), source="web"
)
SetVolumeRequested(volume=to_device_volume(body.percent, general), source="web")
)
# ------------------------------------------------------------- parent mode
@@ -321,12 +311,6 @@ def build_router(
# -------------------------------------------------------------- IR remote
@router.get("/lirc")
def get_lirc_config() -> LircConfigOut:
if app.config.general.lirc is None:
raise HTTPException(status_code=404, detail="lirc not configured")
return LircConfigOut(connected=app.state.lirc_connected)
@router.post("/remote/press/{digit}", status_code=204)
def press_remote(digit: Digit) -> Response:
# A keyboard's number keys, doing what the IR remote's do. An unassigned key

View File

@@ -22,7 +22,19 @@ from typing import Any
from fastapi import WebSocket
from musicmouse.app import App
from musicmouse.events import Event, StateEvent, ToggleClockRequested
from musicmouse.events import (
ActiveFigureChanged,
ClockStateChanged,
ConnectionChanged,
Event,
FigureSelectionChanged,
LedEffectChanged,
PlaybackChanged,
StateEvent,
ToggleClockRequested,
TrackChanged,
VolumeChanged,
)
from musicmouse.services.web.state import snapshot
_log = logging.getLogger(__name__)
@@ -33,6 +45,25 @@ __all__ = ["StateHub"]
#: enough to leave running.
POSITION_INTERVAL = 0.5
#: The state events that change something in :func:`~musicmouse.services.web.state.snapshot`
#: - each one costs every open tab a full snapshot. LED writes are the reason this is a
#: list rather than "every StateEvent": a figure placed and removed is 9 LED writes,
#: and the snapshot has no LED state in it.
SNAPSHOT_EVENTS: tuple[type[StateEvent], ...] = (
PlaybackChanged,
TrackChanged,
VolumeChanged,
ActiveFigureChanged,
ConnectionChanged,
ClockStateChanged,
)
#: State events deliberately not pushed to the browser. Together with
#: :data:`SNAPSHOT_EVENTS` this covers every :class:`StateEvent`; a test holds that, so
#: a new state event has to be sorted into one or the other.
#: The ring's album picker is shown on the LED ring, not in the browser.
IGNORED_EVENTS: tuple[type[StateEvent], ...] = (LedEffectChanged, FigureSelectionChanged)
class StateHub:
def __init__(self, app: App) -> None:
@@ -44,7 +75,7 @@ class StateHub:
def start(self) -> None:
self._unsubscribe = [
self.app.bus.subscribe(StateEvent, self._on_state),
*(self.app.bus.subscribe(kind, self._on_state) for kind in SNAPSHOT_EVENTS),
# The clock overlay is the kiosk's own; the remote's key only asks for it.
self.app.bus.subscribe(ToggleClockRequested, self._on_clock_toggle),
]

View File

@@ -26,7 +26,6 @@ __all__ = [
"HaConfigOut",
"HaDeviceOut",
"LibraryOut",
"LircConfigOut",
"PlayIn",
"PlayerStateOut",
"RemoteMappingIn",
@@ -261,15 +260,13 @@ class SeekIn(BaseModel):
class VolumeIn(BaseModel):
"""An absolute percentage, a relative percentage, or a number of steps; exactly
one of the three.
"""An absolute percentage or a number of steps; exactly one of the two.
``steps`` moves by ``volume_increment`` device units per step - the same step the
IR remote's volume keys and the rotary encoder take - rather than by a percentage.
"""
percent: int | None = Field(default=None, ge=0, le=100)
delta_percent: int | None = Field(default=None, ge=-100, le=100)
steps: int | None = Field(default=None, ge=-10, le=10)
@@ -293,6 +290,7 @@ class SettingsOut(BaseModel):
initial_volume: int
volume_increment: int
button_leds_brightness: float
effects_volume: int
#: ``general.screen.auto_off_minutes``, or ``None`` when there is no screen to
#: switch off (no ``screen`` section, or one without an ``off_command``).
screen_auto_off_minutes: int | None
@@ -304,6 +302,8 @@ class SettingsIn(BaseModel):
initial_volume: int = Field(ge=0, le=200)
volume_increment: int = Field(ge=1, le=100)
button_leds_brightness: float = Field(ge=0, le=1)
#: UI sounds relative to the music, in percent.
effects_volume: int = Field(ge=0, le=100)
#: Ignored without a screen section. ``None`` leaves it as it is.
screen_auto_off_minutes: int | None = Field(default=None, ge=0, le=720)
@@ -322,13 +322,6 @@ class HaConfigOut(BaseModel):
scenes: list[HaDeviceOut]
class LircConfigOut(BaseModel):
"""Presence-only, like ``HaConfigOut``: there is nothing secret in a host/port,
but the frontend only needs to know whether the remote is set up and connected."""
connected: bool
class RemoteSlotOut(BaseModel):
digit: str
target_kind: Literal["album", "series"]

View File

@@ -55,6 +55,7 @@ def read_settings(general: GeneralConfig) -> SettingsOut:
initial_volume=general.initial_volume,
volume_increment=general.volume_increment,
button_leds_brightness=general.button_leds_brightness,
effects_volume=general.effects_volume,
screen_auto_off_minutes=(
screen.auto_off_minutes if screen is not None and screen.off_command else None
),

View File

@@ -2,6 +2,9 @@
``aplay`` in a subprocess, fed the WAV on stdin: no audio library to load, and a hung
or missing ``aplay`` costs a log line, never the event loop.
Every UI sound - the screen's blubb, the parents' timer, the album picker's clicks -
is as loud as :func:`ui_level` says, times its own base gain.
"""
from __future__ import annotations
@@ -11,7 +14,32 @@ import logging
_log = logging.getLogger(__name__)
__all__ = ["play_wav"]
__all__ = ["MIN_PLAYER_VOLUME", "play_wav", "play_wav_soon", "ui_level"]
#: UI sounds are never quieter than at this player volume (0..1): a muted player must
#: not mute the timer, nor leave turning the album picker silent.
MIN_PLAYER_VOLUME = 0.3
#: Sounds started with :func:`play_wav_soon`, kept so they are not collected mid-play.
_playing: set[asyncio.Task[None]] = set()
def ui_level(player_volume: float, effects_volume: float) -> float:
"""How loud a UI sound plays, 0..1, before its own base gain.
``player_volume`` is the music's (0..1), floored at :data:`MIN_PLAYER_VOLUME`;
``effects_volume`` is the parent-mode setting (0..1) for sounds relative to music.
"""
player = max(MIN_PLAYER_VOLUME, min(1.0, player_volume))
return player * max(0.0, min(1.0, effects_volume))
def play_wav_soon(alsa_device: str, wav: bytes) -> None:
"""Start playing ``wav`` without waiting for it - for callers on the event bus,
which must not stall every other event for the length of a sound."""
task = asyncio.create_task(play_wav(alsa_device, wav), name="ui-sound")
_playing.add(task)
task.add_done_callback(_playing.discard)
async def play_wav(alsa_device: str, wav: bytes, *, timeout: float = 10.0) -> None:

View File

@@ -5,8 +5,8 @@ Same convention as everywhere else this codebase persists something outside
missing or corrupt file is just a fresh start, never a crash: nothing here is precious
enough to raise over.
``record_run`` is a near-verbatim port of tippen's own (client-side, until now)
``recordRun`` in ``tippen/src/lib/progress.ts`` - grading a keystroke-by-keystroke run
``record_run`` is a port of the ``recordRun`` the standalone typing app had client-side
before it was merged into the web front-end - grading a keystroke-by-keystroke run
into stars/points/an animal stays entirely client-side (see ``TippenRunIn`` in
``musicmouse.services.web.schemas``); this only owns what happens to progress once a
graded result arrives.

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -1,434 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<div>\n",
"<a href=\"http://www.music-processing.de/\"><img style=\"float:left;\" src=\"../data/FMP_Teaser_Cover.png\" width=40% alt=\"FMP\"></a>\n",
"<a href=\"https://www.audiolabs-erlangen.de\"><img src=\"../data/Logo_AudioLabs_Long.png\" width=59% style=\"float: right;\" alt=\"AudioLabs\"></a>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<div>\n",
"<a href=\"../C5/C5.html\"><img src=\"../data/C5_nav.png\" width=\"100\" style=\"float:right;\" alt=\"C5\"></a>\n",
"<h1>Hidden Markov Model (HMM)</h1> \n",
"</div>\n",
"\n",
"<br/>\n",
"\n",
"<p>\n",
"Motivated by the chord recognition problem, we give in this notebook an overview of hidden Markov models (HMMs) and introduce three famous algorithmic problems related with HMMs following Section 5.3 of <a href=\"http://www.music-processing.de/\">[Müller, FMP, Springer 2015]</a>. For a detailed introduction of HMMs, we refer to the famous tutorial paper by Rabiner.\n",
"\n",
"<ul>\n",
"<li><span style=\"color:black\">\n",
"Lawrence R. Rabiner: <strong>A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition.</strong> Proceedings of the IEEE, 77 (1989), pp. 257&ndash;286. \n",
"<br> \n",
"<a type=\"button\" class=\"btn btn-default btn-xs\" target=\"_blank\" href=\"../data/bibtex/FMP_bibtex_Rabiner89_HMM_IEEE.txt\"> Bibtex </a>\n",
"</span></li>\n",
"</ul>\n",
"</p> "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Markov Chains\n",
"\n",
"Certain transitions from one chord to another are more likely than others. To capture such likelihoods, one can employ a concept called **Markov chains**. Abstracting from our chord recognition scenario, we assume that the chord types to be considered are represented by a set \n",
"\n",
"\\begin{equation}\n",
" \\mathcal{A}:=\\{\\alpha_{1},\\alpha_{2},\\ldots,\\alpha_{I}\\}\n",
"\\end{equation}\n",
"\n",
"of size $I\\in\\mathbb{N}$. The elements $\\alpha_{i}$ for $i\\in[1:I]$ are referred to as **states**. A progression of chords is realized by a system that can be described at any time instance $n=1,2,3,\\ldots$ as being in some state $s_{n}\\in\\mathcal{A}$. The change from one state to another is specified according to a set of probabilities associated with each state. In general, a probabilistic description of such a system can be quite complex. To simplify the model, one often makes the assumption that the probability of a change from the current state $s_{n}$ to the next state $s_{n+1}$ only depends on the current state, and not on the events that preceded it. In terms of conditional probabilities, this property is expressed by\n",
"\n",
"\\begin{equation}\n",
" P[s_{n+1}=\\alpha_{j}|s_{n}=\\alpha_{i},s_{n-1}=\\alpha_{k},\\ldots]\n",
" = P[s_{n+1}=\\alpha_{j}|s_{n}=\\alpha_{i}].\n",
"\\end{equation}\n",
"\n",
"The specific kind of \"amnesia\" is called the **Markov property**. Besides this property, one also often assumes that the system is **invariant under time shifts**, which means by definition that the following coefficients become independent of the index $n$:\n",
"\n",
"\\begin{equation}\n",
" a_{ij} := P[s_{n+1}=\\alpha_{j} | s_{n}=\\alpha_{i}] \\in [0,1]\n",
"\\end{equation}\n",
"\n",
"for $i,j\\in[1:I]$. These coefficients are also called **state transition probabilities**. They obey the standard stochastic constraint $\\sum_{j=1}^{I} a_{ij} = 1$ and can be expressed by an $(I\\times I)$ matrix, which we denote by $A$. A system that satisfies these properties is also called a (discrete-time) **Markov chain**. The following figure illustrates these definitions. It defines a Markov chain that consists of $I=3$ states $\\alpha_{1}$, $\\alpha_{2}$, and $\\alpha_{3}$, which correspond to the major chords $\\mathbf{C}$, $\\mathbf{G}$, and $\\mathbf{F}$, respectively. In the graph representation, the states correspond to the nodes, the transitions to the edges, and the transition probabilities to the labels attached to the edges. For example, the transition probability to remain in the state $\\alpha_{1}=\\mathbf{C}$ is $a_{11}=0.8$, whereas the transition probability of changing from $\\alpha_{1}=\\mathbf{C}$ to $\\alpha_{2}=\\mathbf{G}$ is $a_{12}=0.1$.\n",
"\n",
"<img src=\"../data/C5/FMP_C5_F24.png\" width=\"550px\" align=\"middle\" alt=\"FMP_C5_F24\">\n",
"\n",
"The model expresses the probability of all possible chord changes. To compute the probability of a given chord progression, one also needs the information on how the model gets started. This information is specified\n",
"by additional model parameters referred to as **initial state probabilities**. For a general Markov chain, these probabilities are specified by the numbers\n",
"\n",
"\\begin{equation}\n",
" c_{i} := P[s_{1}=\\alpha_{i}] \\in [0,1]\n",
"\\end{equation}\n",
"\n",
"for $i\\in[1:I]$. These coefficients, which sum up to one, can be expressed by a vector of length $I$ denoted by $C$."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Hidden Markov Models\n",
"\n",
"Based on a Markov chain, one can compute a probability for a given observation consisting of a sequence of states or chord types. In our [chord recognition scenario](../C5/C5S2_ChordRec_Templates.html), however, this is not what we need. Rather than observing a sequence of chord types, we observe a **sequence of chroma vectors** that are somehow related to the chord types. In other words, the state sequence is not directly visible, but only a fuzzier observation sequence that is generated based on the state sequence. This leads to an extension of Markov chains to a statistical model referred to as a **hidden Markov model** (HMM). The idea is to represent the relation between the observed feature vectors and the chord types (the states) using a probabilistic framework. Each state is equipped with a probability function that expresses the likelihood for a given chord type to output or emit a certain feature vector. As a result, we obtain a two-layered process consisting of a **hidden layer** and an **observable layer**. The hidden layer produces a state sequence that is not observable (\"hidden\"), but generates the observation sequence on the basis of the state-dependent probability functions.\n",
"\n",
"The **first layer** of an HMM is a **Markov chain** as introduced above. To define the second layer of an HMM, we need to specify a space of possible output values and a probability function for each state. In general, the output space can be any set including the real numbers, a vector space, or any kind of feature space. For example, in the case of chord recognition, this space may be modeled as the feature space $\\mathcal{F}=\\mathbb{R}^{12}$ consisting of all possible $12$-dimensional chroma vectors. For the sake of simplicity, we only consider the case of a **discrete HMM**, where the output space is assumed to be discrete and even finite. In this case, the space can be modeled as a finite set \n",
"\n",
"\\begin{equation}\n",
" \\mathcal{B} = \\{\\beta_{1},\\beta_{2},\\ldots,\\beta_{K}\\} \n",
"\\end{equation}\n",
"\n",
"of size $K\\in\\mathbb{N}$ consisting of distinct output elements $\\beta_{k}$, $k\\in[1:K]$, which are also referred to as **observation symbols**. An HMM associates with each state a probability function, which is also referred to as the **emission probability** or **output probability**. In the discrete case, the emission probabilities are specified by coefficients\n",
"\n",
"\\begin{equation}\n",
" b_{ik}\\in[0,1]\n",
"\\end{equation}\n",
"\n",
"for $i\\in[1:I]$ and $k\\in[1:K]$. Each coefficient $b_{ik}$ expresses the probability of the system to output the observation symbol $\\beta_{k}$ when in state $\\alpha_{i}$. Similarly to the state transition probabilities, the emission probabilities are required to satisfy the stochastic constraint $\\sum_{k=1}^{K} \\beta_{ik} = 1$ for $i\\in[1:I]$ (thus forming a probability distribution for each state). The coefficients can be expressed by an $(I\\times K)$ matrix, which we denote by $B$. In summary, an HMM is specified by a tuple\n",
"\n",
"\\begin{equation}\n",
" \\Theta:=(\\mathcal{A},A,C,\\mathcal{B},B).\n",
"\\end{equation}\n",
"\n",
"The sets $\\mathcal{A}$ and $\\mathcal{B}$ are usually considered to be fixed components of the model, while the probability values specified by $A$, $B$, and $C$ are the free parameters to be determined. This can be done explicitly by an expert based on his or her musical knowledge or by employing a learning procedure based on suitably labeled training data. Continuing the above example, the following figure illustrates a hidden Markov model, where the state-dependent emission probabilities are indicated by the labels of the dashed arrows.\n",
"\n",
"<img src=\"../data/C5/FMP_C5_F25.png\" width=\"400px\" align=\"middle\" alt=\"FMP_C5_F25\">"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the following code cell, we define the state transition probability matrix $A$ and the output probability $B$ as specified by the figure. \n",
"\n",
"* Here, we assume that $\\alpha_{1}=\\mathbf{C}$, $\\alpha_{2}=\\mathbf{G}$, and $\\alpha_{3}=\\mathbf{F}$. \n",
"* Furthermore, the elements of the output space $\\mathcal{B} = \\{\\beta_{1},\\beta_{2},\\beta_{3}\\}$ represent the three chroma vectors ordered from left to right. \n",
"* Finally, we assume that the initial state probability vector $C$ is given by the values $c_{1}=0.6$, $c_{2}=0.2$, $c_{3}=0.2$."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from sklearn.preprocessing import normalize \n",
"\n",
"A = np.array([[0.8, 0.1, 0.1], \n",
" [0.2, 0.7, 0.1], \n",
" [0.1, 0.3, 0.6]])\n",
"\n",
"C = np.array([0.6, 0.2, 0.2])\n",
"\n",
"B = np.array([[0.7, 0.0, 0.3], \n",
" [0.1, 0.9, 0.0], \n",
" [0.0, 0.2, 0.8]])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## HMM-Based Sequence Generation \n",
"\n",
"Once an HMM is specified by $\\Theta:=(\\mathcal{A},A,C,\\mathcal{B},B)$, it can be used for various analysis and synthesis applications. Since it is very instructive, we now discuss how to (artificially) generate, on the basis of a given HMM, an observation sequence $O=(o_{1},o_{2},\\ldots,o_{N})$ of length $N\\in\\mathbb{N}$ with $o_n\\in \\mathcal{B}$, $n\\in[1:N]$. The generation procedure is as follows:\n",
"\n",
"1. Set $n=1$ and choose an initial state $s_n=\\alpha_i$ for some $i\\in[1:I]$ according to the initial state distribution $C$.\n",
"2. Generate an observation $o_n=\\beta_k$ for some $k\\in[1:K]$ according to the emission probability in state $s_n=\\alpha_i$ (specified by the $i^{\\mathrm{th}}$ row of $B$).\n",
"3. If $n=N$ then terminate the process. Otherwise, if $n<N$, transit to the new state $s_{n+1}=\\alpha_{j}$ according to the state transition probability at state $s_n=\\alpha_i$ (specified by the $i^{\\mathrm{th}}$ row of $A$). Then increase $n$ by one and return to step 2.\n",
"\n",
"In the next code cell, we implement this procedure and apply it to the example HMM specified above. Note that, due to Python conventions, we start in our implementation with index `0`."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"n = 0, S[0] = 0, O[0] = 0\n",
"n = 1, S[1] = 0, O[1] = 0\n",
"n = 2, S[2] = 0, O[2] = 2\n",
"n = 3, S[3] = 0, O[3] = 0\n",
"n = 4, S[4] = 0, O[4] = 0\n",
"n = 5, S[5] = 0, O[5] = 0\n",
"n = 6, S[6] = 0, O[6] = 2\n",
"n = 7, S[7] = 0, O[7] = 0\n",
"n = 8, S[8] = 0, O[8] = 0\n",
"n = 9, S[9] = 0, O[9] = 0\n",
"State sequence S: [0 0 0 0 0 0 0 0 0 0]\n",
"Observation sequence O: [0 0 2 0 0 0 2 0 0 0]\n"
]
}
],
"source": [
"def generate_sequence_hmm(N, A, C, B, details=False):\n",
" \"\"\"Generate observation and state sequence from given HMM\n",
"\n",
" Notebook: C5/C5S3_HiddenMarkovModel.ipynb\n",
"\n",
" Args:\n",
" N (int): Number of observations to be generated\n",
" A (np.ndarray): State transition probability matrix of dimension I x I\n",
" C (np.ndarray): Initial state distribution of dimension I\n",
" B (np.ndarray): Output probability matrix of dimension I x K\n",
" details (bool): If \"True\" then shows details (Default value = False)\n",
"\n",
" Returns:\n",
" O (np.ndarray): Observation sequence of length N\n",
" S (np.ndarray): State sequence of length N\n",
" \"\"\"\n",
" assert N > 0, \"N should be at least one\"\n",
" I = A.shape[1]\n",
" K = B.shape[1]\n",
" assert I == A.shape[0], \"A should be an I-square matrix\"\n",
" assert I == C.shape[0], \"Dimension of C should be I\"\n",
" assert I == B.shape[0], \"Column-dimension of B should be I\"\n",
"\n",
" O = np.zeros(N, int)\n",
" S = np.zeros(N, int)\n",
" for n in range(N):\n",
" if n == 0:\n",
" i = np.random.choice(np.arange(I), p=C)\n",
" else:\n",
" i = np.random.choice(np.arange(I), p=A[i, :])\n",
" k = np.random.choice(np.arange(K), p=B[i, :])\n",
" S[n] = i\n",
" O[n] = k\n",
" if details:\n",
" print('n = %d, S[%d] = %d, O[%d] = %d' % (n, n, S[n], n, O[n]))\n",
" return O, S\n",
"\n",
"N = 10\n",
"O, S = generate_sequence_hmm(N, A, C, B, details=True)\n",
"print('State sequence S: ', S)\n",
"print('Observation sequence O:', O)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As a sanity check for the plausibility of our sequence generation approach, we now check if the generated sequences reflect well the probabilities of our HMM. To this end, we estimate the original transition probability matrix $A$ and the output probability matrix $B$ from a generated observation sequence $O$ and state sequence $S$.\n",
"\n",
"* To obtain an estimate of the entry $a_{ij}$ of $A$, we count all transitions from $n$ to $n+1$ with $S(n)=\\alpha_i$ and $S(n+1)=\\alpha_j$ and then divide this number by the total number of transitions starting with $\\alpha_i$.\n",
"\n",
"* Similarly, to obtain an estimate of the entry $b_{ik}$ of $B$, we count the number of occurrences $n$ with $S(n)=\\alpha_i$ and $O(n)=\\beta_k$ and divide this number by the total number of occurrences of $\\alpha_i$ in $S$.\n",
"\n",
"When generating longer sequences by increasing the number $N$, the resulting estimates should approach the original values in $A$ and $B$. This is demonstrated by the subsequent experiment. \n",
"\n",
"<div class=\"alert alert-block alert-warning\">\n",
"Note: In practice, when estimating HMM model parameters from training data, only <strong>observation sequences</strong> are typically available, and the state sequences (that reflect the hidden generation process) are generally not known. Learning parameters only from observation sequences leads to much harder estimation problems as discussed below. \n",
"</div> "
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"======== Estimation results when using N = 100 ========\n",
"A =\n",
"[[ 0.800 0.100 0.100]\n",
" [ 0.200 0.700 0.100]\n",
" [ 0.100 0.300 0.600]]\n",
"A_est =\n",
"[[ 0.795 0.091 0.114]\n",
" [ 0.172 0.655 0.172]\n",
" [ 0.154 0.269 0.577]]\n",
"B =\n",
"[[ 0.700 0.000 0.300]\n",
" [ 0.100 0.900 0.000]\n",
" [ 0.000 0.200 0.800]]\n",
"B_est =\n",
"[[ 0.705 0.000 0.295]\n",
" [ 0.167 0.833 0.000]\n",
" [ 0.000 0.423 0.577]]\n",
"======== Estimation results when using N = 10000 ========\n",
"A =\n",
"[[ 0.800 0.100 0.100]\n",
" [ 0.200 0.700 0.100]\n",
" [ 0.100 0.300 0.600]]\n",
"A_est =\n",
"[[ 0.799 0.097 0.104]\n",
" [ 0.198 0.696 0.106]\n",
" [ 0.097 0.306 0.597]]\n",
"B =\n",
"[[ 0.700 0.000 0.300]\n",
" [ 0.100 0.900 0.000]\n",
" [ 0.000 0.200 0.800]]\n",
"B_est =\n",
"[[ 0.708 0.000 0.292]\n",
" [ 0.103 0.897 0.000]\n",
" [ 0.000 0.205 0.795]]\n"
]
}
],
"source": [
"def estimate_hmm_from_o_s(O, S, I, K):\n",
" \"\"\"Estimate the state transition and output probability matrices from\n",
" a given observation and state sequence\n",
"\n",
" Notebook: C5/C5S3_HiddenMarkovModel.ipynb\n",
"\n",
" Args:\n",
" O (np.ndarray): Observation sequence of length N\n",
" S (np.ndarray): State sequence of length N\n",
" I (int): Number of states\n",
" K (int): Number of observation symbols\n",
"\n",
" Returns:\n",
" A_est (np.ndarray): State transition probability matrix of dimension I x I\n",
" B_est (np.ndarray): Output probability matrix of dimension I x K\n",
" \"\"\"\n",
" # Estimate A\n",
" A_est = np.zeros([I, I])\n",
" N = len(S)\n",
" for n in range(N-1):\n",
" i = S[n]\n",
" j = S[n+1]\n",
" A_est[i, j] += 1\n",
" A_est = normalize(A_est, axis=1, norm='l1')\n",
"\n",
" # Estimate B\n",
" B_est = np.zeros([I, K])\n",
" for i in range(I):\n",
" for k in range(K):\n",
" B_est[i, k] = np.sum(np.logical_and(S == i, O == k))\n",
" B_est = normalize(B_est, axis=1, norm='l1')\n",
" return A_est, B_est\n",
"\n",
"N = 100\n",
"print('======== Estimation results when using N = %d ========' % N)\n",
"O, S = generate_sequence_hmm(N, A, C, B, details=False)\n",
"A_est, B_est = estimate_hmm_from_o_s(O, S, A.shape[1], B.shape[1])\n",
"np.set_printoptions(formatter={'float': \"{: 7.3f}\".format})\n",
"print('A =', A, sep='\\n')\n",
"print('A_est =', A_est, sep='\\n')\n",
"print('B =', B, sep='\\n')\n",
"print('B_est =', B_est, sep='\\n')\n",
"\n",
"N = 10000\n",
"print('======== Estimation results when using N = %d ========' % N)\n",
"O, S = generate_sequence_hmm(N, A, C, B, details=False)\n",
"A_est, B_est = estimate_hmm_from_o_s(O, S, A.shape[1], B.shape[1])\n",
"np.set_printoptions(formatter={'float': \"{: 7.3f}\".format})\n",
"print('A =', A, sep='\\n')\n",
"print('A_est =', A_est, sep='\\n')\n",
"print('B =', B, sep='\\n')\n",
"print('B_est =', B_est, sep='\\n')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Three Problems for HMMs\n",
"\n",
"We have seen how a given HMM can be used to generate an observation sequence. We will now look at three famous algorithmic problems for HMMs that concern the specification of the free model parameters and the evaluation of observation sequences. \n",
"\n",
"### 1. Evaluation Problem\n",
"\n",
"The first problem is known as **evaluation problem**. Given an HMM specified by $\\Theta=(\\mathcal{A},A,C,\\mathcal{B},B)$ and an observation sequence $O=(o_{1},o_{2},\\ldots,o_{N})$, the task is to compute the probability \n",
"\n",
"\\begin{equation}\n",
" P[O|\\Theta]\n",
"\\end{equation}\n",
"\n",
"of the observation sequence given the model. From a slightly different viewpoint, this probability can be regarded as a score value that expresses how well a given model matches a given observation sequence. This interpretation becomes useful in the case where one is trying to choose among several competing models. The solution would then be to choose the model which best matches the observation sequence. To compute $P[O|\\Theta]$, we first consider a fixed state sequence $S=(s_1,s_2,\\ldots,s_N)$ of length $N$ with $s_n=\\alpha_{i_n}\\in\\mathcal{A}$ for some suitable $i_n\\in[1:I]$, $n\\in[1:N]$. The probability $P[O,S|\\Theta]$ for generating the state sequence $S$ as well as the observation sequence $O$ is given by \n",
"\n",
"$$\n",
"P[O,S|\\Theta] = c_{i_1}\\cdot b_{i_1k_1} \\cdot a_{i_1i_2}\\cdot b_{i_2k_2} \\cdot ...\\cdot a_{i_{N-1}i_N}\\cdot b_{i_Nk_N}\n",
"$$\n",
"\n",
"Next, to obtain the overall probability $P[O|\\Theta]$, one needs to sum up all these probabilities considering all possible state sequences $S$ of length $|S|=N$:\n",
"\n",
"$$\n",
"P[O|\\Theta] = \\sum_{S: |S|=N}P[O,S|\\Theta]\n",
"= \\sum_{i_1=1}^I \\sum_{i_2=1}^I \\ldots \\sum_{i_N=1}^I\n",
"c_{i_1}\\cdot b_{i_1k_1} \\cdot a_{i_1i_2}\\cdot b_{i_2k_2} \\cdot ...\\cdot a_{i_{N-1}i_N}\\cdot b_{i_Nk_N}\n",
"$$\n",
"\n",
"This leads to $I^N$ summands, a number that is exponential in the length $N$ of the observation sequence. Therefore, in practice, this brute-force calculation is computationally infeasible even for a small $N$. The good news is that there is a more efficient way to compute $P[O|\\Theta]$ using an algorithm that is based on the dynamic programming paradigm. This procedure, which is known as [**Forward&ndash;Backward Algorithm**](https://en.wikipedia.org/wiki/Forward%E2%80%93backward_algorithm), requires a number of operations on the order of $I^2N$ (instead of $I^N$). For a detailed description of this algorithm, we refer to the article by [Rabiner](https://ieeexplore.ieee.org/document/18626).\n",
"\n",
"\n",
"### 2. Uncovering Problem\n",
"\n",
"The second problem is the so-called **uncovering problem**. Again, we are given an HMM specified by $\\Theta=(\\mathcal{A},A,C,\\mathcal{B},B)$ and an observation sequence $O=(o_{1},o_{2},\\ldots,o_{N})$. Instead of finding the overall probability $P[O|\\Theta]$ for $O$, where one needs to consider **all** possible state sequences, the goal of the uncovering problem is to find the **single** state sequence $S=(s_{1},s_{2},\\ldots,s_{N})$ that \"best explains\" the observation sequence. The uncovering problem stated so far is not well defined since, in general, there is not a single \"correct\" state sequence generating the observation sequence. Indeed, one needs a kind of optimization criterion that specifies what is meant when talking about a best possible explanation. There are several reasonable choices for such a criterion, and the actual choice will depend on the intended application. In the [FMP notebook on the Viterbi algorithm](../C5/C5S3_Viterbi.html), we will discuss one possible choice as well as an efficient algorithm (called **Viterbi algorithm**). This algorithm, which can be thought of as a kind of context-sensitive smoothing procedure, will apply in the [FMP notebook on HMM-based chord recognition](../C5/C5S3_ChordRec_HMM.html). \n",
"\n",
"\n",
"### 3. Estimation Problem\n",
"\n",
"Besides the evaluation and uncovering problems, the third basic problem for HMMs is referred to as the **estimation problem**. Given an observation sequence $O$, the objective is to determine the free model parameters of $\\Theta$ (specified by by $A$, $C$, and $B$) that maximize the probability $P[O|\\Theta]$. In other words, the free model parameters are to be estimated so as to best describe the observation sequence. This is a typical instance of an **optimization problem** where a set of observation sequences serves as **training material** for adjusting or learning the HMM parameters. The estimation problem is by far the most difficult problem of HMMs. In fact, there is no known way to explicitly solve the given optimization problem. However, iterative procedures that find locally optimal solutions have been suggested. One of these procedures is known as the [**Baum&ndash;Welch Algorithm**](https://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm). Again, we refer to the article by [Rabiner](https://ieeexplore.ieee.org/document/18626) for more details. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<div class=\"alert\" style=\"background-color:#F5F5F5; border-color:#C8C8C8\">\n",
"<strong>Acknowledgment:</strong> This notebook was created by <a href=\"https://www.audiolabs-erlangen.de/fau/professor/mueller\">Meinard Müller</a>.\n",
"</div> "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<table style=\"border:none\">\n",
"<tr style=\"border:none\">\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C0/C0.html\"><img src=\"../data/C0_nav.png\" style=\"height:50px\" alt=\"C0\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C1/C1.html\"><img src=\"../data/C1_nav.png\" style=\"height:50px\" alt=\"C1\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C2/C2.html\"><img src=\"../data/C2_nav.png\" style=\"height:50px\" alt=\"C2\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C3/C3.html\"><img src=\"../data/C3_nav.png\" style=\"height:50px\" alt=\"C3\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C4/C4.html\"><img src=\"../data/C4_nav.png\" style=\"height:50px\" alt=\"C4\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C5/C5.html\"><img src=\"../data/C5_nav.png\" style=\"height:50px\" alt=\"C5\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C6/C6.html\"><img src=\"../data/C6_nav.png\" style=\"height:50px\" alt=\"C6\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C7/C7.html\"><img src=\"../data/C7_nav.png\" style=\"height:50px\" alt=\"C7\"></a></td>\n",
" <td style=\"min-width:50px; border:none\" bgcolor=\"white\"><a href=\"../C8/C8.html\"><img src=\"../data/C8_nav.png\" style=\"height:50px\" alt=\"C8\"></a></td>\n",
"</tr>\n",
"</table>"
]
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

View File

@@ -1,14 +1,10 @@
# Notebooks
Exploratory material, **not part of the running backend** and not imported by it.
Scratch work, **not part of the running backend** and not imported by it.
`audio_analysis.py` and the three `C5S*` notebooks are chroma/chord-recognition course
work (they still reference stale absolute paths). `effect_debug.ipynb` is scratch work
for tinkering with LED effects.
They need `librosa`, `numba` and `numpy`, which are deliberately not in the backend's
dependencies:
`effect_debug.ipynb` is for tinkering with LED effects. It needs `numpy` and `jupyter`,
which are deliberately not in the backend's dependencies:
```sh
pip install librosa numba numpy jupyter
pip install numpy jupyter
```

View File

@@ -1,128 +0,0 @@
import librosa
from numba import jit
import numpy as np
@jit(nopython=True)
def normalize_feature_sequence(X, norm='2', threshold=0.0001, v=None):
"""Normalizes the columns of a feature sequence
Notebook: C3/C3S1_FeatureNormalization.ipynb
Args:
X (np.ndarray): Feature sequence
norm (str): The norm to be applied. '1', '2', 'max' or 'z' (Default value = '2')
threshold (float): An threshold below which the vector ``v`` used instead of normalization
(Default value = 0.0001)
v (float): Used instead of normalization below ``threshold``. If None, uses unit vector for given norm
(Default value = None)
Returns:
X_norm (np.ndarray): Normalized feature sequence
"""
assert norm in ['1', '2', 'max', 'z']
K, N = X.shape
X_norm = np.zeros((K, N))
if norm == '1':
if v is None:
v = np.ones(K, dtype=np.float64) / K
for n in range(N):
s = np.sum(np.abs(X[:, n]))
if s > threshold:
X_norm[:, n] = X[:, n] / s
else:
X_norm[:, n] = v
if norm == '2':
if v is None:
v = np.ones(K, dtype=np.float64) / np.sqrt(K)
for n in range(N):
s = np.sqrt(np.sum(X[:, n]**2))
if s > threshold:
X_norm[:, n] = X[:, n] / s
else:
X_norm[:, n] = v
if norm == 'max':
if v is None:
v = np.ones(K, dtype=np.float64)
for n in range(N):
s = np.max(np.abs(X[:, n]))
if s > threshold:
X_norm[:, n] = X[:, n] / s
else:
X_norm[:, n] = v
if norm == 'z':
if v is None:
v = np.zeros(K, dtype=np.float64)
for n in range(N):
mu = np.sum(X[:, n]) / K
sigma = np.sqrt(np.sum((X[:, n] - mu)**2) / (K - 1))
if sigma > threshold:
X_norm[:, n] = (X[:, n] - mu) / sigma
else:
X_norm[:, n] = v
return X_norm
def compute_chromagram_from_filename(fn_wav,
Fs=22050,
N=4096,
H=2048,
gamma=None,
version='STFT',
norm='2'):
"""Compute chromagram for WAV file specified by filename
Notebook: C5/C5S2_ChordRec_Templates.ipynb
Args:
fn_wav (str): Filenname of WAV
Fs (scalar): Sampling rate (Default value = 22050)
N (int): Window size (Default value = 4096)
H (int): Hop size (Default value = 2048)
gamma (float): Constant for logarithmic compression (Default value = None)
version (str): Technique used for front-end decomposition ('STFT', 'IIS', 'CQT') (Default value = 'STFT')
norm (str): If not 'None', chroma vectors are normalized by norm as specified ('1', '2', 'max')
(Default value = '2')
Returns:
X (np.ndarray): Chromagram
Fs_X (scalar): Feature reate of chromagram
x (np.ndarray): Audio signal
Fs (scalar): Sampling rate of audio signal
x_dur (float): Duration (seconds) of audio signal
"""
x, Fs = librosa.load(fn_wav, sr=Fs)
x_dur = x.shape[0] / Fs
if version == 'STFT':
# Compute chroma features with STFT
X = librosa.stft(x, n_fft=N, hop_length=H, pad_mode='constant', center=True)
if gamma is not None:
X = np.log(1 + gamma * np.abs(X)**2)
else:
X = np.abs(X)**2
X = librosa.feature.chroma_stft(S=X, sr=Fs, tuning=0, norm=None, hop_length=H, n_fft=N)
if version == 'CQT':
# Compute chroma features with CQT decomposition
X = librosa.feature.chroma_cqt(y=x, sr=Fs, hop_length=H, norm=None)
if version == 'IIR':
# Compute chroma features with filter bank (using IIR elliptic filter)
X = librosa.iirt(y=x, sr=Fs, win_length=N, hop_length=H, center=True, tuning=0.0)
if gamma is not None:
X = np.log(1.0 + gamma * X)
X = librosa.feature.chroma_cqt(C=X,
bins_per_octave=12,
n_octaves=7,
fmin=librosa.midi_to_hz(24),
norm=None)
if norm is not None:
X = normalize_feature_sequence(X, norm='2')
Fs_X = Fs / H
return X, Fs_X, x, Fs, x_dur
if __name__ == "__main__":
test_file = "/home/martin/Music/deemix Music/Simone Sommerland - Ki-Ka-Kinderturnen.mp3"
chroma = compute_chromagram_from_filename(test_file)

View File

@@ -55,7 +55,7 @@ filterwarnings = ["error"]
[tool.ruff]
line-length = 100
target-version = "py311"
# Course material and scratch work, not part of the backend. See notebooks/README.md.
# Scratch work, not part of the backend. See notebooks/README.md.
extend-exclude = ["notebooks"]
[tool.ruff.lint]
@@ -77,3 +77,12 @@ warn_unreachable = true
# checkout must still type-check clean, so it needs the same treatment as librosa.
module = ["vlc", "serial_asyncio", "ruamel.*", "librosa.*", "numpy", "feedparser"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
# Recent numpy ships stubs written in 3.12+ syntax (`type X = ...`). Checking against
# the device's 3.11 makes mypy refuse to parse them and abort before checking anything
# at all - which is what happened in a dev venv that has the analysis extra. Treat the
# analysis stack as untyped instead: it is only touched by librosa_analyzer.py.
module = ["numpy", "numpy.*", "librosa", "librosa.*"]
follow_imports = "skip"
follow_imports_for_stubs = true

View File

@@ -15,6 +15,7 @@ from musicmouse.effects import (
EffectCircularConfig,
EffectRandomTwoColorInterpolationConfig,
EffectReverseSwipe,
EffectSegmentSelect,
EffectStaticConfig,
EffectStaticDetailedConfig,
EffectSwipeAndChange,
@@ -135,7 +136,54 @@ def test_swipe_and_change_is_the_two_payloads_concatenated() -> None:
(EffectRandomTwoColorInterpolationConfig(), 35),
(EffectReverseSwipe(), 12),
(EffectSwipeAndChange(), 64),
(EffectSegmentSelect(colors=(ColorRGBW(0, 0, 0, 0),)), 53),
],
)
def test_payload_sizes(effect: LedEffect, size: int) -> None:
assert len(effect.as_bytes()) == size
def test_segment_select() -> None:
effect = EffectSegmentSelect(
colors=(ColorRGBW(1, 0, 0, 0), ColorRGBW(0, 0, 1, 0)),
selected=1,
previous=0,
gap_leds=2,
offset_degrees=180.0,
direction=-1,
dim_factor=0.5,
wobble_amplitude=1.5,
wobble_period_ms=1000.0,
slide_ms=200.0,
edge_blend=0.25,
)
assert effect.as_bytes().hex() == (
"02" "01" "00" "02" # count, selected, previous, gap
"ff000000" "0000ff00" + "00000000" * 4 # always six colours on the wire
+ "00003443" # offset 180.0
"ff" # direction -1
"0000003f" # dim 0.5
"0000c03f" # wobble 1.5
"00007a44" # period 1000.0
"00004843" # slide 200.0
"0000803e" # edge blend 0.25
)
def test_segment_select_without_previous_does_not_glide() -> None:
effect = EffectSegmentSelect(colors=(ColorRGBW(1, 0, 0, 0),) * 3, selected=2)
assert effect.as_bytes()[:3] == bytes([3, 2, 2])
def test_segment_select_round_trips() -> None:
# Every float exact in float32, as the firmware has them.
effect = EffectSegmentSelect(
colors=(ColorRGBW(1, 0, 0, 0), ColorRGBW(0, 1, 0, 0)), previous=1, dim_factor=0.25
)
assert EffectSegmentSelect.from_bytes(effect.as_bytes()) == effect
@pytest.mark.parametrize("count", [0, 7])
def test_segment_select_takes_one_to_six_sections(count: int) -> None:
with pytest.raises(ValueError, match="between 1 and 6"):
EffectSegmentSelect(colors=(ColorRGBW(1, 0, 0, 0),) * count)

View File

@@ -15,7 +15,14 @@ from pathlib import Path
import pytest
from musicmouse.config import DEFAULT_AUDIO_EXTENSIONS, load_config
from musicmouse.library import Album, Analyzer, MusicLibrary, default_worker_count
from musicmouse.effects import SEGMENT_SELECT_MAX_SEGMENTS
from musicmouse.library import (
MAX_FIGURE_ALBUMS,
Album,
Analyzer,
MusicLibrary,
default_worker_count,
)
from musicmouse.library.analysis import ANALYZER_VERSION, BeatGrid, TrackAnalysis, TrackCurves
from musicmouse.library.cache import LibraryCache
from musicmouse.library.colors import colors_from_id
@@ -92,15 +99,72 @@ async def test_figure_playlists_are_keyed_by_figure_name(config_dir: Path) -> No
playlists = library.figure_playlists()
assert set(playlists) == {"fuchs", "eule"}
assert len(playlists["fuchs"]) == 3
assert [track.path.name for track in playlists["fuchs"].tracks] == [
(fuchs,) = playlists["fuchs"]
assert len(fuchs) == 3
assert [track.path.name for track in fuchs.tracks] == [
"00 - track.mp3",
"01 - track.mp3",
"02 - track.mp3",
]
# The playlist carries the album it came from, which is how a front-end answers
# "what is playing?" without keeping its own copy of that state.
assert playlists["fuchs"].album_id == album_named(library, "Fuchs").id
assert fuchs.album_id == album_named(library, "Fuchs").id
def _figure_with_albums(config_dir: Path, figure: str, *folders: str) -> None:
"""Replace a figure's audio with one album per folder (relative to the figure)."""
figure_folder = config_dir / "music" / "Figuren" / figure
for track in figure_folder.glob("*.mp3"):
track.unlink()
for folder in folders:
write_track(figure_folder / folder / "01 - track.mp3", title=f"In {folder}")
async def test_a_figure_with_album_subfolders_offers_each_in_folder_order(
config_dir: Path,
) -> None:
# Depth-first and alphabetical, however deep an album sits.
_figure_with_albums(
config_dir, "fuchs", "B Lieder", "A Geschichten/Teil 2", "A Geschichten/Teil 1"
)
library = await build(config_dir)
choices = library.figure_playlists()["fuchs"]
albums = [library.get(playlist.album_id) for playlist in choices]
assert [album.title if album else None for album in albums] == [
"Fuchs - Teil 1",
"Fuchs - Teil 2",
"Fuchs - B Lieder",
]
assert all(album is not None and album.figure == "fuchs" for album in albums)
async def test_a_figure_album_subfolder_takes_the_figures_kind(config_dir: Path) -> None:
_figure_with_albums(config_dir, "eule", "Folge 1", "Folge 2")
library = await build(config_dir)
albums = [library.get(p.album_id) for p in library.figure_playlists()["eule"]]
assert len(albums) == 2
assert {album.kind if album else None for album in albums} == {"book"}
async def test_a_figure_offers_no_more_albums_than_the_ring_has_room_for(
config_dir: Path, caplog: pytest.LogCaptureFixture
) -> None:
_figure_with_albums(config_dir, "fuchs", *(f"Album {n}" for n in range(MAX_FIGURE_ALBUMS + 2)))
library = await build(config_dir)
choices = library.figure_playlists()["fuchs"]
assert len(choices) == MAX_FIGURE_ALBUMS
assert "only the first 6" in caplog.text
# Browsing still shows every one of them.
assert sum(album.figure == "fuchs" for album in library.albums) == MAX_FIGURE_ALBUMS + 2
def test_the_album_cap_is_the_rings_section_count() -> None:
assert MAX_FIGURE_ALBUMS == SEGMENT_SELECT_MAX_SEGMENTS
async def test_music_takes_its_title_and_artist_from_the_tags(config_dir: Path) -> None:
@@ -852,46 +916,3 @@ class TestCoverDownscaling:
assert thumb.format == "JPEG"
assert "progressive" not in thumb.info
assert abs(thumb.width / thumb.height - 2400 / 1800) < 0.02
def test_thumbnails_are_backfilled_for_covers_stored_without_one(
self, tmp_path: Path
) -> None:
from PIL import Image
from musicmouse.library.cache import THUMB_COVER_PX, LibraryCache
cache = LibraryCache(tmp_path)
cache.prepare()
cache.cover_path("old").write_bytes(self._jpeg((384, 384)))
assert not cache.thumb_path("old").exists()
assert cache.shrink_stored_covers() == 0 # nothing oversized...
with Image.open(cache.thumb_path("old")) as thumb: # ...but the thumb appeared
assert max(thumb.size) == THUMB_COVER_PX
# Steady state: the thumbnail is not regenerated, and is not mistaken for a cover.
before = cache.thumb_path("old").stat().st_mtime_ns
assert cache.shrink_stored_covers() == 0
assert cache.thumb_path("old").stat().st_mtime_ns == before
assert not cache.thumb_path("old.thumb").exists()
def test_covers_written_by_an_older_version_are_migrated(self, tmp_path: Path) -> None:
"""The scanner reuses an unchanged album without re-reading its tags, so a
cover stored before the limit existed would otherwise never be rewritten."""
from PIL import Image
from musicmouse.library.cache import MAX_COVER_PX, LibraryCache
cache = LibraryCache(tmp_path)
cache.prepare()
stale = cache.cover_path("old")
stale.write_bytes(self._jpeg((3000, 3000)))
fresh = cache.cover_path("new")
fresh.write_bytes(self._jpeg((320, 320)))
fresh_before = fresh.read_bytes()
assert cache.shrink_stored_covers() == 1
with Image.open(stale) as image:
assert max(image.size) == MAX_COVER_PX
assert fresh.read_bytes() == fresh_before
# Second pass has nothing left to do - the cheap steady state.
assert cache.shrink_stored_covers() == 0

View File

@@ -14,6 +14,7 @@ from musicmouse.effects import (
OFF,
EffectAlexaSwipeConfig,
EffectCircularConfig,
EffectSegmentSelect,
EffectStaticConfig,
)
from musicmouse.events import (
@@ -23,9 +24,12 @@ from musicmouse.events import (
Event,
LedEffectChanged,
RfidTokenRead,
RotaryTurned,
SelectorPressed,
SelectorTurned,
TouchButtonPressed,
)
from musicmouse.hardware import Button, ButtonAction, LedZone, TouchButton
from musicmouse.hardware import Button, ButtonAction, LedZone, RotaryDirection, TouchButton
from musicmouse.simulator import FakeTransport
FUCHS = bytes.fromhex("04a1b2c3d4")
@@ -82,6 +86,56 @@ async def test_button_press_reaches_the_bus(
]
def _turn(increment: int, direction: RotaryDirection) -> bytes:
return encode_input_event(RotaryTurned(position=0, increment=increment, direction=direction))
def _knob(action: ButtonAction) -> bytes:
return encode_input_event(ButtonEvent(button=Button.ROTARY, action=action))
async def test_a_captured_knob_turns_the_selector_not_the_volume(
bus: EventBus, device: MusicMouseDevice, seen: list[Event]
) -> None:
device.capture_knob(True)
device.feed(_turn(2, RotaryDirection.UP))
device.feed(_turn(1, RotaryDirection.DOWN))
# The first turn after boot reports no increment, but it was still a detent.
device.feed(_turn(0, RotaryDirection.UP))
await bus.drain()
assert [e.steps for e in only(seen, SelectorTurned)] == [2, -1, 1]
assert not only(seen, RotaryTurned)
async def test_a_captured_knob_press_is_the_selectors_alone(
bus: EventBus, device: MusicMouseDevice, seen: list[Event]
) -> None:
device.capture_knob(True)
for action in (ButtonAction.PRESSED, ButtonAction.RELEASED):
device.feed(_knob(action))
# The other buttons are not captured.
device.feed(encode_input_event(ButtonEvent(button=Button.LEFT, action=ButtonAction.PRESSED)))
await bus.drain()
assert len(only(seen, SelectorPressed)) == 1
assert [e.button for e in only(seen, ButtonEvent)] == [Button.LEFT]
async def test_a_released_knob_is_a_volume_knob_again(
bus: EventBus, device: MusicMouseDevice, seen: list[Event]
) -> None:
device.capture_knob(True)
device.capture_knob(False)
device.feed(_turn(1, RotaryDirection.UP))
device.feed(_knob(ButtonAction.PRESSED))
await bus.drain()
assert len(only(seen, RotaryTurned)) == 1
assert len(only(seen, ButtonEvent)) == 1
assert not only(seen, SelectorTurned) and not only(seen, SelectorPressed)
async def test_touch_press_reaches_the_bus(
bus: EventBus, device: MusicMouseDevice, seen: list[Event]
) -> None:
@@ -255,6 +309,19 @@ async def test_an_effect_a_zone_does_not_support_is_reported_not_sent(
assert "cannot be sent to the mouse LEDs" in caplog.text
async def test_only_the_ring_can_be_a_selector(
bus: EventBus, device: MusicMouseDevice, transport: FakeTransport
) -> None:
effect = EffectSegmentSelect(colors=(ColorRGBW(1, 0, 0, 0), ColorRGBW(0, 1, 0, 0)))
device.set_effect(LedZone.RING, effect)
device.set_effect(LedZone.SHELF, effect)
await bus.drain()
sent = transport.effect(LedZone.RING)
assert sent is not None and sent.as_bytes() == effect.as_bytes()
assert transport.effect(LedZone.SHELF) is None
async def test_button_brightness_sets_both_backlights(
device: MusicMouseDevice, transport: FakeTransport
) -> None:

View File

@@ -126,6 +126,22 @@ async def test_putting_the_same_figure_back_resumes_where_it_left_off(
mouse.check("track", "1")
async def test_resuming_survives_a_library_rescan(
mouse: SimulatorDriver, sim: Simulation
) -> None:
"""A rescan rebuilds every Playlist object; resuming must not depend on identity."""
await mouse.place("fuchs")
await mouse.wait(TRACK_SECONDS + 1)
mouse.check("track", "1")
await mouse.remove()
await sim.app.rescan_library()
await mouse.place("fuchs")
mouse.check("playing", "true")
mouse.check("track", "1")
async def test_a_different_figure_starts_from_the_top(mouse: SimulatorDriver) -> None:
await mouse.place("fuchs")
await mouse.wait(TRACK_SECONDS + 1)

View File

@@ -65,13 +65,6 @@ async def album_id(sim: Simulation, title: str) -> str:
return next(a.id for a in sim.app.library.albums if a.title == title)
# ------------------------------------------------------------------------------ lirc
async def test_lirc_is_a_404_when_unconfigured(client: httpx2.AsyncClient) -> None:
assert (await client.get("/api/lirc")).status_code == 404
# --------------------------------------------------------------------------- reading

View File

@@ -0,0 +1,371 @@
"""Picking one of a figure's albums on the LED ring, driven through the simulator."""
from __future__ import annotations
from collections.abc import AsyncIterator
from copy import deepcopy
from pathlib import Path
from typing import Any, TypeVar
import pytest
from musicmouse.blubb import confirm_wav, tick_wav
from musicmouse.config import load_config
from musicmouse.effects import (
EffectRandomTwoColorInterpolationConfig,
EffectReverseSwipe,
EffectSegmentSelect,
EffectSwipeAndChange,
)
from musicmouse.events import ButtonEvent, Event, FigureSelectionChanged, RotaryTurned
from musicmouse.hardware import LedZone
from musicmouse.reactions import selection
from musicmouse.reactions.selection import CONFIRM_GAIN, DEFAULTS, PALETTE, TICK_GAIN
from musicmouse.simulator.driver import SimulatorDriver
from musicmouse.simulator.harness import Simulation, build_simulation
from musicmouse.sound import ui_level
from tests.conftest import VALID_CONFIG, write_config, write_track
TRACK_SECONDS = 10.0
TIMEOUT = DEFAULTS.timeout
T = TypeVar("T", bound=Event)
def _albums(config_dir: Path) -> None:
"""Give the fox three albums of two tracks each; the owl keeps its one."""
fuchs = config_dir / "music" / "Figuren" / "fuchs"
for track in fuchs.glob("*.mp3"):
track.unlink()
for folder in ("A", "B", "C"):
for index in range(2):
write_track(fuchs / folder / f"{index:02d}.mp3", title=f"{folder}{index}")
async def _simulation(config_dir: Path, config: dict[str, Any]) -> Simulation:
_albums(config_dir)
return await build_simulation(
load_config(write_config(config_dir, config)), track_duration=TRACK_SECONDS
)
@pytest.fixture
async def sim(config_dir: Path) -> AsyncIterator[Simulation]:
simulation = await _simulation(config_dir, VALID_CONFIG)
try:
yield simulation
finally:
await simulation.aclose()
@pytest.fixture
def mouse(sim: Simulation) -> SimulatorDriver:
return sim.driver
@pytest.fixture
def seen(sim: Simulation) -> list[Event]:
events: list[Event] = []
sim.bus.subscribe_all(events.append)
return events
def only(events: list[Event], event_type: type[T]) -> list[T]:
return [e for e in events if isinstance(e, event_type)]
def ring(sim: Simulation) -> object:
return sim.app.mouse.effect(LedZone.RING)
def picker(sim: Simulation) -> EffectSegmentSelect:
effect = ring(sim)
assert isinstance(effect, EffectSegmentSelect)
return effect
# ------------------------------------------------------------------------- opening
async def test_a_figure_with_several_albums_plays_the_first_and_offers_all(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.place("fuchs")
mouse.check("playing", "true")
mouse.check("album", "Fuchs - A")
effect = picker(sim)
assert effect.colors == PALETTE[:3]
assert effect.selected == 0
# Only the ring is the picker; the body and shelf run the usual start animation.
assert isinstance(sim.app.mouse.effect(LedZone.MOUSE), EffectSwipeAndChange)
assert isinstance(sim.app.mouse.effect(LedZone.SHELF), EffectSwipeAndChange)
async def test_a_figure_with_one_album_gets_no_picker(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.place("eule")
assert isinstance(ring(sim), EffectSwipeAndChange)
assert sim.app.state.selector is None
assert not sim.app.mouse.knob_captured
async def test_the_config_can_override_how_the_picker_looks(config_dir: Path) -> None:
config = deepcopy(VALID_CONFIG)
config["general"]["figure_selection"] = {"offset_degrees": 90, "wobble_amplitude": 3}
sim = await _simulation(config_dir, config)
try:
await sim.driver.place("fuchs")
effect = picker(sim)
assert effect.offset_degrees == 90
assert effect.wobble_amplitude == 3
assert effect.slide_ms == DEFAULTS.slide_ms
finally:
await sim.aclose()
# --------------------------------------------------------------------------- knob
async def test_turning_moves_the_selection_instead_of_the_volume(
mouse: SimulatorDriver, sim: Simulation, seen: list[Event]
) -> None:
await mouse.place("fuchs")
volume = sim.player.volume
await mouse.turn(1)
effect = picker(sim)
assert (effect.previous, effect.selected) == (0, 1)
assert sim.player.volume == volume
assert not only(seen, RotaryTurned)
assert only(seen, FigureSelectionChanged)[-1].selected == 1
# Turning only moves the highlight; what plays changes on a press.
mouse.check("album", "Fuchs - A")
async def test_turning_wraps_round_the_ring(mouse: SimulatorDriver, sim: Simulation) -> None:
await mouse.place("fuchs")
await mouse.turn(-1)
assert picker(sim).selected == 2
await mouse.turn(2)
assert picker(sim).selected == 1
async def test_pressing_plays_the_selected_album_and_gives_the_knob_back(
mouse: SimulatorDriver, sim: Simulation, seen: list[Event]
) -> None:
await mouse.place("fuchs")
await mouse.turn(2)
await mouse.press("rotary")
mouse.check("album", "Fuchs - C")
mouse.check("track", "0")
mouse.check("playing", "true")
effect = ring(sim)
assert isinstance(effect, EffectRandomTwoColorInterpolationConfig)
assert effect.start_with_existing # a fade from the sections, not a jump
# The press was the picker's: Home Assistant never hears of it.
assert not only(seen, ButtonEvent)
assert only(seen, FigureSelectionChanged)[-1].open is False
volume = sim.player.volume
await mouse.turn(-1)
assert sim.player.volume < volume
async def test_pressing_on_the_album_already_playing_does_not_restart_it(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.place("fuchs")
await mouse.wait(3)
await mouse.press("rotary")
mouse.check("album", "Fuchs - A")
assert sim.player.position == pytest.approx(3.0)
assert sim.app.state.selector is None
async def test_presses_reach_home_assistant_again_once_the_picker_is_closed(
mouse: SimulatorDriver, seen: list[Event]
) -> None:
await mouse.place("fuchs")
await mouse.press("rotary")
await mouse.press("rotary")
assert len(only(seen, ButtonEvent)) == 1
# ------------------------------------------------------------------------ timeout
async def test_left_alone_the_picker_closes_and_the_music_carries_on(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.place("fuchs")
await mouse.turn(1)
await mouse.wait(TIMEOUT + 0.1)
assert sim.app.state.selector is None
assert not sim.app.mouse.knob_captured
assert isinstance(ring(sim), EffectRandomTwoColorInterpolationConfig)
mouse.check("album", "Fuchs - A")
mouse.check("playing", "true")
async def test_every_turn_starts_the_timeout_over(mouse: SimulatorDriver, sim: Simulation) -> None:
await mouse.place("fuchs")
await mouse.wait(TIMEOUT - 2)
await mouse.turn(1)
await mouse.wait(TIMEOUT - 1)
assert sim.app.state.selector is not None
await mouse.wait(1.1)
assert sim.app.state.selector is None
async def test_lifting_the_figure_closes_the_picker(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.place("fuchs")
await mouse.remove()
assert sim.app.state.selector is None
assert not sim.app.mouse.knob_captured
assert isinstance(ring(sim), EffectReverseSwipe)
# ------------------------------------------------------------------------- memory
async def test_putting_the_figure_back_resumes_the_picked_album(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.place("fuchs")
await mouse.turn(1)
await mouse.press("rotary")
await mouse.wait(3)
await mouse.remove()
await mouse.place("fuchs")
mouse.check("album", "Fuchs - B")
assert sim.player.position == pytest.approx(3.0)
assert picker(sim).selected == 1
async def test_another_figure_in_between_keeps_the_album_but_not_the_place(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.place("fuchs")
await mouse.turn(1)
await mouse.press("rotary")
await mouse.wait(TRACK_SECONDS + 3)
await mouse.remove()
await mouse.place("eule")
await mouse.remove()
await mouse.place("fuchs")
mouse.check("album", "Fuchs - B")
mouse.check("track", "0")
assert sim.player.position == pytest.approx(0.0)
assert picker(sim).selected == 1
async def test_an_album_played_to_the_end_is_still_the_one_picked_next_time(
mouse: SimulatorDriver,
) -> None:
await mouse.place("fuchs")
await mouse.turn(1)
await mouse.press("rotary")
await mouse.wait(2 * TRACK_SECONDS + 1)
mouse.check("playing", "false")
await mouse.remove()
await mouse.place("fuchs")
mouse.check("album", "Fuchs - B")
mouse.check("track", "0")
async def test_an_album_started_from_the_web_becomes_the_figures_pick(
mouse: SimulatorDriver, sim: Simulation
) -> None:
await mouse.play_album("Fuchs - C")
await mouse.wait(2)
await mouse.place("fuchs")
mouse.check("album", "Fuchs - C")
assert picker(sim).selected == 2
# -------------------------------------------------------------------------- sound
@pytest.fixture
def sounds(monkeypatch: pytest.MonkeyPatch) -> list[tuple[str, bytes]]:
played: list[tuple[str, bytes]] = []
monkeypatch.setattr(
selection, "play_wav_soon", lambda device, wav: played.append((device, wav))
)
return played
async def _audible(config_dir: Path, effects_volume: int = 100) -> Simulation:
config = deepcopy(VALID_CONFIG)
config["general"]["alsa_device"] = "softvol_effects"
config["general"]["effects_volume"] = effects_volume
return await _simulation(config_dir, config)
async def test_each_step_ticks_but_fast_turns_do_not_pile_up(
config_dir: Path, sounds: list[tuple[str, bytes]]
) -> None:
sim = await _audible(config_dir)
try:
await sim.driver.place("fuchs")
await sim.driver.turn(1)
await sim.driver.turn(1) # no time passed: one tick for both
await sim.driver.wait(0.1)
await sim.driver.turn(1)
level = ui_level(sim.player.volume / 100, 1.0)
assert sounds == [("softvol_effects", tick_wav(gain=TICK_GAIN * level))] * 2
finally:
await sim.aclose()
async def test_picking_an_album_plays_the_confirm_sound(
config_dir: Path, sounds: list[tuple[str, bytes]]
) -> None:
sim = await _audible(config_dir, effects_volume=50)
try:
await sim.driver.place("fuchs")
await sim.driver.press("rotary")
level = ui_level(sim.player.volume / 100, 0.5)
assert sounds == [("softvol_effects", confirm_wav(gain=CONFIRM_GAIN * level))]
finally:
await sim.aclose()
async def test_no_picker_sounds_with_effects_turned_off(
config_dir: Path, sounds: list[tuple[str, bytes]]
) -> None:
sim = await _audible(config_dir, effects_volume=0)
try:
await sim.driver.place("fuchs")
await sim.driver.turn(1)
await sim.driver.press("rotary")
assert sounds == []
finally:
await sim.aclose()

View File

@@ -20,6 +20,7 @@ SETTINGS = {
"initial_volume": 40,
"volume_increment": 5,
"button_leds_brightness": 0.5,
"effects_volume": 100,
}
#: A config file with comments, to prove saving does not flatten them.

View File

@@ -0,0 +1,47 @@
"""How loud UI sounds are, and the album picker's two sounds."""
from __future__ import annotations
import io
import wave
from collections.abc import Callable
import pytest
from musicmouse.blubb import RATE, confirm_wav, tick_wav
from musicmouse.sound import MIN_PLAYER_VOLUME, ui_level
def test_ui_sounds_follow_the_music_volume() -> None:
assert ui_level(0.8, 1.0) == pytest.approx(0.8)
assert ui_level(1.0, 0.5) == pytest.approx(0.5)
def test_ui_sounds_never_drop_below_the_minimum_player_volume() -> None:
# A muted player must not mute the timer, nor make the album picker silent.
assert ui_level(0.0, 1.0) == pytest.approx(MIN_PLAYER_VOLUME)
assert ui_level(0.0, 0.5) == pytest.approx(MIN_PLAYER_VOLUME * 0.5)
def test_parents_can_still_turn_ui_sounds_off() -> None:
assert ui_level(0.8, 0.0) == 0.0
@pytest.mark.parametrize(("make", "longest"), [(tick_wav, 0.05), (confirm_wav, 0.4)])
def test_the_picker_sounds_are_short_mono_wavs(make: Callable[..., bytes], longest: float) -> None:
with wave.open(io.BytesIO(make(gain=0.5))) as wav:
assert (wav.getnchannels(), wav.getsampwidth(), wav.getframerate()) == (1, 2, RATE)
assert 0 < wav.getnframes() / RATE < longest
def test_the_gain_sets_the_peak() -> None:
def peak(data: bytes) -> int:
with wave.open(io.BytesIO(data)) as wav:
frames = wav.readframes(wav.getnframes())
return max(
abs(int.from_bytes(frames[i : i + 2], "little", signed=True))
for i in range(0, len(frames), 2)
)
assert peak(tick_wav(gain=1.0)) == pytest.approx(32767, abs=1)
assert peak(tick_wav(gain=0.25)) == pytest.approx(32767 / 4, abs=1)

View File

@@ -21,15 +21,20 @@ from fastapi import FastAPI
from musicmouse.app import ParentTimer
from musicmouse.config import WebConfig, load_config
from musicmouse.effects import OFF
from musicmouse.events import (
CancelTimerRequested,
ClockStateChanged,
LedEffectChanged,
SetTimerRequested,
StateEvent,
ToggleClockRequested,
ToggleScreenRequested,
)
from musicmouse.hardware import LedZone
from musicmouse.library.analysis import BeatGrid, TrackCurves
from musicmouse.library.models import track_key
from musicmouse.services.web.hub import IGNORED_EVENTS, SNAPSHOT_EVENTS
from musicmouse.services.web.service import build_app
from musicmouse.simulator.harness import Simulation, build_simulation
from tests.conftest import VALID_CONFIG, write_config
@@ -199,15 +204,15 @@ async def test_play_can_start_at_a_track(client: Client, sim: Simulation) -> Non
assert (await client.get("/api/state")).json()["track_index"] == 2
async def test_playing_a_figure_album_reuses_the_figure_playlist(
async def test_playing_a_figure_album_plays_the_figure_playlist(
client: Client, sim: Simulation
) -> None:
"""Identity matters: ``play_figure`` resumes on an ``is`` check."""
"""Equal to the figure's own playlist, so placing the figure afterwards resumes it."""
album = await album_by_title(client, "Fuchs")
await client.post("/api/play", json={"album_id": album["id"]})
await sim.bus.drain()
assert sim.player.playlist is sim.app.playlists["fuchs"]
assert sim.player.playlist == sim.app.playlists["fuchs"][0]
async def test_playing_an_unknown_album_is_a_404(client: Client) -> None:
@@ -283,24 +288,6 @@ async def test_volume_scales_across_the_allowed_range(client: Client, sim: Simul
assert (await client.get("/api/state")).json()["volume"] == 50
async def test_volume_steps_are_relative_to_the_percentage(client: Client, sim: Simulation) -> None:
await client.post("/api/volume", json={"percent": 50})
await sim.bus.drain()
await client.post("/api/volume", json={"delta_percent": 10})
await sim.bus.drain()
assert (await client.get("/api/state")).json()["volume"] == 60
assert sim.player.volume == 36
async def test_volume_steps_clamp_at_the_ends(client: Client, sim: Simulation) -> None:
await client.post("/api/volume", json={"percent": 95})
await sim.bus.drain()
await client.post("/api/volume", json={"delta_percent": 20})
await sim.bus.drain()
assert (await client.get("/api/state")).json()["volume"] == 100
async def test_remote_steps_move_by_the_volume_increment(client: Client, sim: Simulation) -> None:
"""The +/- buttons take the IR remote's step, in device units, not a percentage."""
await client.post("/api/volume", json={"percent": 50})
@@ -442,6 +429,30 @@ async def test_a_state_change_reaches_every_client(api: FastAPI, sim: Simulation
assert message["state"]["active_figure"] == "fuchs"
async def test_led_writes_are_not_pushed_to_the_clients(api: FastAPI, sim: Simulation) -> None:
"""The snapshot has no LED state, so an LED write is no reason to send one."""
async with websocket_connect(api, "/api/ws") as socket:
await socket.next_json()
await sim.bus.emit_and_wait(
LedEffectChanged(zone=LedZone.RING, effect=OFF(), origin="mqtt")
)
assert socket.from_app.empty()
def test_every_state_event_is_either_pushed_or_deliberately_ignored() -> None:
"""A new StateEvent has to be sorted into SNAPSHOT_EVENTS or IGNORED_EVENTS."""
def subclasses(kind: type) -> set[type]:
found = set(kind.__subclasses__())
for sub in list(found):
found |= subclasses(sub)
return found
assert subclasses(StateEvent) == set(SNAPSHOT_EVENTS) | set(IGNORED_EVENTS)
async def test_switching_albums_while_playing_reaches_every_client(
api: FastAPI, client: Client, sim: Simulation
) -> None:

44
scripts/check.sh Executable file
View File

@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# Every check this repo has, in one command. There is no CI, so this is the gate:
# run it before committing, and treat a checker that cannot run as a failure.
#
# scripts/check.sh backend and web
# scripts/check.sh backend just one half
# scripts/check.sh web
#
# PYTHON picks the interpreter (default: python-backend/.venv/bin/python, else python3).
set -euo pipefail
repo="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
what="${1:-all}"
backend() {
cd "$repo/python-backend"
local python="${PYTHON:-}"
if [[ -z "$python" ]]; then
if [[ -x .venv/bin/python ]]; then python=.venv/bin/python; else python=python3; fi
fi
echo "== backend: pytest"
"$python" -m pytest -q
echo "== backend: ruff"
"$python" -m ruff check .
echo "== backend: mypy"
"$python" -m mypy
}
web() {
cd "$repo/web"
echo "== web: vitest"
npm run --silent test
echo "== web: tsc + eslint"
npm run --silent check
}
case "$what" in
all) backend; web ;;
backend) backend ;;
web) web ;;
*) echo "usage: $0 [all|backend|web]" >&2; exit 2 ;;
esac
echo "== all checks passed"

View File

@@ -1 +0,0 @@
target

5908
slint-frontend/Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,65 +0,0 @@
[package]
name = "musicmouse-slint"
version = "0.1.0"
edition = "2021"
description = "Native Slint front-end for the MusicMouse player"
[dependencies]
# default-features off so the renderer is an explicit choice below, never an
# accident: on a Pi 5 (Mesa V3D) the default FemtoVG renderer draws every runtime
# loaded Image as solid black - see slint-ui/slint#11785 - which for this app means
# every album cover. The `skia` feature is the device's way out.
slint = { version = "1.15", default-features = false, features = [
"compat-1-2",
"std",
] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# Blocking HTTP and blocking websockets, each on its own thread, rather than an async
# runtime. The whole client makes one library request, one websocket connection and a
# trickle of covers; tokio would be more machinery than the job has work.
ureq = { version = "2.12", default-features = false, features = ["json", "gzip"] }
tungstenite = { version = "0.24", default-features = false, features = ["handshake"] }
image = { version = "0.25", default-features = false, features = ["jpeg", "png"] }
dirs = "5"
# Only for `normalize()`: folding "Grüffelo" to "gruffelo" needs a real NFD pass, which
# is what the web client gets from String.prototype.normalize for free.
unicode-normalization = "0.1"
[build-dependencies]
slint-build = "1.15"
[features]
default = ["desktop"]
# Development on a normal desktop session. FemtoVG is fine on x86/Mesa and builds in
# seconds, which is what makes it the right default *here* and the wrong one on the Pi.
# `renderer-software` rides along so `SLINT_BACKEND=winit-software` works without a
# rebuild: it is how the UI can be rendered and screenshotted on a machine with no GPU
# (or no session), and it is the fallback the Pi 5 FemtoVG bug leaves standing.
desktop = ["slint/backend-winit", "slint/renderer-femtovg", "slint/renderer-software"]
# Same window, Skia instead - the combination the device runs, so visual checks here
# match what ships. Costs a long first build while rust-skia bootstraps.
skia = ["slint/backend-winit", "slint/renderer-skia"]
# The device: straight onto KMS/DRM with no X, no Wayland and no compositor, input
# via libinput. (`backend-linuxkms-noseat` is deprecated upstream in favour of this.)
#
# Build it with `--no-default-features --features kiosk`: cargo features are additive,
# so leaving `desktop` enabled links winit and linuxkms into the same binary.
kiosk = ["slint/backend-linuxkms-libinput", "slint/renderer-skia"]
[profile.release]
opt-level = 3
lto = "thin"
codegen-units = 1
panic = "abort"
# The UI is laid out and hit-tested in debug builds too; an unoptimised image resize
# on a 343-album first run is otherwise the slowest thing in the app by far.
[profile.dev.package.image]
opt-level = 3
[profile.dev.package.slint]
opt-level = 2

View File

@@ -1,192 +0,0 @@
# Musik Delfin — the Slint front-end
A native front-end for the music player, next to `../web/`. Same backend, same layout,
same German copy; no browser. It exists because the React UI is sluggish on the Pi 5
kiosk, and the things making it sluggish are browser-shaped: a compositor deciding what
gets its own layer, a `requestAnimationFrame` loop that repaints the viewport, and a
JPEG decode per album card per cold start.
The music player and the room panel ("Mein Zimmer"). The typing game (`Tippen`), parent
mode and the IR-remote assignment are still web-only.
## Running it
Start the backend, then:
```sh
nix-shell # cargo, the GL/X11/Wayland libs, and Nunito
cargo run # windowed, FemtoVG
cargo run -- --server http://musicdolphin:8080
cargo run -- --page room # open on the room panel rather than the player
```
Checks:
```sh
cargo test && cargo clippy --all-targets && cargo fmt --check
```
`tools/headless-shots.sh` renders the UI and screenshots it inside a nested headless
compositor, for checking a layout change on a machine with no display (or a locked one).
A Slint window needs a real compositor - under bare Xvfb nothing maps at all.
## Build profiles
| | Backend | Renderer | For |
|---|---|---|---|
| `cargo run` | winit | FemtoVG | development here |
| `cargo run --features skia` | winit | Skia | checking what the device will show |
| `cargo build --release --no-default-features --features kiosk` | linuxkms + libinput | Skia | the Pi |
**The device must not use FemtoVG.** On a Pi 5 (Mesa V3D), FemtoVG draws every
runtime-loaded `Image` as solid black - [slint#11785][bug], open as of this writing -
which for this app means every album cover. Skia and the software renderer are both
unaffected. FemtoVG stays the default *here* because it builds in seconds and is fine on
desktop Mesa; `kiosk` is the profile that ships.
`--no-default-features` is not optional on that last one: the features are additive, so
leaving `desktop` on would link winit *and* linuxkms into the same binary.
`kiosk` takes the framebuffer directly through KMS/DRM: no X, no Wayland, no compositor.
(`backend-linuxkms-noseat` is deprecated upstream in favour of the libinput variant used
here.) `SLINT_BACKEND=winit-software` forces the software renderer without a rebuild.
[bug]: https://github.com/slint-ui/slint/issues/11785
## Where the speed comes from
Four things, in rough order of how much they matter on the device:
**Covers are downscaled once, ever.** The backend serves one size - 640 px on the long
edge - and has no `?size=`. A grid card is 180 px. `src/covers.rs` fetches each cover
once on a worker thread, resizes it to the tier that was asked for, and writes a JPEG
thumbnail to `~/.cache/musicmouse-slint/covers/<album-id>@<px>.jpg`. Every later start
reads that instead. There are two tiers and only two, which is what makes the cache
worth having: a per-widget pixel size would give each album a dozen near-identical files
and a fresh decode for each.
The cache is dropped wholesale when the websocket announces a rescan
(`{"type": "library"}`), because that is the one moment the backend rewrites the art
behind an unchanged cover URL. That is the bug the web client had to learn about the
hard way - see the comment at `python-backend/musicmouse/services/web/api.py:106`.
**The grid is virtualized.** Rust hands the UI a `[GridRow]` - the album list
pre-chunked into rows - and the view puts those in a `ListView`, which only instantiates
the rows on screen. A flat list of 660 cards gives it no rows to skip, hence the
chunking. This is the counterpart of `content-visibility: auto` on `.grid > .card` in
the web UI, which exists for exactly the same measurement: 340 live cards saturated the
Pi's renderer so completely that the main thread stopped scheduling anything.
**The progress bar animates instead of ticking.** Position arrives twice a second. The
web client interpolates it onto animation frames with a JS clock and a render budget;
here the fill is a Slint `animate width` between the pushed values, so interpolation
costs a property evaluation per frame on the render side and there is no clock at all.
It is suppressed for the one frame a track changes on, so a new track jumps rather than
sliding across two unrelated positions.
**Nothing on screen animates by itself.** The web UI's per-track ambient canvas is not
here. Halving its resolution and its frame rate took it from 53.5% of a core to 25.0%
and 25% was still too much: a loop repainting the viewport is a floor you cannot get
under while it runs at all. The stage keeps its gradient, which is what `?pi=1` already
does on the device this replaces.
## What is deliberately different from the web UI
**No glass blur.** Slint has no `backdrop-filter`, so the panels use a flat translucent
fill - exactly the web app's own `data-blur="off"` fallback. Some depth is genuinely
lost. Faking it by compositing a pre-blurred copy would reintroduce the per-frame cost
that dropping the blur was meant to avoid.
Worth knowing if you ever port the blur back: on the Pi, `backdrop-filter` measured five
times *better* than no blur (25% of a core against 118%), because it is what promotes
each panel to its own compositing layer. That is a browser-compositor artifact and does
not transfer here - but it is why the web UI keeps the panel blur under `?pi=1`.
**No dolphin.** The mascot and its swim/bob loops are not ported.
**Layout** is Slint's, not a transcription of the CSS: fixed row heights instead of
`clamp()` and `vh`, a 1080p-shaped fixed measure for the track list rather than
`max-width`. The colours are the same - `ui/theme.slint` carries the web UI's oklch
tokens converted to sRGB, with the oklch source kept beside each one, because oklch is
the form the design is actually maintained in.
## How it is put together
The split that matters is the web UI's, unchanged: **what is playing** comes from the
backend, **what you are looking at** lives here. The buttons on the mouse, a figurine on
the reader and Home Assistant all move playback too, so this follows it rather than
assuming it is in charge.
Three worker threads and no async runtime - the whole client makes one library request,
one websocket connection and a trickle of covers.
| Path | What it is |
|---|---|
| `src/api.rs` | The wire types and the HTTP calls. Mirrors `services/web/schemas.py`. |
| `src/ws.rs` | The state websocket, with reconnect-and-reseed. |
| `src/library.rs` | The search index and the browse hierarchy. Port of `web/src/lib/search.ts`. |
| `src/keymap.rs` | The keyboard model, as a pure function. Port of `web/src/lib/keyboard.ts`. |
| `src/covers.rs` | The thumbnail cache, and the generated stand-in art. |
| `src/ha.rs` | Home Assistant through the backend's proxy, and the shutter/colour conversions. Port of `web/src/hooks/useHomeAssistant.ts` and `lib/shutter.ts`. |
| `src/oklch.rs`, `src/format.rs` | Colour conversion; durations. |
| `src/main.rs` | Wiring: worker channels in, Slint models out. |
| `ui/*.slint` | Layout only. Nothing here formats a number or picks a word. |
`src/keymap.rs` and `src/library.rs` are ports of modules that were already pure
functions with their own tests, and they keep them: 43 tests, no window required.
## Keyboard
The same map as the web UI.
| Key | |
|---|---|
| `SPACE` | play / pause (types a space mid-query) |
| `SHIFT+H` / `SHIFT+L` | previous / next |
| `SHIFT+J` / `SHIFT+K` | quieter / louder |
| `SHIFT+M` | mute |
| `SHIFT+←` / `SHIFT+→` | seek 15 s |
| `A-Z`, `0-9` | search albums as you type |
| `?` | search individual tracks |
| `TAB` | cycle Musik / Hörbücher / Podcasts |
| `← ↑ ↓ →` | move the selection while browsing; transport and volume while playing |
| `CTRL+H/J/K/L` | the same, without leaving the home row |
| `CTRL+D` / `CTRL+U` | half a page |
| `ENTER` | play the selection |
| `SHIFT+ENTER` | open the track list |
| `ESC` | back out one layer at a time |
| `/` | back to browsing |
| `CTRL+TAB` | cycle Musik / Mein Zimmer / Tippen |
Everything is clickable too.
`CTRL+TAB` is the one addition to the web front-end's map. Its tab rail is reachable
only with a pointer, which sits badly with a device whose stated design is that every
screen can be reached from the keyboard. Plain `TAB` was already the group cycle.
## Mein Zimmer
The Home Assistant room panel, in its own violet rather than the player's teal so the
two read as siblings. Lights get a toggle, five brightness steps and six colours;
shutters get four presets, up/stop/down and a window that reflects the reported
position. Nothing is animated locally - a real cover reports its own movement, and
guessing where it will end up is what would fight with that.
The whole page is a poll, not a subscription, and only while it is on screen: Home
Assistant's own auth-and-subscribe websocket is more machinery than this needs, and a
lamp is not worth a request every 2.5 s when nobody is looking. A tap patches the
entity locally and remembers when, so the poll already in flight cannot undo it - the
same reconciliation `useHomeAssistant.ts` documents, and for the same reason.
The tab appears only when the backend answers `GET /api/ha` with a config; a 404 there
means Home Assistant is not set up, which is a normal answer rather than an error.
The special keys are named in `ui/app.slint`, where the `Key.*` constants live, and
reach Rust as `"escape"`, `"left"`, `"enter"` and so on - so `src/keymap.rs` is testable
without a window and without hardcoding keysym values.
## Fonts
Nunito, via `SLINT_DEFAULT_FONT`, which `shell.nix` sets. Slint cannot read the woff2
the web app self-hosts, so this wants the TTF; a deployment can drop one at
`fonts/Nunito.ttf` instead. With neither, everything still renders in the system sans.

View File

@@ -1,3 +0,0 @@
fn main() {
slint_build::compile("ui/app.slint").expect("compiling ui/app.slint");
}

Binary file not shown.

View File

@@ -1,9 +0,0 @@
Nunito, the face the design is drawn in, as a variable TTF.
`web/public/fonts/` has the same family as woff2, which is right for a browser and
useless here: Slint cannot read woff2. This is the upstream variable TTF (weights
200-1000), which `SLINT_DEFAULT_FONT` is pointed at - by `shell.nix` for development,
and by the launcher `roles/pi_slint_frontend` installs on the device.
Nunito is licensed under the SIL Open Font License 1.1:
https://github.com/googlefonts/nunito

View File

@@ -1,62 +0,0 @@
# Dev shell for the Slint front-end.
#
# NixOS has no rustup-installed toolchain here, and Slint's renderers link against
# system GL/X11/Wayland libraries that a plain `cargo build` cannot find on its own -
# hence LD_LIBRARY_PATH below rather than just a list of buildInputs.
{ pkgs ? import <nixpkgs> { } }:
let
# Everything the winit backend dlopen()s at runtime. Missing one of these shows up
# as a panic inside winit, not as a link error, so they are easy to misdiagnose.
runtimeLibs = with pkgs; [
libGL
libxkbcommon
wayland
xorg.libX11
xorg.libXcursor
xorg.libXi
xorg.libXrandr
fontconfig
freetype
];
in
pkgs.mkShell {
nativeBuildInputs = with pkgs; [
cargo
rustc
rustfmt
clippy
pkg-config
# Only needed when building with --features skia: rust-skia compiles its own
# bindings shim and wants a C++ toolchain plus python for the GN build.
clang
python3
];
# Nunito is the face the design is drawn in. `SLINT_DEFAULT_FONT` takes one file and
# makes it the primary font. This points at the copy vendored in `fonts/` rather than
# at nixpkgs, so development and the device render from the identical file.
SLINT_DEFAULT_FONT = toString ./fonts/Nunito.ttf;
buildInputs = runtimeLibs ++ (with pkgs; [
fontconfig
# libinput/libdrm/libseat are the linuxkms (kiosk) backend's dependencies. They
# cost nothing to have present on a desktop build.
libinput
libdrm
seatd
udev
]);
LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath runtimeLibs;
# rust-skia downloads a prebuilt libskia rather than building it, when it can.
SKIA_BINARIES_URL_TEMPLATE = null;
shellHook = ''
echo "musicmouse slint-frontend — cargo $(cargo --version | cut -d' ' -f2)"
echo " cargo run # windowed, femtovg"
echo " cargo run --features skia # windowed, skia (what the Pi uses)"
echo " cargo build --features kiosk # linuxkms + skia, for the device"
'';
}

View File

@@ -1,334 +0,0 @@
//! The shapes the backend serves, and the calls that fetch them.
//!
//! Mirrors `python-backend/musicmouse/services/web/schemas.py`, by way of
//! `web/src/api/types.ts` - the three are hand-kept in step. Only the music player's
//! share of the API is here; settings, the IR remote, Home Assistant and the typing
//! game are all reachable and all deliberately absent.
//!
//! Every field the backend can omit is modelled with `#[serde(default)]` rather than a
//! required `Option`, so a backend that grows a field does not break this client and
//! one that loses a field fails at the one call site that cared.
use serde::Deserialize;
/// Default for a development run; `--server` overrides it.
pub const DEFAULT_BASE_URL: &str = "http://127.0.0.1:8080";
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum AlbumKind {
#[default]
Music,
Book,
}
impl AlbumKind {
/// Shape is how you tell the two apart without reading anything: albums square,
/// audiobooks taller than wide, everywhere they appear. Nothing else in this app
/// may set a cover's aspect ratio. Ported from `web/src/lib/covers.ts`.
pub fn aspect(self) -> f32 {
match self {
Self::Music => 1.0,
Self::Book => 0.82,
}
}
}
#[derive(Debug, Clone, Deserialize)]
pub struct Track {
pub title: String,
/// Seconds, read from the file's tags at scan time. `0.0` when it carries none.
#[serde(default)]
pub duration: f64,
/// A typing-reward track not yet earned. This front-end does not host the typing
/// game, but it still has to not show what the game is withholding.
#[serde(default)]
pub locked: bool,
}
// `series`, and the parts of `PlayerState` the UI does not read, are kept because these
// structs are the record of what the backend serves - dropping a field would make this
// file a list of what this front-end happens to use today, which is a much less useful
// thing to read next to `schemas.py`.
#[allow(dead_code)]
#[derive(Debug, Clone, Deserialize)]
pub struct Album {
pub id: String,
/// One of the four fixed shelf folder names: `Figuren`, `Musik`, `Hörbücher`,
/// `Kinderpodcasts`. What `Group::of` reads to spot a podcast.
pub section: String,
#[serde(default)]
pub kind: AlbumKind,
pub title: String,
#[serde(default)]
pub artist: String,
#[serde(default)]
pub series: Option<String>,
/// The figurine whose folder this is, when it has one.
#[serde(default)]
pub figure: Option<String>,
/// Series for audiobooks and podcasts, artist for music. What browsing groups by.
#[serde(default)]
pub category: String,
/// Exactly three `#rrggbb`: primary, secondary, accent, pulled out of the cover art
/// or synthesised from the id. The generated stand-in cover is painted from these.
#[serde(default)]
pub colors: Vec<String>,
#[serde(default)]
pub has_cover: bool,
#[serde(default)]
pub duration: f64,
#[serde(default)]
pub tracks: Vec<Track>,
#[serde(default)]
pub locked: bool,
}
impl Album {
pub fn is_book(&self) -> bool {
self.kind == AlbumKind::Book
}
/// `"Album · Artist"`, unless a podcast makes those the same string.
pub fn line(&self) -> String {
let prefix = if self.is_book() { "📖 " } else { "" };
if !self.artist.is_empty() && self.artist != self.title {
format!("{prefix}{} · {}", self.title, self.artist)
} else {
format!("{prefix}{}", self.title)
}
}
/// `colors` padded out to the three the painter wants, with the same fallbacks the
/// web client uses, so a library entry missing them looks the same in both.
pub fn palette(&self) -> [&str; 3] {
[
self.colors.first().map_or("#4a6fa5", String::as_str),
self.colors.get(1).map_or("#6a8fc5", String::as_str),
self.colors.get(2).map_or("#a5804a", String::as_str),
]
}
}
#[derive(Debug, Clone, Deserialize)]
struct LibraryOut {
albums: Vec<Album>,
}
#[allow(dead_code)]
#[derive(Debug, Clone, Default, Deserialize)]
pub struct Connection {
#[serde(default)]
pub firmware: bool,
#[serde(default)]
pub mqtt: bool,
#[serde(default)]
pub lirc: bool,
}
/// What is playing. Owned entirely by the backend: the buttons on the mouse, a figurine
/// on the reader and Home Assistant all move it too, so this client follows it rather
/// than assuming it is in charge.
#[allow(dead_code)] // see the note above `Album`
#[derive(Debug, Clone, Default, Deserialize)]
pub struct PlayerState {
#[serde(default)]
pub playing: bool,
#[serde(default)]
pub album_id: Option<String>,
#[serde(default)]
pub album_title: Option<String>,
#[serde(default)]
pub artist: Option<String>,
#[serde(default)]
pub kind: Option<AlbumKind>,
#[serde(default)]
pub track_index: i32,
#[serde(default)]
pub track_title: Option<String>,
#[serde(default)]
pub track_count: i32,
#[serde(default)]
pub position: f64,
#[serde(default)]
pub duration: f64,
/// Percent, 0..100. The device's configured range never leaves the backend.
#[serde(default)]
pub volume: i32,
#[serde(default)]
pub active_figure: Option<String>,
#[serde(default)]
pub connected: Connection,
}
/// The push-only websocket's frames. `library` carries no payload - it is a bare "your
/// index is stale, fetch it again" after a rescan.
#[derive(Debug, Clone, Deserialize)]
#[serde(tag = "type", rename_all = "lowercase")]
pub enum ServerMessage {
State {
state: PlayerState,
},
Position {
position: f64,
#[serde(default)]
duration: f64,
},
Library,
/// A frame this build does not know. Ignored rather than fatal, so the backend can
/// grow a message type without this client falling over on it.
#[serde(other)]
Unknown,
}
// ------------------------------------------------------------------------- client --
#[derive(Debug, Clone)]
pub struct Client {
base: String,
agent: ureq::Agent,
}
/// Enough of an error type to put a cause on screen. Nothing here is recoverable in a
/// way the caller could branch on - the UI either has a library or says it cannot reach
/// the mouse - so the variants are not worth splitting.
pub type Error = String;
impl Client {
pub fn new(base_url: &str) -> Self {
Self {
base: base_url.trim_end_matches('/').to_string(),
agent: ureq::AgentBuilder::new()
.timeout_connect(std::time::Duration::from_secs(4))
.timeout_read(std::time::Duration::from_secs(30))
.build(),
}
}
/// `http://host:port` -> `ws://host:port/api/ws`. The device serves plain HTTP, but
/// deriving the scheme rather than hardcoding it keeps a TLS proxy in front working.
pub fn websocket_url(&self) -> String {
let ws = if let Some(rest) = self.base.strip_prefix("https://") {
format!("wss://{rest}")
} else if let Some(rest) = self.base.strip_prefix("http://") {
format!("ws://{rest}")
} else {
format!("ws://{}", self.base)
};
format!("{ws}/api/ws")
}
pub fn cover_url(&self, album_id: &str) -> String {
format!("{}/api/albums/{album_id}/cover", self.base)
}
/// A GET that decodes JSON, with the status code kept in the error text so a
/// caller that cares about 404 - several do, because "not configured" is a normal
/// answer here - can tell it from a connection failure.
pub(crate) fn get_json<T: serde::de::DeserializeOwned>(&self, path: &str) -> Result<T, Error> {
self.agent
.get(&format!("{}/api{path}", self.base))
.call()
.map_err(|e| format!("GET {path}: {e}"))?
.into_json()
.map_err(|e| format!("GET {path}: bad JSON: {e}"))
}
/// A POST whose response body, if any, is not worth reading. Home Assistant's
/// relayed replies are the only ones with a body, and nothing here reads it.
pub(crate) fn post_json(&self, path: &str, body: serde_json::Value) -> Result<(), Error> {
self.agent
.post(&format!("{}/api{path}", self.base))
.send_json(body)
.map(|_| ())
.map_err(|e| format!("POST {path}: {e}"))
}
pub fn library(&self) -> Result<Vec<Album>, Error> {
let out: LibraryOut = self
.agent
.get(&format!("{}/api/library", self.base))
.call()
.map_err(|e| format!("GET /api/library: {e}"))?
.into_json()
.map_err(|e| format!("GET /api/library: bad JSON: {e}"))?;
Ok(out.albums)
}
pub fn state(&self) -> Result<PlayerState, Error> {
self.agent
.get(&format!("{}/api/state", self.base))
.call()
.map_err(|e| format!("GET /api/state: {e}"))?
.into_json()
.map_err(|e| format!("GET /api/state: bad JSON: {e}"))
}
/// Raw cover bytes, or `Ok(None)` for a 404 - which is a normal answer, not a
/// failure: `has_cover` says so in advance and the client paints the album's own
/// colours instead.
pub fn cover(&self, album_id: &str) -> Result<Option<Vec<u8>>, Error> {
let response = match self.agent.get(&self.cover_url(album_id)).call() {
Ok(response) => response,
Err(ureq::Error::Status(404, _)) => return Ok(None),
Err(e) => return Err(format!("GET cover {album_id}: {e}")),
};
let mut bytes = Vec::new();
std::io::Read::read_to_end(&mut response.into_reader(), &mut bytes)
.map_err(|e| format!("GET cover {album_id}: {e}"))?;
Ok(Some(bytes))
}
// -------------------------------------------------------------------- commands --
//
// Every one of these emits the same *intent* the physical buttons emit, so this UI
// has no privileged path and no way to get out of step with a figure someone puts
// on the reader. They all answer 204 with no body - nothing to parse.
fn command(&self, path: &str, body: serde_json::Value) -> Result<(), Error> {
let request = self.agent.post(&format!("{}/api{path}", self.base));
let result = if body.is_null() {
request.call()
} else {
request.send_json(body)
};
result.map(|_| ()).map_err(|e| format!("POST {path}: {e}"))
}
pub fn play(&self, album_id: &str, track_index: i32) -> Result<(), Error> {
self.command(
"/play",
serde_json::json!({ "album_id": album_id, "track_index": track_index.max(0) }),
)
}
pub fn resume(&self) -> Result<(), Error> {
self.command("/resume", serde_json::Value::Null)
}
pub fn pause(&self) -> Result<(), Error> {
self.command("/pause", serde_json::Value::Null)
}
pub fn next(&self) -> Result<(), Error> {
self.command("/next", serde_json::Value::Null)
}
pub fn previous(&self) -> Result<(), Error> {
self.command("/previous", serde_json::Value::Null)
}
pub fn seek(&self, position: f64) -> Result<(), Error> {
self.command(
"/seek",
serde_json::json!({ "position": position.max(0.0) }),
)
}
pub fn volume(&self, percent: i32) -> Result<(), Error> {
self.command(
"/volume",
serde_json::json!({ "percent": percent.clamp(0, 100) }),
)
}
}

View File

@@ -1,406 +0,0 @@
//! Album art: fetched once, downscaled once, then reused forever.
//!
//! The backend serves one size and only one - `MAX_COVER_PX = 640` on the longest edge,
//! JPEG, at `/api/albums/{id}/cover`, with no `?size=` to ask for less. A browse grid
//! draws that at 192 px and the play view at 384, so painting the served file directly
//! means decoding a 640x640 JPEG for every visible card on every cold start. At 343
//! albums on a Pi that is the whole of "the UI feels slow".
//!
//! So this keeps a thumbnail cache on disk, keyed by album id *and* the size that was
//! asked for, and the decode-and-resize cost is paid once ever rather than once per
//! launch. A miss goes to the network on a worker thread; the UI never blocks on one.
//!
//! **Invalidation.** The bytes behind a cover URL do change while the URL does not -
//! the id addresses which album the art belongs to, not which art. Rather than pay a
//! conditional request per cover per start, this cache is cleared wholesale when the
//! websocket announces a rescan (`{"type": "library"}`), which is the one moment the
//! backend reprocesses cover art. See the long comment at
//! `python-backend/musicmouse/services/web/api.py:106` for how the web client learned
//! this the hard way.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::mpsc::{Receiver, Sender};
use image::imageops::FilterType;
use image::{Rgba, RgbaImage};
use crate::api::{AlbumKind, Client};
use crate::library::Group;
use crate::oklch::{mix, oklch_to_rgb, parse_hex};
/// The two sizes anything on screen actually asks for. Keeping it to two is what makes
/// the disk cache worth having: a per-widget pixel size would give every album a dozen
/// near-identical files and a fresh decode for each.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Tier {
/// Grid cards, shelf tiles, track rows and the player bar's 56 px thumbnail, which
/// scales down from this rather than earning a tier of its own.
Thumb,
/// The play view's cover, and the album sheet's.
Hero,
}
impl Tier {
pub fn px(self) -> u32 {
match self {
Self::Thumb => 192,
Self::Hero => 384,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Key {
pub album_id: String,
pub tier: Tier,
}
/// Everything the worker needs about an album, flattened so it can cross a thread
/// boundary without the `Rc<Album>` the UI side holds.
#[derive(Debug, Clone)]
pub struct Request {
pub album_id: String,
pub tier: Tier,
pub has_cover: bool,
pub is_book: bool,
pub locked: bool,
pub colors: [String; 3],
}
/// A finished bitmap on its way back to the UI thread. Raw RGBA rather than a
/// `slint::Image`, because the image types are built on the UI thread by whoever
/// receives this.
pub struct Ready {
pub key: Key,
pub width: u32,
pub height: u32,
pub rgba: Vec<u8>,
}
// ------------------------------------------------------------------ generated art --
/// Diagonal two-tone stripes: the stand-in for a music cover, and the thing that shows
/// through underneath one that has not arrived yet.
fn stripes(width: u32, height: u32, primary: [u8; 3], secondary: [u8; 3]) -> RgbaImage {
// `max(6, size / 13)` in `web/src/lib/covers.ts` - the stripe scales with the art so
// a 56 px thumbnail does not turn into a grey smear of forty bands.
let band = (width / 13).max(6) as f32;
RgbaImage::from_fn(width, height, |x, y| {
// 135deg in CSS points the gradient line at the bottom-right corner, so the
// position along it is the projection of (x, y) onto (1, 1)/sqrt(2).
let along = (x + y) as f32 * std::f32::consts::FRAC_1_SQRT_2;
let c = if ((along / band) as u32).is_multiple_of(2) {
primary
} else {
secondary
};
Rgba([c[0], c[1], c[2], 255])
})
}
/// A book: page edges down the right, a darker board down the left, over a diagonal
/// wash. Four layers in CSS, four passes here, in the same order.
fn spine(width: u32, height: u32, primary: [u8; 3], secondary: [u8; 3]) -> RgbaImage {
// The audiobook shelf's own hue - warm amber, not a duller yellow-brown - so a
// generated spine and the shelf it sits on are the same family by construction.
let hue = Group::Audiobooks.hue();
let page_light = oklch_to_rgb(0.97, 0.02, hue);
let page_dark = oklch_to_rgb(0.90, 0.03, hue);
RgbaImage::from_fn(width, height, |x, y| {
let fx = x as f32 / width.max(1) as f32;
let fy = y as f32 / height.max(1) as f32;
// Layer 4 (bottom): linear-gradient(155deg, primary -> secondary).
let t = ((fx * 0.42 + fy * 0.91) / 1.33).clamp(0.0, 1.0);
let mut c = mix(primary, secondary, t);
// Layer 3: the board down the left edge.
if fx < 0.11 {
c = secondary;
} else if fx < 0.13 {
c = primary;
}
// Layer 2: two thin highlight rules over the board.
if (0.035..0.043).contains(&fx) || (0.070..0.078).contains(&fx) {
c = mix(c, [255, 255, 255], 0.35);
}
// Layer 1 (top): the page block on the right.
if fx >= 0.975 {
c = page_dark;
} else if fx >= 0.95 {
c = page_light;
}
Rgba([c[0], c[1], c[2], 255])
})
}
/// What an album looks like before - or instead of - its real art.
pub fn generated(request: &Request) -> RgbaImage {
let primary = parse_hex(&request.colors[0], [0x4a, 0x6f, 0xa5]);
let secondary = parse_hex(&request.colors[1], [0x6a, 0x8f, 0xc5]);
let px = request.tier.px();
let (width, height) = dimensions(px, request.is_book);
if request.is_book {
spine(width, height, primary, secondary)
} else {
stripes(width, height, primary, secondary)
}
}
/// Shape comes from the media type and from nowhere else: albums square, audiobooks
/// taller than wide. `px` fixes the longest edge either way.
fn dimensions(px: u32, is_book: bool) -> (u32, u32) {
let kind = if is_book {
AlbumKind::Book
} else {
AlbumKind::Music
};
(((px as f32) * kind.aspect()).round() as u32, px)
}
// ------------------------------------------------------------------------- worker --
pub struct Worker {
client: Client,
dir: Option<PathBuf>,
}
impl Worker {
pub fn new(client: Client) -> Self {
let dir = dirs::cache_dir().map(|d| d.join("musicmouse-slint").join("covers"));
if let Some(dir) = &dir {
if let Err(e) = std::fs::create_dir_all(dir) {
// Not fatal: without a disk cache every cover is simply fetched and
// resized again next launch, which is the old behaviour, not a failure.
eprintln!("cover cache unavailable at {}: {e}", dir.display());
}
}
Self { client, dir }
}
fn path(&self, key: &Key) -> Option<PathBuf> {
self.dir
.as_ref()
.map(|d| d.join(format!("{}@{}.jpg", key.album_id, key.tier.px())))
}
/// Drop every stored thumbnail. Called when the backend says it has rescanned, and
/// therefore possibly rewritten the art behind these ids.
pub fn clear_disk(&self) {
let Some(dir) = &self.dir else { return };
if let Ok(entries) = std::fs::read_dir(dir) {
for entry in entries.flatten() {
let _ = std::fs::remove_file(entry.path());
}
}
}
fn render(&self, request: &Request) -> RgbaImage {
let px = request.tier.px();
let (want_w, want_h) = dimensions(px, request.is_book);
// A locked album is one the typing game is withholding. It has real art, and
// showing it would give away the reward, so it never gets fetched at all.
if request.locked || !request.has_cover {
return generated(request);
}
if let Some(path) = self.path(&Key {
album_id: request.album_id.clone(),
tier: request.tier,
}) {
if let Ok(image) = image::open(&path) {
return image.to_rgba8();
}
}
let bytes = match self.client.cover(&request.album_id) {
Ok(Some(bytes)) => bytes,
// 404 is a normal answer - `has_cover` can be stale by a rescan - and a
// network error should show the album, not a hole.
Ok(None) | Err(_) => return generated(request),
};
let Ok(decoded) = image::load_from_memory(&bytes) else {
return generated(request);
};
// `resize_to_fill` rather than `resize`: a real cover is square but a book's
// slot is not, and letterboxing one inside the other would break the shelf
// metaphor that the whole browse screen is built on.
let thumb = decoded.resize_to_fill(want_w, want_h, FilterType::CatmullRom);
if let Some(path) = self.path(&Key {
album_id: request.album_id.clone(),
tier: request.tier,
}) {
// Storing JPEG, not PNG: these are photographs, so JPEG is both smaller on
// disk and faster to decode on the next start, which is the whole point.
let _ = thumb
.to_rgb8()
.save_with_format(&path, image::ImageFormat::Jpeg);
}
thumb.to_rgba8()
}
}
/// Runs the fetch/decode/resize loop off the UI thread, calling `deliver` for each
/// finished bitmap. `deliver` is expected to hop back onto the Slint event loop.
pub fn spawn(
client: Client,
jobs: Receiver<Job>,
deliver: impl Fn(Ready) + Send + 'static,
) -> std::thread::JoinHandle<()> {
std::thread::Builder::new()
.name("covers".into())
.spawn(move || {
let worker = Worker::new(client);
// A cover asked for while it is already in flight - a card scrolled past
// twice, a grid rebuilt on a keystroke - must not become a second fetch.
let mut done: HashSet<Key> = HashSet::new();
for job in jobs {
match job {
Job::Clear => {
worker.clear_disk();
done.clear();
}
Job::Fetch(request) => {
let key = Key {
album_id: request.album_id.clone(),
tier: request.tier,
};
if !done.insert(key.clone()) {
continue;
}
let image = worker.render(&request);
deliver(Ready {
key,
width: image.width(),
height: image.height(),
rgba: image.into_raw(),
});
}
}
}
})
.expect("spawning the cover worker")
}
pub enum Job {
Fetch(Request),
Clear,
}
/// The UI thread's half: what is already decoded, and what has been asked for.
pub struct Store {
jobs: Sender<Job>,
ready: HashMap<Key, slint::Image>,
requested: HashSet<Key>,
}
impl Store {
pub fn new(jobs: Sender<Job>) -> Self {
Self {
jobs,
ready: HashMap::new(),
requested: HashSet::new(),
}
}
pub fn insert(&mut self, ready: Ready) {
let mut buffer =
slint::SharedPixelBuffer::<slint::Rgba8Pixel>::new(ready.width, ready.height);
buffer.make_mut_bytes().copy_from_slice(&ready.rgba);
self.ready
.insert(ready.key, slint::Image::from_rgba8(buffer));
}
/// The cover for an album if it is decoded, and a fetch queued if it is not. Callers
/// paint the album's own colours in the meantime, so this returning `None` is a
/// normal frame rather than a missing one.
pub fn get(&mut self, request: Request) -> Option<slint::Image> {
let key = Key {
album_id: request.album_id.clone(),
tier: request.tier,
};
if let Some(image) = self.ready.get(&key) {
return Some(image.clone());
}
if self.requested.insert(key) {
let _ = self.jobs.send(Job::Fetch(request));
}
None
}
pub fn clear(&mut self) {
self.ready.clear();
self.requested.clear();
let _ = self.jobs.send(Job::Clear);
}
}
#[cfg(test)]
mod tests {
use super::*;
fn request(is_book: bool, tier: Tier) -> Request {
Request {
album_id: "abc".into(),
tier,
has_cover: false,
is_book,
locked: false,
colors: ["#4a6fa5".into(), "#6a8fc5".into(), "#a5804a".into()],
}
}
#[test]
fn shape_follows_the_media_type() {
assert_eq!(dimensions(192, false), (192, 192));
assert_eq!(dimensions(192, true), (157, 192));
assert_eq!(dimensions(384, true), (315, 384));
}
#[test]
fn generated_art_fills_the_tier_it_was_asked_for() {
let music = generated(&request(false, Tier::Thumb));
assert_eq!((music.width(), music.height()), (192, 192));
let book = generated(&request(true, Tier::Hero));
assert_eq!((book.width(), book.height()), (315, 384));
}
#[test]
fn stripes_actually_alternate() {
let art = generated(&request(false, Tier::Thumb));
let colours: HashSet<[u8; 3]> = (0..192)
.map(|x| {
let p = art.get_pixel(x, 0).0;
[p[0], p[1], p[2]]
})
.collect();
assert_eq!(colours.len(), 2, "expected exactly the two stripe colours");
}
#[test]
fn a_book_has_bright_page_edges_on_its_right() {
let art = generated(&request(true, Tier::Hero));
let (w, h) = (art.width(), art.height());
let edge = art.get_pixel(w - 2, h / 2).0;
let middle = art.get_pixel(w / 2, h / 2).0;
let brightness = |p: [u8; 4]| p[0] as u32 + p[1] as u32 + p[2] as u32;
assert!(
brightness(edge) > brightness(middle),
"page edges should be lighter than the cover"
);
}
#[test]
fn every_generated_pixel_is_opaque() {
let art = generated(&request(true, Tier::Thumb));
assert!(art.pixels().all(|p| p.0[3] == 255));
}
}

View File

@@ -1,61 +0,0 @@
//! Durations, as the screen says them. Ported from `web/src/lib/format.ts`.
/// `m:ss`, or `h:mm:ss` once there is an hour to show. Negative and non-finite inputs
/// collapse to `0:00` rather than rendering `-1:-3`, because both reach here honestly:
/// a position can briefly exceed a tag-derived duration, and a track with no duration
/// tag at all arrives as `0.0`.
pub fn clock(seconds: f64) -> String {
if !seconds.is_finite() || seconds <= 0.0 {
return "0:00".into();
}
let total = seconds.round() as u64;
let (hours, minutes, secs) = (total / 3600, (total % 3600) / 60, total % 60);
if hours > 0 {
format!("{hours}:{minutes:02}:{secs:02}")
} else {
format!("{minutes}:{secs:02}")
}
}
/// How much of the album is left: the rest of this track plus every track after it.
pub fn remaining_in_album(durations: &[f64], track_index: usize, position: f64) -> f64 {
let current = durations.get(track_index).copied().unwrap_or(0.0);
let rest: f64 = durations.iter().skip(track_index + 1).sum();
(current - position).max(0.0) + rest
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn clock_formats_minutes_and_hours() {
assert_eq!(clock(0.0), "0:00");
assert_eq!(clock(9.0), "0:09");
assert_eq!(clock(61.0), "1:01");
assert_eq!(clock(599.0), "9:59");
assert_eq!(clock(3600.0), "1:00:00");
assert_eq!(clock(12130.0), "3:22:10");
}
#[test]
fn clock_survives_the_inputs_a_real_player_produces() {
assert_eq!(clock(-4.0), "0:00");
assert_eq!(clock(f64::NAN), "0:00");
assert_eq!(clock(f64::INFINITY), "0:00");
}
#[test]
fn remaining_counts_this_track_and_the_ones_after_it() {
let durations = [100.0, 200.0, 300.0];
assert_eq!(remaining_in_album(&durations, 0, 0.0), 600.0);
assert_eq!(remaining_in_album(&durations, 1, 50.0), 450.0);
assert_eq!(remaining_in_album(&durations, 2, 300.0), 0.0);
}
#[test]
fn a_position_past_the_tagged_duration_does_not_go_negative() {
assert_eq!(remaining_in_album(&[100.0, 50.0], 0, 130.0), 50.0);
assert_eq!(remaining_in_album(&[], 7, 10.0), 0.0);
}
}

View File

@@ -1,319 +0,0 @@
//! Home Assistant, through the backend's proxy.
//!
//! Ported from `web/src/hooks/useHomeAssistant.ts`. The browser never learns Home
//! Assistant's URL or token and neither does this: the backend keeps both and relays
//! `GET /api/ha/states/{id}` and `POST /api/ha/services/{domain}/{service}`, so all
//! this client knows is which entities exist.
//!
//! A short poll rather than a subscription. HA's own auth/subscribe websocket is more
//! machinery than a room panel needs, and the panel is only on screen while someone is
//! looking at it.
use std::collections::HashMap;
use std::time::{Duration, Instant};
use serde::Deserialize;
use crate::api::{Client, Error};
pub const POLL_INTERVAL: Duration = Duration::from_millis(2500);
#[derive(Debug, Clone, Deserialize)]
pub struct Device {
pub entity_id: String,
#[serde(default)]
pub name: Option<String>,
}
impl Device {
pub fn label(&self) -> &str {
self.name.as_deref().unwrap_or(&self.entity_id)
}
pub fn domain(&self) -> &str {
self.entity_id.split('.').next().unwrap_or("")
}
}
/// Presence-only, like the LIRC config - there is nothing secret in a list of entity
/// ids. `None` from `GET /api/ha` means Home Assistant is not configured at all, which
/// is a normal answer and hides the tab rather than being an error.
#[derive(Debug, Clone, Default, Deserialize)]
pub struct Config {
#[serde(default)]
pub devices: Vec<Device>,
#[serde(default)]
pub scenes: Vec<Device>,
}
/// One entity's `state`/`attributes`, relayed byte-for-byte from Home Assistant.
#[derive(Debug, Clone, Deserialize)]
pub struct EntityState {
#[serde(default)]
pub state: String,
#[serde(default)]
pub attributes: serde_json::Map<String, serde_json::Value>,
}
impl EntityState {
pub fn is_on(&self) -> bool {
self.state == "on"
}
fn number(&self, key: &str) -> Option<f64> {
self.attributes.get(key)?.as_f64()
}
/// 0..255 as Home Assistant reports it.
pub fn brightness(&self) -> f64 {
self.number("brightness").unwrap_or(0.0)
}
/// Percent *open*, which is HA's convention. The room UI thinks in percent closed -
/// see `closed_percent`.
pub fn cover_position(&self) -> Option<f64> {
self.number("current_position")
}
pub fn supported_features(&self) -> u64 {
self.number("supported_features").unwrap_or(0.0) as u64
}
pub fn rgb(&self) -> [u8; 3] {
let Some(values) = self.attributes.get("rgb_color").and_then(|v| v.as_array()) else {
// The warm default the web card falls back to, so an on-but-uncoloured lamp
// is not drawn as if it were off.
return [255, 214, 140];
};
let mut out = [255u8, 214, 140];
for (slot, value) in out.iter_mut().zip(values) {
if let Some(number) = value.as_f64() {
*slot = number.clamp(0.0, 255.0) as u8;
}
}
out
}
/// Whether this light can be given a colour, as opposed to only a brightness. A
/// `brightness_pct` sent to a plain on/off bulb is harmlessly ignored by HA, so
/// only the colour row is worth branching on.
pub fn is_color(&self) -> bool {
const COLOR_MODES: [&str; 5] = ["hs", "rgb", "rgbw", "rgbww", "xy"];
self.attributes
.get("supported_color_modes")
.and_then(|v| v.as_array())
.is_some_and(|modes| {
modes
.iter()
.filter_map(|m| m.as_str())
.any(|m| COLOR_MODES.contains(&m))
})
}
}
// --------------------------------------------------------------------- shutters --
/// Home Assistant's `CoverEntityFeature.SET_POSITION`.
pub const SUPPORT_SET_POSITION: u64 = 4;
/// The four buttons on a shutter card, in percent *closed*.
pub const SHUTTER_PRESETS: [(&str, f64); 4] = [
("Offen", 0.0),
("Halb zu", 50.0),
("Fast zu", 85.0),
("Ganz zu", 100.0),
];
/// HA reports percent *open*; the UI, following the design, thinks in percent *closed*.
/// The inversion lives in one place because it is easy to get backwards.
pub fn closed_percent(position: f64) -> f64 {
100.0 - position
}
pub fn position_from_closed(closed: f64) -> f64 {
100.0 - closed
}
pub fn closed_label(closed: f64) -> String {
if closed <= 1.0 {
"Offen".into()
} else if closed >= 99.0 {
"Ganz zu".into()
} else if closed >= 80.0 {
"Fast zu".into()
} else if closed >= 40.0 {
"Halb zu".into()
} else {
format!("{}% zu", closed.round())
}
}
// ---------------------------------------------------------------------- colours --
/// The six colours the room page offers, with the oklch each was authored as. Kept as
/// literals rather than converted at runtime for the same reason `ui/theme.slint` does:
/// the oklch is the form the design is maintained in, and the sRGB is what gets drawn.
pub const SWATCHES: [(&str, [u8; 3]); 6] = [
("Warmweiß", [0xf0, 0xdc, 0xb1]), // oklch(90% 0.06 85)
("Sonnengelb", [0xef, 0xcc, 0x36]), // oklch(85% 0.16 95)
("Korallenrot", [0xf1, 0x4d, 0x4c]), // oklch(65% 0.20 25)
("Delfinblau", [0x00, 0xab, 0xed]), // oklch(70% 0.15 235)
("Riffgrün", [0x42, 0xcb, 0x80]), // oklch(75% 0.16 155)
("Quallenlila", [0xb1, 0x6a, 0xe0]), // oklch(65% 0.18 310)
];
// ----------------------------------------------------------------------- client --
impl Client {
/// `Ok(None)` when Home Assistant is not configured - a 404 here is a normal
/// answer, and the room tab simply does not appear.
pub fn ha_config(&self) -> Result<Option<Config>, Error> {
match self.get_json::<Config>("/ha") {
Ok(config) => Ok(Some(config)),
Err(error) if error.contains("404") => Ok(None),
Err(error) => Err(error),
}
}
pub fn ha_state(&self, entity_id: &str) -> Result<EntityState, Error> {
self.get_json(&format!("/ha/states/{entity_id}"))
}
pub fn ha_call(
&self,
domain: &str,
service: &str,
body: serde_json::Value,
) -> Result<(), Error> {
self.post_json(&format!("/ha/services/{domain}/{service}"), body)
}
}
/// What the poll thread sends back.
pub struct Snapshot {
pub states: HashMap<String, EntityState>,
/// When the request that produced this went out. The UI drops any entity whose
/// optimistic patch is newer - see the note in `Room::apply`.
pub started_at: Instant,
}
pub enum Job {
/// Poll these entities once, now.
Poll(Vec<String>),
Call {
domain: String,
service: String,
body: serde_json::Value,
},
}
/// One worker for both polling and commands, so a command can never overtake the poll
/// that was meant to confirm it.
pub fn spawn(
client: Client,
jobs: std::sync::mpsc::Receiver<Job>,
deliver: impl Fn(Snapshot) + Send + 'static,
) -> std::thread::JoinHandle<()> {
std::thread::Builder::new()
.name("ha".into())
.spawn(move || {
for job in jobs {
match job {
Job::Call {
domain,
service,
body,
} => {
if let Err(error) = client.ha_call(&domain, &service, body) {
// The next poll corrects whatever the optimistic update
// guessed, so a failed call is worth a line and nothing more.
eprintln!("{domain}.{service}: {error}");
}
}
Job::Poll(entity_ids) => {
let started_at = Instant::now();
let mut states = HashMap::new();
for entity_id in entity_ids {
if let Ok(state) = client.ha_state(&entity_id) {
states.insert(entity_id, state);
}
}
deliver(Snapshot { states, started_at });
}
}
}
})
.expect("spawning the home assistant worker")
}
#[cfg(test)]
mod tests {
use super::*;
fn state(json: serde_json::Value) -> EntityState {
serde_json::from_value(json).unwrap()
}
#[test]
fn closed_percent_inverts_home_assistants_convention() {
assert_eq!(closed_percent(100.0), 0.0); // fully open
assert_eq!(closed_percent(0.0), 100.0); // fully closed
assert_eq!(position_from_closed(85.0), 15.0);
}
#[test]
fn closed_label_names_the_presets_and_falls_back_to_a_number() {
assert_eq!(closed_label(0.0), "Offen");
assert_eq!(closed_label(100.0), "Ganz zu");
assert_eq!(closed_label(85.0), "Fast zu");
assert_eq!(closed_label(50.0), "Halb zu");
assert_eq!(closed_label(20.0), "20% zu");
}
#[test]
fn a_colour_light_is_told_apart_by_its_modes() {
assert!(state(serde_json::json!({
"state": "on",
"attributes": { "supported_color_modes": ["hs", "color_temp"] }
}))
.is_color());
assert!(!state(serde_json::json!({
"state": "on",
"attributes": { "supported_color_modes": ["brightness"] }
}))
.is_color());
assert!(!state(serde_json::json!({ "state": "on", "attributes": {} })).is_color());
}
#[test]
fn rgb_falls_back_to_a_warm_white_rather_than_black() {
assert_eq!(
state(serde_json::json!({"state":"on","attributes":{}})).rgb(),
[255, 214, 140]
);
assert_eq!(
state(serde_json::json!({
"state": "on", "attributes": { "rgb_color": [10, 20, 30] }
}))
.rgb(),
[10, 20, 30]
);
}
#[test]
fn a_missing_attribute_reads_as_zero_rather_than_failing() {
let s = state(serde_json::json!({ "state": "off", "attributes": {} }));
assert_eq!(s.brightness(), 0.0);
assert_eq!(s.cover_position(), None);
assert_eq!(s.supported_features(), 0);
assert!(!s.is_on());
}
#[test]
fn a_device_knows_its_domain_and_its_label() {
let device: Device =
serde_json::from_value(serde_json::json!({ "entity_id": "light.kitchen" })).unwrap();
assert_eq!(device.domain(), "light");
assert_eq!(device.label(), "light.kitchen");
}
}

View File

@@ -1,915 +0,0 @@
//! What you are looking at, and what a keypress does to it.
//!
//! Ported from `web/src/lib/keyboard.ts`, which is already a pure function from a
//! keypress to a list of actions - the single most portable thing in that codebase, and
//! tested there without a DOM for the same reason it is tested here without a window.
//!
//! The split this rests on: **what is playing** comes from the backend, **what you are
//! looking at** lives here. A real player has other front-ends - the buttons on the
//! mouse, a figurine on the reader, Home Assistant - so this UI follows playback rather
//! than owning it, and owns only the browse state.
//!
//! Not ported: the typing game, the room-lights page, the in-app help overlay and the
//! IR-remote assign mode, none of which this front-end hosts.
use crate::library::{Group, Results, GROUPS};
pub const VOLUME_STEP: i32 = 10;
pub const SEEK_STEP: f64 = 15.0;
/// How far Ctrl+d/Ctrl+u jump - vim's half-page scroll, applied to rows of cards.
pub const PAGE_ROWS: i32 = 3;
/// Which top-level page is showing. Orthogonal to everything below it, so switching
/// pages and coming back leaves the music side exactly as it was.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Page {
Music,
Room,
Typing,
}
impl Page {
pub fn icon(self) -> &'static str {
match self {
// A plain Unicode symbol rather than the emoji: that glyph carries its own
// muted colour on some platforms and ignores `color`, which makes it
// illegible on the rail's own background.
Self::Music => "♪",
Self::Room => "💡",
Self::Typing => "⌨️",
}
}
pub fn label(self) -> &'static str {
match self {
Self::Music => "Musik",
Self::Room => "Mein Zimmer",
Self::Typing => "Tippen",
}
}
}
#[derive(Debug, Clone, PartialEq)]
pub struct UiState {
pub page: Page,
pub search: String,
pub tracks_mode: bool,
/// `None` is the bare root screen (three shelves); otherwise which one is open.
pub group: Option<Group>,
pub category: Option<String>,
pub sel_index: usize,
/// Which of the three root shelves is focused. Only meaningful at the bare root,
/// where `sel_index` doubles as "which tile in that row".
pub shelf_row: usize,
pub play_view: bool,
pub open_album: Option<String>,
/// Which track is highlighted in the open album's list.
pub sheet_index: usize,
/// Grid columns, recomputed from the window width - `move_selection` needs it to
/// know what one row down means.
pub cols: usize,
}
impl Default for UiState {
fn default() -> Self {
Self {
page: Page::Music,
search: String::new(),
tracks_mode: false,
group: None,
category: None,
sel_index: 0,
shelf_row: 0,
play_view: false,
open_album: None,
sheet_index: 0,
cols: 4,
}
}
}
impl UiState {
/// The true root: nothing chosen yet, drawn as three shelves rather than a list.
pub fn is_root_shelf(&self) -> bool {
self.group.is_none()
&& self.search.is_empty()
&& !self.tracks_mode
&& self.category.is_none()
}
pub fn browsing(&self) -> bool {
!self.play_view && self.open_album.is_none()
}
/// One step of "back": search, then track-search mode, then group and category
/// together - the order ESC and the corner button both use.
///
/// `group` and `category` clear as one step because a root shelf tile sets both at
/// once, jumping straight to one category's albums; undoing that jump should be one
/// step too, however the category was reached.
pub fn browse_back(&mut self) {
self.sel_index = 0;
if !self.search.is_empty() {
self.search.clear();
} else if self.tracks_mode {
self.tracks_mode = false;
} else {
self.group = None;
self.category = None;
}
}
/// Whether anything is left to back out of - what the corner button's presence is.
pub fn can_go_back(&self) -> bool {
self.page != Page::Music
|| self.play_view
|| self.open_album.is_some()
|| !self.search.is_empty()
|| self.tracks_mode
|| self.group.is_some()
|| self.category.is_some()
}
}
/// What a keypress asks the *player* to do. Everything else a key can do is a change to
/// [`UiState`], applied in place.
#[derive(Debug, Clone, PartialEq)]
pub enum Action {
Play {
album_id: String,
track_index: usize,
},
Toggle,
Next,
Previous,
/// Percentage points, positive or negative.
Volume(i32),
/// Seconds, positive or negative.
Seek(f64),
Mute,
}
/// Where the flat selection index is pointing. Songs, then categories, then albums.
enum Selection {
Song {
album_id: String,
track_index: usize,
},
Category(String),
Album(String),
None,
}
fn selection_at(results: &Results, sel_index: usize) -> Selection {
let total = results.total();
if total == 0 {
return Selection::None;
}
let index = sel_index.min(total - 1);
if let Some(hit) = results.songs.get(index) {
return Selection::Song {
album_id: hit.album.id.clone(),
track_index: hit.index,
};
}
let after_songs = index - results.songs.len();
if let Some(category) = results.categories.get(after_songs) {
return Selection::Category(category.key.clone());
}
match results.albums.get(after_songs - results.categories.len()) {
Some(album) => Selection::Album(album.id.clone()),
None => Selection::None,
}
}
/// The album behind the selection, when "show me the tracks" makes sense for it: an
/// album tile directly, or - so you can reach the rest of it - the album behind a song
/// hit. `None` for a category tile, and for a podcast episode, which is a single track
/// with nothing else to show.
pub fn selection_album(results: &Results, sel_index: usize) -> Option<String> {
let total = results.total();
if total == 0 {
return None;
}
let index = sel_index.min(total - 1);
let album = if let Some(hit) = results.songs.get(index) {
Some(&hit.album)
} else {
let after_songs = index - results.songs.len();
if after_songs < results.categories.len() {
return None;
}
results.albums.get(after_songs - results.categories.len())
}?;
(Group::of(album) != Group::Podcasts).then(|| album.id.clone())
}
/// The bare root is a genuine two-axis layout: three rows, each a differently-long row
/// of tiles. `shelf_lengths` is how long each row actually is, so a vertical move can
/// clamp the column to the row it lands on rather than guessing.
fn move_root_shelf(state: &mut UiState, dx: i32, dy: i32, shelf_lengths: &[usize; 3]) {
if dy != 0 {
state.shelf_row = (state.shelf_row as i32 + dy).clamp(0, GROUPS.len() as i32 - 1) as usize;
state.sel_index = state
.sel_index
.min(shelf_lengths[state.shelf_row].saturating_sub(1));
}
if dx != 0 {
let last = shelf_lengths[state.shelf_row].saturating_sub(1) as i32;
state.sel_index = (state.sel_index as i32 + dx).clamp(0, last.max(0)) as usize;
}
}
fn move_selection(state: &mut UiState, results: &Results, dx: i32, dy: i32) {
let total = results.total();
if total == 0 {
return;
}
let cols = state.cols.max(1) as i32;
let mut index = state.sel_index.min(total - 1) as i32;
index += dx;
if dy != 0 {
// Song hits are a single-column list; everything below them is a grid.
index += if (index as usize) < results.songs.len() {
dy
} else {
dy * cols
};
}
state.sel_index = index.clamp(0, total as i32 - 1) as usize;
}
/// The keys that mean the same thing from anywhere, matching the muscle memory of every
/// other vim-ish media app. Keyed on the exact shifted letter, so this intercepts only
/// these five - every other Shift+letter still reaches the search box. Search is
/// case-insensitive, so nothing searchable is lost.
fn shift_media(key: &str) -> Option<Action> {
match key.to_ascii_uppercase().as_str() {
"H" => Some(Action::Previous),
"L" => Some(Action::Next),
"K" => Some(Action::Volume(VOLUME_STEP)),
"J" => Some(Action::Volume(-VOLUME_STEP)),
"M" => Some(Action::Mute),
_ => None,
}
}
/// A plain character key's default: type it into the search box. Letters and digits
/// only - `char::is_alphanumeric` rather than `[a-z0-9]`, so the umlauts a German title
/// needs are searchable too.
fn type_into_search(state: &mut UiState, key: &str) -> bool {
let mut chars = key.chars();
match (chars.next(), chars.next()) {
(Some(c), None) if c.is_alphanumeric() => {
state.search.push(c);
state.play_view = false;
state.sel_index = 0;
true
}
_ => false,
}
}
/// Translate one keypress: mutate `state`, and return whatever the player should be
/// asked to do.
///
/// `key` is already named by the UI layer - `"escape"`, `"left"`, `"enter"`, … - or is
/// the literal character typed. `shelf_lengths` and `sheet_tracks` are the two counts
/// that `results` does not carry but the clamping needs.
pub fn handle_key(
key: &str,
ctrl: bool,
shift: bool,
state: &mut UiState,
results: &Results,
shelf_lengths: &[usize; 3],
sheet_tracks: usize,
) -> Vec<Action> {
// Reachable from anywhere, including from the play view and with an album sheet open.
if shift && !ctrl {
if let Some(action) = shift_media(key) {
return vec![action];
}
}
if key == "space" {
// Except mid-search, where a space is punctuation the query needs, as in
// "geolino azte".
if state.browsing() && !state.search.is_empty() {
state.search.push(' ');
state.sel_index = 0;
return vec![];
}
return vec![Action::Toggle];
}
// Ctrl+Tab cycles the pages. This is an addition to the web front-end, where the
// tab rail is reachable only with a pointer - which sits badly with a device whose
// stated design is that every screen can be reached from the keyboard. Plain Tab
// is already taken by the group cycle, and Ctrl+Tab is the idiom for the outer
// one everywhere else.
if ctrl && key == "tab" {
state.page = match state.page {
Page::Music => Page::Room,
Page::Room => Page::Typing,
Page::Typing => Page::Music,
};
return vec![];
}
// Ctrl+hjkl moves the highlight in whichever list is on screen, the same job the
// arrows do but without leaving the home row. Ctrl+d/u are vim's half-page jump.
if ctrl {
if state.open_album.is_some() {
match key.to_ascii_lowercase().as_str() {
"j" => {
state.sheet_index = (state.sheet_index + 1).min(sheet_tracks.saturating_sub(1))
}
"k" => state.sheet_index = state.sheet_index.saturating_sub(1),
_ => {}
}
return vec![];
}
if state.play_view {
return vec![];
}
let lower = key.to_ascii_lowercase();
if state.is_root_shelf() {
match lower.as_str() {
"h" => move_root_shelf(state, -1, 0, shelf_lengths),
"l" => move_root_shelf(state, 1, 0, shelf_lengths),
"j" | "d" => move_root_shelf(state, 0, 1, shelf_lengths),
"k" | "u" => move_root_shelf(state, 0, -1, shelf_lengths),
_ => {}
}
return vec![];
}
match lower.as_str() {
"h" => move_selection(state, results, -1, 0),
"l" => move_selection(state, results, 1, 0),
"j" => move_selection(state, results, 0, 1),
"k" => move_selection(state, results, 0, -1),
"d" => move_selection(state, results, 0, PAGE_ROWS),
"u" => move_selection(state, results, 0, -PAGE_ROWS),
_ => {}
}
return vec![];
}
let browsing = state.browsing();
match key {
"tab" => {
let current = state.group.map_or(-1, |g| g.index() as i32);
state.group = Some(Group::from_index(
((current + 1) % GROUPS.len() as i32) as usize,
));
state.category = None;
state.sel_index = 0;
state.play_view = false;
vec![]
}
"/" => {
state.play_view = false;
vec![]
}
// Arrows browse while browsing and drive the player while playing - the one
// pair of keys that changes meaning with the screen, because on the play view
// there is no selection for them to move.
"right" => {
if shift {
return vec![Action::Seek(SEEK_STEP)];
}
if !browsing {
return vec![Action::Next];
}
if state.is_root_shelf() {
move_root_shelf(state, 1, 0, shelf_lengths);
} else {
move_selection(state, results, 1, 0);
}
vec![]
}
"left" => {
if shift {
return vec![Action::Seek(-SEEK_STEP)];
}
if !browsing {
return vec![Action::Previous];
}
if state.is_root_shelf() {
move_root_shelf(state, -1, 0, shelf_lengths);
} else {
move_selection(state, results, -1, 0);
}
vec![]
}
"down" => {
if !browsing {
return vec![Action::Volume(-VOLUME_STEP)];
}
if state.is_root_shelf() {
move_root_shelf(state, 0, 1, shelf_lengths);
} else {
move_selection(state, results, 0, 1);
}
vec![]
}
"up" => {
if !browsing {
return vec![Action::Volume(VOLUME_STEP)];
}
if state.is_root_shelf() {
move_root_shelf(state, 0, -1, shelf_lengths);
} else {
move_selection(state, results, 0, -1);
}
vec![]
}
"?" => {
state.tracks_mode = true;
state.search.clear();
state.sel_index = 0;
state.play_view = false;
vec![]
}
"escape" => {
// Peel one layer at a time rather than dumping you back at the top. The
// page is the outermost layer: ESC on the room or typing page comes back
// to the player, and only then starts unwinding the browse hierarchy.
if state.page != Page::Music {
state.page = Page::Music;
} else if state.open_album.is_some() {
state.open_album = None;
} else if state.play_view {
state.play_view = false;
} else {
state.browse_back();
}
vec![]
}
"backspace" => {
if !state.search.is_empty() {
state.search.pop();
state.sel_index = 0;
}
vec![]
}
"enter" => {
if let Some(album_id) = state.open_album.clone() {
return vec![Action::Play {
album_id,
track_index: state.sheet_index,
}];
}
if browsing && state.is_root_shelf() {
let row = state.shelf_row.min(GROUPS.len() - 1);
state.group = Some(Group::from_index(row));
// A shelf row with no tiles still opens its group; there is simply no
// category to pre-select.
state.category = None;
state.sel_index = 0;
return vec![];
}
if shift {
if let Some(album_id) = selection_album(results, state.sel_index) {
state.open_album = Some(album_id);
state.sheet_index = 0;
return vec![];
}
}
match selection_at(results, state.sel_index) {
Selection::Song {
album_id,
track_index,
} => {
vec![Action::Play {
album_id,
track_index,
}]
}
Selection::Album(album_id) => vec![Action::Play {
album_id,
track_index: 0,
}],
Selection::Category(key) => {
state.category = Some(key);
state.sel_index = 0;
vec![]
}
Selection::None => vec![],
}
}
"f1" => vec![],
_ => {
type_into_search(state, key);
vec![]
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::library::{Category, Library, SongHit};
fn empty() -> Results {
Results::default()
}
fn press(key: &str, state: &mut UiState) -> Vec<Action> {
handle_key(key, false, false, state, &empty(), &[0, 0, 0], 0)
}
#[test]
fn typing_a_letter_searches_and_leaves_the_play_view() {
let mut state = UiState {
play_view: true,
..Default::default()
};
press("k", &mut state);
press("i", &mut state);
assert_eq!(state.search, "ki");
assert!(!state.play_view);
}
#[test]
fn umlauts_are_searchable_like_any_other_letter() {
let mut state = UiState::default();
press("ö", &mut state);
assert_eq!(state.search, "ö");
}
#[test]
fn space_toggles_but_types_mid_search() {
let mut state = UiState::default();
assert_eq!(press("space", &mut state), vec![Action::Toggle]);
state.search = "geolino".into();
assert!(press("space", &mut state).is_empty());
assert_eq!(state.search, "geolino ");
}
#[test]
fn space_toggles_from_the_play_view_even_with_a_query_behind_it() {
let mut state = UiState {
search: "conni".into(),
play_view: true,
..Default::default()
};
assert_eq!(press("space", &mut state), vec![Action::Toggle]);
}
#[test]
fn shift_media_works_from_every_screen() {
for play_view in [false, true] {
let mut state = UiState {
play_view,
..Default::default()
};
let got = handle_key("H", false, true, &mut state, &empty(), &[0, 0, 0], 0);
assert_eq!(got, vec![Action::Previous]);
let got = handle_key("K", false, true, &mut state, &empty(), &[0, 0, 0], 0);
assert_eq!(got, vec![Action::Volume(VOLUME_STEP)]);
}
}
#[test]
fn other_shifted_letters_still_reach_the_search_box() {
let mut state = UiState::default();
handle_key("C", false, true, &mut state, &empty(), &[0, 0, 0], 0);
assert_eq!(state.search, "C");
}
#[test]
fn arrows_browse_while_browsing_and_drive_the_player_while_playing() {
let mut state = UiState {
group: Some(Group::Music),
..Default::default()
};
assert!(press("right", &mut state).is_empty());
state.play_view = true;
assert_eq!(press("right", &mut state), vec![Action::Next]);
assert_eq!(press("up", &mut state), vec![Action::Volume(VOLUME_STEP)]);
assert_eq!(
press("down", &mut state),
vec![Action::Volume(-VOLUME_STEP)]
);
}
#[test]
fn shift_arrows_seek_from_either_screen() {
let mut state = UiState::default();
let got = handle_key("right", false, true, &mut state, &empty(), &[0, 0, 0], 0);
assert_eq!(got, vec![Action::Seek(SEEK_STEP)]);
let got = handle_key("left", false, true, &mut state, &empty(), &[0, 0, 0], 0);
assert_eq!(got, vec![Action::Seek(-SEEK_STEP)]);
}
#[test]
fn the_root_shelf_is_two_axis_and_clamps_to_the_row_it_lands_on() {
let mut state = UiState::default();
let lengths = [5, 2, 0];
handle_key("right", false, false, &mut state, &empty(), &lengths, 0);
handle_key("right", false, false, &mut state, &empty(), &lengths, 0);
handle_key("right", false, false, &mut state, &empty(), &lengths, 0);
assert_eq!((state.shelf_row, state.sel_index), (0, 3));
// Row 1 only has two tiles, so the column has to come back with it.
handle_key("down", false, false, &mut state, &empty(), &lengths, 0);
assert_eq!((state.shelf_row, state.sel_index), (1, 1));
// Row 2 has none at all.
handle_key("down", false, false, &mut state, &empty(), &lengths, 0);
assert_eq!((state.shelf_row, state.sel_index), (2, 0));
// And it does not walk off the bottom.
handle_key("down", false, false, &mut state, &empty(), &lengths, 0);
assert_eq!(state.shelf_row, 2);
}
#[test]
fn enter_at_the_root_opens_the_focused_shelfs_group() {
let mut state = UiState {
shelf_row: 1,
..Default::default()
};
assert!(press("enter", &mut state).is_empty());
assert_eq!(state.group, Some(Group::Audiobooks));
assert_eq!(state.sel_index, 0);
}
#[test]
fn escape_peels_one_layer_at_a_time() {
let mut state = UiState {
group: Some(Group::Music),
category: Some("Rolf".into()),
search: "abc".into(),
tracks_mode: true,
play_view: true,
open_album: Some("x".into()),
..Default::default()
};
press("escape", &mut state);
assert!(state.open_album.is_none(), "the sheet closes first");
press("escape", &mut state);
assert!(!state.play_view, "then the play view");
press("escape", &mut state);
assert_eq!(state.search, "", "then the query");
press("escape", &mut state);
assert!(!state.tracks_mode, "then track-search mode");
press("escape", &mut state);
assert_eq!(
(state.group, state.category.clone()),
(None, None),
"then both at once"
);
assert!(!state.can_go_back());
}
#[test]
fn ctrl_tab_cycles_the_pages_and_plain_tab_still_cycles_groups() {
let mut state = UiState::default();
for expected in [Page::Room, Page::Typing, Page::Music] {
handle_key("tab", true, false, &mut state, &empty(), &[0, 0, 0], 0);
assert_eq!(state.page, expected);
}
// And the outer cycle has not eaten the inner one.
press("tab", &mut state);
assert_eq!(state.group, Some(Group::Music));
assert_eq!(state.page, Page::Music);
}
#[test]
fn escape_comes_back_from_another_page_before_it_touches_the_browse_state() {
let mut state = UiState {
page: Page::Room,
group: Some(Group::Music),
..Default::default()
};
press("escape", &mut state);
assert_eq!(state.page, Page::Music);
assert_eq!(
state.group,
Some(Group::Music),
"the player was left as it was"
);
press("escape", &mut state);
assert_eq!(state.group, None);
}
#[test]
fn tab_cycles_the_three_groups_and_wraps() {
let mut state = UiState::default();
for expected in [
Group::Music,
Group::Audiobooks,
Group::Podcasts,
Group::Music,
] {
press("tab", &mut state);
assert_eq!(state.group, Some(expected));
}
}
#[test]
fn backspace_only_bites_when_there_is_a_query() {
let mut state = UiState {
search: "ab".into(),
..Default::default()
};
press("backspace", &mut state);
assert_eq!(state.search, "a");
press("backspace", &mut state);
press("backspace", &mut state);
assert_eq!(state.search, "");
}
#[test]
fn question_mark_switches_to_track_search_and_clears_the_query() {
let mut state = UiState {
search: "conni".into(),
..Default::default()
};
press("?", &mut state);
assert!(state.tracks_mode);
assert_eq!(state.search, "");
}
// ---- selection over a real results set ----
fn results_with(songs: usize, categories: usize, albums: usize) -> Results {
let library = Library::build(
(0..albums.max(1))
.map(|i| {
serde_json::from_value(serde_json::json!({
"id": format!("al{i}"),
"section": "Musik",
"kind": "music",
"title": format!("Album {i}"),
"artist": "A",
"category": "A",
"tracks": [{ "title": "T", "duration": 10.0 }],
}))
.unwrap()
})
.collect(),
);
Results {
songs: (0..songs)
.map(|i| SongHit {
album: std::rc::Rc::clone(&library.albums[0]),
index: i,
title: format!("S{i}"),
duration: 10.0,
})
.collect(),
categories: (0..categories)
.map(|i| Category {
key: format!("C{i}"),
albums: vec![],
})
.collect(),
albums: library
.albums
.iter()
.take(albums)
.map(std::rc::Rc::clone)
.collect(),
}
}
#[test]
fn the_selection_runs_flat_across_songs_then_categories_then_albums() {
let results = results_with(2, 2, 3);
assert_eq!(results.total(), 7);
let mut state = UiState {
group: Some(Group::Music),
cols: 3,
..Default::default()
};
// Inside the song list a vertical move is one row, not one grid row.
handle_key("down", false, false, &mut state, &results, &[0, 0, 0], 0);
assert_eq!(state.sel_index, 1);
// Past the songs it jumps a whole grid row.
state.sel_index = 2;
handle_key("down", false, false, &mut state, &results, &[0, 0, 0], 0);
assert_eq!(state.sel_index, 5);
// And it never runs off the end.
handle_key("down", false, false, &mut state, &results, &[0, 0, 0], 0);
assert_eq!(state.sel_index, 6);
}
#[test]
fn enter_plays_a_song_hit_at_its_own_track_index() {
let results = results_with(2, 0, 0);
let mut state = UiState {
group: Some(Group::Music),
sel_index: 1,
..Default::default()
};
let got = handle_key("enter", false, false, &mut state, &results, &[0, 0, 0], 0);
assert_eq!(
got,
vec![Action::Play {
album_id: "al0".into(),
track_index: 1
}]
);
}
#[test]
fn enter_on_a_category_drills_in_rather_than_playing() {
let results = results_with(0, 2, 0);
let mut state = UiState {
group: Some(Group::Music),
sel_index: 1,
..Default::default()
};
assert!(handle_key("enter", false, false, &mut state, &results, &[0, 0, 0], 0).is_empty());
assert_eq!(state.category.as_deref(), Some("C1"));
}
#[test]
fn enter_on_an_album_plays_it_from_the_top() {
let results = results_with(0, 0, 3);
let mut state = UiState {
group: Some(Group::Music),
sel_index: 2,
..Default::default()
};
let got = handle_key("enter", false, false, &mut state, &results, &[0, 0, 0], 0);
assert_eq!(
got,
vec![Action::Play {
album_id: "al2".into(),
track_index: 0
}]
);
}
#[test]
fn shift_enter_opens_the_track_list_instead() {
let results = results_with(0, 0, 2);
let mut state = UiState {
group: Some(Group::Music),
sel_index: 1,
..Default::default()
};
assert!(handle_key("enter", false, true, &mut state, &results, &[0, 0, 0], 0).is_empty());
assert_eq!(state.open_album.as_deref(), Some("al1"));
}
#[test]
fn shift_enter_on_a_category_falls_through_to_drilling_in() {
let results = results_with(0, 1, 0);
let mut state = UiState {
group: Some(Group::Music),
sel_index: 0,
..Default::default()
};
handle_key("enter", false, true, &mut state, &results, &[0, 0, 0], 0);
assert!(state.open_album.is_none());
assert_eq!(state.category.as_deref(), Some("C0"));
}
#[test]
fn ctrl_jk_walks_the_open_track_list_and_clamps() {
let mut state = UiState {
open_album: Some("al0".into()),
..Default::default()
};
handle_key("j", true, false, &mut state, &empty(), &[0, 0, 0], 3);
handle_key("j", true, false, &mut state, &empty(), &[0, 0, 0], 3);
handle_key("j", true, false, &mut state, &empty(), &[0, 0, 0], 3);
assert_eq!(state.sheet_index, 2);
handle_key("k", true, false, &mut state, &empty(), &[0, 0, 0], 3);
assert_eq!(state.sheet_index, 1);
}
#[test]
fn enter_with_the_sheet_open_plays_the_highlighted_track() {
let mut state = UiState {
open_album: Some("al0".into()),
sheet_index: 2,
..Default::default()
};
let got = handle_key("enter", false, false, &mut state, &empty(), &[0, 0, 0], 4);
assert_eq!(
got,
vec![Action::Play {
album_id: "al0".into(),
track_index: 2
}]
);
}
}

View File

@@ -1,504 +0,0 @@
//! Browsing and searching the library, entirely on the client.
//!
//! The whole index arrives in one response, so type-to-search has no round trip - which
//! is the point of a keyboard-first UI. Ported from `web/src/lib/search.ts`, including
//! the reason that module is shaped the way it is:
//!
//! The cost that matters is per keystroke, not per library load. A real collection is
//! 343 albums and 4969 tracks, and the naive version re-derives its search keys from
//! scratch on every one of them for every letter typed - `normalize` runs a lowercase,
//! a Unicode NFD decomposition and two filtering passes, so track search would mean
//! five thousand NFD normalizations before a single comparison. That is work whose
//! answer cannot change until the library does, so it happens once, in
//! [`Library::build`], and a keystroke becomes `str::contains` over strings that
//! already exist.
use std::collections::HashMap;
use std::rc::Rc;
use unicode_normalization::UnicodeNormalization;
use crate::api::Album;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Group {
Music,
Audiobooks,
Podcasts,
}
pub const GROUPS: [Group; 3] = [Group::Music, Group::Audiobooks, Group::Podcasts];
impl Group {
/// Which of the three shelves an album belongs on. A podcast is exactly a
/// `Kinderpodcasts`-section album; everything else buckets by `kind`, so a Figuren
/// album joins whichever of music/audiobooks matches what it actually holds.
pub fn of(album: &Album) -> Self {
if album.section == "Kinderpodcasts" {
Self::Podcasts
} else if album.is_book() {
Self::Audiobooks
} else {
Self::Music
}
}
pub fn index(self) -> usize {
match self {
Self::Music => 0,
Self::Audiobooks => 1,
Self::Podcasts => 2,
}
}
pub fn from_index(index: usize) -> Self {
GROUPS[index.min(2)]
}
pub fn label(self) -> &'static str {
match self {
Self::Music => "Musik",
Self::Audiobooks => "Hörbücher",
Self::Podcasts => "Podcasts",
}
}
pub fn icon(self) -> &'static str {
match self {
Self::Music => "🎵",
Self::Audiobooks => "📖",
Self::Podcasts => "🎙️",
}
}
/// Heading over the category tiles inside this group.
pub fn category_label(self) -> &'static str {
match self {
Self::Music => "Künstler",
Self::Audiobooks => "Figuren",
Self::Podcasts => "Sendungen",
}
}
/// Heading over the album grid inside this group.
pub fn section_label(self) -> &'static str {
match self {
Self::Music => "Alben",
Self::Audiobooks => "Hörbücher",
Self::Podcasts => "Episoden",
}
}
/// Per-group hue in oklch degrees, shared with the generated cover art so a shelf
/// already hints at the mood its albums play into.
pub fn hue(self) -> f32 {
match self {
Self::Music => 210.0, // today's sea - the baseline hue
Self::Audiobooks => 55.0, // warm amber
Self::Podcasts => 300.0, // the one otherwise-unused family
}
}
}
/// Diacritics and punctuation are noise when a child is hunting for letters.
pub fn normalize(value: &str) -> String {
value
.to_lowercase()
.nfd()
// Combining marks, i.e. what NFD just split off the base letters.
.filter(|c| !matches!(*c as u32, 0x0300..=0x036F))
.filter(|c| c.is_ascii_alphanumeric())
.collect()
}
/// Words of a query, normalized individually. A space-separated query like "conni rad"
/// should find "Conni lernt Rad fahren" even though "rad" is not right after "conni",
/// so requiring the whole phrase to be one contiguous substring is too strict.
fn search_words(value: &str) -> Vec<String> {
value
.split_whitespace()
.map(normalize)
.filter(|w| !w.is_empty())
.collect()
}
/// Every word has to show up somewhere in the haystack, in any order.
fn matches_words(haystack: &str, words: &[String]) -> bool {
words.iter().all(|word| haystack.contains(word.as_str()))
}
pub struct AlbumEntry {
pub album: Rc<Album>,
/// `normalize(title + artist)`, as a naive `album_matches` would recompute per key.
haystack: String,
}
pub struct TrackEntry {
pub album: Rc<Album>,
/// Index within `album.tracks`, which is what playback wants.
pub index: usize,
pub title: String,
pub duration: f64,
haystack: String,
}
#[derive(Clone)]
pub struct Category {
pub key: String,
pub albums: Vec<Rc<Album>>,
}
#[derive(Clone)]
pub struct SongHit {
pub album: Rc<Album>,
pub index: usize,
pub title: String,
pub duration: f64,
}
/// Everything a search needs, derived once per library payload.
#[derive(Default)]
pub struct Library {
pub albums: Vec<Rc<Album>>,
entries: Vec<AlbumEntry>,
by_group: [Vec<usize>; 3],
/// Each group's categories, already grouped and sorted.
categories_by_group: [Vec<Category>; 3],
/// Unlocked tracks only, album order then track order. A locked track shows as a
/// question mark wherever it appears, so it has no title to match on and is left
/// out of the index entirely rather than skipped at match time.
tracks: Vec<TrackEntry>,
tracks_by_group: [Vec<usize>; 3],
by_id: HashMap<String, Rc<Album>>,
}
impl Library {
pub fn build(albums: Vec<Album>) -> Self {
let mut library = Self::default();
// One map per group rather than one overall: two groups may legitimately hold
// the same category name, and merging them would put an artist's albums on
// the audiobook shelf.
let mut category_maps: [HashMap<String, Vec<Rc<Album>>>; 3] = Default::default();
for album in albums {
let album = Rc::new(album);
let group = Group::of(&album);
library.by_id.insert(album.id.clone(), Rc::clone(&album));
library.albums.push(Rc::clone(&album));
library.by_group[group.index()].push(library.entries.len());
library.entries.push(AlbumEntry {
haystack: normalize(&format!("{}{}", album.title, album.artist)),
album: Rc::clone(&album),
});
category_maps[group.index()]
.entry(album.category.clone())
.or_default()
.push(Rc::clone(&album));
for (index, track) in album.tracks.iter().enumerate() {
if track.locked {
continue;
}
library.tracks_by_group[group.index()].push(library.tracks.len());
library.tracks.push(TrackEntry {
haystack: normalize(&track.title),
album: Rc::clone(&album),
index,
title: track.title.clone(),
duration: track.duration,
});
}
}
for (slot, map) in library.categories_by_group.iter_mut().zip(category_maps) {
let mut categories: Vec<Category> = map
.into_iter()
.map(|(key, albums)| Category { key, albums })
.collect();
// `localeCompare(_, "de")` in the original. A full collator is more than
// this earns; normalizing first folds the umlauts, which is the whole of
// what German ordering asks for here.
categories.sort_by_key(|c| normalize(&c.key));
*slot = categories;
}
library
}
pub fn get(&self, album_id: &str) -> Option<&Rc<Album>> {
self.by_id.get(album_id)
}
pub fn categories(&self, group: Group) -> &[Category] {
&self.categories_by_group[group.index()]
}
}
// --------------------------------------------------------------------- queries --
/// `group: None` means unrestricted - typing at the root searches every group at once.
pub struct Query<'a> {
pub search: &'a str,
pub tracks_mode: bool,
pub group: Option<Group>,
pub category: Option<&'a str>,
}
/// Which of the three lists the browse view is showing.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ListMode {
Tracks,
Albums,
Categories,
}
/// Capped, because an empty query over 900 podcast episodes is not a useful screen.
pub const MAX_SONG_HITS: usize = 40;
/// Selection indices run flat across songs, then categories, then albums, so one pair
/// of arrow keys walks the whole page.
#[derive(Default)]
pub struct Results {
pub songs: Vec<SongHit>,
pub categories: Vec<Category>,
pub albums: Vec<Rc<Album>>,
}
impl Results {
pub fn total(&self) -> usize {
self.songs.len() + self.categories.len() + self.albums.len()
}
}
impl Library {
pub fn list_mode(&self, query: &Query) -> ListMode {
if query.tracks_mode {
ListMode::Tracks
} else if !query.search.is_empty() || query.category.is_some() {
ListMode::Albums
} else {
ListMode::Categories
}
}
fn album_pool(&self, query: &Query) -> Vec<usize> {
match query.group {
None => (0..self.entries.len()).collect(),
Some(group) => self.by_group[group.index()].clone(),
}
}
pub fn category_matches(&self, query: &Query) -> Vec<Category> {
// `group: None` is the bare root screen, rendered as three shelves instead of a
// flat category list - each shelf asks again with its own group filled in.
match (self.list_mode(query), query.group) {
(ListMode::Categories, Some(group)) => self.categories(group).to_vec(),
_ => Vec::new(),
}
}
pub fn album_matches(&self, query: &Query) -> Vec<Rc<Album>> {
if query.tracks_mode {
return Vec::new();
}
let words = search_words(query.search);
if words.is_empty() && query.category.is_none() {
return Vec::new();
}
self.album_pool(query)
.into_iter()
.map(|i| &self.entries[i])
.filter(|entry| match query.category {
Some(category) => entry.album.category == category,
None => true,
})
.filter(|entry| words.is_empty() || matches_words(&entry.haystack, &words))
.map(|entry| Rc::clone(&entry.album))
.collect()
}
pub fn song_matches(&self, query: &Query) -> Vec<SongHit> {
if !query.tracks_mode {
return Vec::new();
}
let words = search_words(query.search);
let pool: &[usize] = match query.group {
None => return self.take_song_hits(self.tracks.iter(), &words),
Some(group) => &self.tracks_by_group[group.index()],
};
self.take_song_hits(pool.iter().map(|&i| &self.tracks[i]), &words)
}
/// Stopping at the cap rather than filling every hit and slicing is what keeps an
/// empty query in tracks mode from walking all 4969 tracks to show 40 rows.
fn take_song_hits<'a>(
&self,
pool: impl Iterator<Item = &'a TrackEntry>,
words: &[String],
) -> Vec<SongHit> {
pool.filter(|track| words.is_empty() || matches_words(&track.haystack, words))
.take(MAX_SONG_HITS)
.map(|track| SongHit {
album: Rc::clone(&track.album),
index: track.index,
title: track.title.clone(),
duration: track.duration,
})
.collect()
}
pub fn results(&self, query: &Query) -> Results {
Results {
songs: self.song_matches(query),
categories: self.category_matches(query),
albums: self.album_matches(query),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn album(id: &str, title: &str, artist: &str, section: &str, tracks: &[&str]) -> Album {
serde_json::from_value(serde_json::json!({
"id": id,
"section": section,
"kind": if section == "Musik" { "music" } else { "book" },
"title": title,
"artist": artist,
"category": artist,
"tracks": tracks.iter().map(|t| serde_json::json!({ "title": t, "duration": 60.0 })).collect::<Vec<_>>(),
}))
.unwrap()
}
fn fixture() -> Library {
Library::build(vec![
album(
"a",
"Grüffelo",
"Julia Donaldson",
"Hörbücher",
&["Kapitel 1", "Kapitel 2"],
),
album("b", "Bibi Blocksberg", "Bibi", "Hörbücher", &["Hexerei"]),
album(
"c",
"Rote Lieder",
"Rolf Zuckowski",
"Musik",
&["Wie schön"],
),
album(
"d",
"Maus Folge 3",
"Sendung mit der Maus",
"Kinderpodcasts",
&["Folge 3"],
),
])
}
#[test]
fn normalize_folds_umlauts_and_punctuation() {
assert_eq!(normalize("Hörbücher!"), "horbucher");
assert_eq!(normalize("Grüffelo"), "gruffelo");
assert_eq!(normalize("A-B C"), "abc");
}
#[test]
fn groups_split_podcasts_out_of_books() {
let library = fixture();
assert_eq!(Group::of(library.get("d").unwrap()), Group::Podcasts);
assert_eq!(Group::of(library.get("a").unwrap()), Group::Audiobooks);
assert_eq!(Group::of(library.get("c").unwrap()), Group::Music);
}
#[test]
fn album_search_is_word_order_independent() {
let library = fixture();
let hits = library.album_matches(&Query {
search: "donaldson gruffelo",
tracks_mode: false,
group: None,
category: None,
});
assert_eq!(hits.len(), 1);
assert_eq!(hits[0].id, "a");
}
#[test]
fn an_empty_query_lists_no_albums_but_does_list_categories() {
let library = fixture();
let query = Query {
search: "",
tracks_mode: false,
group: Some(Group::Audiobooks),
category: None,
};
assert!(library.album_matches(&query).is_empty());
assert_eq!(library.category_matches(&query).len(), 2);
}
#[test]
fn the_root_screen_has_no_flat_category_list_of_its_own() {
let library = fixture();
let query = Query {
search: "",
tracks_mode: false,
group: None,
category: None,
};
assert!(library.category_matches(&query).is_empty());
}
#[test]
fn a_category_lists_its_albums_without_a_search() {
let library = fixture();
let hits = library.album_matches(&Query {
search: "",
tracks_mode: false,
group: Some(Group::Audiobooks),
category: Some("Bibi"),
});
assert_eq!(hits.len(), 1);
assert_eq!(hits[0].id, "b");
}
#[test]
fn track_search_is_capped_and_group_scoped() {
let library = fixture();
let hits = library.song_matches(&Query {
search: "kapitel",
tracks_mode: true,
group: None,
category: None,
});
assert_eq!(hits.len(), 2);
assert!(hits.len() <= MAX_SONG_HITS);
let scoped = library.song_matches(&Query {
search: "",
tracks_mode: true,
group: Some(Group::Music),
category: None,
});
assert_eq!(scoped.len(), 1);
assert_eq!(scoped[0].title, "Wie schön");
}
#[test]
fn categories_sort_by_folded_key() {
let library = fixture();
let keys: Vec<&str> = library
.categories(Group::Audiobooks)
.iter()
.map(|c| c.key.as_str())
.collect();
assert_eq!(keys, ["Bibi", "Julia Donaldson"]);
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,95 +0,0 @@
//! oklch -> sRGB.
//!
//! The web front-end's palette is oklch throughout (`web/src/styles/app.css`) and
//! Slint has no such colour space, so every value crossing over is converted here.
//! Ported from `web/src/lib/oklch.ts`.
//!
//! The static palette is converted once at build time into the literals in
//! `ui/theme.slint`; this module exists for the values that are only known at runtime -
//! the per-group hues behind the generated book-spine art.
/// `l` is 0..1 (not the 0..100% that CSS writes), `c` is chroma, `h` is degrees.
pub fn oklch_to_rgb(l: f32, c: f32, h: f32) -> [u8; 3] {
let (sin, cos) = (h.to_radians().sin(), h.to_radians().cos());
let (a, b) = (c * cos, c * sin);
let l_ = l + 0.396_337_78 * a + 0.215_803_76 * b;
let m_ = l - 0.105_561_34 * a - 0.063_854_17 * b;
let s_ = l - 0.089_484_18 * a - 1.291_485_5 * b;
let (l3, m3, s3) = (l_ * l_ * l_, m_ * m_ * m_, s_ * s_ * s_);
let lin = [
4.076_741_7 * l3 - 3.307_711_6 * m3 + 0.230_969_94 * s3,
-1.268_438 * l3 + 2.609_757_4 * m3 - 0.341_319_38 * s3,
-0.004_196_086_3 * l3 - 0.703_418_6 * m3 + 1.707_614_7 * s3,
];
lin.map(|v| {
// Gamut clipping is per channel and deliberately crude: every colour this app
// actually asks for is inside sRGB, so a channel landing outside it means a
// typo, and a hard clamp shows that as a flat primary rather than hiding it.
let v = v.clamp(0.0, 1.0);
let encoded = if v <= 0.003_130_8 {
12.92 * v
} else {
1.055 * v.powf(1.0 / 2.4) - 0.055
};
(encoded * 255.0).round().clamp(0.0, 255.0) as u8
})
}
/// `#rrggbb` as the backend writes it. Anything unparseable falls back rather than
/// failing: the colours are decoration, and an album with a malformed one should still
/// appear on the shelf.
pub fn parse_hex(hex: &str, fallback: [u8; 3]) -> [u8; 3] {
let digits = hex.trim().trim_start_matches('#');
if digits.len() != 6 {
return fallback;
}
let byte = |i: usize| u8::from_str_radix(&digits[i..i + 2], 16).ok();
match (byte(0), byte(2), byte(4)) {
(Some(r), Some(g), Some(b)) => [r, g, b],
_ => fallback,
}
}
/// Linear blend, for the spine's translucent highlight lines.
pub fn mix(base: [u8; 3], over: [u8; 3], alpha: f32) -> [u8; 3] {
let alpha = alpha.clamp(0.0, 1.0);
[0, 1, 2].map(|i| (base[i] as f32 * (1.0 - alpha) + over[i] as f32 * alpha).round() as u8)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn known_anchors_round_trip_into_srgb() {
// Pure white and pure black are the two values with no room to be subtly wrong.
assert_eq!(oklch_to_rgb(1.0, 0.0, 0.0), [255, 255, 255]);
assert_eq!(oklch_to_rgb(0.0, 0.0, 0.0), [0, 0, 0]);
}
#[test]
fn the_accent_pink_lands_where_the_web_ui_puts_it() {
// oklch(70% 0.16 340) is `--accent`; the browser resolves it to ~#e A pink in
// the red-magenta corner. Assert the shape of it rather than an exact byte.
let [r, g, b] = oklch_to_rgb(0.70, 0.16, 340.0);
assert!(r > g && b > g, "expected a pink, got {r},{g},{b}");
assert!(r > 200, "expected a light pink, got {r},{g},{b}");
}
#[test]
fn hex_parsing_falls_back_rather_than_failing() {
assert_eq!(parse_hex("#4a6fa5", [0, 0, 0]), [0x4a, 0x6f, 0xa5]);
assert_eq!(parse_hex("4a6fa5", [0, 0, 0]), [0x4a, 0x6f, 0xa5]);
assert_eq!(parse_hex("nope", [1, 2, 3]), [1, 2, 3]);
assert_eq!(parse_hex("", [1, 2, 3]), [1, 2, 3]);
}
#[test]
fn mix_interpolates_between_the_two_ends() {
assert_eq!(mix([0, 0, 0], [255, 255, 255], 0.0), [0, 0, 0]);
assert_eq!(mix([0, 0, 0], [255, 255, 255], 1.0), [255, 255, 255]);
assert_eq!(mix([0, 0, 0], [200, 100, 50], 0.5), [100, 50, 25]);
}
}

View File

@@ -1,97 +0,0 @@
//! The state websocket, with reconnect-and-reseed.
//!
//! Ported from `web/src/hooks/usePlayerState.ts`. The backend pushes and never listens,
//! so this is a read loop and nothing else. Two details are load-bearing:
//!
//! * **Reseed on open, not just on connect.** A reconnect can have missed every change
//! between the drop and the new socket, and the server only sends a snapshot to a
//! client it already considers connected. So each successful open is followed by a
//! `GET /api/state`.
//! * **Position is not on the event bus.** The server polls its player on a 0.5 s timer
//! and sends `position` frames only while something is playing - so the absence of
//! them is not a stall, and the UI interpolates between them rather than waiting.
use std::sync::mpsc::Sender;
use std::time::Duration;
use crate::api::{Client, PlayerState, ServerMessage};
/// What the UI thread hears from this connection. Wider than `ServerMessage` because
/// connectivity is itself a thing the screen shows.
pub enum Update {
Connected,
Disconnected,
State(Box<PlayerState>),
Position {
position: f64,
duration: f64,
},
/// The library was rescanned: re-fetch it, and drop the cover thumbnails, since a
/// rescan is exactly when the art behind an unchanged URL gets rewritten.
LibraryChanged,
}
/// Matches the web client's retry interval. Short enough that the screen comes back
/// before a child gives up on it, long enough not to spin while the backend restarts.
const RETRY: Duration = Duration::from_millis(1500);
pub fn spawn(client: Client, updates: Sender<Update>) -> std::thread::JoinHandle<()> {
std::thread::Builder::new()
.name("ws".into())
.spawn(move || loop {
match connect_once(&client, &updates) {
Ok(()) | Err(()) => {}
}
// `send` failing means the UI is gone, which is the one reason to stop.
if updates.send(Update::Disconnected).is_err() {
return;
}
std::thread::sleep(RETRY);
})
.expect("spawning the websocket thread")
}
fn connect_once(client: &Client, updates: &Sender<Update>) -> Result<(), ()> {
let (mut socket, _) = tungstenite::connect(client.websocket_url()).map_err(|_| ())?;
if updates.send(Update::Connected).is_err() {
return Err(());
}
// Reseed: see the note at the top of this module.
if let Ok(state) = client.state() {
if updates.send(Update::State(Box::new(state))).is_err() {
return Err(());
}
}
loop {
let message = match socket.read() {
Ok(message) => message,
Err(_) => return Err(()),
};
let text = match message {
tungstenite::Message::Text(text) => text,
tungstenite::Message::Close(_) => return Ok(()),
// Ping/Pong are answered by tungstenite itself on the next write; binary
// frames are not part of this protocol.
_ => continue,
};
let parsed: ServerMessage = match serde_json::from_str(&text) {
Ok(parsed) => parsed,
// A frame this build cannot read is not a reason to drop the connection -
// the next one is probably fine.
Err(_) => continue,
};
let update = match parsed {
ServerMessage::State { state } => Update::State(Box::new(state)),
ServerMessage::Position { position, duration } => {
Update::Position { position, duration }
}
ServerMessage::Library => Update::LibraryChanged,
ServerMessage::Unknown => continue,
};
if updates.send(update).is_err() {
return Err(());
}
}
}

View File

@@ -1,56 +0,0 @@
#!/usr/bin/env bash
#
# Render the UI and screenshot it without touching your session.
#
# A Slint window needs a compositor: under bare Xvfb nothing maps, and on a locked or
# remote machine there is no session to draw into. So this starts a nested headless
# sway, runs the app inside it, drives it with synthetic keys and grabs a frame per
# screen. It is how the shots in README.md were made, and how to check a layout change
# on a machine with no display.
#
# nix-shell -E '(import ./shell.nix {}).overrideAttrs (o: {
# nativeBuildInputs = o.nativeBuildInputs ++ (with (import <nixpkgs> {}); [ sway grim wtype ]);
# })' --run 'tools/headless-shots.sh out/ http://127.0.0.1:8080'
#
set -euo pipefail
OUTDIR="${1:?usage: headless-shots.sh <outdir> [server-url]}"
SERVER="${2:-http://127.0.0.1:8080}"
BIN="${BIN:-./target/release/musicmouse-slint}"
mkdir -p "$OUTDIR"
export XDG_RUNTIME_DIR="$(mktemp -d)"
unset WAYLAND_DISPLAY DISPLAY
export WLR_BACKENDS=headless WLR_LIBINPUT_NO_DEVICES=1 WLR_RENDERER=pixman
trap 'kill %1 2>/dev/null || true; rm -rf "$XDG_RUNTIME_DIR"' EXIT
launcher="$XDG_RUNTIME_DIR/launch.sh"
printf '#!/usr/bin/env bash\nexec %q --server %q >%q/app.log 2>&1\n' \
"$(realpath "$BIN")" "$SERVER" "$XDG_RUNTIME_DIR" > "$launcher"
chmod +x "$launcher"
printf 'output HEADLESS-1 mode 1920x1080\ndefault_border none\nexec %s\n' "$launcher" \
> "$XDG_RUNTIME_DIR/sway.cfg"
sway -c "$XDG_RUNTIME_DIR/sway.cfg" >"$XDG_RUNTIME_DIR/sway.log" 2>&1 &
for _ in $(seq 30); do [ -S "$XDG_RUNTIME_DIR/wayland-1" ] && break; sleep 1; done
export WAYLAND_DISPLAY=wayland-1
sleep 16 # library fetch plus the first covers
# Each step is ONE wtype invocation, and starts with a throwaway keypress.
#
# wtype creates a virtual keyboard, sends, and exits. The client needs a moment to
# notice the seat gained a keyboard, so the first key or two of any invocation is
# dropped - which makes a per-key loop lose every key it sends. Batching a step into
# one invocation and opening it with "x" + BackSpace spends that warm-up on a keypress
# that cancels itself out.
WARM=(-s 260 "x" -k BackSpace)
shot() { sleep 2.5; grim "$OUTDIR/$1.png"; echo " $1"; }
shot 01-root
wtype "${WARM[@]}" -k Down -k Return; shot 02-categories
wtype "${WARM[@]}" -k Return; shot 03-grid
wtype "${WARM[@]}" -k Escape -k Escape "maja"; shot 04-search
wtype "${WARM[@]}" -k Escape -k question "brumm"; shot 05-tracksearch
wtype "${WARM[@]}" -k Return; sleep 4; shot 06-play
wtype "${WARM[@]}" -k Escape; shot 07-bar
echo "wrote 7 frames to $OUTDIR"

View File

@@ -1,347 +0,0 @@
import { Theme } from "theme.slint";
import { RoundButton } from "widgets.slint";
import { ShelfView, ResultsView } from "browse.slint";
import { PlayerBar, PlayView, AlbumSheet } from "player.slint";
import { RoomView } from "room.slint";
import { AlbumTile, GridRow, CategoryTile, CategoryRow, Shelf, TrackRow, NowPlaying, RoomCard, RoomRow, SceneTile, Swatch, Tab } from "models.slint";
export { AlbumTile, GridRow, CategoryTile, CategoryRow, Shelf, TrackRow, NowPlaying, RoomCard, RoomRow, SceneTile, Swatch, Tab, Theme }
export component AppWindow inherits Window {
title: "Musik Delfin";
default-font-family: Theme.font;
default-font-size: Theme.text-body;
// The stage. In the web app a per-track animated canvas sits on top of this; here
// it is the whole background, on purpose. A full-viewport repaint loop is a floor
// you cannot get under while it runs at all, and it is the first thing `?pi=1`
// already removes on the device this is meant to replace.
background: @linear-gradient(180deg,
Theme.stage-top 0%,
Theme.stage-mid 45%,
Theme.sea-deep 100%);
// ------------------------------------------------------------------- inputs --
// "music" | "room" | "typing" - named rather than an enum so Rust owns the
// ordering and the labels, as it does for every other string on screen.
in property <string> page: "music";
in property <[Tab]> tabs;
in property <[RoomRow]> room-rows;
in property <[SceneTile]> room-scenes;
in property <[Swatch]> room-swatches;
in property <[string]> shutter-presets;
in property <bool> room-loading: true;
in property <bool> loading: true;
in property <string> splash-text;
in property <string> status-label;
in property <bool> online: false;
in property <bool> play-view: false;
in property <bool> show-sheet: false;
in property <bool> can-go-back: false;
in property <[Shelf]> shelves;
in property <bool> root-screen: true;
in property <int> shelf-row: 0;
in property <string> group-label;
in property <string> category-heading;
in property <string> album-heading;
in property <bool> show-search: false;
in property <string> search-text;
in property <string> mode-label;
in property <string> count-label;
in property <[TrackRow]> songs;
in property <[CategoryRow]> category-grid;
in property <[GridRow]> grid;
in property <bool> no-results: false;
in property <int> sel-index: 0;
in property <int> sel-row: 0;
in property <NowPlaying> now;
in property <bool> playing: false;
in property <float> progress: 0.0;
in property <bool> smooth: true;
in property <int> volume: 60;
in property <bool> has-bar: false;
in property <string> sheet-title;
in property <string> sheet-subtitle;
in property <string> sheet-meta;
in property <image> sheet-cover;
in property <bool> sheet-is-book: false;
in property <[TrackRow]> sheet-tracks;
in property <int> sheet-index: 0;
// How wide the grid may be. Rust reads this back to decide the column count, so the
// chunking into `GridRow`s and the layout agree about how many fit.
out property <length> grid-width: self.width - 32px - Theme.rail-clearance;
// ---------------------------------------------------------------- callbacks --
// The special keys are named here rather than in Rust because this is where the
// `Key.*` constants exist; Rust receives "escape"/"left"/"enter"/… or the literal
// character that was typed.
callback key(string, bool, bool);
callback toggle();
callback next();
callback previous();
callback seek(float);
callback set-volume(int);
callback mute();
callback go-back();
callback enter-group(int);
callback enter-category(int, string);
callback open-category(string);
callback open-album(string);
callback play-album(string);
callback play-song(string, int);
callback open-play-view();
callback close-sheet();
callback play-sheet-track(int);
callback select-page(string);
callback toggle-light(string);
callback set-light-level(string, int);
callback set-light-swatch(string, int);
callback set-shutter-preset(string, int);
callback shutter(string, string);
callback activate-scene(string);
forward-focus: keys;
keys := FocusScope {
key-pressed(event) => {
root.key(
event.text == Key.Escape ? "escape"
: event.text == Key.Return ? "enter"
: event.text == Key.Backspace ? "backspace"
: event.text == Key.Tab ? "tab"
: event.text == Key.LeftArrow ? "left"
: event.text == Key.RightArrow ? "right"
: event.text == Key.UpArrow ? "up"
: event.text == Key.DownArrow ? "down"
: event.text == Key.F1 ? "f1"
: event.text == Key.Space ? "space"
: event.text,
event.modifiers.control,
event.modifiers.shift);
accept
}
VerticalLayout {
// ------------------------------------------------------------ header --
if root.page == "music" && !root.play-view: HorizontalLayout {
alignment: center;
spacing: 16px;
padding-top: 18px;
padding-bottom: 6px;
Text {
text: "Musik Delfin";
font-size: 34px;
font-weight: 900;
color: Theme.paper;
}
if root.status-label != "": VerticalLayout {
alignment: center;
Rectangle {
background: Theme.paper.with-alpha(0.16);
border-radius: Theme.pill;
width: status.preferred-width;
height: status.preferred-height;
status := HorizontalLayout {
padding-left: 14px;
padding-right: 14px;
padding-top: 6px;
padding-bottom: 6px;
Text {
text: root.status-label;
font-size: Theme.text-small + 1px;
font-weight: 800;
color: root.online ? Theme.paper : #ffb4a2;
}
}
}
}
}
// -------------------------------------------------------------- body --
if root.page == "music" && !root.play-view && root.root-screen: ShelfView {
shelves: root.shelves;
shelf-row: root.shelf-row;
sel-index: root.sel-index;
enter-group(s) => { root.enter-group(s); }
enter-category(s, key) => { root.enter-category(s, key); }
}
if root.page == "music" && !root.play-view && !root.root-screen: ResultsView {
group-label: root.group-label;
category-heading: root.category-heading;
album-heading: root.album-heading;
show-search: root.show-search;
search-text: root.search-text;
mode-label: root.mode-label;
count-label: root.count-label;
songs: root.songs;
category-grid: root.category-grid;
grid: root.grid;
sel-index: root.sel-index;
sel-row: root.sel-row;
empty: root.no-results;
play-song(id, i) => { root.play-song(id, i); }
open-category(key) => { root.open-category(key); }
open-album(id) => { root.open-album(id); }
play-album(id) => { root.play-album(id); }
}
if root.page == "room": RoomView {
rows: root.room-rows;
scenes: root.room-scenes;
swatches: root.room-swatches;
presets: root.shutter-presets;
loading: root.room-loading;
toggle-light(id) => { root.toggle-light(id); }
set-level(id, n) => { root.set-light-level(id, n); }
set-swatch(id, i) => { root.set-light-swatch(id, i); }
set-preset(id, i) => { root.set-shutter-preset(id, i); }
shutter(id, what) => { root.shutter(id, what); }
activate-scene(id) => { root.activate-scene(id); }
}
if root.page == "typing": VerticalLayout {
alignment: center;
Text {
text: "Tippen";
font-size: 34px;
font-weight: 900;
color: Theme.paper;
horizontal-alignment: center;
}
Text {
text: "Noch nicht portiert.";
font-size: Theme.text-heading;
font-weight: 700;
color: Theme.text-dim;
horizontal-alignment: center;
}
}
if root.page == "music" && root.play-view: PlayView {
now: root.now;
playing: root.playing;
progress: root.progress;
smooth: root.smooth;
volume: root.volume;
toggle => { root.toggle(); }
next => { root.next(); }
previous => { root.previous(); }
seek(f) => { root.seek(f); }
set-volume(v) => { root.set-volume(v); }
mute => { root.mute(); }
open-album => { root.open-album(root.now.title); }
}
}
}
// The player bar floats over the browse screen rather than displacing it, which is
// why the scrollers above reserve bottom padding for it. Geometry is set here
// rather than inside the component: a plain element takes its *preferred* width
// from its content, not its parent's width, so a bar that only declared a height
// left gaps for the grid behind it to show through.
if root.page == "music" && !root.play-view && root.has-bar: PlayerBar {
x: 0;
y: parent.height - self.height;
width: parent.width;
now: root.now;
playing: root.playing;
progress: root.progress;
smooth: root.smooth;
volume: root.volume;
toggle => { root.toggle(); }
next => { root.next(); }
previous => { root.previous(); }
seek(f) => { root.seek(f); }
set-volume(v) => { root.set-volume(v); }
mute => { root.mute(); }
open-play-view => { root.open-play-view(); }
}
// One fixed round button in the top-left: back, everywhere it applies, so "top
// left" always means the same thing.
if root.can-go-back: RoundButton {
x: 24px;
y: 18px;
label: "‹";
clicked => { root.go-back(); }
}
// The vertical tab rail. Right edge, vertically centred - the one fixed control
// that is in the same place on every page.
tab-rail := Rectangle {
x: parent.width - self.width - 16px;
y: (parent.height - self.height) / 2;
width: 60px;
height: rail.preferred-height;
border-radius: 30px;
background: Theme.paper.with-alpha(0.22);
border-width: 1px;
border-color: Theme.paper.with-alpha(0.3);
rail := VerticalLayout {
padding: 8px;
spacing: 6px;
for tab[i] in root.tabs: Rectangle {
width: 44px;
height: 44px;
border-radius: 22px;
background: root.page == tab.id ? Theme.accent : transparent;
animate background { duration: 150ms; }
Text {
text: tab.icon;
font-size: 20px;
color: root.page == tab.id ? white : Theme.paper;
vertical-alignment: center;
horizontal-alignment: center;
}
TouchArea {
clicked => { root.select-page(tab.id); }
}
}
}
}
if root.show-sheet: AlbumSheet {
width: 100%;
height: 100%;
title: root.sheet-title;
subtitle: root.sheet-subtitle;
meta: root.sheet-meta;
cover: root.sheet-cover;
is-book: root.sheet-is-book;
tracks: root.sheet-tracks;
sel-index: root.sheet-index;
close => { root.close-sheet(); }
play(i) => { root.play-sheet-track(i); }
play-all => { root.play-sheet-track(0); }
}
if root.loading: Rectangle {
background: Theme.sea-deep.with-alpha(0.92);
TouchArea { }
VerticalLayout {
alignment: center;
spacing: 18px;
Text {
text: "🐬";
font-size: 96px;
horizontal-alignment: center;
}
Text {
text: root.splash-text;
font-size: 22px;
font-weight: 800;
color: Theme.paper;
horizontal-alignment: center;
}
}
}
}

View File

@@ -1,567 +0,0 @@
import { ListView } from "std-widgets.slint";
import { Theme } from "theme.slint";
import { GlassPanel, Cover } from "widgets.slint";
import { AlbumTile, GridRow, CategoryTile, CategoryRow, Shelf, TrackRow } from "models.slint";
// Heights are fixed rather than derived from content, and that is the whole reason the
// grid can be virtualized: a `ListView` that has to measure a row before it can decide
// whether to skip it has not skipped anything. The web grid does the same thing with
// `contain-intrinsic-size` for the same reason.
//
// A book is taller than an album, so the cover *slot* is sized for the taller of the
// two and each cover is centred in it. That keeps every row identical without
// flattening the shape difference that tells the two media apart.
export global Metrics {
out property <length> cover-slot: 200px;
out property <length> card-text: 78px;
out property <length> card-height: 278px; // cover-slot + card-text
out property <length> row-height: 302px; // card-height + grid-gap
out property <length> shelf-height: 236px;
out property <length> category-row-height: 260px;
}
component SelectionRing inherits Rectangle {
in property <bool> selected;
in property <bool> current;
// Selection outranks "currently playing": when the keyboard is on the album that is
// also playing, the ring it needs to see is the one that moves.
border-width: root.selected ? 5px : (root.current ? 4px : 0px);
border-color: root.selected ? Theme.paper : Theme.accent;
border-radius: Theme.radius-card + 6px;
}
component AlbumCard inherits Rectangle {
in property <AlbumTile> tile;
in property <bool> selected;
callback open();
callback play();
height: Metrics.card-height;
ring := SelectionRing {
selected: root.selected;
current: root.tile.current;
x: -7px;
y: -7px;
width: parent.width + 14px;
height: parent.height + 14px;
}
card := Rectangle {
background: root.tile.is-book ? Theme.card-book : Theme.card-music;
border-radius: Theme.radius-card;
drop-shadow-color: #00101759;
drop-shadow-blur: 18px;
drop-shadow-offset-y: 6px;
// The lift on selection: 120ms, the same as the web card's transition.
y: root.selected ? -3px : 0px;
animate y { duration: 120ms; easing: ease-out; }
VerticalLayout {
padding: 10px;
spacing: 6px;
// The cover, centred in a slot tall enough for the taller media type.
Rectangle {
height: Metrics.cover-slot - 20px;
HorizontalLayout {
alignment: center;
Cover {
source: root.tile.cover;
is-book: root.tile.is-book;
height: parent.height;
width: root.tile.is-book ? parent.height * 0.82 : parent.height;
}
}
TouchArea {
// Cover opens the track list; the title below starts it playing.
// Two targets, because "show me what is in this" and "play this"
// are different intentions and a child has both.
clicked => { root.open(); }
}
}
Rectangle {
height: Metrics.card-text - 26px;
VerticalLayout {
spacing: 1px;
Text {
text: root.tile.title;
font-size: Theme.text-body - 1px;
font-weight: 800;
color: Theme.ink;
wrap: word-wrap;
overflow: elide;
// Two lines always, whether or not the title needs them, so
// every card in the row ends at the same place.
height: (Theme.text-body - 1px) * 2.4;
}
Text {
text: root.tile.artist;
font-size: Theme.text-small - 1px;
font-weight: 700;
color: Theme.ink.with-alpha(0.65);
overflow: elide;
}
}
TouchArea {
clicked => { root.play(); }
}
}
Text {
text: root.tile.meta;
font-size: Theme.text-tiny;
font-weight: 800;
color: Theme.count-pink;
overflow: elide;
}
}
}
}
component CategoryCard inherits Rectangle {
in property <CategoryTile> tile;
in property <bool> selected;
in property <length> tile-width: Theme.shelf-tile;
callback open();
width: root.tile-width;
height: Metrics.shelf-height;
SelectionRing {
selected: root.selected;
current: false;
x: -7px;
y: -7px;
width: parent.width + 14px;
height: parent.height + 14px;
}
Rectangle {
background: root.tile.is-book ? Theme.card-book : Theme.card-music;
border-radius: Theme.radius-card;
drop-shadow-color: #00101759;
drop-shadow-blur: 18px;
drop-shadow-offset-y: 6px;
y: root.selected ? -3px : 0px;
animate y { duration: 120ms; easing: ease-out; }
VerticalLayout {
padding: 8px;
spacing: 5px;
// One album fills the tile; several show as a 2x2 contact sheet, which is
// how a category reads as "a stack of these" without a label saying so.
Rectangle {
height: root.tile-width - 16px;
clip: true;
if root.tile.cover-count <= 1: Cover {
source: root.tile.cover1;
is-book: root.tile.is-book;
width: 100%;
height: 100%;
}
if root.tile.cover-count > 1: VerticalLayout {
spacing: 4px;
HorizontalLayout {
spacing: 4px;
Cover { source: root.tile.cover1; radius: 6px; }
Cover { source: root.tile.cover2; radius: 6px; }
}
HorizontalLayout {
spacing: 4px;
Cover { source: root.tile.cover3; radius: 6px; }
Cover { source: root.tile.cover4; radius: 6px; }
}
}
}
Text {
text: root.tile.key;
font-size: Theme.text-body;
font-weight: 800;
color: Theme.ink;
overflow: elide;
}
Text {
text: root.tile.count-label;
font-size: Theme.text-tiny;
font-weight: 800;
color: Theme.count-pink;
overflow: elide;
}
}
TouchArea {
clicked => { root.open(); }
}
}
}
component TrackListRow inherits Rectangle {
in property <TrackRow> track;
in property <bool> selected;
callback play();
height: 58px;
background: root.track.current ? Theme.row-active : Theme.row-idle;
border-radius: Theme.radius-row;
border-width: root.selected ? 3px : 1px;
border-color: root.selected ? Theme.paper : (root.track.current ? Theme.accent : Theme.glass-border);
HorizontalLayout {
padding-left: 12px;
padding-right: 16px;
spacing: 12px;
VerticalLayout {
alignment: center;
Cover {
source: root.track.cover;
is-book: root.track.is-book;
radius: 8px;
height: 40px;
width: root.track.is-book ? 33px : 40px;
}
}
VerticalLayout {
alignment: center;
Text {
text: root.track.title;
font-size: Theme.text-body;
font-weight: 800;
color: Theme.paper;
overflow: elide;
}
Text {
text: root.track.subtitle;
font-size: Theme.text-tiny;
font-weight: 700;
color: Theme.text-sub.with-alpha(0.75);
overflow: elide;
}
}
VerticalLayout {
alignment: center;
Text {
text: root.track.duration;
font-family: Theme.mono;
font-size: Theme.text-small;
font-weight: 700;
color: Theme.text-dim.with-alpha(0.8);
}
}
}
TouchArea {
clicked => { root.play(); }
}
}
// The root screen: three shelves, each scrolling sideways.
export component ShelfView inherits Flickable {
in property <[Shelf]> shelves;
in property <int> shelf-row;
in property <int> sel-index;
callback enter-group(int); // shelf index
callback enter-category(int, string); // shelf index, category key
content-height: layout.preferred-height;
content-width: self.width;
layout := VerticalLayout {
padding-left: 32px;
padding-right: Theme.rail-clearance;
padding-top: 14px;
// Clear of the player bar, which floats over this rather than displacing it.
padding-bottom: 210px;
spacing: 25px;
for shelf[s] in root.shelves: GlassPanel {
height: Metrics.shelf-height + 74px;
// The panel itself is a target: it opens the whole group.
TouchArea {
clicked => { root.enter-group(s); }
}
VerticalLayout {
padding: 14px;
spacing: 10px;
HorizontalLayout {
spacing: 10px;
alignment: start;
// A round frosted badge behind the emoji, which otherwise renders
// thin and low-contrast straight on the stage gradient.
Rectangle {
width: 34px;
height: 34px;
border-radius: 17px;
background: Theme.paper.with-alpha(0.2);
border-width: 1px;
border-color: Theme.paper.with-alpha(0.4);
Text {
text: shelf.icon;
font-size: 17px;
vertical-alignment: center;
horizontal-alignment: center;
}
}
VerticalLayout {
alignment: center;
Text {
text: shelf.label;
font-size: Theme.text-heading;
font-weight: 900;
color: Theme.paper;
}
}
VerticalLayout {
alignment: center;
Text {
text: "›";
font-size: 17px;
font-weight: 900;
color: Theme.paper.with-alpha(0.6);
}
}
}
if shelf.tiles.length == 0: Text {
text: shelf.empty-label;
font-size: Theme.text-small;
font-weight: 700;
color: Theme.text-dim.with-alpha(0.6);
horizontal-alignment: center;
vertical-alignment: center;
}
if shelf.tiles.length > 0: Flickable {
content-width: shelf.tiles.length * (Theme.shelf-tile + 12px) + 20px;
content-height: self.height;
// Keep the focused tile in view as the arrow keys walk past the fold.
content-x: root.shelf-row != s
? 0px
: -max(0px, min(
self.content-width - self.width,
root.sel-index * (Theme.shelf-tile + 12px) - self.width / 2 + Theme.shelf-tile / 2));
animate content-x { duration: 180ms; easing: ease-out; }
HorizontalLayout {
spacing: 12px;
padding: 10px;
alignment: start;
for tile[i] in shelf.tiles: CategoryCard {
tile: tile;
selected: root.shelf-row == s && root.sel-index == i;
open => { root.enter-category(s, tile.key); }
}
}
}
}
}
}
}
// Inside a group, or showing search results.
//
// Exactly one of the three lists is ever populated, and that is a property of the query
// model rather than a coincidence worth defending against: track hits only exist in
// track-search mode, which forces the album list empty; the category list only exists
// when there is neither a query nor a chosen category, which is exactly when the album
// list is empty. So this shows one list filling the page, instead of stacking three
// inside a scroller and nesting a second scroller in it.
export component ResultsView inherits VerticalLayout {
in property <string> group-label;
in property <string> category-heading;
in property <string> album-heading;
in property <bool> show-search;
in property <string> search-text;
in property <string> mode-label;
in property <string> count-label;
in property <[TrackRow]> songs;
in property <[CategoryRow]> category-grid;
in property <[GridRow]> grid;
in property <int> sel-index;
/// Which row of the visible grid holds the selection. Every row is the same height
/// by construction, so following the keyboard is arithmetic, not measurement.
in property <int> sel-row;
in property <bool> empty;
callback play-song(string, int);
callback open-category(string);
callback open-album(string);
callback play-album(string);
spacing: 6px;
padding-left: 32px;
padding-right: Theme.rail-clearance;
if root.group-label != "": Text {
text: root.group-label;
font-size: 15px;
font-weight: 800;
color: Theme.paper;
horizontal-alignment: center;
}
if root.show-search: HorizontalLayout {
alignment: center;
spacing: 12px;
padding-bottom: 4px;
Rectangle {
background: Theme.paper;
border-radius: Theme.pill;
drop-shadow-color: #00101759;
drop-shadow-blur: 14px;
drop-shadow-offset-y: 4px;
width: chip.preferred-width;
height: chip.preferred-height;
chip := HorizontalLayout {
padding-left: 10px;
padding-right: 20px;
padding-top: 10px;
padding-bottom: 10px;
spacing: 10px;
Rectangle {
background: Theme.ink;
border-radius: Theme.pill;
width: label.preferred-width;
label := HorizontalLayout {
padding-left: 10px;
padding-right: 10px;
padding-top: 4px;
padding-bottom: 4px;
Text {
text: root.mode-label;
font-size: Theme.text-small + 1px;
font-weight: 800;
color: white;
}
}
}
Text {
text: root.search-text;
font-size: Theme.text-heading;
font-weight: 800;
color: Theme.ink;
vertical-alignment: center;
}
}
}
VerticalLayout {
alignment: center;
Text {
text: root.count-label;
font-size: Theme.text-small + 1px;
font-weight: 700;
color: Theme.text-dim.with-alpha(0.85);
}
}
}
if root.empty: VerticalLayout {
alignment: center;
Text {
text: "Nichts gefunden — probier andere Buchstaben!";
font-size: Theme.text-heading;
font-weight: 800;
color: Theme.paper;
horizontal-alignment: center;
}
}
// --- track hits -----------------------------------------------------------
// Capped at 40 rows upstream, so this one does not need virtualizing. Held to a
// readable column rather than the full width, which is what lets a long album
// line elide instead of running the width of a 1080p screen. The heading rides
// inside that column so the two are aligned with each other, not with the page.
if root.songs.length > 0: Flickable {
content-height: songs-column.preferred-height + 210px;
content-width: self.width;
HorizontalLayout {
alignment: center;
songs-column := VerticalLayout {
// Without this the column is centred in whatever vertical space is
// left over, which on a short result list leaves it floating.
alignment: start;
width: 860px;
spacing: 8px;
Text {
text: "Titel";
font-size: Theme.text-heading;
font-weight: 900;
color: Theme.paper;
}
GlassPanel {
height: song-rows.preferred-height;
song-rows := VerticalLayout {
padding: 12px;
spacing: 6px;
for song in root.songs: TrackListRow {
track: song;
selected: root.sel-index == song.nav;
play => { root.play-song(song.album-id, song.index); }
}
}
}
}
}
}
// --- categories -----------------------------------------------------------
if root.category-grid.length > 0: Text {
text: root.category-heading;
font-size: Theme.text-heading;
font-weight: 900;
color: Theme.paper;
}
if root.category-grid.length > 0: ListView {
for row in root.category-grid: HorizontalLayout {
height: Metrics.category-row-height;
spacing: Theme.grid-gap;
alignment: start;
for tile in row.tiles: CategoryCard {
tile: tile;
tile-width: Theme.card-width;
selected: root.sel-index == tile.nav;
open => { root.open-category(tile.key); }
}
}
}
// --- albums ---------------------------------------------------------------
if root.grid.length > 0: Text {
text: root.album-heading;
font-size: Theme.text-heading;
font-weight: 900;
color: Theme.paper;
}
if root.grid.length > 0: album-list := ListView {
// Follow the keyboard without measuring anything.
content-y: -max(0px, min(
max(0px, self.content-height - self.height),
root.sel-row * Metrics.row-height));
animate content-y { duration: 180ms; easing: ease-out; }
for row in root.grid: HorizontalLayout {
height: Metrics.row-height;
spacing: Theme.grid-gap;
alignment: start;
for tile in row.tiles: AlbumCard {
width: Theme.card-width;
tile: tile;
selected: root.sel-index == tile.nav;
open => { root.open-album(tile.id); }
play => { root.play-album(tile.id); }
}
}
}
}

View File

@@ -1,151 +0,0 @@
// The shapes that cross between Rust and the UI.
//
// Everything here is already *presentation*: labels are formatted, durations are
// strings, covers are decoded images. Nothing in the .slint files formats a number or
// picks a word, so the German copy and the duration format live in one place (Rust)
// rather than being split across two languages.
//
// `nav` is each item's position in the single flat selection order that runs songs ->
// categories -> albums, so one pair of arrow keys walks the whole page. It is assigned
// in Rust, where the three lists are built together.
export struct AlbumTile {
id: string,
title: string,
artist: string,
// "12 Songs · 45:03 · 🧸" - already assembled, including the figurine marker.
meta: string,
cover: image,
is-book: bool,
locked: bool,
// This album is the one playing; drawn with the accent ring rather than the
// selection ring, and outranked by it when they coincide.
current: bool,
nav: int,
}
// The grid is handed to the UI pre-chunked into rows, because a `ListView` virtualizes
// its model and a flat list of 343 cards has no rows to skip. This is the direct
// counterpart of the web app's `content-visibility: auto` on `.grid > .card`, which
// exists for exactly the same measurement: 340 live cards saturated the Pi's renderer.
export struct GridRow {
tiles: [AlbumTile],
}
export struct CategoryTile {
key: string,
// "7 Hörbücher"
count-label: string,
// Up to four covers in a 2x2, or one filling the tile when the category holds a
// single album. Four fields rather than an array: the count is fixed and small, and
// this keeps the tile a plain struct.
cover1: image,
cover2: image,
cover3: image,
cover4: image,
cover-count: int,
is-book: bool,
nav: int,
}
export struct CategoryRow {
tiles: [CategoryTile],
}
// One of the three shelves on the root screen.
export struct Shelf {
label: string,
icon: string,
tiles: [CategoryTile],
empty-label: string,
}
export struct TrackRow {
title: string,
subtitle: string,
duration: string,
cover: image,
is-book: bool,
current: bool,
locked: bool,
album-id: string,
index: int,
nav: int,
}
// What the player bar and the full-screen view both say. Shared so the two cannot
// drift apart on a copy change.
export struct NowPlaying {
title: string,
subtitle: string,
// "Song 3 von 8" / "Kapitel 3 von 8"
counter: string,
cover: image,
hero-cover: image,
is-book: bool,
has-album: bool,
// "−4:12", the time left in this track.
clock-label: string,
// "Noch 38:20 im Album" - empty for a podcast, where it means nothing.
remaining-label: string,
}
// ------------------------------------------------------------- Mein Zimmer --
// One device card on the room page. Lights and shutters share a struct rather than
// having one each, because the grid renders them in the order Home Assistant's config
// lists them and two models would lose that order. `kind` is what the view branches on.
export struct RoomCard {
entity-id: string,
name: string,
// "light" | "shutter"
kind: string,
on: bool,
// --- light ---
is-color: bool,
// 0..5, matching the five brightness buttons.
level: int,
tint: color,
// Which of the six swatches the light is currently set to, or -1.
swatch: int,
// --- shutter ---
// Percent *closed*: 0 is fully open. Home Assistant reports the opposite; the
// inversion happens once, in Rust.
closed-percent: float,
// "Fährt runter …", or the preset name the position matches.
status: string,
// "" | "up" | "down"
moving: string,
supports-position: bool,
// Which of the four presets the position matches, or -1.
preset: int,
}
// The room grid, pre-chunked into rows for the same reason the album grid is: the
// view lays rows out, and how many fit is a function of the window width, which Rust
// is the one that knows.
export struct RoomRow {
cards: [RoomCard],
}
export struct SceneTile {
entity-id: string,
name: string,
active: bool,
}
// One of the six colours a colour-capable light can be set to.
export struct Swatch {
name: string,
tint: color,
}
// One entry in the vertical tab rail. Built in Rust like every other string on
// screen, so the page order, the icons and the German labels live in one place.
export struct Tab {
id: string,
icon: string,
label: string,
}

View File

@@ -1,413 +0,0 @@
import { Theme } from "theme.slint";
import { Cover, Transport, ProgressBar, VolumeBars, RoundButton, GlassPanel } from "widgets.slint";
import { NowPlaying, TrackRow } from "models.slint";
// The bar along the bottom of the browse screen.
export component PlayerBar inherits Rectangle {
in property <NowPlaying> now;
in property <bool> playing;
in property <float> progress;
in property <bool> smooth;
in property <int> volume;
callback toggle();
callback next();
callback previous();
callback seek(float);
callback set-volume(int);
callback mute();
callback open-play-view();
height: 96px;
background: Theme.bar-bg;
// The accent rule along the top is what separates the bar from the stage without a
// shadow, which over a gradient reads as a smudge.
Rectangle {
y: 0;
height: 3px;
width: 100%;
background: Theme.accent.with-alpha(0.5);
}
HorizontalLayout {
padding-left: 24px;
padding-right: 24px;
padding-top: 16px;
padding-bottom: 16px;
spacing: 18px;
// Cover: the way back into the full-screen view.
VerticalLayout {
alignment: center;
Rectangle {
width: root.now.is-book ? 46px : 56px;
height: 56px;
Cover {
source: root.now.cover;
is-book: root.now.is-book;
radius: 10px;
width: 100%;
height: 100%;
}
TouchArea {
clicked => { root.open-play-view(); }
}
}
}
VerticalLayout {
alignment: center;
width: 240px;
Text {
text: root.now.title;
font-size: Theme.text-body;
font-weight: 800;
color: Theme.paper;
overflow: elide;
}
Text {
text: root.now.subtitle;
font-size: Theme.text-small;
font-weight: 600;
color: Theme.text-sub.with-alpha(0.75);
overflow: elide;
}
if root.now.has-album: Text {
text: root.now.counter;
font-size: Theme.text-tiny;
font-weight: 800;
color: Theme.accent-dim;
overflow: elide;
}
}
VerticalLayout {
alignment: center;
horizontal-stretch: 1;
ProgressBar {
progress: root.progress;
smooth: root.smooth;
on-dark: true;
bar-height: 10px;
seek(fraction) => { root.seek(fraction); }
}
}
VerticalLayout {
alignment: center;
Transport {
playing: root.playing;
size: 48px;
toggle => { root.toggle(); }
next => { root.next(); }
previous => { root.previous(); }
}
}
VerticalLayout {
alignment: center;
VolumeBars {
volume: root.volume;
bar-width: 11px;
bar-height: 30px;
set-volume(v) => { root.set-volume(v); }
mute => { root.mute(); }
}
}
}
}
// The full-screen now-playing view.
export component PlayView inherits VerticalLayout {
in property <NowPlaying> now;
in property <bool> playing;
in property <float> progress;
in property <bool> smooth;
in property <int> volume;
callback toggle();
callback next();
callback previous();
callback seek(float);
callback set-volume(int);
callback mute();
callback open-album();
alignment: center;
spacing: 18px;
padding: 32px;
HorizontalLayout {
alignment: center;
Rectangle {
height: 320px;
width: root.now.is-book ? 262px : 320px;
drop-shadow-color: #0016208c;
drop-shadow-blur: 40px;
drop-shadow-offset-y: 20px;
if root.now.has-album: Cover {
source: root.now.hero-cover;
is-book: root.now.is-book;
radius: 28px;
width: 100%;
height: 100%;
}
if !root.now.has-album: Rectangle {
background: #2a4f57;
border-radius: 28px;
}
TouchArea {
clicked => { root.open-album(); }
}
}
}
VerticalLayout {
spacing: 6px;
Text {
text: root.now.title;
font-size: Theme.text-hero;
font-weight: 900;
color: Theme.paper;
horizontal-alignment: center;
wrap: word-wrap;
}
Text {
text: root.now.subtitle;
font-size: Theme.text-heading;
font-weight: 700;
color: Theme.text-dim.with-alpha(0.8);
horizontal-alignment: center;
overflow: elide;
}
if root.now.has-album: Text {
text: root.now.counter;
font-size: 15px;
font-weight: 800;
color: Theme.accent-dim;
horizontal-alignment: center;
}
}
HorizontalLayout {
alignment: center;
VerticalLayout {
width: 680px;
spacing: 8px;
ProgressBar {
progress: root.progress;
smooth: root.smooth;
bar-height: 14px;
seek(fraction) => { root.seek(fraction); }
}
HorizontalLayout {
Text {
text: root.now.clock-label;
font-family: Theme.mono;
font-size: Theme.text-small + 1px;
font-weight: 800;
color: Theme.text-dim.with-alpha(0.8);
}
Rectangle { horizontal-stretch: 1; }
Text {
text: root.now.remaining-label;
font-size: Theme.text-small + 1px;
font-weight: 800;
color: Theme.text-dim.with-alpha(0.8);
}
}
}
}
Transport {
playing: root.playing;
size: 72px;
toggle => { root.toggle(); }
next => { root.next(); }
previous => { root.previous(); }
}
VolumeBars {
volume: root.volume;
bar-width: 18px;
bar-height: 44px;
set-volume(v) => { root.set-volume(v); }
mute => { root.mute(); }
}
}
// The track list for one album, over a dimmed stage.
export component AlbumSheet inherits Rectangle {
in property <string> title;
in property <string> subtitle;
in property <string> meta;
in property <image> cover;
in property <bool> is-book;
in property <[TrackRow]> tracks;
in property <int> sel-index;
callback close();
callback play(int);
callback play-all();
background: Theme.backdrop.with-alpha(0.6);
// Catches every click that misses the sheet, so tapping the dimmed area closes it.
TouchArea {
clicked => { root.close(); }
}
Rectangle {
width: 640px;
height: min(root.height - 80px, 720px);
background: Theme.sheet-bg;
border-radius: Theme.radius-sheet;
drop-shadow-color: #0016208c;
drop-shadow-blur: 60px;
drop-shadow-offset-y: 24px;
// Without this the backdrop's TouchArea above would receive clicks aimed at the
// sheet's own background and close it.
TouchArea { }
VerticalLayout {
padding: 26px;
spacing: 16px;
HorizontalLayout {
spacing: 18px;
Cover {
source: root.cover;
is-book: root.is-book;
width: root.is-book ? 123px : 150px;
height: 150px;
}
VerticalLayout {
alignment: start;
spacing: 6px;
Text {
text: root.title;
font-size: Theme.text-title;
font-weight: 900;
color: Theme.ink;
wrap: word-wrap;
}
Text {
text: root.subtitle;
font-size: Theme.text-body;
font-weight: 700;
color: Theme.ink.with-alpha(0.7);
overflow: elide;
}
Text {
text: root.meta;
font-size: Theme.text-small;
font-weight: 800;
color: Theme.count-pink;
overflow: elide;
}
Rectangle {
height: 8px;
}
Rectangle {
background: play-all-touch.pressed ? Theme.accent-dim : Theme.accent;
border-radius: Theme.pill;
width: play-all.preferred-width;
height: play-all.preferred-height;
play-all := HorizontalLayout {
padding-left: 18px;
padding-right: 18px;
padding-top: 9px;
padding-bottom: 9px;
Text {
text: "▶ Alle abspielen";
font-size: Theme.text-small + 1px;
font-weight: 800;
color: white;
}
}
play-all-touch := TouchArea {
clicked => { root.play-all(); }
}
}
}
VerticalLayout {
alignment: start;
RoundButton {
label: "×";
size: 40px;
clicked => { root.close(); }
}
}
}
Flickable {
content-height: list.preferred-height;
content-width: self.width;
// Follow the keyboard through the list without measuring: every row is
// the same height, so the selected one's offset is arithmetic.
content-y: -max(0px, min(
max(0px, self.content-height - self.height),
root.sel-index * 52px - self.height / 2));
animate content-y { duration: 150ms; easing: ease-out; }
list := VerticalLayout {
spacing: 4px;
for track[i] in root.tracks: Rectangle {
height: 48px;
background: track.current ? Theme.accent.with-alpha(0.18) : transparent;
border-radius: 14px;
border-width: root.sel-index == i ? 3px : 0px;
border-color: Theme.accent;
HorizontalLayout {
padding-left: 14px;
padding-right: 14px;
spacing: 12px;
VerticalLayout {
alignment: center;
Rectangle {
width: 30px;
height: 30px;
border-radius: 15px;
background: track.current ? Theme.accent : Theme.ink.with-alpha(0.12);
Text {
text: i + 1;
font-size: Theme.text-tiny;
font-weight: 800;
color: track.current ? white : Theme.ink;
vertical-alignment: center;
horizontal-alignment: center;
}
}
}
VerticalLayout {
alignment: center;
Text {
text: track.title;
font-size: Theme.text-small + 1px;
font-weight: 700;
color: Theme.ink;
overflow: elide;
}
}
VerticalLayout {
alignment: center;
Text {
text: track.duration;
font-family: Theme.mono;
font-size: Theme.text-tiny;
font-weight: 700;
color: Theme.ink.with-alpha(0.6);
}
}
}
TouchArea {
clicked => { root.play(i); }
}
}
}
}
}
}
}

View File

@@ -1,460 +0,0 @@
// "Mein Zimmer" - the Home Assistant room panel.
//
// Its own violet hue rather than the player's teal, so the two pages read as siblings.
// Lights and shutters are drawn from one `RoomCard` model in Home Assistant's own
// config order; `kind` is what each card branches on.
import { Theme } from "theme.slint";
import { RoomCard, RoomRow, SceneTile, Swatch } from "models.slint";
component ToggleSwitch inherits Rectangle {
in property <bool> on;
callback clicked();
width: 74px;
height: 42px;
border-radius: 21px;
background: root.on ? Theme.room-accent : Theme.room-toggle-off;
animate background { duration: 140ms; }
Rectangle {
x: root.on ? parent.width - self.width - 4px : 4px;
y: 4px;
width: 34px;
height: 34px;
border-radius: 17px;
background: white;
drop-shadow-color: #0f051d66;
drop-shadow-blur: 6px;
drop-shadow-offset-y: 2px;
animate x { duration: 140ms; easing: ease-out; }
}
TouchArea {
clicked => { root.clicked(); }
}
}
// A flat labelled button - the shutter presets and the up/stop/down row.
component RoomButton inherits Rectangle {
in property <string> label;
in property <bool> active;
in property <brush> active-fill: Theme.room-accent;
in property <length> text-size: 14px;
callback clicked();
border-radius: 14px;
background: root.active ? root.active-fill : (touch.pressed ? Theme.room-off-bg : Theme.room-btn);
animate background { duration: 120ms; }
Text {
text: root.label;
font-size: root.text-size;
font-weight: 800;
color: root.active ? white : Theme.room-btn-fg;
vertical-alignment: center;
horizontal-alignment: center;
}
touch := TouchArea {
clicked => { root.clicked(); }
}
}
component RoomCardFrame inherits Rectangle {
in property <bool> glow;
in property <color> glow-tint;
background: Theme.room-card.with-alpha(0.55);
border-radius: 22px;
// Standing in for the web card's `0 0 0 3px color-mix(...)` ring: a lit lamp's own
// colour around its card is how the page reads at a glance from across the room.
border-width: root.glow ? 3px : 0px;
border-color: root.glow-tint;
drop-shadow-color: #0f051d59;
drop-shadow-blur: 24px;
drop-shadow-offset-y: 8px;
}
component LightCard inherits RoomCardFrame {
in property <RoomCard> card;
in property <[Swatch]> swatches;
callback toggle();
callback set-level(int);
callback set-swatch(int);
glow: root.card.on;
glow-tint: root.card.tint;
VerticalLayout {
padding-left: 20px;
padding-right: 20px;
padding-top: 18px;
padding-bottom: 20px;
spacing: 14px;
HorizontalLayout {
spacing: 14px;
Rectangle {
width: 56px;
height: 56px;
border-radius: 16px;
background: root.card.on ? root.card.tint : Theme.room-off-bg;
drop-shadow-color: root.card.on ? root.card.tint : transparent;
drop-shadow-blur: 22px;
// A pendant for a colour lamp, a ceiling fitting for a plain one -
// drawn rather than iconographic, because there is no icon font here.
Rectangle {
width: 34px;
height: 34px;
x: 11px;
y: 11px;
if root.card.is-color: Path {
width: 100%;
height: 100%;
viewbox-width: 48;
viewbox-height: 48;
stroke: root.card.on ? Theme.room-btn-fg : Theme.room-off-fg;
stroke-width: 3px;
commands: "M 24 7 A 17 17 0 1 1 23.9 7 Z";
}
if !root.card.is-color: Path {
width: 100%;
height: 100%;
viewbox-width: 48;
viewbox-height: 48;
stroke: root.card.on ? Theme.room-btn-fg : Theme.room-off-fg;
stroke-width: 3px;
commands: "M 9 30 L 24 11 L 39 30 Z M 24 4 L 24 11";
}
}
}
VerticalLayout {
alignment: center;
horizontal-stretch: 1;
Text {
text: root.card.name;
font-size: 19px;
font-weight: 900;
color: Theme.room-ink;
overflow: elide;
}
}
VerticalLayout {
alignment: center;
ToggleSwitch {
on: root.card.on;
clicked => { root.toggle(); }
}
}
}
// Five steps rather than a slider: a child aiming at a bar hits a level.
HorizontalLayout {
spacing: 6px;
height: 46px;
opacity: root.card.on ? 1.0 : 0.55;
for step[i] in [1, 2, 3, 4, 5]: Rectangle {
border-radius: 12px;
background: root.card.on && root.card.level >= step
? root.card.tint
: Theme.room-off-bg;
animate background { duration: 120ms; }
TouchArea {
clicked => { root.set-level(step); }
}
}
}
if root.card.is-color: HorizontalLayout {
spacing: 10px;
height: 46px;
alignment: start;
for swatch[i] in root.swatches: Rectangle {
width: 46px;
height: 46px;
border-radius: 23px;
background: swatch.tint;
border-width: 4px;
border-color: root.card.on && root.card.swatch == i
? Theme.room-btn-fg
: Theme.room-card.with-alpha(0.8);
drop-shadow-color: #0f051d59;
drop-shadow-blur: 8px;
drop-shadow-offset-y: 3px;
TouchArea {
clicked => { root.set-swatch(i); }
}
}
}
}
}
component ShutterCard inherits RoomCardFrame {
in property <RoomCard> card;
in property <[string]> presets;
callback set-preset(int);
callback open();
callback stop();
callback close();
VerticalLayout {
padding-left: 20px;
padding-right: 20px;
padding-top: 18px;
padding-bottom: 20px;
spacing: 16px;
HorizontalLayout {
spacing: 14px;
alignment: start;
Rectangle {
width: 56px;
height: 56px;
border-radius: 16px;
background: Theme.room-icon-bg;
Path {
x: 11px;
y: 11px;
width: 34px;
height: 34px;
viewbox-width: 48;
viewbox-height: 48;
stroke: Theme.room-icon-fg;
stroke-width: 3px;
commands: "M 7 7 L 41 7 L 41 41 L 7 41 Z M 7 16 L 41 16 M 7 24 L 41 24 M 7 32 L 41 32";
}
}
VerticalLayout {
alignment: center;
Text {
text: root.card.name;
font-size: 19px;
font-weight: 900;
color: Theme.room-ink;
overflow: elide;
}
Text {
text: root.card.status;
font-size: 13px;
font-weight: 700;
color: Theme.room-sub.with-alpha(0.8);
}
}
}
HorizontalLayout {
spacing: 18px;
// The window itself: sky behind, slats drawn down from the top to however
// closed the shutter is. Real covers report their own position, so this
// only ever reflects Home Assistant - no client-side movement animation.
Rectangle {
width: 104px;
height: 140px;
border-radius: 12px;
border-width: 3px;
border-color: Theme.room-icon-fg;
clip: true;
background: @linear-gradient(180deg, Theme.room-window-a 0%, Theme.room-window-b 100%);
Rectangle {
x: 0;
y: 0;
width: 100%;
height: parent.height * clamp(root.card.closed-percent / 100, 0.0, 1.0);
clip: true;
animate height { duration: 200ms; easing: linear; }
// The slats, as a fixed ladder clipped by the parent's height.
for slat[i] in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]: Rectangle {
y: i * 12px;
width: 100%;
height: 12px;
background: Theme.room-slat-a;
Rectangle {
y: 9px;
width: 100%;
height: 3px;
background: Theme.room-slat-b;
}
}
}
}
VerticalLayout {
spacing: 10px;
horizontal-stretch: 1;
if root.card.supports-position: VerticalLayout {
spacing: 8px;
for row[r] in [0, 1]: HorizontalLayout {
spacing: 8px;
height: 48px;
for col[c] in [0, 1]: RoomButton {
label: root.presets[r * 2 + c];
active: root.card.preset == r * 2 + c;
clicked => { root.set-preset(r * 2 + c); }
}
}
}
HorizontalLayout {
spacing: 8px;
height: 52px;
RoomButton {
label: "▲";
text-size: 20px;
active: root.card.moving == "up";
clicked => { root.open(); }
}
RoomButton {
label: "■";
text-size: 18px;
active: true;
active-fill: Theme.room-stop;
clicked => { root.stop(); }
}
RoomButton {
label: "▼";
text-size: 20px;
active: root.card.moving == "down";
clicked => { root.close(); }
}
}
}
}
}
}
export component RoomView inherits Rectangle {
in property <[RoomRow]> rows;
in property <[SceneTile]> scenes;
in property <[Swatch]> swatches;
in property <[string]> presets;
in property <bool> loading;
callback toggle-light(string);
callback set-level(string, int);
callback set-swatch(string, int);
callback set-preset(string, int);
callback shutter(string, string); // entity, "open" | "stop" | "close"
callback activate-scene(string);
background: @linear-gradient(180deg,
Theme.room-top 0%,
Theme.room-mid 45%,
Theme.room-deep 100%);
VerticalLayout {
padding-top: 18px;
spacing: 6px;
Text {
text: "Mein Zimmer";
font-size: 34px;
font-weight: 900;
color: Theme.room-paper;
horizontal-alignment: center;
}
if root.loading: VerticalLayout {
alignment: center;
Text {
text: "Einen Moment …";
font-size: 22px;
font-weight: 800;
color: Theme.room-paper;
horizontal-alignment: center;
}
}
if !root.loading: Flickable {
content-height: body.preferred-height;
content-width: self.width;
body := VerticalLayout {
padding-left: 32px;
padding-right: Theme.rail-clearance;
padding-top: 14px;
padding-bottom: 64px;
spacing: 30px;
alignment: start;
if root.scenes.length > 0: VerticalLayout {
spacing: 14px;
Text {
text: "Szenen";
font-size: 19px;
font-weight: 900;
color: Theme.room-paper;
horizontal-alignment: center;
}
HorizontalLayout {
alignment: center;
spacing: 12px;
for scene in root.scenes: Rectangle {
height: 52px;
width: pill.preferred-width;
border-radius: 26px;
background: scene.active
? Theme.room-paper
: Theme.room-paper.with-alpha(0.18);
animate background { duration: 140ms; }
pill := HorizontalLayout {
padding-left: 20px;
padding-right: 20px;
spacing: 8px;
VerticalLayout {
alignment: center;
Text {
text: "✨";
font-size: 22px;
}
}
VerticalLayout {
alignment: center;
Text {
text: scene.name;
font-size: 15px;
font-weight: 800;
color: scene.active ? Theme.room-btn-fg : Theme.room-paper;
}
}
}
TouchArea {
clicked => { root.activate-scene(scene.entity-id); }
}
}
}
}
// Both kinds share one cell width, so the grid stays a grid whatever
// mix of devices Home Assistant reports.
for row in root.rows: HorizontalLayout {
alignment: center;
spacing: 20px;
for card in row.cards: Rectangle {
width: Theme.room-card-width;
height: card.kind == "shutter" ? 268px : 246px;
if card.kind == "light": LightCard {
width: 100%;
card: card;
swatches: root.swatches;
toggle => { root.toggle-light(card.entity-id); }
set-level(n) => { root.set-level(card.entity-id, n); }
set-swatch(i) => { root.set-swatch(card.entity-id, i); }
}
if card.kind == "shutter": ShutterCard {
width: 100%;
card: card;
presets: root.presets;
set-preset(i) => { root.set-preset(card.entity-id, i); }
open => { root.shutter(card.entity-id, "open"); }
stop => { root.shutter(card.entity-id, "stop"); }
close => { root.shutter(card.entity-id, "close"); }
}
}
}
}
}
}
}

View File

@@ -1,107 +0,0 @@
// The palette, converted once from the web front-end's oklch tokens
// (`web/src/styles/app.css`) into the sRGB hex Slint speaks. The oklch source value is
// kept beside each one, because that is the form the design is actually maintained in -
// if a colour needs adjusting, adjust the oklch and re-convert (`src/oklch.rs`), rather
// than nudging the hex and letting the two drift.
export global Theme {
// ------------------------------------------------------------------ colours --
out property <color> ink: #113339; // oklch(30% 0.04 210)
out property <color> paper: #eef7f9; // oklch(97% 0.01 210)
out property <color> accent: #de73bd; // oklch(70% 0.16 340)
out property <color> accent-dim: #f292d3; // oklch(78% 0.14 340)
out property <color> sea-deep: #001b21; // oklch(20% 0.045 210)
out property <color> shadow: #001017; // oklch(15% 0.05 210)
// The stage gradient, which in the web app sits under an animated canvas. There is
// no canvas here: a full-viewport repaint loop is the one cost a Pi cannot absorb,
// and `?pi=1` already removes it on the device this replaces.
out property <color> stage-top: #397d88; // oklch(55% 0.07 210)
out property <color> stage-mid: #0e4b54; // oklch(38% 0.06 210)
out property <color> text-dim: #c9dbdf; // oklch(88% 0.02 210)
out property <color> text-sub: #c0d2d5; // oklch(85% 0.02 210)
out property <color> count-pink: #7d0563; // oklch(40% 0.17 340)
out property <color> card-music: #e4f1f4; // oklch(95% 0.015 210)
out property <color> card-book: #ffd5ab; // oklch(92% 0.09 55)
out property <color> book-edge-1: #ffc298; // oklch(86% 0.09 55)
out property <color> book-edge-2: #e5a880; // oklch(78% 0.09 55)
out property <color> bar-bg: #00171e; // oklch(18% 0.05 210)
out property <color> sheet-bg: #e9f4f6; // oklch(96% 0.012 210)
out property <color> backdrop: #000e12; // oklch(15% 0.03 210)
// Two track colours, not one with an alpha: the seek bar sits on the pale stage in
// the play view and on the near-black player bar in the browse view, and a single
// translucent fill that reads correctly on one is invisible on the other.
out property <color> progress-track: #1b3236; // oklch(30% 0.03 210)
out property <color> progress-track-dark: #2e4f57;
out property <color> vol-empty: #c9dbdf; // oklch(88% 0.02 210)
// Glass, without the glass. `backdrop-filter` has no counterpart here, so these are
// exactly the web app's own `data-blur="off"` fallback: a flat translucent fill.
// That is a real loss of depth and the honest substitute for it - faking a blur by
// compositing a pre-blurred copy would reintroduce the per-frame cost the blur was
// dropped to avoid.
out property <brush> glass: @linear-gradient(160deg, #eef7f924 0%, #eef7f90d 100%);
out property <color> glass-border: #eef7f929;
out property <color> row-idle: #f7fcfd1a;
out property <color> row-active: #f7fcfd38;
// --------------------------------------------------- Mein Zimmer (violet) --
// Its own hue (~298-310) rather than the player's teal (~210), so the two pages
// read as siblings rather than one bleeding into the other. Same oklch-converted-
// once treatment as above.
out property <color> room-ink: #181322; // oklch(20% 0.03 300)
out property <color> room-paper: #f6f4fb; // oklch(97% 0.01 302)
out property <color> room-accent: #986dd0; // oklch(62% 0.15 302)
out property <color> room-card: #f3f0f9; // oklch(96% 0.012 300)
out property <color> room-top: #885cb5; // oklch(56% 0.14 305)
out property <color> room-mid: #4d2f77; // oklch(38% 0.12 300)
out property <color> room-deep: #1e0c37; // oklch(21% 0.08 298)
out property <color> room-icon-bg: #dad3e9; // oklch(88% 0.03 300)
out property <color> room-icon-fg: #3e3451; // oklch(35% 0.05 300)
out property <color> room-off-bg: #d9d5e3; // oklch(88% 0.02 300)
out property <color> room-off-fg: #736f7c; // oklch(55% 0.02 300)
out property <color> room-btn: #dcd8e6; // oklch(89% 0.02 300)
out property <color> room-btn-fg: #2c243a; // oklch(28% 0.04 300)
out property <color> room-stop: #e85854; // oklch(65% 0.18 25)
out property <color> room-toggle-off: #cbc9d0;
out property <color> room-sub: #585264; // oklch(45% 0.03 300)
out property <color> room-slat-a: #a19baf; // oklch(70% 0.03 300)
out property <color> room-slat-b: #7d778a; // oklch(58% 0.03 300)
out property <color> room-window-a: #87d1e8; // oklch(82% 0.08 220)
out property <color> room-window-b: #50afb4; // oklch(70% 0.09 200)
out property <length> room-card-width: 380px;
// ------------------------------------------------------------------- metrics --
out property <length> radius-panel: 22px;
out property <length> radius-card: 16px;
out property <length> radius-row: 12px;
out property <length> radius-sheet: 26px;
out property <length> pill: 999px;
// The album grid's track width. The web grid is `minmax(180px, 1fr)`; keeping the
// same number keeps the same number of columns at 1920, which is what the shelf
// rows and the measured card height below were both tuned against.
out property <length> card-width: 180px;
out property <length> grid-gap: 24px;
out property <length> shelf-tile: 148px;
// How much of the right edge the page must leave clear for nothing to render
// underneath the corner controls.
out property <length> rail-clearance: 88px;
// ---------------------------------------------------------------- typography --
// Heavy throughout: 700/800/900. Durations and counters use the monospace family so
// a ticking clock does not reflow the row it sits in.
out property <string> font: "Nunito";
out property <string> mono: "monospace";
out property <length> text-hero: 44px;
out property <length> text-title: 26px;
out property <length> text-heading: 20px;
out property <length> text-body: 16px;
out property <length> text-small: 13px;
out property <length> text-tiny: 12px;
}

View File

@@ -1,299 +0,0 @@
import { Theme } from "theme.slint";
// A frosted surface wrapped around a shelf or a list, so it reads as something
// floating over the stage rather than text on a gradient.
export component GlassPanel inherits Rectangle {
background: Theme.glass;
border-width: 1px;
border-color: Theme.glass-border;
border-radius: Theme.radius-panel;
drop-shadow-color: #00101759;
drop-shadow-blur: 24px;
drop-shadow-offset-y: 8px;
}
// Album art. `source` is whatever the cover store has decoded; until it has one, the
// Rust side hands over generated art built from the album's own three colours, so this
// never has an empty frame to show.
//
// The shape comes from the media type and from nowhere else: square for an album,
// taller than wide for an audiobook, everywhere either appears.
export component Cover inherits Rectangle {
in property <image> source;
in property <bool> is-book: false;
in property <length> radius: Theme.radius-card;
clip: true;
border-radius: root.radius;
// A book keeps a sliver of its generated spine showing past real square artwork, so
// the shelf metaphor survives contact with a photograph.
background: root.is-book ? Theme.book-edge-2 : transparent;
Image {
source: root.source;
image-fit: cover;
width: root.is-book ? parent.width * 1.05 : parent.width;
height: parent.height;
x: 0;
}
}
// Prev / next / play. Every glyph is drawn from rectangles and a clip, not an icon
// font: there is no icon font on the device, and a missing glyph is a blank button.
export component TransportButton inherits Rectangle {
in property <length> size: 48px;
in property <bool> primary: false;
in property <bool> playing: false;
in property <string> direction: "play"; // "play" | "next" | "previous"
callback clicked();
width: root.size;
height: root.size;
border-radius: root.size / 2;
background: root.primary
? (touch.pressed ? Theme.accent-dim : Theme.accent)
: (touch.pressed ? #2a4f57 : Theme.ink);
drop-shadow-color: root.primary ? #de73bd80 : #00101766;
drop-shadow-blur: root.primary ? 20px : 10px;
drop-shadow-offset-y: 4px;
animate background { duration: 100ms; }
// Pause: two bars. Play and the skips: a triangle, clipped out of a square.
if root.direction == "play" && root.playing: HorizontalLayout {
alignment: center;
spacing: root.size * 0.1;
Rectangle {
width: root.size * 0.1;
height: root.size * 0.34;
border-radius: 2px;
background: white;
}
Rectangle {
width: root.size * 0.1;
height: root.size * 0.34;
border-radius: 2px;
background: white;
}
}
if root.direction != "play" || !root.playing: HorizontalLayout {
alignment: center;
spacing: root.size * 0.06;
// "Previous" is the same triangle mirrored, plus the bar it butts against.
if root.direction == "previous": Rectangle {
width: root.size * 0.07;
height: root.size * 0.3;
border-radius: 1px;
background: white;
}
Rectangle {
width: root.size * 0.3;
height: root.size * 0.3;
Path {
width: 100%;
height: 100%;
fill: white;
viewbox-width: 100;
viewbox-height: 100;
commands: root.direction == "previous"
? "M 100 0 L 100 100 L 0 50 Z"
: "M 6 0 L 100 50 L 6 100 Z";
}
}
if root.direction == "next": Rectangle {
width: root.size * 0.07;
height: root.size * 0.3;
border-radius: 1px;
background: white;
}
}
touch := TouchArea {
clicked => { root.clicked(); }
}
}
export component Transport inherits HorizontalLayout {
in property <bool> playing;
in property <length> size: 48px;
callback toggle();
callback next();
callback previous();
alignment: center;
spacing: root.size * 0.21;
VerticalLayout {
alignment: center;
TransportButton {
direction: "previous";
size: root.size;
clicked => { root.previous(); }
}
}
VerticalLayout {
alignment: center;
TransportButton {
direction: "play";
primary: true;
playing: root.playing;
// The play button is the one target a child aims at from across the room.
size: root.size * 1.26;
clicked => { root.toggle(); }
}
}
VerticalLayout {
alignment: center;
TransportButton {
direction: "next";
size: root.size;
clicked => { root.next(); }
}
}
}
// The seek bar.
//
// `position` is only pushed twice a second, so the fill is *animated* between frames
// rather than recomputed on a client-side clock. That puts the interpolation on the
// render side, where it costs a property evaluation per frame instead of a layout pass,
// and it is why there is no equivalent of the web client's `usePlaybackClock` here.
//
// The animation is suppressed whenever `smooth` goes false - while dragging, and for
// the frame a track changes on - because sliding the fill across two unrelated
// positions is exactly the "it moves slightly backwards" artefact the web UI has.
export component ProgressBar inherits Rectangle {
in property <float> progress; // 0..1
in property <bool> interactive: true;
in property <length> bar-height: 10px;
in property <bool> smooth: true;
/// Set on the player bar, whose background is darker than the seek bar's own
/// track would otherwise show against.
in property <bool> on-dark: false;
callback seek(float);
height: max(root.bar-height, 28px);
track := Rectangle {
y: (parent.height - root.bar-height) / 2;
height: root.bar-height;
width: 100%;
border-radius: root.bar-height / 2;
background: root.on-dark ? Theme.progress-track-dark : Theme.progress-track.with-alpha(0.6);
clip: true;
fill := Rectangle {
x: 0;
width: parent.width * clamp(root.progress, 0.0, 1.0);
height: 100%;
border-radius: root.bar-height / 2;
background: Theme.accent;
animate width {
duration: root.smooth ? 500ms : 0ms;
easing: linear;
}
}
}
// A grab handle only while it is worth having one: at player-bar size the bar is
// the target, and a knob would be smaller than the thing it sits on.
if root.interactive && root.bar-height >= 12px: Rectangle {
x: max(0px, min(parent.width - self.width, track.width * clamp(root.progress, 0.0, 1.0) - self.width / 2));
y: (parent.height - self.height) / 2;
width: root.bar-height * 1.7;
height: root.bar-height * 1.7;
border-radius: self.width / 2;
background: Theme.paper;
drop-shadow-color: #00101773;
drop-shadow-blur: 8px;
}
if root.interactive: TouchArea {
width: 100%;
height: 100%;
// Both a click and a drag resolve to "the position under the finger", so
// scrubbing works without a separate drag state machine.
moved => {
if (self.pressed) { root.seek(clamp(self.mouse-x / track.width, 0.0, 1.0)); }
}
clicked => { root.seek(clamp(self.mouse-x / track.width, 0.0, 1.0)); }
}
}
// Volume as five discrete steps, not a continuous slider: a child aiming at a bar hits
// a level, where a slider would hand back whatever pixel they landed on.
export component VolumeBars inherits HorizontalLayout {
in property <int> volume; // 0..100
in property <length> bar-width: 11px;
in property <length> bar-height: 30px;
callback set-volume(int);
callback mute();
spacing: root.bar-width * 0.3;
alignment: center;
VerticalLayout {
alignment: center;
Rectangle {
width: root.bar-height * 0.8;
height: root.bar-height * 0.8;
border-radius: self.width / 2;
background: speaker.has-hover ? #ffffff26 : transparent;
Text {
text: root.volume == 0 ? "🔇" : "🔊";
font-size: root.bar-width * 1.6;
vertical-alignment: center;
horizontal-alignment: center;
}
speaker := TouchArea {
clicked => { root.mute(); }
}
}
}
for level[i] in [20, 40, 60, 80, 100]: VerticalLayout {
alignment: center;
Rectangle {
width: root.bar-width;
// Stepped heights, so the control reads as a volume ramp at a glance.
height: root.bar-height * (0.45 + 0.55 * (i + 1) / 5);
border-radius: 5px;
background: root.volume >= level ? Theme.accent : Theme.vol-empty.with-alpha(0.4);
animate background { duration: 120ms; }
TouchArea {
clicked => { root.set-volume(level); }
}
}
}
}
// A round icon button - back, close, and the fixed corner controls.
export component RoundButton inherits Rectangle {
in property <string> label;
in property <length> size: 44px;
in property <bool> highlighted: false;
callback clicked();
width: root.size;
height: root.size;
border-radius: root.size / 2;
background: root.highlighted
? Theme.accent
: (touch.pressed ? Theme.text-dim : Theme.paper.with-alpha(0.95));
drop-shadow-color: #00101766;
drop-shadow-blur: 18px;
drop-shadow-offset-y: 6px;
Text {
text: root.label;
font-size: root.size * 0.45;
font-weight: 900;
color: root.highlighted ? white : Theme.ink;
vertical-alignment: center;
horizontal-alignment: center;
}
touch := TouchArea {
clicked => { root.clicked(); }
}
}

View File

@@ -33,9 +33,11 @@ For the device, `npm run build` writes `dist/`, and `general.web.static_dir` in
```sh
npm run test # vitest over src/lib
npx tsc --noEmit # strict, noUncheckedIndexedAccess
npm run check # tsc (strict, noUncheckedIndexedAccess) and eslint
```
`../scripts/check.sh web` runs both.
Only the pure modules are tested - the search, the keyboard machine and the time
formatting. That is where the behaviour lives; the components are mostly layout.
@@ -61,11 +63,12 @@ are local.
|---|---|
| `src/lib/keyboard.ts` | The key map, as a pure function from a keypress to a list of actions. Testable without a DOM. |
| `src/lib/search.ts` | Filtering and the three-level browse hierarchy. The whole library is in memory, so this is instant - see the note on `buildSearchIndex` below. |
| `src/lib/lowPower.ts` | The `?pi=1` profile: what gets cheaper, and by how much. |
| `src/lib/perfSettings.ts` | The performance knobs: the `FULL` and `PI` presets, and per-browser overrides from the ⚡ dialog. |
| `src/lib/lowPower.ts` | The measurements behind the `?pi=1` preset, and the ambient-canvas quality numbers. |
| `src/lib/covers.ts` | Generated cover art from the album's three colours, and the book-spine treatment. |
| `src/hooks/usePlayerState.ts` | The websocket, with reconnect-and-reseed. |
| `src/hooks/usePlaybackClock.ts` | Interpolates the 2 Hz position onto animation frames. |
| `src/components/` | Layout only. |
| `src/components/` | Layout only. `RoomView` is the "Mein Zimmer" page, `TippenApp` the typing game. |
`src/api/types.ts` mirrors `musicmouse/services/web/schemas.py`; the two are hand-kept in
step.
@@ -73,26 +76,29 @@ step.
## Keyboard
The device is meant to be driven from a keyboard, so every screen is reachable without a
pointer. `F1` shows the same list in-app.
pointer. `F1` shows the full list in-app; `src/components/HelpOverlay.tsx` is that list
and `src/lib/keyboard.ts` is the binding - change both together.
| Key | |
|---|---|
| `SPACE` | play / pause |
| `CTRL+H` / `CTRL+L` | previous / next track |
| `CTRL+J` / `CTRL+K` | quieter / louder |
| `SHIFT+H` / `SHIFT+L` | previous / next track |
| `SHIFT+J` / `SHIFT+K` | quieter / louder |
| `SHIFT+M` | mute |
| `SHIFT+←` / `SHIFT+→` | seek 15 s |
| `A-Z`, `0-9` | search albums as you type |
| `?` | switch to searching individual tracks |
| `TAB` | cycle all / music / audiobooks |
| `← ↑ ↓ →` | move the selection while browsing; transport and volume while playing |
| `ENTER` | play the selection |
| `ESC` | back out one layer at a time |
| `/` | back to browsing |
| `0-9` | what the IR remote's number key does |
| `← ↑ ↓ →`, `CTRL+HJKL` | move the selection |
| `ENTER` / `SHIFT+ENTER` | play the selection / open its track list |
| `TAB` | cycle music / audiobooks / podcasts |
| `A-Z` | search albums as you type |
| `?` | search individual tracks |
| `/` | open search, or switch albums ↔ tracks |
| `A` then `0-9` | put the playing album on that remote key (`SHIFT+A`: just this song) |
| `INSERT` | clock on / off |
| `DELETE` | screen off / on (only on the device itself) |
| `ESC` | close, or clear the search |
| `F1` | this list |
`SHIFT+arrows` is the one addition to the mockup: plain arrows were already taken, and
seeking is the one thing a real player can do that the mockup could not.
## The Raspberry Pi profile
`?pi=1` loads the same app with nothing on screen animating itself. It exists for
@@ -125,15 +131,18 @@ sea behind everything - it just no longer moves or follows the track.
`AMBIENCE_QUALITY` in `src/lib/lowPower.ts` is kept accurate for whoever turns the
canvas back on.
The flag is read once at module load from the URL and nowhere else - no persistence, no
auto-detection. That is what makes "is this the profile or the hardware?" answerable by
editing the address bar. The kiosk gets it from `pi_kiosk_url` in the ansible repo.
`?pi=1` selects the `PI` preset in `src/lib/perfSettings.ts`, and each knob can then be
overridden from the ⚡ dialog, stored in that browser's localStorage. Without `?pi=1`
the stored values are ignored and the full version applies, which keeps "is this the
profile or the hardware?" answerable by editing the address bar. Settings are read once
at load, so a change applies on reload. The kiosk gets the flag from `pi_kiosk_url` in
the ansible repo.
## Parent mode
`?parentMode=1` opens a settings panel - volume limits, the rotary step, button
brightness, and a button to re-read the library. Saving writes the device's
`config.yml`.
`?parentMode=1` adds a way into a settings panel - volume limits, the rotary step, button
brightness, the screen's auto-off, the parents' timer, and a button to re-read the
library. Saving writes the device's `config.yml`.
It is hidden from the child, not protected from them: there is no authentication on the
endpoints behind it. See `../python-backend/README.md`.
@@ -159,7 +168,41 @@ Nunito is self-hosted in `public/fonts/` (one variable file per subset, ~74 kB)
than pulled from Google. The mouse may have no internet, and a cached shell rendering in
a fallback face looks broken.
## Not implemented
## Layout on phones
The mockup's second page, "Mein Zimmer" (Hue lights, a roller shutter and scenes), is
not built. Nothing here depends on it.
Phones (`usePhoneLayout`: narrow, or short landscape on touch) get the touch layout in
`components/mobile/`; tablets and the kiosk keep the keyboard one. The home-screen app
on iOS has caught every shortcut taken here, so these are the rules - follow them rather
than patching one element at a time:
- **The status bar is opaque** (`apple-mobile-web-app-status-bar-style: black`). The page
starts below it, so nothing can end up under the clock. `black-translucent` looks
nicer, but it ran the page behind the status bar, and since iOS 26 it also sizes the
app one status bar short, leaving a strip at the bottom that CSS cannot paint
([WebKit 301108](https://bugs.webkit.org/show_bug.cgi?id=301108)). The workarounds for
that (`100lvh`, measuring `screen` in JS) each broke something else. Installed apps
read this tag once: after changing it, remove the app and add it again.
- **`viewport-fit=cover` stays**, so the sea fills the rounded corners and the home
indicator area, which means:
- **Anything placed against a screen edge adds that edge's inset**: `--safe-top`,
`--safe-right`, `--safe-bottom` and `--safe-left` in `app.css`, always the variables
and never a bare `env()`. That covers `top`/`bottom`/`left`/`right` offsets of
positioned elements, and the padding of bars and scroll containers that reach the
edge. Scrolling content clears what floats over it (`--m-dock-clearance`). The top
inset is 0 with the opaque status bar, but stays in the formulas for Safari tabs and
landscape.
- **The stage is pinned, not sized**: `position: fixed; inset: 0` on phones, with the sea
colour on `html`/`body` so any gap would show sea, not white. No viewport-height unit
is trusted in the home-screen app.
- **The keyboard is measured, not assumed**: iOS slides it over the page without resizing
anything, so the dock is raised by `visualViewport` (`useKeyboardInset`) and drops its
home-indicator padding while the keyboard is up
([safe-area-inset-bottom does not update for the keyboard](https://webventures.rejh.nl/blog/2025/safe-area-inset-bottom-does-not-update/)).
## Other tabs
- **Mein Zimmer** (`RoomView`): the mockup's room page - lights, a roller shutter and
scenes - shown when the backend has an `ha` section. Every Home Assistant call goes
through the backend's `/api/ha/...` proxy, so the browser never sees the token.
- **Tippen** (`TippenApp`): the typing game, when the backend has a `tippen` section.
Passing lessons unlocks tracks in the library.

View File

@@ -2,7 +2,8 @@
<html lang="de">
<head>
<meta charset="utf-8" />
<!-- viewport-fit=cover so the sea reaches under the iPhone's rounded corners;
<!-- viewport-fit=cover so the sea reaches under the rounded corners and the home
indicator - edge-anchored elements clear them with the --safe-* insets;
user-scalable=no because a pinch-zoom on a big pink play button is a misfire. -->
<meta
name="viewport"
@@ -18,8 +19,13 @@
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-title" content="Musik" />
<!-- black-translucent lets the page's own gradient run behind the status bar. -->
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<!-- An opaque status bar, so the page starts below it - not black-translucent. That
one ran the page under the clock, which put every top-anchored control behind
it unless each remembered its inset, and since iOS 26 it also sizes the app one
status bar short with an unpaintable strip at the bottom (WebKit bug 301108).
See "Layout on phones" in README.md. Home-screen installs read this tag once:
remove and re-add the app after changing it. -->
<meta name="apple-mobile-web-app-status-bar-style" content="black" />
<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
<link rel="icon" href="/icon-192.png" type="image/png" />
<link rel="stylesheet" href="/fonts/nunito.css" />

View File

@@ -643,7 +643,7 @@ export function App() {
<div
style={{
position: "absolute",
bottom: 24,
bottom: "calc(24px + var(--safe-bottom))",
left: "50%",
transform: "translateX(-50%)",
zIndex: 7,
@@ -762,7 +762,16 @@ export function App() {
that corner, and parent mode is a tab in it instead. The F1 sheet's button is
not on the typing page either (TippenApp has its own help and owns every key). */}
{!phone && (
<div style={{ position: "absolute", left: 8, bottom: 8, zIndex: 5, display: "flex", gap: 8 }}>
<div
style={{
position: "absolute",
left: "calc(8px + var(--safe-left))",
bottom: "calc(8px + var(--safe-bottom))",
zIndex: 5,
display: "flex",
gap: 8,
}}
>
{PERF_ENABLED && !perfOpen && (
<ToolButton label="Leistungseinstellungen" onClick={() => setPerfOpen(true)}>
⚡
@@ -844,8 +853,8 @@ function CornerButton({
title={label}
style={{
position: "absolute",
top: 18,
[side]: 24,
top: "calc(18px + var(--safe-top))",
[side]: `calc(24px + var(--safe-${side}))`,
zIndex: 3,
width: 44,
height: 44,

View File

@@ -4,7 +4,6 @@ import type {
Album,
HaConfig,
HaEntityState,
LircConfig,
PlayerState,
RemoteMapping,
RemoteSlotInput,
@@ -55,9 +54,6 @@ async function fetchOrNullOnError<T>(path: string): Promise<T | null> {
/** `null` means the room-control page isn't configured, not an error. */
const fetchHaConfig = (): Promise<HaConfig | null> => fetchOrNullOn404<HaConfig>("/ha");
/** `null` means the IR remote isn't configured, not an error. */
const fetchLircConfig = (): Promise<LircConfig | null> => fetchOrNullOn404<LircConfig>("/lirc");
/** `null` means the typing game isn't configured, not an error - the tab still shows,
* just with a "not set up" placeholder instead of a lesson map. */
const fetchTippenCurriculum = (): Promise<TippenCurriculum | null> =>
@@ -96,7 +92,6 @@ export const api = {
previous: () => post("/previous"),
seek: (position: number) => post("/seek", { position }),
setVolume: (percent: number) => post("/volume", { percent }),
nudgeVolume: (deltaPercent: number) => post("/volume", { delta_percent: deltaPercent }),
/** One step of the IR remote's volume keys per unit, in the device's own increment. */
stepVolume: (steps: number) => post("/volume", { steps }),
toggleScreen: () => post("/screen/toggle"),
@@ -118,7 +113,6 @@ export const api = {
trackDetail: fetchTrackDetail,
lircConfig: fetchLircConfig,
pressRemote: (digit: string) => post(`/remote/press/${digit}`),
remoteMapping: () => request<RemoteMapping>("/remote/mapping"),
saveRemoteMapping: (slots: Record<string, RemoteSlotInput>) =>

View File

@@ -111,6 +111,8 @@ export interface Settings {
initial_volume: number;
volume_increment: number;
button_leds_brightness: number;
/** UI sounds (screen, timer, album picker) relative to the music, in percent. */
effects_volume: number;
/** Minutes without music before the device's screen switches itself off, 0 = never.
* `null` when there is no screen to switch (no `off_command` configured). */
screen_auto_off_minutes: number | null;
@@ -169,11 +171,6 @@ export interface RemoteSlotInput {
track_index?: number;
}
/** Presence-only, like `HaConfig` - there is nothing secret in a host/port. */
export interface LircConfig {
connected: boolean;
}
// --------------------------------------------------------------------------- tippen
//
// The typing game. `null` from `GET /tippen/curriculum` means the whole feature isn't

View File

@@ -54,7 +54,7 @@ export function AlbumModal({
style={{
zIndex: 4,
background: "oklch(15% 0.03 var(--hue) / .6)",
...(phone && { alignItems: "flex-end", padding: "calc(40px + env(safe-area-inset-top)) 0 0" }),
...(phone && { alignItems: "flex-end", padding: "calc(40px + var(--safe-top)) 0 0" }),
}}
onClick={onClose}
>
@@ -65,7 +65,7 @@ export function AlbumModal({
style={{
background: "oklch(96% 0.012 var(--hue))",
borderRadius: phone ? "26px 26px 0 0" : 26,
padding: phone ? "20px 16px calc(20px + env(safe-area-inset-bottom))" : 26,
padding: phone ? "20px 16px calc(20px + var(--safe-bottom))" : 26,
width: phone ? "100%" : 640,
maxWidth: "100%",
maxHeight: "100%",

View File

@@ -27,8 +27,8 @@ export function AppHeader({
display: "flex",
justifyContent: "center",
padding: status
? `calc(${compact ? 8 : 10}px + env(safe-area-inset-top)) 16px 0`
: "env(safe-area-inset-top) 0 0",
? `calc(${compact ? 8 : 10}px + var(--safe-top)) 16px 0`
: "var(--safe-top) 0 0",
flex: "none",
}}
>
@@ -47,8 +47,8 @@ export function AppHeader({
gap: compact ? 10 : 16,
// The status bar is translucent in standalone mode, so make room for it.
padding: compact
? "calc(12px + env(safe-area-inset-top)) 16px 4px"
: "calc(18px + env(safe-area-inset-top)) 32px 6px",
? "calc(12px + var(--safe-top)) 16px 4px"
: "calc(18px + var(--safe-top)) 32px 6px",
flex: "none",
position: "relative",
}}

View File

@@ -55,6 +55,7 @@ const FIELDS: Array<[AlwaysSetting, string, string, number, number, number]> = [
["initial_volume", "Start-Lautstärke", "Beim Einschalten", 0, 200, 1],
["volume_increment", "Schritt am Drehknopf", "Pro Rasterung", 1, 100, 1],
["button_leds_brightness", "Tastenbeleuchtung", "0 bis 1", 0, 1, 0.05],
["effects_volume", "Effekt-Lautstärke", "Klicks und Signaltöne, in % der Musik", 0, 100, 5],
];
function NumberField({
@@ -142,7 +143,7 @@ export function ParentPage({ clock, onOpenPerf }: { clock: ClockState | null; on
<div className="m-scroll" style={{ padding: "0 var(--m-gutter) calc(var(--m-dock-clearance) + 24px)" }}>
<div
style={{
padding: "calc(16px + env(safe-area-inset-top)) 0 16px",
padding: "calc(16px + var(--safe-top)) 0 16px",
fontSize: 24,
fontWeight: 900,
color: "var(--paper)",

View File

@@ -240,8 +240,8 @@ export function PlayView({
title="Einer Fernbedienungs-Taste zuweisen (oder: A und dann eine Zahl, Shift+A für nur dieses Lied)"
style={{
position: "absolute",
bottom: 28,
right: 32,
bottom: "calc(28px + var(--safe-bottom))",
right: "calc(32px + var(--safe-right))",
border: "none",
cursor: "pointer",
width: 52,

Some files were not shown because too many files have changed in this diff Show More