From a737ccf1088e214af71c4de21f864a25b9087851 Mon Sep 17 00:00:00 2001 From: Martin Bauer Date: Sun, 4 Oct 2026 00:28:37 +0200 Subject: [PATCH] Bring the docs in line with the code - python-backend/README: track analysis is implemented (2e0e6ad); the cache layout, the module table (lirc, screen, timers, podcasts, tippen) and the route table (track analysis, HA proxy, typing game) were missing pieces. - docs/REPO_OVERVIEW: Mein Zimmer and the clock are implemented; Home Assistant is also reached over HTTP through the room page's proxy, not MQTT only; the IR remote, scripts/, the optional config sections and figure kind were not mentioned. - web/README: the key table was the mockup's (Ctrl transport keys, TAB over 'all'); ?pi=1 is now a preset with per-browser overrides, not a read-once flag; the room page and the typing game exist; eslint is part of the checks. - Docstrings that described earlier designs: HaConfig (written as browser-direct in the same commit that built the proxy, a8ed350), services/__init__, events, playback, media, colors, tippen progress and AppHeader, lowPower.ts, and the 'Hoerbuecher' spelling. Co-Authored-By: Claude Opus 5.5 --- docs/REPO_OVERVIEW.md | 38 +++++++----- python-backend/README.md | 30 ++++++--- python-backend/musicmouse/config.py | 10 +-- python-backend/musicmouse/events.py | 4 +- python-backend/musicmouse/library/colors.py | 2 +- python-backend/musicmouse/media.py | 6 +- .../musicmouse/reactions/playback.py | 4 +- .../musicmouse/services/__init__.py | 5 +- python-backend/musicmouse/tippen/progress.py | 4 +- web/README.md | 62 +++++++++++-------- web/src/components/tippen/AppHeader.tsx | 4 +- web/src/lib/lowPower.ts | 13 ++-- 12 files changed, 105 insertions(+), 77 deletions(-) diff --git a/docs/REPO_OVERVIEW.md b/docs/REPO_OVERVIEW.md index d6ebfaf..141ac48 100644 --- a/docs/REPO_OVERVIEW.md +++ b/docs/REPO_OVERVIEW.md @@ -20,7 +20,9 @@ Assistant integration. | `web/` | React + TypeScript front-end (Vite). Browse the whole library and play any of it from a browser. Built output is served by `python-backend` itself. | | `esp-firmware/` | ESP32 firmware (C++, Arduino framework via PlatformIO). Reads the RFID reader and buttons, drives the LED strips, talks to `python-backend` over serial. | | `hardware/` | 3D-print models for the figurines and enclosure (FreeCAD/Blender/OBJ/STL), a Fritzing electronics sketch, datasheets, and `pinout.md` (RFID reader + button-board wiring). | -| `claude-design/` | The interaction and visual spec the web front-end was built from ("Dolphin Beats"), as a standalone HTML mockup with hardcoded data. `web/` is the real implementation; the mockup is kept as the reference for the keyboard model and the styling. It also contains a second page, "Mein Zimmer" (room lights), which is **not** implemented. | +| `claude-design/` | The interaction and visual specs the web front-end was built from, as standalone HTML mockups with hardcoded data: "Dolphin Beats" (the player), "Mein Zimmer" (the room page) and "Delphin Uhr" (the clock overlay). `web/` is the real implementation of all three; the mockups are kept as the reference for the keyboard model and the styling. | +| `scripts/` | `check.sh` (every check, see below) and `sync-library.sh` (seed a device's library and analysis cache). | +| `docs/` | This file. | ## How the pieces talk to each other @@ -41,12 +43,18 @@ Assistant integration. out of step with a figure someone puts on the reader. See `python-backend/README.md` for the endpoint list. -- **`python-backend` ↔ Home Assistant**: MQTT only. The backend publishes three - discoverable lights, a player sensor, a volume number, transport buttons, device - triggers for every button and touch area, and a tag scanner for the RFID reader. It - does *not* call Home Assistant services directly any more; behaviour like "the left - ear turns the room light pink" is an HA automation. See `python-backend/README.md` - for the trigger topics and the old colour mapping. +- **`python-backend` ↔ Home Assistant**: two separate paths. + - MQTT, for the mouse itself. The backend publishes three discoverable lights, a + player sensor, a volume number, transport buttons, device triggers for every button + and touch area, and a tag scanner for the RFID reader. Behaviour like "the left ear + turns the room light pink" is an HA automation, not backend code. See + `python-backend/README.md` for the trigger topics and the old colour mapping. + - HTTP, for the web front-end's room page only: `/api/ha/...` proxies state reads and + service calls to Home Assistant's REST API with the token from `general.ha`, so the + token never reaches the browser. + +- **`python-backend` ↔ IR remote**: a TCP client for lircd (`general.lirc`), turned into + the same intents as every other front-end. ## Running it @@ -89,10 +97,13 @@ production so one process serves both the UI and the API. - `general.{library.*, serial_port, baudrate, reconnect_interval, alsa_device, min_volume, max_volume, initial_volume, volume_increment, button_leds_brightness, - audio_extensions, mqtt.*, web.*}` -- `figures..{id, colors}` — `id` is a 5-byte hex RFID tag, unique per figure; - `colors` is exactly four (`primary, secondary, bg, accent`), each `"#rrggbb"` or - `"wNN"`. + audio_extensions, podcast_episode_limit}`, plus optional sections `mqtt`, `web`, `ha` + (room page), `lirc` (IR remote), `screen` (monitor on/off), `tippen` (typing game) + and `clock` (school bus countdown). +- `figures..{id, colors, kind}` — `id` is a 5-byte hex RFID tag, unique per + figure; `colors` is exactly four (`primary, secondary, bg, accent`), each `"#rrggbb"` + or `"wNN"`; `kind` is `music` (default) or `book`. +- `remote.` — what each IR-remote number key plays. Written by the web UI. `general.library.root` is the one path to the music. The shelves under it are fixed names, not settings, because each has quirks the scanner knows about: @@ -118,9 +129,8 @@ reported at once. the behaviour is end-to-end — those files are executed by the test suite. - `python-backend/notebooks/` is LED-effect scratch work, not part of the app. - The firmware has no automated tests beyond a PlatformIO `native` env for LED effects. -- The front-end has `vitest` over the pure modules (`web/src/lib/`) and `tsc --noEmit`; - there is no component-level test harness. `npm run test` and `npx tsc --noEmit` are - the two checks. +- The front-end has `vitest` over the pure modules (`web/src/lib/`), `tsc` and `eslint`; + there is no component-level test harness. - The library index is cached under `general.library.cache`. It is keyed by file mtime and size, so **a change to how the scanner derives a title, artist or series is invisible until the cache is invalidated** — bump `_INDEX_VERSION` in diff --git a/python-backend/README.md b/python-backend/README.md index a1c9642..e720da1 100644 --- a/python-backend/README.md +++ b/python-backend/README.md @@ -94,7 +94,10 @@ the prompt becomes a regression test by pasting it into a `.txt` file. | `musicmouse/reactions/` | The policy. `@on(SomeEvent)` functions that get the app and act. | | `musicmouse/library/` | The music collection: scanning, tags, cover art and its colours, the cache, and the seams for future track analysis. | | `musicmouse/services/mqtt/` | Home Assistant entities: three lights, a player sensor, a volume number, transport buttons, device triggers, a tag scanner. | -| `musicmouse/services/web/` | The browser front-end's API: the library, a state websocket, command endpoints, and parent-mode settings. | +| `musicmouse/services/web/` | The browser front-end's API: the library, a state websocket, command endpoints, parent-mode settings, the typing game and the Home Assistant proxy. | +| `musicmouse/services/lirc/` | The IR remote, as a lircd TCP client. | +| `musicmouse/services/` (`screen.py`, `timers.py`, `podcasts.py`) | The monitor's on/off and idle auto-off; the parents' timer and school-day calendar; the podcast feed poller. | +| `musicmouse/tippen/` | The typing game's curriculum, progress and track unlocks. | | `musicmouse/simulator/` | Fake transport and player, the driver vocabulary, the REPL and the script runner. | | `musicmouse/config.py` | Pydantic schema, validation, and human-readable error formatting. | @@ -186,8 +189,9 @@ different amounts to produce: ``` index.json cheap: tags and structure. Rebuilt freely. -covers/.jpg medium: art extracted from an ID3 APIC frame -analysis/.json expensive: reserved for offline audio analysis +covers/.jpg medium: the album's art, downscaled to 384 px +covers/.thumb.jpg medium: the same at 256 px, for cards and rows +analysis/.json expensive: track analysis (plus .beats.json, .curve.json) ``` Deleting the whole directory is safe; deleting it throws away analysis that is minutes @@ -199,13 +203,16 @@ change to how the scanner derives a title, artist or series is invisible until t is invalidated: bump `_INDEX_VERSION` in `musicmouse/library/cache.py` when you touch that logic. -**Track analysis is not implemented.** `musicmouse/library/analysis.py` fixes the shape -of the results - scalars (`tempo`, `energy`, `valence`, `brightness`) travel inline with -the index, and a beat grid lives in its own file and is fetched per track - so an -analyzer can be added later without touching the scanner, the API or the frontend. It -will go behind an optional dependency group, and because results are content-keyed -files, they can equally well be computed on a workstation and the `analysis/` folder -copied to the device. +**Track analysis** drives the web front-end's ambient background. It is optional: install +the `analysis` extra (`pip install -e '.[analysis]'`, librosa and numpy) and +`musicmouse/library/librosa_analyzer.py` works through the music shelf in the +background, in a pool of niced worker processes (`musicmouse/library/workers.py`), and +pauses while anything is playing. Without the extra, `build_analyzer()` returns a no-op +and the background stays static. Scalars (`tempo`, `energy`, `valence`, `brightness`, +`pulse`) travel inline with the index; the beat grid and per-second curves live in their +own files and are fetched only for the playing track. Results are content-keyed, so they +can be computed on a workstation and the `analysis/` folder copied to the device +(`scripts/sync-library.sh` does that). ## The web front-end @@ -226,6 +233,9 @@ copied to the device. | `POST /api/screen/toggle` `/screen/activity` | the kiosk's Del key; its key presses and taps, holding off the idle screen-off | | `GET` `PUT /api/settings` | parent mode | | `POST` `DELETE /api/timer` | parent mode's countdown: `{minutes}` or `{until: "HH:MM"}`; stop or silence it | +| `GET /api/tracks/{album_id}/{index}/analysis` | beat grid and mood curves of one track, or 404 | +| `GET /api/ha`, `GET /api/ha/states/{entity}`, `POST /api/ha/services/{domain}/{service}` | the room page's Home Assistant proxy; 404 without an `ha` section | +| `GET /api/tippen/curriculum` `/progress`, `PUT /api/tippen/settings`, `POST /api/tippen/runs` | the typing game; 404 without a `tippen` section | Two decisions worth knowing: diff --git a/python-backend/musicmouse/config.py b/python-backend/musicmouse/config.py index 4062ab8..856767a 100644 --- a/python-backend/musicmouse/config.py +++ b/python-backend/musicmouse/config.py @@ -205,7 +205,7 @@ class ClockConfig(_Strict): class LibraryConfig(_Strict): """Where the music lives. - One path. The shelves underneath it - ``Figuren``, ``Musik``, ``Hoerbuecher``, + One path. The shelves underneath it - ``Figuren``, ``Musik``, ``Hörbücher``, ``Kinderpodcasts`` - are fixed names, not settings; see :mod:`musicmouse.library.sections`. """ @@ -259,10 +259,10 @@ class HaDeviceConfig(_Strict): class HaConfig(_Strict): """Home Assistant integration for the room-control page ("Mein Zimmer"). - The backend never calls Home Assistant itself - it only hands the browser the - server URL, the token, and these two ordered lists. Control happens directly from - the browser to Home Assistant's own REST API, so this token grants full HA control - to anything on the LAN that can reach musicmouse. See config.yml.example. + The browser only gets the two ordered lists. Every state read and service call goes + through the backend's ``/api/ha/...`` proxy, which attaches the token - so the token + never leaves the backend. The proxy itself is unauthenticated, though: anything on + the LAN that can reach musicmouse can use it. See config.yml.example. """ url: str diff --git a/python-backend/musicmouse/events.py b/python-backend/musicmouse/events.py index 30fe922..5ce7cd2 100644 --- a/python-backend/musicmouse/events.py +++ b/python-backend/musicmouse/events.py @@ -7,8 +7,8 @@ Three flavours, distinguished by base class: * :class:`StateEvent` - something changed. The intent layer is what lets several front-ends drive the same behaviour: a button -press, an MQTT command and a future web request all emit ``NextTrackRequested`` and a -single reaction acts on it. +press, an MQTT command, a web request and the IR remote all emit ``NextTrackRequested`` +and a single reaction acts on it. """ from __future__ import annotations diff --git a/python-backend/musicmouse/library/colors.py b/python-backend/musicmouse/library/colors.py index 0044c1d..a36bcec 100644 --- a/python-backend/musicmouse/library/colors.py +++ b/python-backend/musicmouse/library/colors.py @@ -2,7 +2,7 @@ The frontend paints an album card with these and the LED strips run an effect in them, so a web-started album looks the same on the shelf as it does on the screen. Extraction -happens once per album ever - the result is cached next to the cover. +happens when an album is scanned; the result is stored in the cached index. """ from __future__ import annotations diff --git a/python-backend/musicmouse/media.py b/python-backend/musicmouse/media.py index 621af25..69ace99 100644 --- a/python-backend/musicmouse/media.py +++ b/python-backend/musicmouse/media.py @@ -1,8 +1,8 @@ """Playlist model. -Deliberately a real type rather than a bare ``list[str]``: it is what the browse API -serves, and it keeps track metadata in one place. Building one from a folder is -:mod:`musicmouse.library`'s job. +What the player plays: a name and the files, in order. The browse API serves +:class:`~musicmouse.library.models.Album` instead, which carries the tag metadata; +``Album.to_playlist()`` builds one of these from it. """ from __future__ import annotations diff --git a/python-backend/musicmouse/reactions/playback.py b/python-backend/musicmouse/reactions/playback.py index 3881181..8d392e9 100644 --- a/python-backend/musicmouse/reactions/playback.py +++ b/python-backend/musicmouse/reactions/playback.py @@ -1,8 +1,8 @@ """What the mouse plays, and when. Physical inputs are turned into *intents*, and the intents are what actually drive the -player. That indirection is the point: an MQTT command or a future web request emits -the same intent and lands in the same handler, so there is one place per behaviour. +player. That indirection is the point: an MQTT command, a web request or an IR remote key +emits the same intent and lands in the same handler, so there is one place per behaviour. """ from __future__ import annotations diff --git a/python-backend/musicmouse/services/__init__.py b/python-backend/musicmouse/services/__init__.py index d00f59d..4e746f0 100644 --- a/python-backend/musicmouse/services/__init__.py +++ b/python-backend/musicmouse/services/__init__.py @@ -1,7 +1,8 @@ """Front-ends: things that mirror state outwards and turn requests into intents. -``MqttService`` is the reference implementation. A web service would be another file -here plus one line in the app - devices and reactions would not change. +MQTT (Home Assistant), web (the browser), lirc (the IR remote) and the screen, timer and +podcast services each live here; adding another is a file here plus a line in +``__main__._build_services`` - devices and reactions do not change. """ from musicmouse.services.base import Service diff --git a/python-backend/musicmouse/tippen/progress.py b/python-backend/musicmouse/tippen/progress.py index 45b9050..7fea8b2 100644 --- a/python-backend/musicmouse/tippen/progress.py +++ b/python-backend/musicmouse/tippen/progress.py @@ -5,8 +5,8 @@ Same convention as everywhere else this codebase persists something outside missing or corrupt file is just a fresh start, never a crash: nothing here is precious enough to raise over. -``record_run`` is a near-verbatim port of tippen's own (client-side, until now) -``recordRun`` in ``tippen/src/lib/progress.ts`` - grading a keystroke-by-keystroke run +``record_run`` is a port of the ``recordRun`` the standalone typing app had client-side +before it was merged into the web front-end - grading a keystroke-by-keystroke run into stars/points/an animal stays entirely client-side (see ``TippenRunIn`` in ``musicmouse.services.web.schemas``); this only owns what happens to progress once a graded result arrives. diff --git a/web/README.md b/web/README.md index 77618f3..bd351a8 100644 --- a/web/README.md +++ b/web/README.md @@ -33,9 +33,11 @@ For the device, `npm run build` writes `dist/`, and `general.web.static_dir` in ```sh npm run test # vitest over src/lib -npx tsc --noEmit # strict, noUncheckedIndexedAccess +npm run check # tsc (strict, noUncheckedIndexedAccess) and eslint ``` +`../scripts/check.sh web` runs both. + Only the pure modules are tested - the search, the keyboard machine and the time formatting. That is where the behaviour lives; the components are mostly layout. @@ -61,11 +63,12 @@ are local. |---|---| | `src/lib/keyboard.ts` | The key map, as a pure function from a keypress to a list of actions. Testable without a DOM. | | `src/lib/search.ts` | Filtering and the three-level browse hierarchy. The whole library is in memory, so this is instant - see the note on `buildSearchIndex` below. | -| `src/lib/lowPower.ts` | The `?pi=1` profile: what gets cheaper, and by how much. | +| `src/lib/perfSettings.ts` | The performance knobs: the `FULL` and `PI` presets, and per-browser overrides from the ⚡ dialog. | +| `src/lib/lowPower.ts` | The measurements behind the `?pi=1` preset, and the ambient-canvas quality numbers. | | `src/lib/covers.ts` | Generated cover art from the album's three colours, and the book-spine treatment. | | `src/hooks/usePlayerState.ts` | The websocket, with reconnect-and-reseed. | | `src/hooks/usePlaybackClock.ts` | Interpolates the 2 Hz position onto animation frames. | -| `src/components/` | Layout only. | +| `src/components/` | Layout only. `RoomView` is the "Mein Zimmer" page, `TippenApp` the typing game. | `src/api/types.ts` mirrors `musicmouse/services/web/schemas.py`; the two are hand-kept in step. @@ -73,26 +76,29 @@ step. ## Keyboard The device is meant to be driven from a keyboard, so every screen is reachable without a -pointer. `F1` shows the same list in-app. +pointer. `F1` shows the full list in-app; `src/components/HelpOverlay.tsx` is that list +and `src/lib/keyboard.ts` is the binding - change both together. | Key | | |---|---| | `SPACE` | play / pause | -| `CTRL+H` / `CTRL+L` | previous / next track | -| `CTRL+J` / `CTRL+K` | quieter / louder | +| `SHIFT+H` / `SHIFT+L` | previous / next track | +| `SHIFT+J` / `SHIFT+K` | quieter / louder | +| `SHIFT+M` | mute | | `SHIFT+←` / `SHIFT+→` | seek 15 s | -| `A-Z`, `0-9` | search albums as you type | -| `?` | switch to searching individual tracks | -| `TAB` | cycle all / music / audiobooks | -| `← ↑ ↓ →` | move the selection while browsing; transport and volume while playing | -| `ENTER` | play the selection | -| `ESC` | back out one layer at a time | -| `/` | back to browsing | +| `0-9` | what the IR remote's number key does | +| `← ↑ ↓ →`, `CTRL+HJKL` | move the selection | +| `ENTER` / `SHIFT+ENTER` | play the selection / open its track list | +| `TAB` | cycle music / audiobooks / podcasts | +| `A-Z` | search albums as you type | +| `?` | search individual tracks | +| `/` | open search, or switch albums ↔ tracks | +| `A` then `0-9` | put the playing album on that remote key (`SHIFT+A`: just this song) | +| `INSERT` | clock on / off | +| `DELETE` | screen off / on (only on the device itself) | +| `ESC` | close, or clear the search | | `F1` | this list | -`SHIFT+arrows` is the one addition to the mockup: plain arrows were already taken, and -seeking is the one thing a real player can do that the mockup could not. - ## The Raspberry Pi profile `?pi=1` loads the same app with nothing on screen animating itself. It exists for @@ -125,15 +131,18 @@ sea behind everything - it just no longer moves or follows the track. `AMBIENCE_QUALITY` in `src/lib/lowPower.ts` is kept accurate for whoever turns the canvas back on. -The flag is read once at module load from the URL and nowhere else - no persistence, no -auto-detection. That is what makes "is this the profile or the hardware?" answerable by -editing the address bar. The kiosk gets it from `pi_kiosk_url` in the ansible repo. +`?pi=1` selects the `PI` preset in `src/lib/perfSettings.ts`, and each knob can then be +overridden from the ⚡ dialog, stored in that browser's localStorage. Without `?pi=1` +the stored values are ignored and the full version applies, which keeps "is this the +profile or the hardware?" answerable by editing the address bar. Settings are read once +at load, so a change applies on reload. The kiosk gets the flag from `pi_kiosk_url` in +the ansible repo. ## Parent mode -`?parentMode=1` opens a settings panel - volume limits, the rotary step, button -brightness, and a button to re-read the library. Saving writes the device's -`config.yml`. +`?parentMode=1` adds a way into a settings panel - volume limits, the rotary step, button +brightness, the screen's auto-off, the parents' timer, and a button to re-read the +library. Saving writes the device's `config.yml`. It is hidden from the child, not protected from them: there is no authentication on the endpoints behind it. See `../python-backend/README.md`. @@ -190,7 +199,10 @@ than patching one element at a time: home-indicator padding while the keyboard is up ([safe-area-inset-bottom does not update for the keyboard](https://webventures.rejh.nl/blog/2025/safe-area-inset-bottom-does-not-update/)). -## Not implemented +## Other tabs -The mockup's second page, "Mein Zimmer" (Hue lights, a roller shutter and scenes), is -not built. Nothing here depends on it. +- **Mein Zimmer** (`RoomView`): the mockup's room page - lights, a roller shutter and + scenes - shown when the backend has an `ha` section. Every Home Assistant call goes + through the backend's `/api/ha/...` proxy, so the browser never sees the token. +- **Tippen** (`TippenApp`): the typing game, when the backend has a `tippen` section. + Passing lessons unlocks tracks in the library. diff --git a/web/src/components/tippen/AppHeader.tsx b/web/src/components/tippen/AppHeader.tsx index 3f067c8..32e295a 100644 --- a/web/src/components/tippen/AppHeader.tsx +++ b/web/src/components/tippen/AppHeader.tsx @@ -1,5 +1,5 @@ -/** The header, matching ../../web/src/components/AppHeader.tsx so the two apps open the - * same way. The mascot bobs; `SHOW_DRIFT` doubles as the decorative-motion switch. */ +/** The typing tab's header, matching ../AppHeader.tsx so both tabs open the same + * way. The mascot bobs; `SHOW_DRIFT` doubles as the decorative-motion switch. */ import type { ReactNode } from "react"; import { SKIN } from "../../lib/skins"; diff --git a/web/src/lib/lowPower.ts b/web/src/lib/lowPower.ts index 4e6f145..82da17e 100644 --- a/web/src/lib/lowPower.ts +++ b/web/src/lib/lowPower.ts @@ -41,19 +41,14 @@ * "decorative loops" row above: noise). Turning the canvas back on from the dialog * wants the blur back on with it - the 118% row is that combination. * - * Read once at module load, from the URL and nowhere else: no persistence, no - * auto-detection. The kiosk points Firefox at `http://localhost:8080/?pi=1` (see the - * `pi_kiosk_url` host var in the ansible repo) and a laptop opening the same page gets - * the full-fat version, which is what makes "is this the profile or the hardware?" - * answerable by editing the address bar. + * The kiosk points Firefox at `http://localhost:8080/?pi=1` (see the `pi_kiosk_url` host + * var in the ansible repo). That flag is no longer a single switch: it selects the `PI` + * preset in `lib/perfSettings.ts`, which the ⚡ dialog can override per knob, and every + * knob here and in `theme.ts` reads `PERF`. This header is kept for the measurements. */ import { PERF, PERF_ENABLED } from "./perfSettings"; -// Superseded as a single switch: every knob below and in theme.ts now comes from -// `PERF` (lib/perfSettings.ts), which `?pi=1` seeds and the settings dialog overrides. -// This header is kept for the measurements it records. - export interface AmbienceQuality { /** Multiplies the canvas backing store relative to its on-screen size, before * `devicePixelRatio`. At 0.5 the loop fills a quarter of the pixels and the browser