214 lines
11 KiB
Markdown
214 lines
11 KiB
Markdown
# Musik Delphin — the web front-end
|
|
|
|
Browse the whole music collection and play any of it from a browser: the six figurines
|
|
are only five of the twenty-odd albums on the shelf, and everything else was previously
|
|
unreachable without swapping files around on disk.
|
|
|
|
Built from `../claude-design/Dolphin Beats.dc.html`, which is a finished interaction and
|
|
visual spec rather than a sketch. The keyboard model, the layout, the colours and the
|
|
German copy are ported from it; what changed is that a real player sits behind it.
|
|
|
|
## Running it
|
|
|
|
The backend serves the API. Either of these will do - `--simulate` needs no audio device
|
|
and no libVLC at all, which makes it the easier one to develop against:
|
|
|
|
```sh
|
|
cd ../python-backend
|
|
python -m musicmouse --config ./config.yml --no-hardware # real audio
|
|
python -m musicmouse --config ./config.yml --simulate # fake player
|
|
```
|
|
|
|
Then:
|
|
|
|
```sh
|
|
npm install
|
|
npm run dev # http://localhost:5173, /api and the websocket proxied to :8080
|
|
```
|
|
|
|
For the device, `npm run build` writes `dist/`, and `general.web.static_dir` in
|
|
`config.yml` points at it so one process serves both.
|
|
|
|
## Checks
|
|
|
|
```sh
|
|
npm run test # vitest over src/lib
|
|
npm run check # tsc (strict, noUncheckedIndexedAccess) and eslint
|
|
```
|
|
|
|
`../scripts/check.sh web` runs both.
|
|
|
|
Only the pure modules are tested - the search, the keyboard machine and the time
|
|
formatting. That is where the behaviour lives; the components are mostly layout.
|
|
|
|
`src/lib/search.ts` builds its normalized search keys once per library payload
|
|
(`buildSearchIndex`) rather than per keystroke. It matters more than it sounds: a real
|
|
collection is 343 albums and 4969 tracks, and `normalize()` does a Unicode NFD
|
|
decomposition, so track search used to mean five thousand of those before a single
|
|
comparison - 4 ms per keystroke on a laptop, and this runs on a Pi. The same build also
|
|
partitions albums by shelf and pre-sorts each shelf's categories, which `App` and
|
|
`BrowseView` were otherwise deriving separately from the same data.
|
|
|
|
## How it is put together
|
|
|
|
The split that matters: **what is playing** comes from the backend, **what you are
|
|
looking at** lives in the browser. The mockup kept both in one state object, but a real
|
|
player has other front-ends - the buttons on the mouse, a figurine on the reader, Home
|
|
Assistant - and this UI has to follow them rather than assume it is in charge. So
|
|
`isPlaying`, `volume`, the position and the current album all arrive over the websocket,
|
|
and only `search / mode / filter / category / selIndex / view / openAlbumId / showHelp`
|
|
are local.
|
|
|
|
| Path | What it is |
|
|
|---|---|
|
|
| `src/lib/keyboard.ts` | The key map, as a pure function from a keypress to a list of actions. Testable without a DOM. |
|
|
| `src/lib/search.ts` | Filtering and the three-level browse hierarchy. The whole library is in memory, so this is instant - see the note on `buildSearchIndex` below. |
|
|
| `src/lib/perfSettings.ts` | The performance knobs: the `FULL` and `PI` presets, and per-browser overrides from the ⚡ dialog. |
|
|
| `src/lib/lowPower.ts` | The measurements behind the `?pi=1` preset, and the ambient-canvas quality numbers. |
|
|
| `src/lib/covers.ts` | Generated cover art from the album's three colours, and the book-spine treatment. |
|
|
| `src/hooks/usePlayerState.ts` | The websocket, with reconnect-and-reseed. |
|
|
| `src/hooks/usePlaybackClock.ts` | Interpolates the 2 Hz position onto animation frames. |
|
|
| `src/components/` | Layout only. `RoomView` is the "Mein Zimmer" page, `TippenApp` the typing game. |
|
|
|
|
`src/api/types.ts` mirrors `musicmouse/services/web/schemas.py`; the two are hand-kept in
|
|
step.
|
|
|
|
## Keyboard
|
|
|
|
The device is meant to be driven from a keyboard, so every screen is reachable without a
|
|
pointer. `F1` shows the full list in-app; `src/components/HelpOverlay.tsx` is that list
|
|
and `src/lib/keyboard.ts` is the binding - change both together.
|
|
|
|
| Key | |
|
|
|---|---|
|
|
| `SPACE` | play / pause |
|
|
| `SHIFT+H` / `SHIFT+L` | previous / next track |
|
|
| `SHIFT+J` / `SHIFT+K` | quieter / louder |
|
|
| `SHIFT+M` | mute |
|
|
| `SHIFT+←` / `SHIFT+→` | seek 15 s |
|
|
| `0-9` | what the IR remote's number key does |
|
|
| `← ↑ ↓ →`, `CTRL+HJKL` | move the selection |
|
|
| `ENTER` / `SHIFT+ENTER` | play the selection / open its track list |
|
|
| `TAB` | cycle music / audiobooks / podcasts |
|
|
| `A-Z` | search albums as you type |
|
|
| `?` | search individual tracks |
|
|
| `/` | open search, or switch albums ↔ tracks |
|
|
| `A` then `0-9` | put the playing album on that remote key (`SHIFT+A`: just this song) |
|
|
| `INSERT` | clock on / off |
|
|
| `DELETE` | screen off / on (only on the device itself) |
|
|
| `ESC` | close, or clear the search |
|
|
| `F1` | this list |
|
|
|
|
## The Raspberry Pi profile
|
|
|
|
`?pi=1` loads the same app with nothing on screen animating itself. It exists for
|
|
musicdolphin - a Pi 4 driving a 1920x1080 kiosk screen.
|
|
|
|
| | Full | `?pi=1` |
|
|
|---|---|---|
|
|
| Ambient canvas (gradient + bubbles) | on | **not rendered** |
|
|
| Decorative CSS loops, view transitions | on | off |
|
|
| Typing game bubbles, next-key pulse | on | off |
|
|
| Typing game pets | swimming | placed, held still |
|
|
| `backdrop-filter` on cards and list rows | on | off |
|
|
| `backdrop-filter` on panels | on | **on** |
|
|
| Progress-bar re-renders | every animation frame | 10 per second |
|
|
|
|
Two of those rows are worth knowing the reasons for, because both are the opposite of
|
|
what they look like.
|
|
|
|
**The panel blur stays on.** Dropping it measured *five times worse* on the device -
|
|
118% of a core against 25%. `backdrop-filter` is what promotes each glass panel to its
|
|
own compositing layer; without it the panels and everything under them collapse into
|
|
one layer that repaints wholesale. The card blur goes because that is a different
|
|
problem: three hundred of them in one grid, not one per panel.
|
|
|
|
**The canvas goes away rather than getting cheaper.** Half resolution and 30 fps took
|
|
it from 53.5% of a core to 25.0%, and 25% was still not good enough to use. A
|
|
`requestAnimationFrame` loop repainting the viewport is a floor you cannot get under
|
|
while it runs at all. `.stage` keeps its own static CSS gradient, so there is still a
|
|
sea behind everything - it just no longer moves or follows the track.
|
|
`AMBIENCE_QUALITY` in `src/lib/lowPower.ts` is kept accurate for whoever turns the
|
|
canvas back on.
|
|
|
|
`?pi=1` selects the `PI` preset in `src/lib/perfSettings.ts`, and each knob can then be
|
|
overridden from the ⚡ dialog, stored in that browser's localStorage. Without `?pi=1`
|
|
the stored values are ignored and the full version applies, which keeps "is this the
|
|
profile or the hardware?" answerable by editing the address bar. Settings are read once
|
|
at load, so a change applies on reload. The kiosk gets the flag from `pi_kiosk_url` in
|
|
the ansible repo.
|
|
|
|
## Parent mode
|
|
|
|
`?parentMode=1` adds a way into a settings panel - volume limits, the rotary step, button
|
|
brightness, the screen's auto-off, the parents' timer, and a button to re-read the
|
|
library. Saving writes the device's `config.yml`.
|
|
|
|
It is hidden from the child, not protected from them: there is no authentication on the
|
|
endpoints behind it. See `../python-backend/README.md`.
|
|
|
|
## Installing it on a phone
|
|
|
|
It is a PWA: `public/manifest.webmanifest`, icons generated from the mascot, and a small
|
|
service worker in `public/sw.js`. On iOS, Share → *Zum Home-Bildschirm*; it then opens
|
|
full-screen with no Safari chrome, because iOS reads the `apple-mobile-web-app-*` meta
|
|
tags in `index.html` rather than the manifest.
|
|
|
|
A few things worth knowing:
|
|
|
|
- **Install from `?parentMode=1` to get a parents' app.** iOS starts an installed app at
|
|
the manifest's `start_url`, not at the address it was added from, so the plain
|
|
manifest's `/` would drop the flag. Opened with it, `index.html` swaps in
|
|
`public/manifest-parents.webmanifest` (`start_url: /?parentMode=1`, named "Musik
|
|
Eltern"). An app installed before this change has to be removed and added again.
|
|
- **The service worker only registers over HTTPS or on `localhost`.** At
|
|
`http://<mouse>:8080` on the LAN, iOS will still install the icon and run it
|
|
standalone - that part is the meta tags - but there is no offline caching. Put the
|
|
mouse behind a TLS-terminating proxy if you want that.
|
|
- **The worker deliberately caches very little**: the app shell, the font and cover art.
|
|
Not `/api/state`, not the library, not anything else live. A player that shows stale
|
|
state is worse than one that says it cannot reach the mouse.
|
|
|
|
Nunito is self-hosted in `public/fonts/` (one variable file per subset, ~74 kB) rather
|
|
than pulled from Google. The mouse may have no internet, and a cached shell rendering in
|
|
a fallback face looks broken.
|
|
|
|
## Layout on phones
|
|
|
|
Phones (`usePhoneLayout`: narrow, or short landscape on touch) get the touch layout in
|
|
`components/mobile/`; tablets and the kiosk keep the keyboard one. The home-screen app
|
|
on iOS has caught every shortcut taken here, so these are the rules - follow them rather
|
|
than patching one element at a time:
|
|
|
|
- **The status bar is opaque** (`apple-mobile-web-app-status-bar-style: black`). The page
|
|
starts below it, so nothing can end up under the clock. `black-translucent` looks
|
|
nicer, but it ran the page behind the status bar, and since iOS 26 it also sizes the
|
|
app one status bar short, leaving a strip at the bottom that CSS cannot paint
|
|
([WebKit 301108](https://bugs.webkit.org/show_bug.cgi?id=301108)). The workarounds for
|
|
that (`100lvh`, measuring `screen` in JS) each broke something else. Installed apps
|
|
read this tag once: after changing it, remove the app and add it again.
|
|
- **`viewport-fit=cover` stays**, so the sea fills the rounded corners and the home
|
|
indicator area, which means:
|
|
- **Anything placed against a screen edge adds that edge's inset**: `--safe-top`,
|
|
`--safe-right`, `--safe-bottom` and `--safe-left` in `app.css`, always the variables
|
|
and never a bare `env()`. That covers `top`/`bottom`/`left`/`right` offsets of
|
|
positioned elements, and the padding of bars and scroll containers that reach the
|
|
edge. Scrolling content clears what floats over it (`--m-dock-clearance`). The top
|
|
inset is 0 with the opaque status bar, but stays in the formulas for Safari tabs and
|
|
landscape.
|
|
- **The stage is pinned, not sized**: `position: fixed; inset: 0` on phones, with the sea
|
|
colour on `html`/`body` so any gap would show sea, not white. No viewport-height unit
|
|
is trusted in the home-screen app.
|
|
- **The keyboard is measured, not assumed**: iOS slides it over the page without resizing
|
|
anything, so the dock is raised by `visualViewport` (`useKeyboardInset`) and drops its
|
|
home-indicator padding while the keyboard is up
|
|
([safe-area-inset-bottom does not update for the keyboard](https://webventures.rejh.nl/blog/2025/safe-area-inset-bottom-does-not-update/)).
|
|
|
|
## Other tabs
|
|
|
|
- **Mein Zimmer** (`RoomView`): the mockup's room page - lights, a roller shutter and
|
|
scenes - shown when the backend has an `ha` section. Every Home Assistant call goes
|
|
through the backend's `/api/ha/...` proxy, so the browser never sees the token.
|
|
- **Tippen** (`TippenApp`): the typing game, when the backend has a `tippen` section.
|
|
Passing lessons unlocks tracks in the library.
|