6 Commits

Author SHA1 Message Date
55bcc9448c color parsing cleanup 2026-08-26 11:28:48 +02:00
1cb3387fbe Fix: lights now should turn off correctly when playlist ends 2026-08-26 10:54:40 +02:00
7aa7fe4693 Claude cleanup 2026-08-26 10:38:56 +02:00
bd8925a278 Cleaned up repository
- only moving files around
2026-08-25 17:06:53 +02:00
Martin Bauer
b2c060fcc9 Adapted to new aiomqtt version 2024-03-07 16:38:05 +01:00
Martin Bauer
11bf0505fb Changes for python3.11 2024-03-06 17:15:07 +01:00
90 changed files with 512252 additions and 153 deletions

1
.gitignore vendored
View File

@@ -4,3 +4,4 @@ build
*.FCStd1
*.blend1
__pycache__
.ipynb_checkpoints

42
docs/REPO_OVERVIEW.md Normal file
View File

@@ -0,0 +1,42 @@
# MusicMouse — repo overview
Orientation doc for AI agents (or humans) working on this repo for the first time. Written from what's actually in the repo — no roadmap speculation beyond `esp-firmware/todo.md`.
## What this is
MusicMouse is a DIY, Toniebox-style physical music player for kids, shaped like a mouse and living on a shelf. Small 3D-printed animal figurines (fox, owl, dog, elephant, squirrel, crocodile, rabbit, snowman, puppy — see `hardware/3dprints/figures/`) each carry an RFID tag. Placing a figurine on the mouse triggers an RFID read, which starts that figure's music playlist. The mouse also has a rotary encoder + touch buttons (ears/feet) for volume/skip control, addressable RGBW LED rings with animated effects, and MQTT/Home Assistant integration so a "shelf light" shows up as a smart-home device.
## Repo layout
| Path | What it is |
|---|---|
| `python-backend/` | Python host application — the main runtime. Reads RFID/button/encoder events from the ESP32 over serial, drives playback via VLC, sends LED effect commands, bridges to MQTT/Home Assistant. Start here for backend work. |
| `esp-firmware/` | ESP32 firmware (C++, Arduino framework via PlatformIO). Reads the RFID reader and buttons, drives the LED strips, talks to `python-backend` over serial. |
| `hardware/` | 3D-print models for the figurines and enclosure (FreeCAD/Blender/OBJ/STL), a Fritzing electronics sketch, datasheets, and `pinout.md` (RFID reader + button-board wiring). |
| `claude-design/` | An unrelated exploratory web-UI mockup ("Dolphin Beats Music Player" / rebrand concept). Not integrated with the rest of the repo — no build system ties it in. Don't assume it reflects current product direction. |
| `.vscode/` | Editor settings (C++ header associations). |
There is no top-level README elsewhere in the repo; this file plus `esp-firmware/todo.md` and `hardware/pinout.md` are the only prose docs.
## How the pieces talk to each other
- **ESP32 firmware ↔ `python-backend`**: a length-prefixed binary protocol over serial (`pyserial-asyncio`), implemented in `python-backend/host_driver.py`. Messages are framed with magic tokens (`MAGIC_TOKEN_HOST_TO_FW`/`MAGIC_TOKEN_FW_TO_HOST`) and a `struct`-packed header. The message-ID maps and struct formats in `host_driver.py` must stay byte-for-byte in sync with the firmware's `esp-firmware/src/Messages.h` — there's no shared schema or test verifying this cross-language contract, so a firmware protocol change can silently desync the Python side.
- **`python-backend` ↔ MQTT/Home Assistant**: `python-backend/mqtt_json.py` exposes the shelf LED strip as a Home-Assistant-discoverable JSON-schema MQTT light (`ShelveLightMqtt`), and `main.py` also calls Home Assistant services directly (e.g. toggling room lights) via `hass-client`.
## Running it
```
python python-backend/main.py <config_dir>
```
`main.py` expects `<config_dir>/config.yml` (schema documented in the new `python-backend/config.yml.example` — no real config was previously checked in or documented). In production this is deployed as a systemd service reading music from `/media/musicmouse/`; see `esp-firmware/musicmouse.service` for the unit template — **note its `ExecStart` path (`.../espmusicmouse/host_driver/main.py`) is stale**, referencing the pre-reorg directory layout from before the `bd8925a "Cleaned up repository"` commit moved things to `python-backend/`. Update that path before relying on the service file.
## Config schema (see `python-backend/config.yml.example`)
- `general.{alsa_device, serial_port, hass_url, hass_token, mqtt.{server,user,password}, min_volume, max_volume, volume_increment, button_leds_brightness}`
- `figures.<name>.{id, colors, media_files}``id` is a hex RFID tag id, `colors` is a list of 4 colors (`primary, secondary, background, accent`, each `"#rrggbb"` or `"wNN"`), `media_files` is optional (auto-globbed from the config dir by figure name if omitted).
## Known gaps / notes for agents
- **No automated tests, no lint/formatter config, no CI** anywhere in the repo (neither `python-backend/` nor `esp-firmware/`, aside from a PlatformIO `native` build env for firmware unit testing).
- `python-backend/audio_analysis.py` and the three chord-recognition notebooks (`C5S2_ChordRec_Templates.ipynb`, `C5S3_ChordRec_HMM.ipynb`, `C5S3_HiddenMarkovModel.ipynb`) are university-course exploratory material (chroma/chord-recognition DSP), **not imported by `main.py`** and not part of the running app. They reference stale personal absolute paths.

View File

@@ -1,74 +0,0 @@
Reader
----------
- GND black
- RST blue 3.3V
- 3.3V red
- MISO brown 21
- SDA green 19
- SCK yellow 18
- MOSI orange 5
- IRQ green single cable not connected
Button Board:
-------------
- rot in | white 13
- btn2 led | grey 12
- btn2 in | purple 14
- rotB | blue 27
- rotA | green 26
- btn1 in | yellow 25
- btn1 led | orange 33
rot="rotary encoder"
in=button sense in
led = 5V pwm
Firmware Planning
-----------------
- input commands:
- led: effect + parameters
- off
- single color
- multiple color HSV fade, list of colors with timings
- circular motion (already exists)
- chained events? e.g. circle two times then fade
- effects:
- welle fuer an und aus
- breathe waehrend an, oder farbgradient
-
- output infos:
- nfc read: with id
- nfc remove
- button presses, (possible also long press, double click, etc)
- rotary encoder up down + current numeric state
- on led effect end?
TODO
----
1) case redesign
- slightly smaller led ring (10mm -> 9mm) [ok]
- thicker top of inner ring, but cutouts for reader [ok]
- adjust reader stands position [ok]
- bottom for led ring snap-in [ok]
- bottom for inner ring [ok]
- stands for own "pcb" [ok]
- 2 cutouts for cables [ok]
- checks, compared to existing print
- same diameter, very slightly smaller
- larger overlap of LED ring
- minimal wall thickness for led ring top and side
- check total height - compare to existing

View File

@@ -1,3 +0,0 @@
pyserial-asyncio==0.6
python-vlc==3.0.12118
hass-client==0.1.2

View File

Before

Width:  |  Height:  |  Size: 1.3 MiB

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

Binary file not shown.

Binary file not shown.

28
hardware/pinout.md Normal file
View File

@@ -0,0 +1,28 @@
Reader
----------
- GND black
- RST blue 3.3V
- 3.3V red
- MISO brown 21
- SDA green 19
- SCK yellow 18
- MOSI orange 5
- IRQ green single cable not connected
Button Board:
-------------
- rot in | white 13
- btn2 led | grey 12
- btn2 in | purple 14
- rotB | blue 27
- rotA | green 26
- btn1 in | yellow 25
- btn1 led | orange 33
rot="rotary encoder"
in=button sense in
led = 5V pwm

BIN
hardware/sketch.fzz Normal file

Binary file not shown.

View File

@@ -1,4 +1,3 @@
from typing import overload
import librosa
from numba import jit
import numpy as np

View File

@@ -0,0 +1,51 @@
# Example config.yml for the MusicMouse python-backend.
#
# Reverse-engineered from main.py (load_config/Controller) since no schema
# was previously documented. Copy this file to config.yml in the directory
# passed as the CLI argument to main.py, e.g.:
#
# python main.py /media/musicmouse/
#
# main.py reads "<config_dir>/config.yml". Real credentials (hass_token,
# mqtt.password) should never be committed - keep the real config.yml
# outside the repo (e.g. only on the deployed device).
general:
# ALSA output device passed to python-vlc, e.g. "hw:0,0"; omit/null for VLC's default.
alsa_device: "softvol_effects"
# Serial port the ESP32 firmware is connected on.
serial_port: "/dev/ttyUSB0"
# Home Assistant connection used for light/service calls (hass_service()).
hass_url: "http://homeassistant.local:8123"
hass_token: "REPLACE_WITH_LONG_LIVED_ACCESS_TOKEN"
# MQTT broker used for the Home-Assistant-discoverable "shelf light".
mqtt:
server: "homeassistant.local"
user: "musicmouse"
password: "REPLACE_WITH_MQTT_PASSWORD"
# Optional playback/UI tuning (all have defaults if omitted).
min_volume: 0
max_volume: 32
volume_increment: 5 # per rotary-encoder tick
button_leds_brightness: 0.5 # 0..1, brightness of the prev/next button backlight
# One entry per figurine. The key is an arbitrary figure name (also used as
# the subdirectory name under the config dir when media_files is omitted).
figures:
fuchs:
# RFID tag id as a hex string (matched against bytes read from the reader).
id: "04a1b2c3d4"
# Exactly 4 colors: [primary, secondary, background, accent].
# Accepted formats: "#rrggbb" (RGB hex) or "wNN" (white channel hex, e.g. "wff").
colors: ["#ff6600", "#ffcc00", "#331100", "#ffffff"]
# Optional explicit list of media file paths for this figure's playlist.
# If omitted, main.py globs os.path.join(config_dir, "<figure_name>").
media_files: []
eule:
id: "04b2c3d4e5"
colors: ["#3355ff", "#66aaff", "#001133", "#ffffff"]

View File

@@ -23,7 +23,8 @@ if __name__ == "__main__":
url = sys.argv[1]
token = sys.argv[2]
loop = asyncio.get_event_loop()
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
hass = HomeAssistantClient(url, token)
async def hass_event(event, event_details):

View File

@@ -2,7 +2,9 @@ import asyncio
from enum import Enum
import struct
from led_cmds import *
from led_cmds import (EffectStaticConfig, EffectStaticDetailedConfig, EffectAlexaSwipeConfig,
EffectCircularConfig, EffectRandomTwoColorInterpolationConfig,
EffectSwipeAndChange, EffectReverseSwipe)
MAGIC_TOKEN_HOST_TO_FW = 0x1d6379e3
MAGIC_TOKEN_FW_TO_HOST = 0x10c65631

View File

@@ -1,4 +1,4 @@
from dataclasses import dataclass
from dataclasses import dataclass, field
import struct
import colorsys
@@ -26,12 +26,6 @@ class ColorRGBW:
assert 0<= other <= 1
return ColorRGBW(self.r * other, self.g * other, self.b * other, self.w * other)
def __eq__(self, other:'ColorRGBW'):
return self.r == other.r and self.g == other.g and self.b == other.b and self.w == other.w
def __neq__(self, other:'ColorRGBW'):
return not self == other
def without_white_channel(self, scale=1):
args = (min(1, e + self.w) for e in (self.r, self.g, self.b) )
return ColorRGBW(*args, 0)
@@ -98,8 +92,8 @@ class EffectAlexaSwipeConfig:
bell_curve_width_in_leds: float = 3
start_position: float = 180 # in degrees
forward: bool = True
primary_color: ColorRGBW = ColorRGBW(0, 0, 1, 0)
secondary_color: ColorRGBW = ColorRGBW(0, 200 / 255, 1, 0)
primary_color: ColorRGBW = field(default_factory=lambda: ColorRGBW(0, 0, 1, 0))
secondary_color: ColorRGBW = field(default_factory=lambda: ColorRGBW(0, 200 / 255, 1, 0))
def as_bytes(self) -> bytes:
return struct.pack(
@@ -118,8 +112,8 @@ class EffectRandomTwoColorInterpolationConfig:
num_segments: int = 3
hue1_random: bool = False
hue2_random: bool = False
color1: ColorHSV = ColorHSV(240, 1, 1)
color2: ColorHSV = ColorHSV(192, 1, 1)
color1: ColorHSV = field(default_factory=lambda: ColorHSV(240, 1, 1))
color2: ColorHSV = field(default_factory=lambda: ColorHSV(192, 1, 1))
def as_bytes(self) -> bytes:
c1 = ColorHSV.fromRGB(self.color1) if isinstance(self.color1, ColorRGBW) else self.color1
@@ -136,7 +130,7 @@ class EffectRandomTwoColorInterpolationConfig:
class EffectCircularConfig:
speed: float = 360 # in degrees per second
width: float = 180 # in degrees
color: ColorRGBW = ColorRGBW(0, 0, 1, 0)
color: ColorRGBW = field(default_factory=lambda: ColorRGBW(0, 0, 1, 0))
def as_bytes(self) -> bytes:
return struct.pack("<ff", self.speed, self.width) + self.color.as_bytes()
@@ -144,8 +138,8 @@ class EffectCircularConfig:
@dataclass
class EffectSwipeAndChange:
swipe: EffectAlexaSwipeConfig = EffectAlexaSwipeConfig()
change: EffectRandomTwoColorInterpolationConfig = EffectRandomTwoColorInterpolationConfig()
swipe: EffectAlexaSwipeConfig = field(default_factory=lambda: EffectAlexaSwipeConfig())
change: EffectRandomTwoColorInterpolationConfig = field(default_factory=lambda: EffectRandomTwoColorInterpolationConfig())
def as_bytes(self) -> bytes:
return self.swipe.as_bytes() + self.change.as_bytes()

View File

@@ -3,7 +3,7 @@
import asyncio
import sys
import serial_asyncio
from led_cmds import (ColorRGBW, ColorHSV, EffectCircularConfig, EffectStaticConfig,
from led_cmds import (ColorRGBW, EffectCircularConfig, EffectStaticConfig,
EffectRandomTwoColorInterpolationConfig, EffectAlexaSwipeConfig,
EffectSwipeAndChange, EffectReverseSwipe)
from host_driver import MusicMouseProtocol, RfidTokenRead, RotaryEncoderEvent, ButtonEvent, TouchButton, TouchButtonPress, TouchButtonRelease, mouse_leds_index_ranges
@@ -12,11 +12,9 @@ from glob import glob
from copy import deepcopy
import os
from hass_client import HomeAssistantClient
import argparse
from ruamel.yaml import YAML
import warnings
from pprint import pprint
from typing import Optional
from typing import Optional, NamedTuple
from mqtt_json import start_mqtt
yaml = YAML(typ='safe')
@@ -24,6 +22,13 @@ yaml = YAML(typ='safe')
OFF_COLOR = ColorRGBW(0, 0, 0, 0)
class FigureColors(NamedTuple):
primary: ColorRGBW
secondary: ColorRGBW
bg: ColorRGBW
accent: ColorRGBW
def parse_color(color_str: str):
if isinstance(color_str, ColorRGBW):
return color_str
@@ -34,13 +39,16 @@ def parse_color(color_str: str):
elif color_str.startswith("w"):
color_str = color_str.lstrip("w")
return ColorRGBW(0, 0, 0, int(color_str, 16) / 255)
else:
raise ValueError(f"Unrecognized color format: {color_str!r}")
def load_config(config_path):
# Schema documented in config.yml.example.
with open(os.path.join(config_path, "config.yml")) as cfg_file:
cfg = yaml.load(cfg_file)
for figure_name, figure_cfg in cfg["figures"].items():
figure_cfg["colors"] = [parse_color(c) for c in figure_cfg["colors"]]
figure_cfg["colors"] = FigureColors(*(parse_color(c) for c in figure_cfg["colors"]))
if 'media_files' not in figure_cfg:
figure_cfg['media_files'] = sorted(glob(os.path.join(config_path, figure_name)))
return cfg
@@ -132,9 +140,8 @@ class Controller:
self.mmstate.active_figure = None
elif tagid in self._rfid_to_figure_name:
newly_placed_figure = self._rfid_to_figure_name[tagid]
primary_color, secondary_color, *rest = self.cfg["figures"][newly_placed_figure][
"colors"]
self._start_animation(primary_color, secondary_color)
colors = self.cfg["figures"][newly_placed_figure]["colors"]
self._start_animation(colors.primary, colors.secondary)
self.mmstate.button_leds(self.cfg["general"].get("button_leds_brightness", 0.5))
if newly_placed_figure in self.cfg['figures']:
@@ -144,8 +151,7 @@ class Controller:
else:
print("Restarting playlist")
self.audio_player.set_playlist(
self.audio_player.create_playlist(
self.cfg['figures'][newly_placed_figure]['media_files']))
self.audio_player.create_playlist(self.cfg['figures'][newly_placed_figure]['media_files']))
self.audio_player.play_from_start()
self.mmstate.active_figure = newly_placed_figure
@@ -157,8 +163,7 @@ class Controller:
if isinstance(message, RfidTokenRead):
self.handle_rfid_event(message.id)
elif isinstance(message, RotaryEncoderEvent):
volume_increment = self.cfg["general"].get("volume_increment", 2) * abs(
message.increment)
volume_increment = self.cfg["general"].get("volume_increment", 2) * abs(message.increment)
if message.direction == 2:
self.audio_player.change_volume(volume_increment)
elif message.direction == 1:
@@ -174,9 +179,9 @@ class Controller:
elif isinstance(message, TouchButtonPress):
figure = self.mmstate.active_figure
if figure and self.audio_player.is_playing():
primary_color, secondary_color, bg, accent = self.cfg["figures"][figure]["colors"]
figure_colors = self.cfg["figures"][figure]["colors"]
self.protocol.mouse_led_effect(
EffectStaticConfig(accent, *mouse_leds_index_ranges[message.touch_button]))
EffectStaticConfig(figure_colors.accent, *mouse_leds_index_ranges[message.touch_button]))
colors = {
TouchButton.RIGHT_FOOT: {
@@ -204,15 +209,15 @@ class Controller:
eff_change = EffectRandomTwoColorInterpolationConfig()
eff_static = EffectStaticConfig(ColorRGBW(0, 0, 0, 0),
*mouse_leds_index_ranges[message.touch_button])
if self.audio_player.is_playing():
primary_color, secondary_color, bg, accent = self.cfg["figures"][figure]["colors"]
eff_static.color = primary_color
if figure and self.audio_player.is_playing():
colors = self.cfg["figures"][figure]["colors"]
eff_static.color = colors.primary
self.protocol.mouse_led_effect(eff_static)
if self.audio_player.is_playing():
primary_color, secondary_color, bg, accent = self.cfg["figures"][figure]["colors"]
eff_change.color1 = primary_color
eff_change.color2 = secondary_color
if figure and self.audio_player.is_playing():
colors = self.cfg["figures"][figure]["colors"]
eff_change.color1 = colors.primary
eff_change.color2 = colors.secondary
eff_change.start_with_existing = True
self.protocol.mouse_led_effect(eff_change)
@@ -246,8 +251,9 @@ class Controller:
def main(config_path):
cfg = load_config(config_path)
loop = asyncio.get_event_loop()
hass = HomeAssistantClient(cfg["general"]["hass_url"], cfg["general"]["hass_token"], loop)
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
hass = HomeAssistantClient(cfg["general"]["hass_url"], cfg["general"]["hass_token"], loop=loop)
coro = serial_asyncio.create_serial_connection(loop,
MusicMouseProtocol,
@@ -256,7 +262,7 @@ def main(config_path):
transport, protocol = loop.run_until_complete(coro)
controller = Controller(protocol, hass, cfg)
mqtt_cfg = cfg["general"]["mqtt"]
loop.create_task(start_mqtt(protocol, mqtt_cfg["server"], mqtt_cfg["user"],mqtt_cfg["password"] ))
loop.create_task(start_mqtt(protocol, mqtt_cfg["server"], mqtt_cfg["user"], mqtt_cfg["password"] ))
loop.create_task(hass.connect())
return controller, loop

View File

@@ -1,12 +1,11 @@
from led_cmds import ColorRGBW, EffectStaticConfig, EffectStaticDetailedConfig, EffectCircularConfig, EffectRandomTwoColorInterpolationConfig, EffectAlexaSwipeConfig, EffectSwipeAndChange
import asyncio
import asyncio_mqtt
import aiomqtt
import json
from copy import deepcopy
class ShelveLightMqtt:
def __init__(self, protocol, client: asyncio_mqtt.Client):
def __init__(self, protocol, client: aiomqtt.Client):
self._protocol = protocol
self._mqtt_client = client
@@ -33,7 +32,7 @@ class ShelveLightMqtt:
await self._notify_mqtt_state({"state": "OFF"})
async def handle_light_message(self, msg):
if msg.topic == self._discovery_spec['command_topic']:
if msg.topic.value == self._discovery_spec['command_topic']:
payload = msg.payload.decode()
new_state = json.loads(payload)
print("IN ", new_state)
@@ -153,22 +152,6 @@ class ShelveLightMqtt:
async def _notify_mqtt_state(self, state):
state_payload = json.dumps(self._state)
print("OUT ", state_payload)
await self._mqtt_client.publish(self._discovery_spec['state_topic'], state_payload.encode())
return
direct_ack = False
if direct_ack == True:
state_payload = json.dumps(state)
else:
s = deepcopy(self._state)
if s['state'] == "OFF":
state_payload = json.dumps({"state": "OFF"})
else:
s['color_mode'] = "rgbw"
state_payload = json.dumps(s)
print("OUT ", state_payload)
await self._mqtt_client.publish(self._discovery_spec['state_topic'], state_payload.encode())
@@ -176,14 +159,13 @@ async def start_mqtt(music_mouse_protocol, server, username, password):
reconnect_interval = 10 # [seconds]
while True:
try:
async with asyncio_mqtt.Client(hostname=server, username=username, password=password) as client:
async with aiomqtt.Client(hostname=server, username=username, password=password) as client:
shelve_light = ShelveLightMqtt(music_mouse_protocol, client)
await shelve_light.init()
async with client.filtered_messages("musicmouse_json/#") as messages:
await client.subscribe("musicmouse_json/#")
async for message in messages:
async for message in client.messages:
await shelve_light.handle_light_message(message)
except asyncio_mqtt.MqttError as error:
except aiomqtt.MqttError as error:
print(f'Error "{error}". Reconnecting in {reconnect_interval} seconds')
finally:
await asyncio.sleep(reconnect_interval)

View File

@@ -27,9 +27,9 @@ class AudioPlayer:
evm = result.event_manager()
evm.event_attach(vlc.EventType.MediaListEndReached,
lambda e: print("Ml CB", str(vlc.EventType(e.type))))
lambda e: print("Ml CB", str(e.type)))
evm.event_attach(vlc.EventType.MediaListItemAdded,
lambda e: print("Ml ia CB", str(vlc.EventType(e.type))))
lambda e: print("Ml ia CB", str(e.type)))
return result
@@ -57,14 +57,11 @@ class AudioPlayer:
self.media_list_player.pause()
def _callback(self, event, *args, **kwargs):
eventStr = str(vlc.EventType(event.type))
print(f"Got vlc event type {event.type} {eventStr} , event {event}")
print(f"Got vlc event type {event.type}")
if event.type == vlc.EventType.MediaPlayerStopped:
if self.on_playlist_end_callback:
print("Calling playlist end cb")
self.on_playlist_end_callback()
#print("Callback from VLC", event, args, kwargs)
#print(event.meta_type, event.obj, event.type)
def set_volume(self, volume):
if self.volume_min and volume < self.volume_min:

View File

@@ -0,0 +1,5 @@
pyserial-asyncio==0.6
python-vlc==3.0.20123
hass-client==0.1.2
ruamel.yaml==0.18.6
aiomqtt==2.0.0