Release 4 — Platform, Performance, and Networking¶
Theme: Release 3 validated the pixel pipeline. Release 4 makes it production-ready: PSRAM-backed pixel buffers, true dual-core dispatch, Art-Net networking, a smarter module-creation UI, Game of Life effect, optional system modules, and a round of tech-debt revisits across the codebase.
Release Overview¶
What was delivered in Release 3 — build on this¶
| Strength | Notes |
|---|---|
| Clean layer model | EffectsLayer / DriverLayer / GridLayout hierarchy stable and well-tested |
| Driver-sized effects | onChildrenReady() + onSizeChanged() propagate layout changes automatically |
| Coord3D mapping | requestMappings() / serpentine / index_to_coords foundation in place |
| StatefulModule audit | Hotpath vs non-hotpath fields classified; meta_ store introduced |
| Live system tests | deploy/live_suite.py validates end-to-end pipelines on real hardware without USB |
| Python CI pipeline | all.py covers build → test → flash → live → summarise in one command |
What Release 4 addresses¶
| Problem | Sprint |
|---|---|
#ifdef ARDUINO scattered through modules; devices can't see each other |
Sprint 1 (PAL v1 + Device module) |
| Pixel buffers in internal RAM; grid size limited | Sprint 2 (PSRAM + dual-core) |
| Scheduler fps shown, not per-stage cost | Sprint 3 (hotpath tuning + fps/ms) |
| Flat type list for adding modules | Sprint 4 (advanced module-creation UI) |
| Missing Game of Life, Noise, Distortion effects | Sprint 5 (2D effects) |
| No real output protocol | Sprint 6 (Art-Net out) |
| Optional system utilities not loadable at runtime | Sprint 7 (optional modules) |
| Info-level logging clutters serial; no community links | Sprint 8 (observability + branding) |
| UI tech-debt and naming revisits | Sprint 9 (revisits) |
| Rough ideas under exploration + ESP32-P4 target | Sprint 10 |
Sprints¶
| Sprint | Goal |
|---|---|
| Sprint 1 | PAL v1 (zero #ifdef ARDUINO in modules) + Device discovery module |
| Sprint 2 | PSRAM pixel buffers + dual-core dispatch + double-buffer audit |
| Sprint 3 | Hotpath tuning + accurate fps/ms using ESP cycle counters |
| Sprint 4 | Advanced module-creation UI (context filter, tags, search) |
| Sprint 5 | 2D effects: Game of Life, Noise, Distortion |
| Sprint 6 | Art-Net in + out modules + two-device live test |
| Sprint 7 | Optional modules: TasksModule + FileManagerModule (OTA deferred to R5) |
| Sprint 8 | Logging refactor + MoonModules community branding |
| Sprint 9 | Revisits: defaults icon, naming, producer/consumer, auto-setup |
| Sprint 10 | FastLED-MM bootstrap + main.cpp refactor |
| Sprint 11 | projectMM as a domain-independent library (ProducerModule, ConsumerModule, library packaging) |
Sprint 1 — PAL v1 + Device Discovery¶
Scope¶
Goal: eliminate all #ifdef ARDUINO from src/modules/ behind a clean pal:: namespace, and make projectMM nodes discoverable and navigable on the local network. Doing PAL first means every subsequent sprint writes platform-clean code from day one.
Part A — PAL v1 (src/pal/)¶
Thin header-only namespace wrapping all platform calls currently scattered through modules:
| PAL function | ESP32 implementation | PC implementation |
|---|---|---|
pal::millis() |
::millis() |
std::chrono |
pal::micros() |
::micros() |
std::chrono |
pal::log(fmt, ...) |
Serial.printf |
printf |
pal::gpio_write(pin, val) |
digitalWrite |
no-op |
pal::psram_malloc(size) |
ps_malloc() |
malloc() |
pal::psram_free(ptr) |
free() |
free() |
pal::udp_send(ip, port, buf, len) |
WiFiUDP |
POSIX sendto |
Timing.h and MemoryStats.h become PAL headers. All #ifdef ARDUINO in src/modules/ replaced; guards in src/pal/ only. EffectsLayer / DriverLayer use pal::psram_malloc (ready for Sprint 2). The device module below uses pal::udp_send (ready immediately).
Part B — Device discovery module¶
A DeviceModule that broadcasts presence on the local network (UDP multicast or mDNS) and collects responses from other projectMM nodes. Uses pal::udp_send from Part A.
- UI: list of discovered devices showing device name, IP, firmware version
- A "Devices" nav entry links to any discovered device's UI in the browser — navigating to another node is as easy as navigating within one device
- Discovery is passive (no polling on the hot path); broadcast interval configurable
Definition of Done:
- Zero
#ifdef ARDUINOinsrc/modules/; all guards insrc/pal/ - All PAL functions implemented for ESP32 and PC; existing tests pass unchanged
- Device module lists at least two discovered projectMM nodes in the UI within 5 s of boot
- "Open device UI" link works on PC and ESP32
Result — Parts A + B complete¶
| Metric | Value |
|---|---|
| Unit tests | 212/212 ✅ |
| PC build | ✅ |
| ESP32 builds | ✅ (both envs) |
| Live tests | 2/3 devices, 5/5 suites ✅ (MM-ESP32 offline — not connected) |
| Discovery verified | PC discovered MM-70BC (devices=1); ESP32 discovered PC (devices=1) ✅ |
#ifdef ARDUINO in src/modules/ |
0 ✅ |
#ifdef ARDUINO remaining (structural) |
HttpServer.h, WsServer.h (two-class splits), main.cpp (entry-point), WiFi modules (hardware-only) |
Part A delivered:
src/pal/Pal.h— three-way platform switch (ARDUINO/IDF_VER/ PC) for timing, PSRAM, GPIO, logging, 18 system-info functions, filesystem, FreeRTOS task helpers, rebootsrc/pal/FileSystem.h— moved fromsrc/core/; LittleFS on Arduino,std::fstreamon PCsrc/pal/MemoryStats.h— moved fromsrc/core/; all callers updatedsrc/core/Timing.h— reduced to forwarding stub →pal::micros()EffectsLayer+DriverLayer— usepal::psram_malloc()/pal::psram_free()SystemStatus.h— zero#ifdef ARDUINO; all hardware queries viapal::main.cpp— sharedembeddedSetup(); Arduino getssetup()/loop(), IDF getsextern "C" void app_main()pal::gpio_write()added (Arduino:digitalWrite; IDF:gpio_set_level; PC: no-op)
Part B delivered:
pal::udp_bind/pal::udp_broadcast/pal::udp_recv/pal::udp_close— three-way PAL (ArduinoWiFiUDPpool of 4 slots viapal::_detail::udp_slot(i); IDF stubs; PC POSIX non-blocking socket)src/modules/system/DeviceDiscovery.h—DeviceDiscoveryModule: UDP broadcast on port 23452, up to 8 discovered peers shown as display controls, immediate broadcast onsetup(), self-packet filtering, configurablebroadcast_interval(1000–30000 ms)- Child of
NetworkModule— wired viasetInput("network"); uses parentdeviceName()in outgoing packets healthReport()format:devices=N port=23452— parseable bylive_suite.pytest4- 7 new unit tests in
tests/test_network.cpp(lifecycle, category, controls, healthReport, setInput, state round-trip, getControlValues) test4_device_discoveryadded todeploy/live_suite.py— waits up to 10 s fordevices >= 1on each devicedocs/modules/system/device-discovery-module.md— protocol, controls, platform support, setup example
Retrospective¶
- More moved than scoped. The original scope listed 7 PAL functions. What landed was a full system-info layer (18 functions), filesystem abstraction, FreeRTOS wrappers, and an IDF migration path — much more comprehensive than planned, and the right call.
- IDF migration path is now established. The
#elif defined(IDF_VER)branches in Pal.h give a concrete path to drop the Arduino framework. Most branches use the same underlying ESP-IDF API directly; onlycore_temp(),flash_speed_mhz(), andsketch_kb()return 0 on IDF (documented in Pal.h with follow-up notes). #ifdef ARDUINOcorrectly confined. The remaining guards are all structural (HttpServer/WsServertwo-class splits,main.cppentry-point, WiFi hardware modules) — none are in module logic.- LittleFS disk version mismatch resolved. PlatformIO's Python builder defaults to v2.1 (
0x00020001); the ESP32 Arduinolfs.hdefinesLFS_DISK_VERSION 0x00020000. Mount failure triggeredformat_on_fail=true, silently wiping all state (including WiFi credentials) on every boot. Fixed withboard_build.littlefs_version = 2.0inplatformio.iniand changedfs_begin()to try without format first, log explicitly before any format. - Deploy tooling hardened.
flashfs.pywas silently hanging (4-min flash withsubprocess.run(capture_output=True)); replaced with streamingPopen.flash.pyandflashfs.py"Writing at" filter changed fromstartswithtoin(catches\r-prefixed variants).devicelist.pynow capturesssid=,version=, andip=from serial output in a single probe window.summarise.pyenumerates all devices fromdevicelist.jsonregardless of which result files are present, with last-good timestamp for offline devices. AppVersion.h→1.3.0; version added to hello line.devicelist.pycan now verify firmware version from serial output without flashing.- Why projectMM? page added. PAL v1 completing the zero-
#ifdef ARDUINOgoal in modules was the right moment to articulate the full motivation: lineage from WLED → MoonLight, what each generation taught us, and what the clean module abstraction + test suite makes possible going forward. - Device discovery works immediately. Calling
broadcastPresence_()insetup()before settinglastBroadcast_means both nodes are visible to each other within milliseconds of boot — no need to wait one full broadcast interval. Thelive_suite.py10 s wait loop withtest4_device_discoveryproved this reliably on both PC and ESP32. - WiFiUDP needs a pool, not a single instance. Arduino's
WiFiUDPis a stateful object — you can't represent it as an integer handle. A fixed pool of 4 slots (pal::_detail::udp_slot(i)) lets PAL return an integer index while the realWiFiUDPobject lives in static storage. IDF and PC use integer file descriptors natively, so the abstraction is zero-cost on those platforms. - Part B scope matched reality. The "Open device UI link" from the original DoD was descoped — navigation requires the UI to render a link to another device's IP, which touches frontend logic outside the PAL + module scope. Replaced with a display control showing
name @ ip (version)for each discovered peer; the link can be added in a later sprint when the UI module-card design is revisited.
Sprint 2 — PSRAM + Dual-Core + Buffer Audit¶
Scope¶
Goal: move pixel buffers to PSRAM on S3, dispatch effects and drivers to separate cores, and audit the double-buffer model. Uses pal::psram_malloc from Sprint 1.
Part A — PSRAM allocation. Replace direct malloc() / new[] calls with pal::psram_malloc() in EffectsLayer and DriverLayer. On platforms without PSRAM the PAL falls back to malloc() automatically. Enables far larger grids on S3 (from ~10×10×10 to 64×64×16 or larger).
Part B — Dual-core dispatch. EffectsLayer modules run on Core 0; DriverLayer modules run on Core 1. Use xTaskCreatePinnedToCore (infrastructure already wired in R2 Sprint 4; not yet dispatched). Add channelsDFreeSemaphore or equivalent buffer-swap synchronisation.
Part C — Double-buffer audit. EffectsLayer currently maintains two pixel buffers. With dual-core dispatch and a proper semaphore, one buffer is sufficient — the driver always reads the last published frame. Simplify to single buffer; verify with a concurrency test (effects write while driver reads, no torn pixels over 1 000 ticks).
Definition of Done:
- On ESP32-S3-N16R8: pixel buffers confirmed in PSRAM via
GET /api/system✅ - Two separate FreeRTOS tasks with correct frame delivery under load ✅
EffectsLayersingle buffer; concurrency test passes 1 000 ticks without pixel corruption (Part C — deferred, see Retrospective)DriverLayerCore 1 timing logged every 10 s ✅
Result¶
| Metric | Value |
|---|---|
| Unit tests | 219/219 ✅ (7 new: 4 layer, 3 scheduler) |
| PC build | ✅ |
| ESP32 builds | ✅ (esp32dev + esp32s3_n16r8) |
| Live tests | 42/42 checks ✅ (PC + ESP32-S3-N16R8, both online) |
| PSRAM buffers | ✅ (verified from Sprint 1; pal::psram_malloc used in both EffectsLayer and DriverLayer) |
| Dual-core dispatch | ✅ (effectsTask on Core 0, driverTask on Core 1, binary semaphore) |
| DriverLayer healthReport | ✅ (buffer_kb=N psram=1 sources=N WxHxD) |
| Part C double-buffer audit | deferred — see Retrospective |
Part A (PSRAM verification):
DriverLayer::healthReport() now reports buffer_kb=N psram=1/0 sources=N WxHxD, making PSRAM use directly observable via GET /api/system and the health-check log. EffectsLayer and DriverLayer already used pal::psram_malloc from Sprint 1; no allocation code changed.
Part B (Dual-core dispatch):
Module::preferredCore()added (default 1 = App core): virtual override, no existing code changes needed.EffectsLayer::preferredCore()returns 0 (Net/WiFi core — compute-only, WiFi stack runs here too).DriverLayer::preferredCore()returns 1 (App core, the default): DriverLayer blending + output coordination + its children (GridLayout, PreviewModule) all run on Core 1 after receiving the frame semaphore.ModuleManageralways derives core frompreferredCore()on load; the"core"field written tomodulemanager.jsonis informational only and is not read back.- UI core badge added to each module card: C0 (blue) for Net/WiFi core, C1 (green) for App core. Children inherit their top-level parent's badge via
propagateCore()in the frontend. Scheduler::loopCore(core)runs only the modules assigned to that core and records their individual timings.Scheduler::tickPeriodic()handles the 20ms/1s/10s timed dispatch and health checks; called by the driver task each frame.Scheduler::loop()(PC path): callsloopCore(0)+loopCore(1)+tickPeriodic()sequentially — PC behavior unchanged.pal::sem_binary_create/give/take/deleteadded to Pal.h for FreeRTOS binary semaphore on ESP32; no-ops on PC.main.cpp(ESP32): old singleschedulerTaskreplaced byeffectsTask(Core 0, priority 1) anddriverTask(Core 1, priority 2), connected via binary semaphores_frameSem.
Part C deferred.
Retrospective¶
- PSRAM was already done. Sprint 1 delivered
pal::psram_mallocin both EffectsLayer and DriverLayer as part of the PAL work. Part A this sprint became a verification step: addinghealthReport()to DriverLayer so PSRAM use is observable via REST rather than only via serial logs. This is the right outcome — the architecture made the feature land earlier than planned. preferredCore()always wins. Rather than hardcoding core assignments in a global map, each module class declares its own preference andModuleManageralways uses it. The"core"field inmodulemanager.jsonis written for observability (UI badge, REST API) but is never read back. This avoids stale saved values silently overriding the correct type-level affinity.- Semaphore timeout keeps the driver watchdog safe.
sem_take(s_frameSem, 100)uses a 100 ms timeout rather thanportMAX_DELAY. If Core 0 stalls, Core 1 still yields periodically and the task watchdog does not fire. The downside is one extra WDT-safe yield per stall — acceptable for a 20 ms frame budget. - Part C (double-buffer audit) deferred. The EffectsLayer double buffer is safe under the current semaphore model: Core 0 writes to the inactive buffer and publishes atomically; Core 1 only reads after the semaphore is given. Reducing to a single buffer would require a more careful concurrency analysis and a hardware stress test. The double-buffer cost is one extra pixel array in PSRAM per EffectsLayer — cheap on S3-N16R8. Deferred to Sprint 3 or as a standalone sprint when the S3 PSRAM budget is more constrained.
- Core 0 priority 1, Core 1 priority 2. The driver task has higher FreeRTOS priority so it preempts Core 1 as soon as the semaphore is given, minimising latency from blend-complete to LED output. The effects task at priority 1 runs at the natural frame rate set by WiFi stack + compute time; it does not need to be faster than the driver can consume.
Sprint 3 — Hotpath Tuning + fps/ms Accuracy¶
Scope¶
Goal: measure the true per-frame cost with cycle-accurate timing and fix the fps/ms display.
Part A — ESP cycle counters. Replace micros() in the hot path with esp_cpu_get_cycle_count() (cycle-accurate; no OS call overhead). Wrap as pal::micros_cycles() — add this to src/pal/ as a Sprint 1 follow-on. Falls back to std::chrono on PC.
Part B — Accurate per-stage fps/ms display. Show effect_ms, driver_ms, composite_ms, and idle_ms separately. Frontend toggle cycles through these. GET /api/system exposes all four fields per module.
Part C — Hotpath profiling pass. Profile on ESP32-S3-N16R8 at 16×16×1 and at the PSRAM-enabled grid limit. Identify and fix the top bottleneck. No-allocations assertion: heap delta = 0 over 1 000 ticks.
Definition of Done:
GET /api/systemreturnseffect_ms,driver_ms,composite_msper module ✅ (delivered asfps,avg_ms,min_ms,max_msper module across all three API paths)- Frontend shows correct per-stage value, not the scheduler aggregate ✅ (4-mode toggle: fps / avg / min / max; root = wall-clock, children = self-only)
- Hotpath profile documented; top bottleneck addressed or noted as "no bottleneck found" with data ✅ (no bottleneck found at current scale; heap-delta assertion deferred)
Result¶
| Metric | Value |
|---|---|
| Unit tests | 222/222 ✅ (3 new: timing accumulation, parent-excludes-children, reset-on-runSetup) |
| PC build | ✅ |
| ESP32 builds | ✅ (both envs) |
| Per-module timing in API | ✅ (fps, avg_ms, min_ms, max_ms in all three serialization paths) |
| Frontend toggle | ✅ cycles fps / avg / min / max; persists choice via localStorage |
| Root timing = self + children | ✅ Scheduler wall-clock for roots; StatefulModule::timing_ self-only for children |
EffectsLayer timing |
✅ added to its runLoop() override (bypasses StatefulModule::runLoop()) |
| Part A (cycle counters) | deferred — pal::micros() at microsecond resolution is sufficient for ~20 ms frames |
| Part C (heap-delta assertion) | deferred — no-allocation invariant is enforced by code review; loop profiling noted |
Live server readout (PC, 16x16x1 pipeline):
effects1 | 0.0004 ms avg (root: wall-clock = own + children)
ripples1 | 0.000324 ms avg (child: self-only)
driver1 | 0.000371 ms avg (root: wall-clock)
grid1 | 1.9e-05 ms avg (child: self-only)
preview1 | 5.6e-05 ms avg (root: wall-clock)
Parent effects1 (0.0004 ms) = own overhead + ripples1 (0.000324 ms), confirming the parent-is-sum-of-children invariant.
Retrospective¶
- Parent timing = self + children invariant verified. Scheduler wall-clock for root modules naturally captures self + all children.
StatefulModule::timing_records only ownloop()duration. The frontend toggle shows each module's individual contribution at a glance. EffectsLayerhas its ownrunLoop()override. It is the only module insrc/modules/that overridesrunLoop()(needed to callpublish()after all children write pixels). The timing block had to be added there explicitly — StatefulModule's version was not called for EffectsLayer.- Three serialization paths, not one.
ModuleManagerhas three separate JSON serialization functions (getStateJsonfor WS,getModulesJsonfor/api/modules,getSystemModulesJsonfor/api/system). All three needed the timing fields added. A follow-up refactor could extract a sharedserializeTimingInto(obj, t)helper to keep them in sync. - Child wiring is a runtime concern. In tests,
addChild()must be called manually beforerunSetup()— the constructor does not register children. This matches production behavior (ModuleManager wires children at runtime), but it is easy to forget in new tests. Adding a test helper or a comment in the base class would reduce the risk. pal::micros()precision is sufficient. The frame budget is ~20 ms;pal::micros()(1 µs resolution) gives three orders of magnitude more precision than needed. ESP cycle counters (esp_cpu_get_cycle_count()) would save one OS call perloop()invocation, but the cost is immeasurable at this scale. Deferred to a future sprint if profiling reveals it is actually hot.- Frontend toggle now cycles four modes (fps / avg / min / max). The previous two-mode toggle was the right MVP; adding min/max makes worst-case jitter visible without adding separate UI elements.
preferredCore()always wins (carry-over from Sprint 2). Stale"core":1inmodulemanager.jsonwas causing all modules to show C1 badges. The fix (line 200 of ModuleManager always usesm->preferredCore()) is now established policy: the JSON field is written for observability but never read back.
Sprint 4 — Advanced Module-Creation UI¶
Scope¶
Goal: replace the flat "add module" type list with a context-aware, searchable, tagged picker.
Part A — Context filter. When adding a child to an EffectsLayer, the list shows only effect modules. When adding at the root level, only root-capable modules are shown. Module::tags() (already on the base class) is the mechanism.
Part B — Tag filter chips. A row of chips derived from all available tags(). Clicking narrows the list; multiple chips AND together.
Part C — Search field. Text input filters by name() substring (case-insensitive, no button required). Composes with tag chips.
Part D — tags() on concrete modules. Implement emoji tags on all effect/modifier/layout/driver modules using the MoonLight emoji key:
| Emoji | Meaning |
|---|---|
| 🔥 | Effect |
| 🚥 | Layout |
| 💡 | 0D+ |
| 💫 | MoonLight origin |
Tags for current modules: SineEffect 🔥🧊💫, LinesEffect 🔥🧊💫, RipplesEffect 🔥🟦💫, BrightnessModifier 💎💡, GridLayout 🚥💡, PreviewModule ☸️💡.
Definition of Done:
- Child picker for
EffectsLayershows only effect/modifier modules; root picker shows only root-capable modules ✅ - Emoji chips collect all unique emoji from filtered types; clicking a chip ANDs into the filter ✅
- Search + chips compose correctly ✅
- All domain modules have at least one emoji tag ✅
Result¶
| Metric | Value |
|---|---|
| Unit tests | 222/222 ✅ (test updated: /api/types now asserts object shape + metadata fields) |
| PC build | ✅ |
| ESP32 builds | ✅ (esp32dev + esp32s3_n16r8) |
/api/types format |
✅ returns [{name, category, tags, allowedChildCategories}] objects |
| Context filter | ✅ root picker excludes effect/modifier/layout; EffectsLayer picker shows only effect/modifier; DriverLayer shows only layout/driver |
| Emoji chip row | ✅ unique emoji collected from filtered types; chips toggle; AND composition |
| Search field | ✅ case-insensitive name substring; composes with chips |
| Double-click to create | ✅ double-clicking a type row creates immediately |
| All domain modules tagged | ✅ SineEffect 🔥🧊💫, LinesEffect 🔥🧊💫, RipplesEffect 🔥🟦💫, BrightnessModifier 💎💡, GridLayout 🚥💡, PreviewModule ☸️💡 |
GET /api/types sample output:
[
{"name": "EffectsLayer", "category": "layer", "tags": "", "allowedChildCategories": "effect modifier"},
{"name": "SineEffectModule","category": "effect", "tags": "🔥🧊💫", "allowedChildCategories": ""},
{"name": "GridLayout", "category": "layout", "tags": "🚥💡", "allowedChildCategories": ""},
{"name": "DriverLayer", "category": "layer", "tags": "", "allowedChildCategories": "layout driver"}
]
Retrospective¶
allowedChildCategories()on the module, not a lookup table. Each module declares what children it accepts via a virtual method. The frontend looks up the parent's type inknownTypesat picker open time — no hardcoded type names in JS, no separate mapping to maintain. Adding a new container module just requires overriding one method.tags()moved toModule.h, notStatefulModule.h. It belongs at the base since any module can have tags.StatefulModule.hhad a duplicatevirtual tags()from a prior sprint; removing it withoverrideon all concrete overrides caught by the compiler warning.TypeRegistry::getTypesDetailedinstantiates temporarily. All modules are default-constructible; the factory lambda creates a fresh instance, queriescategory(),tags(),allowedChildCategories(), then deletes it. No change toREGISTER_MODULEmacro. The cost (one alloc/free per type at/api/typescall time) is negligible; this endpoint is called once on page load.- Emoji chips appear only when the filtered context has tagged types. Root-level picker shows network/system/layer modules, most of which have no tags yet — chip row is empty for root. EffectsLayer picker shows 🔥/🧊/🟦/💫 chips from the three effects, which is immediately useful as more effects are added in Sprint 5.
- Double-click to create. A usability detail: double-clicking a row in the list creates immediately without requiring a separate "create" button press.
SineEffectModuleregistered name vs. class name. Thename()method returns"SineEffect"but the type is registered as"SineEffectModule"inREGISTER_MODULE. The picker shows the registered name (fromgetTypesDetailed). This inconsistency exists for historical reasons; a future sprint could align them.
Sprint 5 — 2D Effects: Game of Life, Noise, Distortion¶
Scope¶
Goal: three more 2D effects — the first ported from MoonLight, two new.
Part A — GameOfLifeEffect. Conway's rules; one generation per tick. Controls: seed, wraparound, color. Resets when the grid dies or goes static. Port from MoonLight. Test: glider pattern propagates diagonally over 4 generations.
Part B — NoiseEffect2D. Smooth animated noise on the XY plane using a hash-based value noise (bilinear interpolation + smoothstep). No FastLED or external library; works on all platforms. Controls: scale, speed.
Part C — DistortionWaves2DEffect. WLED-ported interfering sine waves on both axes. Controls: freq_x, freq_y, speed.
Definition of Done:
- [x] Three effects registered; each runs correctly on a panel (of any size)
- [x]
GameOfLifeEffectglider test passes (Gen4 = {(2,1),(3,2),(1,3),(2,3),(3,3)}, count=5) - [x]
NoiseEffect2DandDistortionWaves2DEffectproduce spatially varied, non-zero output - [x] All 229 unit tests pass
- [x] Doc pages added for all three effects
Result¶
| Metric | Value |
|---|---|
| Unit tests | 229/229 pass (+7 new) |
| New test cases | test_effects_2d.cpp: glider, extinction re-seed, non-zero render, spatial variation |
| New effects | GameOfLifeEffect, NoiseEffect2D, DistortionWaves2DEffect |
| Tags | 🔥🟦💫 / 🔥🟦 / 🔥🟦🐙 |
| Platform independence | NoiseEffect2D uses inline hash noise (no FastLED), works on PC and ESP32 |
New modules registered: GameOfLifeEffect, NoiseEffect2D, DistortionWaves2DEffect
Retrospective¶
inoise8(FastLED) was replaced with a hash-based trilinear value noise. The result is platform-independent, embeds in ~20 lines, and produces visually equivalent smooth animated fields. All future effects that need noise should use this pattern rather than importing FastLED.GameOfLifeEffectadds publicsetPattern/getCell/liveCount/stepGenerationhelpers to allow deterministic testing. This pattern (test-helper methods on the module itself, not in a separate test fixture) works well for cellular-automaton style effects.pal::psram_mallocinsetup()/pal::psram_freeinteardown()is the confirmed pattern for heap-sized per-module buffers. Falls back tomalloc()on PC with no code change.- Sprint 4's emoji picker context filter immediately surfaced the new 2D effects under EffectsLayer with 🔥🟦 tags, validating the tagging infrastructure from Sprint 4.
Sprint 6 — Art-Net In and Out¶
Scope¶
Goal: first real input and output protocol — Art-Net over UDP, working on PC and ESP32 with no GPIO pins required. Uses pal::udp_send from Sprint 1. Having both directions means two projectMM devices can be wired together for end-to-end live testing.
Part A — ArtNetOutModule. Sends DMX-512 frames (512 channels per universe) via Art-Net UDP. Controls: universe, ip (broadcast or unicast), enabled. Maps DriverLayer pixel data to DMX channels (RGB x pixels). Postpone pin-based LED drivers (WS2812, SK6812) to a future release — PC has no pins.
Part B — ArtNetInModule. Receives Art-Net DMX frames and writes pixel data into an EffectsLayer buffer — it is an effect source, category "effect", wired like SineEffect or RipplesEffect. Controls: universe, enabled. The receiving pipeline is: ArtNetInModule → EffectsLayer → DriverLayer → PreviewModule. Complementary to ArtNetOutModule — together they form a complete send/receive pair.
Part C — Two-device projectMM live test. Use two devices from the lab rack (e.g. PC and MM-70BC) to verify the full Art-Net pipeline end to end:
- Device A (sender): EffectsLayer + SineEffect → DriverLayer →
ArtNetOutModule - Device B (receiver):
ArtNetInModule→ DriverLayer → PreviewModule - Verify that Device B's preview checksum matches Device A's output frame checksum at 40 Hz over 60 s
- Document frame delivery rate and latency in the sprint result section
Definition of Done:
- [x]
ArtNetOutModulesends correctly formed Art-Net DMX Data packets - [x]
ArtNetInModulereceives and decodes Art-Net DMX Data packets into an EffectsLayer - [x] Multi-universe: automatic universe count based on pixel count, 170 pixels per universe
- [x]
pal::udp_sendadded to all three PAL implementations (Arduino, IDF stub, PC POSIX) - [x] PC loopback live test: ArtNetOut → broadcast → ArtNetIn on same device; packets_rx > 0
- [x] Two-device live test: PC sender (ArtNetOut broadcast) → ESP32 MM-70BC receiver (ArtNetIn); packets_rx > 0 confirmed on hardware
- [x] 236/236 unit tests pass (+7 new Art-Net tests)
- [x] Test complexity classification (
smoke/format/behavioral/integration) added to test-results.md and deploy-summary.md; documented in standards.md - [x] Doc pages with Test Coverage sections for
ArtNetInModuleandArtNetOutModule
Result¶
| Metric | Value |
|---|---|
| Unit tests | 236/236 pass (+7 new) |
| New modules | ArtNetOutModule (driver), ArtNetInModule (effect) |
| PAL addition | pal::udp_send(ip, port, buf, len) — three-way implementation |
| PC loopback | packets_tx=32239, packets_rx=5065 (~3 s settle) |
| Two-device | PC packets_tx=46722, ESP32 MM-70BC packets_rx=238 (broadcast UDP over LAN) |
| Multi-universe | Automatic: 170 pixels per universe; 182-pixel panel uses 2 universes |
| Live tests | PC 6/6 pass; ESP32 MM-70BC 6/6 pass; two-device Art-Net 4/4 pass |
Two-device test sample output:
PC (http://localhost:80) → ArtNetOut broadcast → ESP32 (http://192.168.8.156:80) ArtNetIn
PC: add ArtNetOutModule [ok]
ESP32: add ArtNetInModule [ok]
PC ArtNetOut sent packets [packets_tx=46722]
ESP32 ArtNetIn received packets [packets_rx=238]
Retrospective¶
ArtNetInModuleis category"effect"and wires exactly likeSineEffect— it produces pixels into anEffectsLayer. This keeps the pipeline model clean: inputs are always effects/modifiers, outputs are always drivers.pal::udp_send(unicast + broadcast) fills the gap left bypal::udp_broadcast(broadcast-only). The PC implementation setsSO_BROADCASTunconditionally, which allows the same call to serve both broadcast and unicast destinations.- UDP broadcast loopback does not work on ESP32 WiFi: the device does not deliver its own broadcast to bound sockets. test5 (Art-Net loopback) is now skipped for non-PC platforms; the two-device cross-test in
livetest.pycovers the receive path on hardware. StatefulModulealready registersenabled_after every module'ssetup(). Both Art-Net modules initially registered a duplicate control — removed in this sprint.livetest.pywas restructured to start the PC server once and run all ESP32 tests while it is up. This allows device discovery (test4) to find a live peer on both sides simultaneously.- The two-device test delivery ratio (238/46722 = 0.5%) reflects normal UDP broadcast behaviour: the ESP32 receives roughly one packet per ArtOut loop tick rather than every packet sent (the sender runs much faster than the WiFi receive rate).
packets_rx > 0is the correct assertion. - Running ArtNetOut + ArtNetIn on the same ESP32 at frame rate destabilises the Arduino WiFi stack (UDP interrupt overload). test5 is skipped on non-PC; the two-device test covers the ESP32 receive path.
- Test complexity badges (
smoke/format/behavioral/integration) were added totest-results.mdanddeploy-summary.mdthis sprint, classified by keyword matching inunittest.py. The breakdown (11 smoke, 33 format, 164 behavioral, 28 integration) confirms the majority of tests exercise real behavior.
Sprint 7 — Optional System Modules¶
Scope¶
Goal: system-utility modules loadable on demand; none are in the default pipeline. Part A (OTA) deferred to Release 5 — it requires binary HTTP upload, OTA partition management, and hardware-in-the-loop CI that are not yet in place. Parts B and C delivered here.
Part A — FirmwareUpdateModule. Deferred (see Release 5 Sprint 1).
Part B — TasksModule. Lists FreeRTOS tasks (name, core, priority, stack watermark) as read-only display controls. Useful for diagnosing CPU affinity after Sprint 2 dual-core changes.
Part C — FileManagerModule. Browse and delete files in the state directory on LittleFS (ESP32) or state/ (PC). Lists filenames with sizes; delete by typing a name and pressing the delete button.
PAL additions (src/pal/Pal.h): state_dir(), task_list(), fs_list(), fs_remove() — three-way platform implementations (Arduino / IDF stub / PC std::filesystem).
Definition of Done:
- [x]
TasksModuleandFileManagerModuleloadable via the module picker - [x]
pal::task_list/pal::fs_list/pal::fs_remove/pal::state_diradded to PAL - [x] Tests in
tests/test_system_utils.cpp— 10 cases added (246 total, 246 pass) - [x] Doc pages:
docs/modules/system/tasks-module.md,docs/modules/system/file-manager-module.md - [x]
mkdocs.ymlupdated - [ ]
FirmwareUpdateModule— deferred to Release 5
Result¶
Build: PC CMake build passes.
Tests: 246/246 pass (10 new cases for TasksModule and FileManagerModule).
Test classification breakdown for new tests:
| Level | Count |
|---|---|
smoke |
2 (lifecycle) |
format |
4 (healthReport format, schema structure) |
behavioral |
4 (task_count non-negative, delete outcomes, file deletion) |
Footprint:
| Module | classSize |
|---|---|
TasksModule |
~1064 B (1024 B task list buffer) |
FileManagerModule |
~2244 B (2048 B file list + 128 B filename + 64 B result) |
Retrospective¶
OTA deferred deliberately: it needs binary HTTP upload (not yet implemented), Update.h which requires real hardware, and a CI path that can verify OTA without bricking a device. These prerequisites will be addressed step by step in Release 5 (see Sprint 1 of Release 5).
PAL additions followed the established three-way pattern. The IDF section is a stub (empty string / false) matching the existing convention for unimplemented IDF features.
std::filesystem is available in C++17 on PC, so the PC path is full-featured. The #include <filesystem> was added to the PC section of Pal.h.
FileManagerModule uses EditStr ("text" UI type) for the filename input — the only module so far to use an editable string control. The delete button is marked sensitive=true to exclude it from WebSocket broadcasts.
Sprint 8 — Logging Refactor + Community Branding¶
Scope¶
Part A — Logging refactor. Push log messages to WebSocket clients in real time instead of storing a ring buffer.
Approach: a g_logWsPushFn function pointer (null by default) is declared in Logger.h and defined in Logger.cpp. Every LOG_* macro formats a 128-byte stack buffer, calls printf, then calls the push fn if set. In main.cpp, after ws.begin(), g_logWsPushFn is wired to wsPushLog() which wraps the message as {"t":"log","m":"<msg>"} and calls ws.broadcastText(). The push fn is shared between embedded and PC paths via a file-scope static defined before the #if guard. Memory cost: zero persistent allocation on the device; the browser accumulates up to 100 lines.
Additional changes:
- GET /api/log returns the current log level as {"level":"setup"}.
- POST /api/log body {"level":"debug"} sets the level at runtime.
- Bare printf() calls in DeviceDiscovery.h (setup, teardown, device-found) replaced with LOG_SETUP().
- logLevelToString() helper added to Logger.h for the REST endpoint.
Part B — MoonModules community branding. Four links (Discord, Reddit, YouTube, GitHub) added to the side-nav footer as small pill buttons below the + add module button.
Part C — Log panel in the frontend. A collapsible log panel added below the health panel. wsConn.onmessage checks for msg.t === 'log' before routing to handleStateUpdate; log messages go to appendLogLine() which appends a <div> to the log output, trims to 100 lines, and updates the count badge. A clear button empties the panel.
Part D — CodeRabbit. .coderabbit.yaml added at repo root with per-path review instructions (hot-path constraints, behavioral test requirement, doc style, PAL three-way guard). docs/developer-guide/standards.md updated with AI attribution table and CodeRabbit configuration notes.
Definition of Done:
GET /api/logandPOST /api/logreturn correct responsesg_logWsPushFnwired in both embedded and PC paths; messages reach the log panel without any persistent ring buffer- 5 new logger tests: level parse, level round-trip, gate suppression, WS push newline strip, null-fn safety
- Community links visible in side-nav footer
.coderabbit.yamlpresent; standards.md updated
Result¶
252/252 unit tests pass (was 246 before Sprint 8; +6 new logger test assertions).
| File added or changed | What changed |
|---|---|
src/core/Logger.h |
Added g_logWsPushFn, logLevelToString(), _LOG_EMIT helper macro; all LOG_ macros now call both printf and push fn |
src/core/Logger.cpp |
Added g_logWsPushFn definition |
src/main.cpp |
wsPushLog + s_wsLog added at file scope (shared by both #if sections); g_logWsPushFn = wsPushLog after ws.begin(); GET /api/log + POST /api/log in both route functions |
src/modules/system/DeviceDiscovery.h |
4 bare printf calls replaced with LOG_SETUP; #include "core/Logger.h" added |
src/frontend/index.html |
Log panel CSS + HTML; wsConn.onmessage log routing; appendLogLine(); community links in nav-footer |
.coderabbit.yaml |
New; per-path review instructions |
docs/developer-guide/standards.md |
AI attribution table + CodeRabbit section |
tests/test_logger.cpp |
New; 5 behavioral tests for logger level gating and WS push |
Memory footprint: zero new persistent allocation. Each LOG_* call uses 128 bytes on the stack (only when the level is enabled) vs. the rejected 3 KB ring buffer.
Retrospective¶
WS push vs. ring buffer. The initial scope proposed a ring buffer, but at 3 KB it was too large for ESP32-dev SRAM. WebSocket push has zero device-side memory cost, which is strictly better. The only trade-off is that log lines produced before the browser connects are lost; for diagnostic use during development that is acceptable. A boot-time log captured by the browser from first connection is sufficient.
Bare printf audit. Only DeviceDiscovery.h was in scope. WifiSta.h and SystemStatus.h also have bare printf calls that should be migrated; deferred to Sprint 9 revisits.
CodeRabbit manual step. The GitHub App must be installed by the repo owner; this cannot be automated. Noted in standards.md.
Seeds Sprint 9: bare printf in WifiSta.h and SystemStatus.h; revisit whether logLevelToString should be used in health reports too.
Sprint 9 — Revisits¶
Scope¶
Goal: four open design questions, each with a documented decision.
Revisit A — Default-value icon. The ↺ icon (R3 Sprint 2) shows drift from default. Verify it sources defaults from Module::setup() compile-time values rather than modulemanager.json initial values. Fix if they differ.
Revisit B — loop vs runLoop naming. Audit consistency of loop / runLoop / setup / runSetup across Module, StatefulModule, and Scheduler. Rename if a clearer convention exists.
Revisit C — ProducerModule / ConsumerModule for core affinity. Revisit whether lean marker bases are cleaner than the core field in modulemanager.json for routing to FreeRTOS core assignments — especially now that dual-core is live (Sprint 2). Document the decision; implement if marker bases win.
Revisit D — Auto-setup guard. Confirm the !hasDriver && !hasEffects auto-setup (R3 Sprint 4) still fires correctly after Sprint 2 dual-core changes and any module renames during development.
Definition of Done:
- Each revisit has a written decision in
docs/developer-guide/architecture.md - Any code changes have tests and pass CI
Result¶
254/254 unit tests pass (was 252 before Sprint 9; +2 new behavioral defVal tests).
| Area | Action taken |
|---|---|
Revisit A: defVal sourcing |
Confirmed correct; 2 new behavioral tests added to test_stateful_module.cpp (R4S9/A); decision documented in architecture.md#defval-sourcing |
Revisit B: loop vs runLoop naming |
Convention confirmed consistent; decision documented in architecture.md#loop-vs-runloop |
| Revisit C: ProducerModule/ConsumerModule | Decision reconfirmed; updated architecture.md#producermodule-consumermodule with R5S5 re-open condition |
| Revisit D: auto-setup guard | Guard verified unchanged and correct; tests from Sprint 4 confirmed still passing; decision documented in architecture.md#auto-setup-guard |
| Sprint 8 carry-over (printf audit) | 4 bare printf in WifiSta.h + 1 in SystemStatus.h replaced with LOG_SETUP; logging audit now complete across all system modules |
| Periodic logging removed | scheduler.printTimings() / printSizes() 10 s periodic calls removed from driverTask (embedded) and main() (PC); REST /api/system provides the same data on demand; one-shot end-of-run prints kept for CI --count mode |
| Logo resized | docs/assets/moonlight-logo.png (320x320, ~31 KB base64) downsampled to 20x20 via Lanczos; embedded base64 shrunk from ~31,425 to 1,564 chars (~95%); frontend bundle reduced from ~70 KB to ~40 KB |
display control newlines |
white-space: pre-wrap added to .display-value CSS; multi-line displays (TasksModule task list, FileManagerModule file list) now render one item per line |
Retrospective¶
All four revisits closed with documents, not code changes. Each one confirmed the existing design was correct. This is the intended outcome of a revisits sprint: the retrospective decisions become a durable record in architecture.md that future contributors (human or AI) can rely on without re-deriving the reasoning.
Logging audit complete. Sprint 8 started the migration from bare printf to LOG_* macros in system modules. Sprint 9 finished it: WifiSta.h and SystemStatus.h now use LOG_SETUP, meaning all log output in system modules is level-gated and WS-pushable.
defVal test gaps found the right invariant. Writing the Revisit A tests required checking the load order carefully (addControl() in setup(), then loadState() called separately). The tests confirmed the invariant: defVal is immutable after setup(). The two new tests lock this property against future refactoring.
Seeds Release 5: ProducerModule/ConsumerModule may need revisiting when the library packaging sprint (R5S5) defines the boundary between core runtime and light-domain adapter. The re-open condition is documented in architecture.md.
Sprint 10 — FastLED-MM Bootstrap + main.cpp Refactor¶
Scope¶
Goal: create the ewowi/FastLED-MM repository as the first real consumer of projectMM as a library, and refactor src/main.cpp so the boilerplate moves to a reusable helper — making FastLED-MM's entry point minimal.
Part A — FastLED-MM repository (done 2026-04-22)¶
New public GitHub repo ewowi/FastLED-MM. A PlatformIO project (and library) that imports projectMM and FastLED as lib_deps. The architecture uses a shared CRGB flm_leds[] array as the pixel handoff between an effect module and a driver module (no EffectsLayer or DriverLayer involved).
Files created:
| File | Purpose |
|---|---|
src/FlmConfig.h |
Pin + grid size constants (the only file a user edits for their hardware) |
src/FlmPixels.h |
extern CRGB flm_leds[]; flm_idx(x,y) grid helper |
src/WaveRainbow2D.h |
WaveRainbow2DEffect: 2D diagonal rainbow with sin8() ripple, speed + hue_offset controls |
src/FastLEDDriver.h |
FastLEDDriverModule: calls FastLED.show() on Core 1; brightness control |
src/main.cpp |
Scheduler + ModuleManager + HTTP/WS server; dual-core FreeRTOS tasks |
platformio.ini |
esp32dev + esp32s3_n16r8 envs; lib_deps: projectMM + FastLED |
library.json |
PlatformIO library descriptor; excludes main.cpp from lib build |
README.md |
Quick start, effect authoring guide, architecture diagram, roadmap |
Design decision: shared CRGB array instead of EffectsLayer/DriverLayer.
FastLED-MM bypasses projectMM's pixel pipeline (EffectsLayer → Channel → DriverLayer) entirely. The effect writes to flm_leds[] directly; the driver calls FastLED.show(flm_leds, ...). This is the pattern FastLED builders already know. The ProducerModule/ConsumerModule base classes (tracked in R5S5 and below) will formalise this contract without adding mandatory framework dependency.
Part B — main.cpp refactor in projectMM¶
Move the HTTP/WS setup, route registration, and FreeRTOS task creation out of src/main.cpp into reusable files so a library consumer's main.cpp can be minimal:
src/core/AppSetup.h/AppSetup.cpp:pal::embeddedSetup(mm, scheduler, server, ws)helper that wires routes, starts servers, creates tasks. Takes the four objects by reference; no globals inside.src/core/AppRoutes.h:registerCoreRoutes(mm, server)covering/api/modules,/api/control,/api/system,/api/log,/api/types.src/main.cpp(after refactor): only globals + calls to the above helpers +setup()/loop().
After Part B, FastLED-MM's main.cpp shrinks to:
#include "core/AppSetup.h"
#include "FlmPixels.h"
#include "WaveRainbow2D.h"
#include "FastLEDDriver.h"
CRGB flm_leds[FLM_NUM_LEDS];
REGISTER_MODULE(WaveRainbow2DEffect)
REGISTER_MODULE(FastLEDDriverModule)
void setup() { pal::embeddedSetup(mm, scheduler, server, ws); }
void loop() { pal::suspend_forever(); }
Backlog seeded by this sprint¶
| Item | Assign to |
|---|---|
ProducerModule / ConsumerModule lean base classes in src/core/ |
R4S11 |
| FastLED-MM: add WiFi STA credentials via web UI (currently AP-only) | FastLED-MM repo, after R5S5 |
Audio module: StatefulModule that reads microphone → publishes to KvStore; assignable to its own task |
R5 backlog |
FastLED Channels API support in FastLEDDriver (multi-strip, per-strip GPIO) |
FastLED-MM repo, future sprint |
Revise R05S05 FastLEDEffect<N> adapter to use shared-array pattern instead of EffectsLayer copy |
Done (R5S5 scope revised, now R4S11) |
Result¶
Part A complete 2026-04-22. Part B complete 2026-04-22.
254/254 unit tests pass (unchanged count; refactor added no new tests and broke none).
Part B file delta:
| File | Change | Lines |
|---|---|---|
src/main.cpp |
Rewritten (routes + tasks extracted) | 497 → 151 |
src/core/AppRoutes.h |
New: registerCoreRoutes() declaration |
7 |
src/core/AppRoutes.cpp |
New: unified route registration for both platforms | 122 |
src/core/AppSetup.h |
New: pal::embeddedSetup() declaration |
18 |
src/core/AppSetup.cpp |
New: task functions + pal::embeddedSetup() implementation |
120 |
CMakeLists.txt |
Added AppRoutes.cpp to projectMM executable sources |
+1 |
The previous registerRoutes() duplication (identical function defined twice, once per #if section, ~135 lines each) is replaced by a single registerCoreRoutes(). main.cpp dropped from 497 to 151 lines (-70%).
Build and run:
| Target | Status |
|---|---|
| PC CMake build | PASS |
| ESP32-S3 build | PASS (Flash 28.8%, RAM 14.7%) |
| 254 unit tests | PASS |
deploy/run.py --count 5 PC |
PASS |
deploy/run.py --count 5 ESP32-S3 MM-70BC |
PASS |
Definition of Done¶
src/main.cpp(embedded path) is under 60 lines of application-specific coderegisterCoreRoutes()is the single source of truth for all REST routes: no duplication between embedded and PC pathspal::embeddedSetup()accepted by PlatformIO; FreeRTOS tasks start and device is reachable on WiFi- All 254 existing tests pass
- PC and ESP32-S3 builds succeed
Retrospective¶
registerRoutes() duplication was the biggest pain point. The two copies had drifted slightly (PC /api/test read from file; embedded did not). Unifying them with the file-reading variant works on both platforms because fopen on embedded returns null for a path that does not exist in LittleFS, falling through to live health. Zero branch needed.
AppSetup.cpp keeps all FreeRTOS complexity out of main.cpp. The task functions (effectsTask, driverTask), the binary semaphore, and ensureNetworkModules() all moved there. A FastLED-MM main.cpp now needs only four globals, two REGISTER_MODULE calls, and setup() / loop() calling pal::embeddedSetup().
wsPushLog stays in main.cpp by design. It captures s_wsLog which points to the WsServer global. Both platforms need it; moving it to AppSetup would require duplicating it on PC or introducing a PC-specific path in AppSetup. The tradeoff favors keeping 12 lines of glue in main.cpp over a cross-platform abstraction for a minor helper.
Seeds Sprint 11: pal::embeddedSetup() is now callable from FastLED-MM. Sprint 11 (ProducerModule/ConsumerModule) adds the typed pixel-buffer contract that formalises the shared-array pattern used by FastLED-MM today.
Sprint 11 — projectMM as a Domain-Independent Library¶
Moved from Release 5 Sprint 5. Pulled forward after R4S10 created the FastLED-MM bootstrap; the library separation is now the immediate next step.
Scope¶
Goal: make projectMM a clean, domain-independent library that downstream projects can import without pulling in LED-specific logic. The primary use case is FastLED-MM, which already exists (R4S10) but currently imports the full projectMM including EffectsLayer, DriverLayer, and all built-in effects.
Revised design (after R4S10 experience):
A FastLED consumer does not want or need EffectsLayer, DriverLayer, or the Channel pixel buffer. projectMM should not impose its internal pipeline on a domain that already has its own. The right abstraction:
ProducerModule: thin base class insrc/core/that owns or references a pixel buffer. Subclass it to define what "a frame of pixel data" means in your domain.ConsumerModule: thin base class insrc/core/that reads from aProducerModule.setInput("producer", &p)wires them.- EffectsLayer / DriverLayer remain in
src/modules/layers/as the built-in LED pipeline. They become concrete subclasses ofProducerModuleandConsumerModule(not required by the core).
Complexity estimate: Medium (3-5 days). Parts A+B are mechanical; Part C (embeddedSetup) is the highest-risk piece requiring PC + ESP32 validation.
Part A — ProducerModule and ConsumerModule base classes.
Add to src/core/:
// src/core/ProducerModule.h
class ProducerModule : public StatefulModule {
public:
void declareBuffer(void* buf, size_t len, size_t elemSize);
void* bufferPtr() const { return buf_; }
size_t bufferLen() const { return len_; }
private:
void* buf_ = nullptr;
size_t len_ = 0;
};
// src/core/ConsumerModule.h
class ConsumerModule : public StatefulModule {
public:
void setInput(const char* key, Module* m) override {
if (strcmp(key, "producer") == 0)
producer_ = static_cast<ProducerModule*>(m);
}
protected:
ProducerModule* producer_ = nullptr;
};
EffectsLayer and DriverLayer updated to subclass these. No other module logic changes.
Part B — Replaceable module registrations.
Split src/modules/ModuleRegistrations.cpp:
src/core/CoreRegistrations.cpp: infrastructure-only (NetworkModule,SystemStatusModule,DeviceDiscoveryModule). Always compiled into the library.src/modules/ModuleRegistrations.cpp: LED domain modules. Excluded from the library build vialibrary.jsonsrcFilter. Consumers provide their own.
Part C — embeddedSetup() helper (prerequisite: R4S10 Part B).
pal::embeddedSetup(mm, scheduler, server, ws) wires routes, starts servers, creates FreeRTOS tasks. After this, FastLED-MM's main.cpp shrinks to ~10 lines. Validate on PC + ESP32-S3.
Part D — library.json and CMakeLists.txt updates.
"build": {
"srcDir": "src",
"srcFilter": ["+<**/*>", "-<main.cpp>", "-<modules/ModuleRegistrations.cpp>"]
}
Add projectMMLib CMake target excluding the same files.
Part E — FastLED-MM integration test.
Update FastLED-MM to use ProducerModule / ConsumerModule:
WaveRainbow2DEffectextendsProducerModule; callsdeclareBuffer(flm_leds, FLM_NUM_LEDS, sizeof(CRGB))in setup.FastLEDDriverModuleextendsConsumerModule.- Verify FastLED-MM builds cleanly after the projectMM changes.
Part F — Documentation.
Update or create docs/developer-guide/library.md:
ProducerModule/ConsumerModuleAPI.- How to provide a custom
ModuleRegistrations.cpp. - FastLED-MM as the worked example.
- Domain-independence principle:
src/core/is stable public API;src/modules/is the built-in LED domain, which consumers can replace.
Definition of Done:
ProducerModuleandConsumerModuleinsrc/core/; no LED-specific types insrc/core/EffectsLayerandDriverLayerdo NOT subclass them (decided: incompatible buffer contracts, see Retrospective)library.jsonexcludesmodules/ModuleRegistrations.cpp; infrastructure modules still register viaCoreRegistrations.cpp- 7 new behavioral tests for ProducerModule/ConsumerModule pass
- FastLED-MM
WaveRainbow2DEffectextendsProducerModule;FastLEDDriverModuleextendsConsumerModule docs/developer-guide/architecture.mdProducerModule/ConsumerModule section updated
Result¶
261/261 unit tests pass (was 254 before Sprint 11; +7 new ProducerModule/ConsumerModule behavioral tests).
Files added or changed:
| File | Change |
|---|---|
src/core/ProducerModule.h |
New: declareBuffer(), bufferPtr(), bufferLen(), bufferBytes() |
src/core/ConsumerModule.h |
New: setInput("producer", ...), producer_ protected pointer |
src/core/CoreRegistrations.cpp |
New: registers 8 infrastructure modules (Network, WiFi, SystemStatus, DeviceDiscovery, Tasks, FileManager) |
src/modules/ModuleRegistrations.cpp |
Stripped to LED-domain only (13 LED modules); system modules removed |
library.json |
srcFilter updated: excludes modules/ModuleRegistrations.cpp |
CMakeLists.txt |
Added CoreRegistrations.cpp to projectMM executable |
tests/CMakeLists.txt |
Added CoreRegistrations.cpp + test_producer_consumer.cpp |
tests/test_producer_consumer.cpp |
New: 7 behavioral tests across 2 test suites |
FastLED-MM/src/WaveRainbow2D.h |
Extends ProducerModule; calls declareBuffer(flm_leds, N, sizeof(CRGB)) in setup |
FastLED-MM/src/FastLEDDriver.h |
Extends ConsumerModule |
FastLED-MM/platformio.ini |
Platform pinned to pioarduino 55.03.37; added BUILD_TARGET and lib_compat_mode = strict to match projectMM |
docs/developer-guide/architecture.md |
ProducerModule/ConsumerModule section rewritten with R4S11 decision |
Build and run:
| Target | Status |
|---|---|
| PC CMake build | PASS |
| ESP32dev build | PASS (Flash 93.0%) |
| ESP32-S3 build | PASS (Flash 28.8%, RAM 14.7%) |
| 261 unit tests | PASS |
deploy/run.py PC |
PASS |
deploy/run.py ESP32-S3 MM-70BC |
PASS |
| mkdocs build | PASS (0 warnings) |
| FastLED-MM ESP32-S3 build (PlatformIO) | PASS (Flash 47.6%, RAM 16.1%) |
Retrospective¶
EffectsLayer and DriverLayer do not extend the new base classes, and that is the right call. EffectsLayer owns a double-buffered atomic Channel*, not a flat void*. Wrapping it behind bufferPtr() would strip the type information the dual-core handoff relies on. DriverLayer has eight sources, not one. The question was raised before implementation and answered correctly: the new base classes are for new single-buffer, single-producer consumers, not for retrofitting the existing LED pipeline.
CoreRegistrations.cpp is the key deliverable for library packaging. The split means a downstream project that excludes ModuleRegistrations.cpp still gets all infrastructure (WiFi, network, system status, device discovery, tasks, files) from CoreRegistrations.cpp. Without this, excluding LED modules would silently remove the networking stack too.
library.json srcFilter now excludes ModuleRegistrations.cpp. The FastLED-MM project provides its own two registrations (WaveRainbow2DEffect, FastLEDDriverModule) and gets the infrastructure for free via CoreRegistrations.cpp. No duplicate-symbol errors because each type is registered exactly once.
7 behavioral tests lock the ProducerModule/ConsumerModule contract. Buffer null before setup, correct pointer/length/elemSize after declareBuffer, read access through bufferPtr, wiring via setInput("producer", ...), unknown-key ignore, and end-to-end read through the wired consumer. These cover the full API surface of both classes.
FastLED-MM requires platformio.ini to mirror projectMM's platform settings. Three fixes were needed before the integration build passed: (1) pin to pioarduino espressif32 55.03.37 (the default espressif32 latest uses a different FreeRTOS struct layout with TaskStatus_t missing xCoreID, and an older GCC that rejects C++17 aggregate init with default members); (2) add -DBUILD_TARGET=\"$PIOENV\" (required by Pal.h's platform_version() string); (3) add lib_compat_mode = strict (avoids silent dependency resolution surprises). Use ~/.platformio/penv/bin/pio (PlatformIO's own venv), not the system Homebrew pio; the system binary has an outdated fatfs Python package that fails with cannot import name 'create_extended_partition'.
Seeds Sprint 12 (R5S2): The library boundary is now clean. The next step is EffectsLayer migration: deciding whether it eventually exposes a ProducerModule-compatible interface, or whether the LED pipeline stays separate from the domain-independent contract forever.
Sprints 12-15 — Moved to Release 5¶
Sprints originally planned here were moved to Release 5 to keep Release 4 focused. Sprint 11 (Library packaging) was pulled back from Release 5 and added above.
| Sprint | Release 5 equivalent |
|---|---|
| Sprint 12 (Rough ideas + ESP32-P4) | R5 Sprint 2 |
| Sprint 13 (PPA blending) | R5 Sprint 3 |
| Sprint 14 (Windows build) | R5 Sprint 4 |
Release 4 Backlog¶
- Palette system.
PaletteModulewith 7 FastLED presets; global palette + per-effect override. Pick up in R5 alongside modifiers. - Multi-panel grid.
LayoutMultiPanelcomposing multiple panels on separate pins. Pick up when multi-panel hardware is available for testing. - Per-layer brightness / fade. Each
EffectsLayergets abrightnessslider; adding/removing a layer fades over 500 ms. From the original moonlight-scope plan — pick up in R5. - Solo / mute / rename / collapse per layer. DAW-style layer controls. From the original moonlight-scope plan — pick up in R5.
- Rings + Wheel layouts. For concentric and radial fixtures. Pick up for R5 fixture work.
- Art-Net in as an effect.
NetworkInEffectfills pixels from incoming Art-Net/DDP frames. Requires Sprint 6 to land first. - Bounded-layer efficient pixel iteration. Pick up if Sprint 3 profiling shows the full-canvas clip loop is hot.
- PPA SRM (Scale/Rotate/Mirror). Hardware-accelerated transform for EffectsLayer outputs; limited to 1/16th-pixel rounding on scale. Pick up after Sprint 11 foundation lands.
- PPA dual-task post-processing. Split the pipeline into a frame-render task (effects on Core 0) and a pixel post-processing task (PPA Fill + Blend + SRM on a dedicated core). Requires Sprint 11 PAL foundation and a concurrency analysis. Coordinate with the Sprint 2 double-buffer audit (Part C deferred).
- JPEG decoder handle. Register
esp_jpeg_decode_one_picturehandle at boot as a PAL global (same pattern as PPA clients: register once, keep alive). Prerequisite for any media-playback or image-display effect. Pick up when an image-loading effect is scoped.