Development History¶

This section records how projectMM is built: the process, the decisions, and the history of each release. Vision, principles, and system overview live in the home page; architecture, design, and implementation details live in their respective documents.

How to Read This Section¶

projectMM follows an Agile development process adapted for AI-collaborative development. The terminology:

Term	Meaning
Release	A major deliverable milestone. Each release has a theme (e.g. "Proof of Concept") and corresponds to a GitHub Release tag when complete.
Sprint	A scope-boxed work cycle within a release. Each sprint has a goal, a definition of done, a result, and a retrospective.
Backlog	Items raised but explicitly deferred. Each entry notes when it should be picked up — a backlog item is not a forgotten one.
Retrospective	End-of-sprint reflection: what was learned, what questions were raised, and what seeds the next sprint.

For new contributors: start with the latest release document to understand where the project is. Read the Backlog for contribution opportunities — items marked with a "pick up when" condition that is now met are good candidates.

Development Process¶

Agile sprints. Each sprint begins by reviewing the previous sprint's retrospective and closes by writing a new one. Sprints are scope-boxed (not time-boxed): a sprint ends when its definition of done is met.
Top-down refinement. Ideas flow from vision to architecture to design to implementation, with each layer free to push back on the one above.
Version control. GitHub is the primary host. The repository is ewowi/projectMM (will be moved/renamed to projectMM under MoonModules when appropriate).
CI/CD. Automated build, test, and (where possible) deploy across PC, rPi, and ESP32. Every PR gets a footprint delta report.
Automated testing. Unit, integration, and (future) hardware-in-the-loop. Coverage is part of the definition of done.
AI in the workflow. Code generation and review are AI-assisted. We stay tool-agnostic: currently Claude Code for generation and CodeRabbit for PR review, with active experiments on others (e.g. Mistral Devstral).
Open to contributors. The project welcomes pull requests from day one. The sprint structure, backlog, and documentation are designed so a new contributor (human or AI) can find meaningful work and understand why decisions were made.

Sprint Lifecycle¶

This is the standard sequence for running a sprint — followed by the maintainer and by AI coding assistants alike. Each step has a prompt or action associated with it.

Step 1 — Pre-sprint analysis¶

Before the implementation prompt, raise any observations, findings, or new requirements that have come up since the last sprint. These are analysed, documented in the relevant docs (architecture.md, design.md, implementation.md, backlog), and linked from the sprint section.

Prompt pattern: "Before starting the next sprint, I have a few things to analyse — add them to the relevant docs and link from this sprint: [list]"

Step 2 — Implementation prompt¶

Kick off the sprint with a standard prompt that includes: - Which sprint to implement - Check previous retrospectives and backlog for relevant deferred items - Ask clarifying questions before starting; update the scope or architecture/design/implementation docs based on answers - When done: ask to build and run on PC and ESP32, run the tests, capture and analyse logs, update the sprint's Result section, and prove that the issues targeted by the sprint are resolved - After build-and-run: add Result, DoD verification, and Retrospective to the sprint section - If new modules were created: add a doc page in docs/modules/ and register it in mkdocs.yml

Prompt pattern: "Implement the next sprint. Check previous retrospectives and the backlog for anything relevant. Ask me any questions (provide options and propose the simplest and backlog the rest by default ) and update the scope (or architecture/design/implementation docs, with links). When implemented, ask if you can build and run for PC and ESP32 and run the tests, capture and analyse the logs, update relevant documents and the sprint's Result section (refer to other pages where possible), (follow AI-Specific Writing Rules for all generated docs and comments) and prove the sprint issues are solved. Then add Result, DoD, and Retrospective. Add a complexity estimate"

Step 3 — Q&A¶

The AI asks scoping questions; the maintainer answers. Answers narrow the scope, defer to backlog, or confirm. This keeps sprints small and reviewable. Aim to focus on small steps, so if alternitives are presented answer with: do the most simple alternative first and add the more complex items to future sprints or to the backlog.

Step 4 — Watch the run¶

While the AI builds and runs, monitor progress in the git client (e.g. GitKraken): stage files as they appear to get a clearer view of what has changed. Use the diff view to verify that only the expected files are touched.

Step 5 — Approve or redirect tool calls¶

When the AI requests permission for a tool call (shell command, file write, etc.): - Yes — approve if the action looks correct. - Other — redirect with a reason if the approach is wrong or unsafe. - Never approve git commit — always evaluate the staged changes yourself first and do the commit manually after review.

Step 6 — Sprint close¶

Before committing, verify the sprint is complete: If complete, ask an AI agent to do the commit with a descriptive commit message.

Prompt (maintainer will ask AI!): "Did we finish this sprint? Did you run build and run and log including live tests on PC and ESP32 devices. If not please run it. Did you update DoD, Results, Retrospective? Is mkdocs serve running fine? Anything else before commit? if not run the commit command with a descriptive message"

Style¶

This section uses we throughout (not "I"). projectMM is intended as a community project from day one, and the voice should reflect that. Personal anecdotes and single-maintainer history belong in the Background section of the home page, framed as learnings, not as authorship.

Standards and Guidelines¶

The dedicated Standards and Guidelines document covers coding standards for C/C++ and JS, the anti-debt checklist, the human/AI readability rule, non-trivial testing, AI workflow rules, and the contribution workflow.

Releases¶

Release	Theme	Status	Tag
Release 1	Proof of Concept	Complete	v1.0.0
Release 2	Observable, Hierarchical, and Network-Ready	Complete	v1.1.0
Release 3	Pixel Pipeline, MoonLight Effects, and Continuous Integration	In progress	—
Release 4	Platform Reach	In progress	—
Release 5	OTA, Platform Reach, and Library Packaging	Planned	—

The original Release 3–10 roadmap (pixel pipeline, 2D effects, modifiers, 3D, drivers, network protocols) is preserved in the MoonLight Scope subfolder for reference.

Backlog¶

Items raised but deferred. Each entry notes when it should be picked up. Sprint-specific and release-specific items live in the relevant sprint/release doc; this is the cross-release long-term list.

Runtime and infrastructure¶

Supersync. Multi-instance synchronization (passive mirror → active control → group control → shared processing with synchronized clocks). Includes discovery (mDNS / MQTT / explicit config) and clock sync. See architecture.md. When: single-instance runtime is stable on all three platforms and real users run more than one device together.
REST and MQTT endpoints. External control surface. See architecture.md. When: WiFiModule is stable (done Release 2 Sprint 8); REST first, MQTT when Home Assistant integration is on the table.
State pub/sub. Lightweight publish/subscribe to replace direct reads. Candidates: callback list, observer pattern, event queue. See architecture.md. When: a Module needs to react to state changes from two independent sources.
FreeRTOS Core 0 dispatch. cores_ vector is populated but xTaskCreatePinnedToCore dispatch is not yet wired. Start with the MoonLight producer/consumer split; generalise once stable. See architecture.md. When: WiFiModule is stable (Core 0 is used by WiFi; priorities need tuning).
Pending-writes queue. Replace controlMutex_ in the HTTP→scheduler hot path with a lock-free queue drained at the top of each tick. When: profiling shows mutex contention at realistic request rates.
Graceful teardown / dirty-flag save on ESP32. teardown() never runs in Arduino; saveAllState() at end of setup() covers first boot. Wire a dirty-flag + debounce timer (2 s after last control change) for runtime saves. When: live control mutation frequency makes per-change saves a flash-wear concern.
Multiple WiFi configs + scan-to-select. WifiStaModule currently holds a single SSID/password pair. Add support for a priority-ordered list of networks (e.g. home, office, field) and a WiFi scan endpoint (GET /api/wifi/scan) that returns visible SSIDs so the UI can present a picker instead of requiring manual SSID entry. deploy/wifi.py would gain a --profile <name> flag; deploy/flashfs.py would bake all profiles. When: a device needs to move between networks regularly, or a user-friendly onboarding flow is needed.*
Password / credential encryption at rest. Sprint 8 marks passwords sensitive (never broadcast); encrypting the LittleFS state files requires a key management story. When: security requirements are clearer (multi-user / cloud deployment).
KvStore hash table. Linear scan is fine for ≤16 entries. When: entry count exceeds 16 and profiling shows KvStore in the hot path.
~~Minimize floats in the hot path.~~ Picked up in Release 2 Sprint 9 Part G. Integer ControlDescriptor variant with min/max/step; float controls remain available for sub-integer precision. Full DMX/Art-Net 0–255 / 0–65535 alignment is a Release 3 driver concern.
Consolidating ModuleManager and Scheduler — or letting one implement the other. When: experience with both makes the right abstraction boundary obvious.
Module footprint optimization — non-hot-path metadata to JSON only. isPermanent(), platform constraints, display label, category — strip from compiled class, store in JSON manifest. Hot path stays in C++. When: Release 3 module count and footprint budgets make this worthwhile.

Hardware and CI¶

WS2812Driver — GPIO LED driver module using FastLED (or ESP-IDF RMT directly). Reads DriverLayer output and GridLayout mapping to write physical pixel data. Stub on PC. Art-NET out is the higher-priority output path; this is deferred until the Art-NET driver story is clearer. When: Art-NET out is shipping and a GPIO hardware test rig is available.
Art-NET output driver — ArtNetDriver module that sends DriverLayer pixels as Art-NET UDP frames. Requires GridLayout.requestMappings() (Sprint 6) and a UDP send PAL call. When: Sprint 7 PAL is in place (UDP send is a PAL function).
FastLED integration — required by WS2812Driver (see above). ESP-IDF vs Arduino framework question affects this. When: WS2812Driver is picked up.
ESP32 CI build: WiFiUDP.h not found. The ESP32 CI job fails with WiFiUDP.h: No such file or directory on core files (ModuleManager.cpp etc.). Multiple fixes attempted: lib_deps = WiFi in platformio.ini, moving #include <WiFiUDP.h> from the global ARDUINO block to the UDP implementation section in Pal.h. Neither resolved it. Root cause likely involves how PlatformIO's library scanner resolves transitive includes from project headers vs. library headers. When: someone can reproduce the failure locally with PlatformIO and trace the exact scanner path.
rPi and Linux/Windows builds in CI. When: a rPi CI runner is available.
Hardware-in-the-loop testing. Automated flash + boot-output assertion on real devices. When: PC/simulator tests stop catching hardware regressions.
Multi-board CI test rig. USB hub (16 ports) with classic, S3, P4 boards. deploy/all.py already supports multi-device runs; CI hardware rig requires a self-hosted runner. See deploy.md. When: USB hub assembled and at least two board variants available.
ESP32-S3 N16R8 board support. New [env:esp32s3_n16r8] PlatformIO environment; verify PSRAM init, adjust flash/RAM budgets. When: physical N16R8 board available.
Board farm port management. deploy/devicelist.py enumerates boards by USB serial port → deploy/devicelist.json; all deploy scripts select by any field (-env, -ip, -device_name, etc.). When: second board added to the farm.
devicelist.py port-change merging. When a known device (identified by device_name from the hello line) reappears on a different USB port, the script currently creates a duplicate bare entry instead of updating the port on the existing rich entry. Should merge by device_name and update port and last_seen in place. When: devices are regularly moved between USB ports or machines.
ESP-IDF vs Arduino framework. Affects FastLED compatibility. See implementation.md. When: first ESP32 LED driver Module is needed.

Frontend¶

~~Navigation beyond flat list~~ — Picked up in Release 2 Sprint 9 Part H: hamburger menu with category-based scroll navigation, responsive collapse on mobile.
Human-readable control labels. Separate "label" field in ControlDescriptor (e.g. "heap_free_kb" → "Heap Free (KB)"). Schema change; all addControl() call sites need updating. When: raw key names become usability friction.
Alternative frontend technologies — Svelte 5 (requires Node.js build pipeline) or CDN widgets (requires internet; not suitable for standalone ESP32). Evaluate with flash-cost measurements. When: plain HTML frontend becomes a scaling bottleneck.
Live control values on page load. GET /api/modules returns last-saved state file values. Truly live values require a scheduler-hot-path read or routing through the WS push path. When: stale-on-load is reported as a user-visible bug.
ESP32 WebSocket intermittency. Occasional multi-second delay on first F5 after flash — likely a race between browser fast-reconnect and ESPAsyncWebServer tear-down. Sprint 8 improved the JS reconnect path; root cause needs a packet capture or hardware-in-the-loop F5 test. When: reliable repro sequence found.
Slider hang browser automation test. Sprint 3 dragTs suppression prevents mid-drag WS state overwriting the slider, but no automated test covers it. Needs Playwright/Puppeteer drag-while-broadcasting test. When: browser automation harness is in place.
CMake auto-rebuild of frontend bundle — frontend_bundle.h as a proper CMake DEPENDS target. When: frontend edits become frequent enough that manual rebuilds are friction.
Frontend source maps — DEBUG_FRONTEND CMake option embeds original JS for readable DevTools traces. When: debugging the frontend is recurring friction.

Observability¶

PC/rPi native system info. SystemInfoModule fields cpu_freq_mhz, flash_size_mb, sdk_version, chip_model are 0/empty on PC. Needs sysctl (macOS) / /proc/cpuinfo (Linux) implementations. When: permanent rPi deployment needs accurate hardware metrics.
Health check RAM footprint. Each module's healthReport() currently uses a Scheduler-owned scratch buffer (already shared). No per-module char[] cost. ✅ Resolved Sprint 7.
Remove periodic serial logging; verify REST captures everything. Serial logs currently contain periodic background writes (module state, health snapshots). Now that deploy/run.py captures device state via GET /api/system and live tests assert on REST responses, the periodic serial writes are redundant and add noise. Audit what is still only visible via serial and migrate it to REST-accessible state before removing the periodic writes. When: a sprint adds REST endpoints or logging improvements.
Revisit the runtime logging system. The current approach writes info-level entries periodically in the hot path, which inflates loop() timing measurements. Design a log system that: (a) writes nothing in loop() by default, (b) buffers log entries in a ring buffer drained by a background task or on REST request, (c) exposes entries via GET /api/log. Structured entries (timestamp, module id, level, message) enable dashboard display and post-run analysis. When: serial logging is no longer the primary observability path (see item above).

Project identity¶

Installer / updater. Easy first install and in-place upgrade across platforms. When: first demo runs end-to-end and external contributors want to try it without a build toolchain.
Domain / site hosting. moonmodules.org/projectMM as initial home. When: project name is final and docs ready to publish under MoonModules.
Final project name. projectMM is a working title. When: proof of concept runs on all three platforms.
Repository move. ewowi/projectMM → MoonModules/projectMM. When: name is final and CI/CD is in place on all platforms.

Resolved (kept for reference)¶

~~controls_[] embedded cost~~ → resolved Release 2 Sprint 1c (lazy heap allocation)
~~WiFiModule as StatefulModule~~ → done Release 2 Sprint 8
~~Sensitive control flag~~ → done Release 2 Sprint 8
~~Layer hierarchy (parent-child modules)~~ → done Release 2 Sprint 6
~~Runtime add/remove of modules from UI~~ → done Release 2 Sprint 5 + 7
~~WebSocket pause when tab hidden~~ → done Release 2 Sprint 8 (close on hide, reconnect on show)