Skip to content

Release 5 — Virtual/Physical Pipeline, Observability, and Agentic Testing

Status: Complete | Sprints: 10 | GitHub Release tag: v1.5.0

Release 5 rebuilds the pixel pipeline around a clean Virtual/Physical layer split (PhysMap, per-layer buffers, modifier chains), adds non-rectangular layout fixtures, a full observability stack (MemBoot balance sheet, MemLive warnings, per-second CPU accounting), and closes the agentic feedback loop with a scenario benchmarking system shared by unit and live tests. The sprint count grew from the original six because observability and memory hardening proved foundational before the performance work could land.

Release Overview

Area Highlights
Virtual/Physical split PhysMap (1:0, 1:1, 1:N), DriverLayer two-buffer design, EffectsLayer per-layer buffers
Modifier pipeline ModifierModule base; MirrorModifier, CheckerboardModifier, ScrollModifier, RotateModifier, TileModifier; static bake vs dynamic per-frame paths
Non-rectangular layouts RingLayout, WheelLayout, XmasTreeLayout; sparse virtual canvas; multi-layout union bounding box
Observability MemBoot boot balance sheet; MemLive fragmentation warnings; GET /api/memory; per-second TIMING hierarchy with self_ms_per_tick
Platform hardening 5 PSRAM crashes fixed; pre-allocated WS pixel frame buffer; EffectsLayer allocate-before-free; psramSize() virtual
UI/UX Module reorder persistence; WiFi PAL (zero #ifdef ARDUINO); frontend split to index.html + style.css + app.js; category emoji badges
Scenario benchmarking Declarative JSON pipelines; deploy/scenario.py live runner; test_scenarios.cpp in-process C++ replay; baseline comparison

Sprint 1 — PreviewModule

Goal: replace ad-hoc snapshot() scanning with a dedicated PreviewModule that explicitly owns pixel preview and WebSocket delivery.

  • PreviewModule wired to any ProducerModule; caches the packed wire frame (0x02 7-byte header) in a pre-allocated buffer.
  • ModuleManager::pixelSnapshot() targets PreviewModule by type name; fallback scan removed.
  • snapshot() / snapshotWidth/Height/Depth() virtuals removed from ProducerModule and DriverLayer.
  • Key design: always uses actual source geometry from pixelBuf(), ignoring persisted control values — prevents stale state from suppressing preview.
Metric Value
Tests 261 total (8 new in test_websocket.cpp)
Live tests 6/6 on PC, MM-70BC (S3), MM-ESP32 (esp32dev)

Sprint 2 — Virtual/Physical Layer Split + PhysMap

Goal: decouple where effects render (virtual space) from where pixels are physically driven (LED arrangement), enabling non-rectangular fixtures.

  • PhysMap flat table: 4 bytes per physical LED; 1:0 (NO_VIRT sentinel), 1:1, 1:N all handled by the same table. Rebuilt only on layout change.
  • Layout::buildPhysMap() virtual; GridLayout implements identity and serpentine.
  • DriverLayer two-buffer design: virtBlend_ accumulates effect blends, physical_ projected via PhysMap in loop().
  • onChildrenReady() only rebuilds PhysMap if dims change; serpentine toggle fires without teardown.
Metric Value
Tests 279 total (17 new test_physmap.cpp + 1 fix)
ESP32 footprint Flash 66.6%, RAM 14.9% (second buffer at 768 B negligible on PSRAM boards)

Sprint 3 — Layers + Modifiers + Mapping Pipeline

Goal: restructure into four decoupled stages — Layout, EffectsLayer+modifiers, mapping, drivers — so per-layer buffers, modifier stacks, and blend modes have a clean home.

  • ModifierModule base: modifyDims() (dim transform), modifyXYZ() (index transform), isStatic(). Static chains baked into per-layer PhysMap at setup; any dynamic modifier falls back to per-pixel per-frame composition.
  • MirrorModifier (static; dim-halving + 1:N reflection), CheckerboardModifier (NO_VIRT masking), ScrollModifier (dynamic reference).
  • DriverLayer absorbs mapping: virtBlend_ removed; blend accumulates directly into physical_; perLayerMaps_[] per EffectsLayer.
  • HttpServer now serves gzip-compressed frontend bundle (71 KB to 19 KB); eliminates 71 KB internal heap copy that silently broke esp32dev.
  • BrightnessModifierModule removed (did not fit coordinate-transform contract).
Metric Value
Tests 288 total (36 new in test_modifiers.cpp)

Sprint 4 — Non-Rectangular Layouts + 3D Preview

Goal: layouts for fixtures that are not rectangles; WebGL renderer already handles sparse virtual grids — no new wire format needed.

  • RingLayout: N LEDs on a circle; (2r+1)^2 virtual grid; trig-based buildPhysMap().
  • WheelLayout: spokes × ledsPerSpoke; radial coordinate placement.
  • XmasTreeLayout: triangular tree; row r has 2r+1 LEDs; pure integer arithmetic.
  • Existing renderPixelFrame() skips black pixels and normalises positions — ring renders as a circle without frontend changes.
Metric Value
Tests 302 total (14 new layout tests)
New doc pages 3 (ring-layout, wheel-layout, xmas-tree-layout)

Sprint 5 — Multi-Layout Installation Demo

Goal: let a single DriverLayer host multiple Layout children forming one combined installation.

  • Layout base gains offsetX_/Y_/Z_ controls; addOffsetControls() called in all four layout classes.
  • DriverLayer::onChildrenReady() two-pass: union bounding box, then concatenated PhysMap with offset-shifted global virtual indices.
  • PreviewModule reverse-maps sparse layouts via physicalMap() virtual: memset virtual frame to 0, scatter physical LED colours into virtual grid positions.
  • Bug fixed: pixelBuf() now returns physMap_.physCount() length (not w*h*d), preventing 11 KB buffer overread for non-rectangular layouts.
Metric Value
Tests 305 total (3 new multi-layout tests)
Live tests 8/8 on PC, MM-70BC, MM-ESP32 (79/79 assertions per device)

Sprint 6 — Modifier Library

Goal: extend the modifier library with the remaining high-value modifiers.

  • RotateModifier: 2D Z-axis; static at speed=0 (bakes into PhysMap), dynamic at speed>0; bounding-box expand; cosf/sinf cached once per frame per (dims, angle).
  • ScrollModifier fix: isStatic() returns true when all speeds are 0 (enables static-bake path); default speedY_ changed from 1.0 to 0.0 (no accidental animation on add).
  • TileModifier deferred to Sprint 9.
Metric Value
Tests 314 total (9 new RotateModifier + 2 ScrollModifier)
ESP32 flash 64.6% (unchanged)

Sprint 7 — UI/UX Polish, Platform Hardening, Frontend Split

Goal: module reorder persistence, WiFi PAL, frontend maintainability, and PSRAM crash hardening from 5 production crashes.

  • POST /api/modules/reorder; Scheduler::reorderModules(); ModuleManager::reorderChildren(); order persists across reboot. localStorage order overrides removed.
  • WiFi PAL: 8 new pal::wifi_* functions; WifiAp.h and WifiSta.h fully rewritten with zero #ifdef ARDUINO.
  • Frontend split: style.css + app.js + HTML shell; gen_frontend_bundle.py inlines back before gzip; PlatformIO pre: script handles SCons __file__ absence.
  • sensitive field removed from ControlDescriptor; type="password" is the correct abstraction.
  • 5 PSRAM crash fixes: PhysMap PSRAM raw pointer (std::vector caused 48 KB internal-SRAM alloc); perLayerMaps_[] pre-allocated in onChildrenReady() (not in loop()); PreviewModule::snapBuf_ PSRAM raw pointer; WS broadcast zero-copy via pixelSnapshotRaw(); GameOfLifeEffect detects post-modifier layer resize before memset.
  • psramSize() virtual; frontend card shows "X heap + Y PSRAM".
  • replaceModule() 5-case test coverage added.
Metric Value
Tests 328 total (11 new reorder + 5 replaceModule; net +13 after 3 removed)

Sprint 8 — Memory and Time Observability

Goal: make heap pressure and CPU cost visible per module, per phase, as a hierarchical balance sheet.

  • pal::memEvent(label): logs heap delta with a name; called from AppSetup.cpp around WiFi init, HTTP start, WS start.
  • StatefulModule: setupHeapDelta_ / setupPsramDelta_ measured in runSetup(). Serial boot output: indented hierarchy with KB delta per module.
  • GET /api/memory: transient JsonDocument; returns boot balance sheet + free/largest/frag fields.
  • MemLive fragmentation monitoring: largest block < 50% of free logs [MemLive] WARNING frag=X% ... last_module=Y; memWarnings_ counter in Scheduler.
  • Per-second TIMING hierarchy: windowed accumulators per root module + self-only times; ms_per_tick / self_ms_per_tick pushed via WebSocket; total + [idle] = budget balance verified < 0.5% error.
  • test_observability.cpp: 16 tests covering timing arithmetic, balance invariant, fragmentation threshold, setupHeapDelta_ formula.
  • --count N live test: asserts timing balance and memory growth; exits non-zero on violation.
Metric Value
Tests 347 total (16 new in test_observability.cpp)
First MemBoot balance error < 0.5% on both esp32s3_n16r8 and esp32dev

esp32dev crash pattern identified: frag=51-64% at rest; bad_alloc on WS connect when largest contiguous block = 51 KB. Sprint 9 target: pre-allocate the WS frame buffer before the heap fragments.

Sprint 9 — Hotpath and Memory Optimisations

Goal: eliminate bad_alloc crashes under WebSocket load on esp32dev (no PSRAM) and reduce per-frame allocations.

  • Pre-allocated WS pixel frame buffer (AsyncWebSocketSharedBuffer pixBuf_ at ws.begin()): max_alloc_kb on esp32dev improved from 51-65 KB to 84 KB; zero bad_alloc crashes in 40-second WS load test.
  • Broadcast rate reduction: pixel preview 50fps to 10fps; state JSON 50fps to 5fps. 5x fewer allocations per second with no visible UI degradation.
  • EffectsLayer::allocate_() allocate-before-free: OOM keeps previous buffer; previous crash path left pixels = nullptr and crashed in RipplesEffectModule::loop().
  • TileModifier: static; ceiling-div modifyDims; modulo wrap modifyXYZ; bakes into PhysMap.
  • PATCH /api/modules/<id>: rewires inputs on an existing module via ModuleManager::rewireModule(); test0_infrastructure uses it to wire preview1 to the active DriverLayer.
  • src/core/ audit: confirmed domain-free; PhysMap.h reviewed; note added to architecture.md.
Metric Value
Tests 354 total (unchanged from Sprint 8 for this pass)
esp32dev WS crash Zero in 40 s (was reproducible within 1-15 s in Sprint 8)
max_alloc_kb 84 KB (was 51-65 KB)

Sprint 10 — Scenario Benchmarking

Goal: declarative JSON pipelines that drive both unit tests (in-process C++) and live tests (REST against a running device), with baseline comparison for regression detection.

  • Scenario schema: add_module, set_control, set_input ops; "measure": true steps trigger metric capture.
  • deploy/test/scenarios/: base-pipeline.json, speed sweep, 32x32 resize, two-layers, xmas-tree-3d (5 files).
  • deploy/scenario.py: standalone live runner; --compare-baseline / --update-baseline flags; cumulative scenario-results.json.
  • tests/test_scenarios.cpp: replays scenario files in-process via ModuleManager; asserts pipeline builds without crash; reports fps table.
  • deploy/live_suite.py: auto-registers one test per scenario file via glob.
Metric Value
Tests 357 total (+3 scenario replay)
Live tests 13/13 (12 pre-existing + 5 scenario tests)
PC fps at 16x16 155,000 (bound min=1000: pass)

Retrospective — Release 5 complete

What was proven:

  • Virtual/Physical split with PhysMap is correct and efficient: all-static modifier chains pay one table lookup per LED; dynamic chains fall back per-pixel per-frame without code changes.
  • Non-rectangular fixtures (ring, wheel, tree) integrate into the pipeline with zero changes to DriverLayer or EffectsLayer.
  • The MemBoot balance equation (total + idle = budget) verified < 0.5% error on first measurement.
  • Pre-allocating the WS frame buffer before heap fragmentation is the key insight for embedded WebSocket stability.

Watch points going into Release 6:

  • esp32dev frag_pct=40% at rest under WS load — approaching the 50% warning threshold; future 4096-LED pipelines will push further.
  • EffectsLayer single-buffer change (replace ping/pong with mutex) still deferred — the highest-risk item.
  • system_overhead_kb (48-60 KB) still partially opaque; more pal::memEvent() calls would attribute the WiFi/WS init cost.
  • SystemStatus / SystemStatusModule type mismatch in AppSetup means a fresh device boots without a system status module — one-line fix.

Ready for Release 6: toolchain hardening (uv, pre-commit), fast inner loop, USB hub scaling, MCP server for AI-driven build/flash/test.