Pulse Sequencer

The Pulse Sequencer is Ant64's real-time musical interface — MIDI, jog dials, audio synthesis, a sequencer UI rendered into the display, and a dedicated supervisor for everything performance-related.

Overview

Pulse is built around the ESP32-P4 — the same SoC as DeMon — paired with 32MB of embedded PSRAM and a MIPI display output that feeds directly into the FireStorm FPGA as a display layer.

Why ESP32-P4 for Pulse?

The Pulse role is the real-time musical brain of the Ant64 — sequencer logic, MIDI processing, jog dial state, sample manipulation, AMY-based audio synthesis, software audio emulators, and now its own sequencer UI rendered into the display compositor. (Speech synthesis is handled separately by SAM on DeMon, accessible to Pulse over the bidirectional SPI link.) The earlier supervisor was capable but tight on memory (8MB) and limited for UI rendering. The ESP32-P4 brings:

• 400 MHz HP RISC-V core with PIE extensions (Espressif's SIMD-like instructions for AI/DSP) — runs the sequencer engine and UI rendering; 40 MHz LP core for low-power background tasks
• 32MB PSRAM — large enough for substantial sample libraries, multi-track arrangements, granular synthesis buffers
• MIPI display engine — Pulse renders its own sequencer / mixer / sample-browser UI and feeds it into FireStorm as a display layer
• Identical to DeMon — same toolchain, same hardware know-how, same recovery model

Both DeMon and Pulse are now ESP32-P4-based, with similar internal architecture. They differ in role, peripheral wiring, and PSRAM use — not in core silicon.

Roles & Responsibilities

Pulse (ESP32-P4)
 |
 ├── Audio sequencer (multi-track, MIDI, CV/trigger)
 ├── MIDI router (DIN + USB, MPE) — full decode on Pulse
 ├── AMY synth library — additive / FM / PCM / partials / filters / envelopes
 ├── Classic synth emulators in firmware
 ├── Triggers DeMon SID and SAM engines via SPI ↔ DeMon link
 ├── Jog dial / control surface state aggregation
 ├── Sticky interface (5V joypad I/O)
 ├── MIPI UI + sprite layer → FireStorm (sequencer screen, mixer, sample browser)
 ├── QSPI → FireStorm (register access, audio parameter writes)
 ├── Bidirectional SPI ↔ DeMon (peer command/data link)
 ├── Audio buffer streaming to FireStorm audio engine for mixing
 └── Round TFT display (jog dial labels / overlays)

Audio Path

The Ant64's audio architecture has multiple generation sources that meet at FireStorm's hardware mixer:

1. FireStorm chipset audio engine (in the FPGA) — the primary audio generator: up to 128+ voices of sample playback, FM, granular, physical modelling, hardware-rendered and hardware-mixed at the codec's sample rate. The engine is a chipset block in the same FPGA as the FireStorm Execution Engine and is driven directly by whatever has access to the chipset registers:

   • FireStorm EE itself — applications, games, demos, AntOS-launched programs write to the audio engine via MMIO with single-cycle latency
   • Pulse — sends note on/off and parameter updates over its QSPI register window (sequencer-driven voices)
   • DeMon — can use its QSPI window for system sounds, alerts, AntOS-side audio effects

   This is the high-voice-count workhorse for game audio, mixed scores, and any patch that maps cleanly to the chipset's sub-engines.

2. Pulse AMY voices (software-rendered on Pulse) — software synthesis on Pulse's 400 MHz ESP32-P4 core, accelerated by the PIE extensions. AMY (github.com/shorepine/amy) is an open-source additive / FM / PCM / partial / sample synth library that runs natively on the ESP32-P4. Pulse renders audio buffers into its 32 MB PSRAM and streams them to FireStorm over the MIPI bulk-data path; FireStorm's mixer treats them as another stereo voice source alongside the chipset voices. Adds patch-style synthesis for timbres that don't map cleanly to the chipset (heavy partials, free-routing FM, filter-heavy patches).

3. DeMon Triple SID engine + SAM speech synthesizer — both shared audio resources hosted on DeMon, PIE-accelerated, streamed to FireStorm via MIPI. Pulse can trigger SID voices and SAM phrases directly via the bidirectional SPI link to DeMon — so the sequencer can route MIDI tracks to the SID engine alongside chipset and AMY voices, and dispatch text or phoneme events to SAM for sequencer-driven vocoder-style leads and dialogue tracks. FireStorm EE code can also trigger both (chipset event mailbox + IRQ to DeMon). See DeMon Triple SID and DeMon SAM.

4. FireStorm EE application audio — bare-metal or AntOS-launched programs on the FireStorm EE can also generate PCM directly (software synthesis, mod players, file-based audio playback) and route it through the same mixer.

5. Other Pulse-rendered audio — tracker engines and classic synth emulators on Pulse stream to FireStorm the same way AMY does.

All five meet at the FireStorm mixer (also in the FPGA), which combines them with per-source gain / pan / send before driving the WM8958 / WM8960 codec.

So Pulse is both a controller and one synth source among several. A typical MIDI-driven workflow:

• Incoming MIDI (DIN or USB) is decoded entirely on Pulse.
• For voices that suit the chipset engine (sample playback, FM operator chains, granular), Pulse issues per-voice parameter writes to FireStorm over QSPI. FireStorm renders + mixes them in hardware.
• For voices that suit AMY (additive, partials-rich timbres, filter-heavy patches that don't map to the chipset's fixed sub-engines), Pulse synthesises them locally and streams the resulting audio to FireStorm via MIPI. FireStorm mixes them with the rest.
• Sample / wavetable / patch data shared by both paths is uploaded once to FireStorm DDR3 via the MIPI bulk path.
• Sequencer events are timed by Pulse's MIDI scheduler — the same scheduler dispatches events to chipset voices (QSPI writes) and to AMY voices (local AMY note-on).

Pulse's other audio sources — tracker engines and the classic synth emulators — coexist with AMY as additional Pulse-side voices. The 32 MB PSRAM is large enough to hold a respectable sample library and several AMY patch banks simultaneously. Speech synthesis is provided by SAM on DeMon — Pulse dispatches text or phoneme events over the SPI link and DeMon renders the audio, streaming it to FireStorm independently.

The codec (WM8958 / WM8960) is wired directly to FireStorm. Pulse does not drive the codec — it sends audio to FireStorm and FireStorm drives the codec. The architectural reason: the chipset audio engine, the AMY stream, any application-generated audio from FireStorm itself, and the audio side of HDMI output all meet at the FireStorm mixer; the codec sees the final mix only.

UI Path

Pulse's MIPI TX drives one of the FPGA's two MIPI RX hardcells (the other is DeMon's). FireStorm composites Pulse's MIPI feed as a display layer — typically a dedicated UI region for the sequencer, mixer, or sample browser. Pulse renders this UI on the ESP32-P4 itself, using its 32MB PSRAM as a framebuffer (and double-buffer / scroll buffers as needed).

The ESP32-P4 includes a full 2D Pixel Processing Accelerator (2D-PPA) with hardware rotation, scaling, mirroring, alpha blending, and colour-key support. This means Pulse's MIPI feed is not just a flat UI surface — Pulse can also act as a sprite layer source for FireStorm's compositor:

• Music-performance sprites — VU meters, scope traces, level indicators, animated mixer faders rendered with rotation
• Sample / pattern browser sprites — icons, waveform thumbnails, animated track strips
• Live performance visuals — sequencer-synchronised graphics composited into the main display, driven by the same scheduler that drives the audio
• Jog-dial parameter overlays — rotated dial graphics, animated when knobs are turned, fed up into the main screen alongside the round display labels

The 2D-PPA does its work on framebuffers in Pulse's 32MB PSRAM; the composited image is streamed over MIPI to FireStorm, which treats it as a regular display layer (priority register, colour key, Copper-controllable per-scanline). DeMon's MIPI feed (MIPI RX #0) has the same capability — between the two supervisors, the Ant64 has two hardware-accelerated sprite-layer sources feeding the main display in addition to the FPGA's native sprite engine.

This gives the user two concurrent UI surfaces above and beyond FireStorm's own output:

• The DeMon overlay for system status, recovery, and supervisor menus
• The Pulse overlay for music sequencing, mixing, and live performance

Both are composited by FireStorm alongside the running application's main display output.

Controls

• 8× RGB-illuminated jog dials with integrated push buttons — simulate analogue instruments such as the TB-303
• 4× 3.5 mm trigger / CV inputs (Kick · Snare · Sync · Foot)
• Round TFT display — small, dedicated, mounted on the far left of the case, just above the keyboard, for jog dial labels and per-knob state (SPI-driven from Pulse, separate from the main MIPI feed; always visible regardless of FPGA state)

MIDI

• DIN MIDI — In / Out / Thru via UART, Ant64C only (galvanically isolated optoisolators)
• USB MIDI host — connect MIDI keyboards / controllers
• USB MIDI device — Ant64 appears as a USB MIDI device to a host PC
• MPE support — per-note expression
• USB OTG — host mode default, device mode via runtime switch

On-Pulse Synthesis

The 400 MHz HP core, accelerated by the PIE extensions, runs several audio engines whose output is streamed to FireStorm for mixing:

AMY — Primary Synth Library

AMY is an open-source synth library designed to run on the ESP32-P4. It provides:

• Additive synthesis — partials-based sound with per-partial envelopes
• FM synthesis — operator chains in the spirit of classic FM synths
• PCM playback — sample-based voices, including supplied factory patches
• Wavetable / partial-set instruments
• Per-voice filters — low-pass, high-pass, band-pass with resonance
• ADSR envelopes and LFO modulation on amplitude, pitch, filter, etc.
• MIDI-controllable — Pulse's MIDI decoder feeds AMY directly

AMY voices count against Pulse's CPU budget rather than the chipset's voice count — the two synthesis paths are independent. A typical patch costs a few percent of the HP core per voice, with PIE-accelerated additive partials being the most expensive case.

Tracker Engines

For MOD / S3M / IT-style sequencing. Pattern data is held in PSRAM; sample playback can be done either by AMY (PCM voices) or by chipset voices via QSPI parameter writes — whichever fits the patch better.

Classic Synth Emulators

Specific emulations of well-known instruments outside the AMY model — for example, TB-303-style bassline emulators driven by the jog dials.

Speech Synthesis (on DeMon)

Speech synthesis is not hosted on Pulse — it lives on DeMon as the SAM engine, shared across all three CPUs. Pulse uses it musically by sending text or phoneme events over the bidirectional SPI link; DeMon renders the audio and streams it to FireStorm. From the sequencer's point of view, SAM is just another voice destination alongside chipset voices, AMY, and SID.

Memory Model

The ESP32-P4 has 32MB of embedded PSRAM mapped into its address space, plus 16MB of external SPI flash. There is no separate FRAM (the earlier FRAM is gone — its role is taken by PSRAM for working data and flash for non-volatile storage).

A configurable region of PSRAM is reserved as the MIPI framebuffer, accessed by the MIPI engine via DMA. The remainder holds:

• AMY patch banks and per-voice state
• Sample libraries
• Sequencer / pattern data
• Working buffers for software synthesis (AMY audio output queues, etc.)
• Any large data structures the firmware needs

Communication Summary

Prototype

The prototype uses an Olimex ESP32-P4-DEV board. The mipi dsi also goes to a hdmi adapter, io36 is pulled to 3v, io35 (boot) is pulled to 3v. The JTAG USB goes to the DeMon for debugging. The full USB goes to the Midi input. The HS USB goes to the cartridge port (pins 12 to 15).

SPI3 is QPI to FireStorm LPSPI is slave to DeMon

Reference Links

• ESP32-P4 — espressif.com/en/products/socs/esp32-p4
• PicoADK Hardware — github.com
• ml_synth_organ_example — github.com
• Audcalc — github.com