Pulse Sequencer

The Pulse Sequencer is Ant64's real-time musical interface — MIDI, jog dials, audio synthesis, a sequencer UI rendered into the display, and a dedicated supervisor for everything performance-related.

Overview

Pulse is built around the ESP32-P4 — the same SoC as DeMon — paired with 32MB of embedded PSRAM and a MIPI display output that feeds directly into the FireStorm FPGA as a display layer.

Why ESP32-P4 for Pulse?

The Pulse role is the real-time musical brain of the Ant64 — sequencer logic, MIDI processing, jog dial state, sample manipulation, AMY-based audio synthesis, software audio emulators, and now its own sequencer UI rendered into the display compositor.

Unlike DeMon, Pulse runs no user scripting and no general-purpose OS. It is a real-time RTOS (FreeRTOS / ESP-IDF) running a fixed set of dedicated tasks — sequencer, MIDI, synthesis, UI composition, FireStorm delivery — never user-launched programs. The focus is deliberate: with no OS load to time-slice against, Pulse can give its cycles over to pushing data to FireStorm at higher throughput and lower jitter than the busier, script-running DeMon (both reach FireStorm over a QSPI register window).

(Speech synthesis is handled separately by SAM on DeMon, accessible to Pulse over the bidirectional SPI link.) The earlier supervisor was capable but tight on memory (8MB) and limited for UI rendering. The ESP32-P4 brings:

• 400 MHz HP RISC-V core with PIE extensions (Espressif's SIMD-like instructions for AI/DSP) — runs the sequencer engine and UI rendering; 40 MHz LP core for low-power background tasks
• 32MB PSRAM — large enough for substantial sample libraries, multi-track arrangements, granular synthesis buffers
• MIPI display engine — Pulse renders its own sequencer / mixer / sample-browser UI and feeds it into FireStorm as a display layer
• Identical to DeMon — same toolchain, same hardware know-how, same recovery model

Both DeMon and Pulse are now ESP32-P4-based, with similar internal architecture. They differ in role, peripheral wiring, and PSRAM use — not in core silicon.

Roles & Responsibilities

Pulse (ESP32-P4)
 |
 ├── Audio sequencer (multi-track, MIDI, CV/trigger)
 ├── MIDI router (DIN + USB, MPE) — AMY MIDI engine + Pulse port/routing glue
 ├── I3C controller → [Audio Expansion Header](audio_more#audio-expansion-header) DAC daughterboard (DAC register setup, ID-EEPROM probe; I²C-legacy mode for non-I3C parts)
 ├── I²S master to Audio Expansion Header (MCLK / BCLK / LRCLK) — optional daughterboard ADC; SDATA_IN returns direct to Pulse, feeding AMY's audio-input oscillator, the sampler, and live resampling
 ├── AMY synth library — additive / FM / PCM / partials / filters / envelopes
 ├── Classic synth emulators in firmware
 ├── Triggers DeMon SID and SAM engines via SPI ↔ DeMon link
 ├── Jog dial / control surface state aggregation (via Crank, see below)
 ├── Sticky interface (5V joypad I/O — joysticks, paddles, mice over SPI)
 ├── Crank interface (8 RGB-illuminated jog dials with push buttons over SPI)
 ├── UPDI master for Sticky and Crank — bit-bang firmware update + verify
 │      under DeMon's orchestration (see DeMon UPDI Orchestration)
 ├── Composes UI + sprite + tilemap layer descriptions → FireStorm RX #1 (sequencer, mixer, sample browser); FireStorm renders at scanout
 ├── Streams aggregated controller state (jog dials, joypads) to FireStorm over MIPI — lower priority than the display description
 ├── QSPI → FireStorm (register access, audio parameter writes; small-packet path parallel to MIPI)
 ├── Bidirectional SPI ↔ DeMon (peer command/data link)
 ├── 5 Mbit/s asynchronous, GDMA-driven UART ↔ DeMon (additional peer link, complements the SPI channels)
 ├── Audio buffer streaming to Tempest for mixing
 └── Round TFT display under dome lens (jog dial labels / overlays — at-a-glance only, far too small for control panels)

Audio Path

The Ant64's audio architecture has multiple generation sources that meet at Tempest's mixer:

1. Tempest — the FireStorm chipset audio engine (in the FPGA) — the primary audio generator: up to 128+ voices of sample playback, FM, granular, physical modelling, hardware-rendered and hardware-mixed at the codec's sample rate. Tempest is a chipset block in the same FPGA as the FireStorm Execution Engine and is driven directly by whatever has access to its registers:

   • FireStorm EE itself — applications, games, demos, AntOS-launched programs write to Tempest via MMIO with single-cycle latency
   • Pulse — sends note on/off and parameter updates over its QSPI register window (sequencer-driven voices)
   • DeMon — can use its QSPI window for system sounds, alerts, AntOS-side audio effects

   This is the high-voice-count workhorse for game audio, mixed scores, and any patch that maps cleanly to the chipset's sub-engines.

2. Pulse AMY voices (software-rendered on Pulse) — software synthesis on Pulse's 400 MHz ESP32-P4 core, accelerated by the PIE extensions. AMY (github.com/shorepine/amy) is an open-source additive / FM / PCM / partial / sample synth library that runs natively on the ESP32-P4. Pulse renders audio buffers into its 32 MB PSRAM and streams them to FireStorm over the MIPI bulk-data path; FireStorm's mixer treats them as another stereo voice source alongside the Tempest voices. Adds patch-style synthesis for timbres that don't map cleanly to the chipset (heavy partials, free-routing FM, filter-heavy patches).

3. DeMon Triple SID engine + SAM speech synthesizer — both shared audio resources hosted on DeMon, PIE-accelerated, streamed to FireStorm via MIPI. Pulse can trigger SID voices and SAM phrases directly via the bidirectional SPI link to DeMon — so the sequencer can route MIDI tracks to the SID engine alongside chipset and AMY voices, and dispatch text or phoneme events to SAM for sequencer-driven vocoder-style leads and dialogue tracks. FireStorm EE code can also trigger both (chipset event mailbox + IRQ to DeMon). See DeMon Triple SID and DeMon SAM.

4. FireStorm EE application audio — bare-metal or AntOS-launched programs on the FireStorm EE can also generate PCM directly (software synthesis, mod players, file-based audio playback) and route it through the same mixer.

5. Other Pulse-rendered audio — tracker engines and classic synth emulators on Pulse stream to FireStorm the same way AMY does.

All five meet at the Tempest mixer (also in the FPGA), which combines them with per-source gain / pan / send before driving the WM8958 / WM8960 codec.

So Pulse is both a controller and one synth source among several. A typical MIDI-driven workflow:

• Incoming MIDI (DIN or USB) arrives on Pulse and is processed by AMY's MIDI engine, which maps channels to synths and applies note/velocity/pitch-bend/CC/program-change events.
• For voices that suit the chipset engine (sample playback, FM operator chains, granular), Pulse issues per-voice parameter writes to FireStorm over QSPI. FireStorm renders + mixes them in hardware.
• For voices that suit AMY (additive, partials-rich timbres, filter-heavy patches that don't map to the chipset's fixed sub-engines), Pulse synthesises them locally and streams the resulting audio to FireStorm via MIPI. FireStorm mixes them with the rest.
• Sample / wavetable / patch data shared by both paths is uploaded once to FireStorm DDR3 via the MIPI bulk path.
• Sequencer events are timed by Pulse's MIDI scheduler — the same scheduler dispatches events to Tempest voices (QSPI writes) and to AMY voices (local AMY note-on).

Pulse's other audio sources — tracker engines and the classic synth emulators — coexist with AMY as additional Pulse-side voices. The 32 MB PSRAM is large enough to hold a respectable sample library and several AMY patch banks simultaneously. Speech synthesis is provided by SAM on DeMon — Pulse dispatches text or phoneme events over the SPI link and DeMon renders the audio, streaming it to FireStorm independently.

The codec (WM8958 / WM8960) is wired directly to FireStorm. Pulse does not drive the codec — it sends audio to FireStorm and FireStorm drives the codec. The architectural reason: Tempest, the AMY stream, any application-generated audio from FireStorm itself, and the audio side of HDMI output all meet at the Tempest mixer; the codec sees the final mix only.

UI Path

Pulse's MIPI TX drives one of the FPGA's two MIPI RX hardcells (the other is DeMon's). FireStorm composites Pulse's MIPI feed as a display layer — typically a dedicated UI region for the sequencer, mixer, or sample browser. Pulse composes this UI as layer descriptions (text cell grid, tilemap tile-index map, sprite display list) and streams them to FireStorm, which renders the pixels at scanout; Pulse holds the layer state in its 32MB PSRAM but never produces a finished framebuffer (see Composer). In the reverse direction, FireStorm presents a USB 2.0 High-Speed device to Pulse, hosted on Pulse's HS USB OTG controller — a direct two-wire (D+/D−) link, 480 Mbps (~45 MB/s), asynchronous, bringing Tempest mixer output back to Pulse for resampling, recording, and sample capture into the sequencer (DeMon has the same arrangement). This replaces the earlier LVDS-to-MIPI CSI bridge — no bridge chip, no LVDS TX pins.

Pulse's MIPI traffic is frame-locked to FireStorm's system vsync output — a dedicated signal from FireStorm to a Pulse GPIO that pulses once per display frame. On each pulse Pulse bursts the next frame's layer descriptions (cell grid, tile-index map, sprite display list) plus that frame's AMY / emulator audio chunk over MIPI; the link is idle between pulses. The descriptions are tiny — index-per-cell grids and a sprite list, a small fraction of the per-frame MIPI budget — and FireStorm renders and scales them to the active output resolution at scanout.

Because the layer is a description rather than a flat surface, Pulse is a full sprite and tilemap layer source for FireStorm's compositor, not just a UI overlay. Tilemaps suit the sequencer's natural grid UIs especially well — the P4 streams a tile-index map and FireStorm renders the tiles from a tileset, so a step matrix or mixer strip is just per-cell tile indices:

• Music-performance sprites — VU meters, scope traces, level indicators, animated mixer faders (position + scale/flip/rotation in the display list)
• Sample / pattern browser sprites — icons, waveform thumbnails, animated track strips
• Live performance visuals — sequencer-synchronised graphics composited into the main display, driven by the same scheduler that drives the audio
• Jog-dial parameter overlays — rotated dial graphics, animated when knobs are turned, fed up into the main screen alongside the round display labels

Typical tilemap uses on Pulse: the step-sequencer grid (16–256-step matrix; per-step state by tile index), drum-machine pattern matrices, sample-browser tile grids, and mixer channel strips composed as tiles. Like DeMon, Pulse's supervisor-side tilemaps are for UI-scale grids — modest in size and update rate. Full-screen game-style tilemap backgrounds belong on FireStorm's native tilemap layers (graphics), driven by the running application, not here.

These descriptions are rendered by the same FireStorm tile and sprite hardware as its native layers; the difference is only the authoring path (a streamed description vs chipset-register writes). DeMon's feed (MIPI RX #0) does the same — between the two supervisors, the Ant64 has two independent description-stream layer sources feeding the main display in addition to the FPGA's native sprite and tilemap engines. See Composer for the shared 4bpp texture format and the open render-split decisions.

This gives the user two concurrent UI surfaces above and beyond FireStorm's own output:

• The DeMon overlay for system status, recovery, and supervisor menus
• The Pulse overlay for music sequencing, mixing, and live performance

Both are composited by FireStorm alongside the running application's main display output.

Controls

• 8× RGB-illuminated jog dials with integrated push buttons — simulate analogue instruments such as the TB-303. The dials are managed by Crank, a dedicated AVR128DB-family microcontroller next to Pulse on the board, talking to Pulse over SPI. See Crank
• 4× 3.5 mm trigger / CV inputs (Kick · Snare · Sync · Foot)
• Round TFT display — small, dedicated, mounted on the far left of the case, just above the keyboard, for jog dial labels and per-knob state (SPI-driven from Pulse, separate from the main MIPI feed; always visible regardless of FPGA state)

UPDI — Programming Sticky and Crank

Pulse hosts the UPDI bit-bang service that programs the two AVR128DB-family microcontrollers in the system — Sticky (joystick / paddle I/O) and Crank (jog dials). Both AVRs sit on Pulse's side of the board with their UPDI lines wired to Pulse GPIOs; Pulse is physically adjacent to them and is the natural host for the timing-sensitive single-wire protocol.

DeMon owns the firmware files and the toolchain; Pulse owns the wire. The two divide work as follows:

UPDI runs at ~225 kbaud half-duplex; the ESP32-P4's UART in half-duplex mode handles the line natively if a UART is wired to the UPDI pin, otherwise a GPIO bit-bang with timer-driven edges works at the same rate. Each AVR has its own UPDI line — Sticky and Crank are programmed independently, never both at once.

Scope is firmware update and possible debug only — not runtime control. Routine joystick data (Sticky → Pulse) and jog dial state (Crank → Pulse) flows on the same SPI peer links that the AVRs use for normal operation; UPDI is reserved for the comparatively rare update events. The update sticky and update crank AntOS commands are the user-facing entry points.

MIDI

• DIN MIDI — In / Out via UART1, software-driven Thru 1 via UART2, plus UART3 / UART4 reserved for an optional 2nd and 3rd MIDI Thru on future studio-router daughterboards. Each Thru is independently filterable: drop channels, transpose, remap CCs, merge Pulse-generated MIDI in, gate on song state. The control surface for these Thru rules lives in the Workstation app's MIDI panel — rendered into Pulse's MIPI feed and composited on the main display by FireStorm (the round TFT next to the keyboard is far too small, and reserved for jog-dial labels and per-knob status). Per-Thru filter lists, transpose tables, CC remap tables, merge-source toggles, and song-state gating are configured there and saved per project, so a live setup (e.g. Thru 1 = drums routed to a hardware drum machine with cymbals dropped; Thru 2 = bass channel transposed to a second synth) loads with the song. Routed through the Audio Expansion Header daughterboard; populated on the Studio I/O variant that ships with the Ant64C, optional on other models (galvanically isolated optoisolators)
• USB MIDI host — a single USB-MIDI port for a keyboard / controller; host-only, no USB MIDI device / slave mode
• FS USB OTG — hosts that single MIDI device directly (no hub). Pulse's HS USB OTG is committed to the FireStorm return link; the mouse and other HID live on DeMon's hub
• MPE support — per-note expression

On/Off button light

The power-button RGB LED moved from DeMon to Pulse. It is driven by three PWM GPIOs:

On-Pulse Synthesis

The 400 MHz HP core, accelerated by the PIE extensions, runs several audio engines whose output is streamed to FireStorm for mixing:

AMY — Primary Synth Library

AMY is an open-source synthesis engine designed to run on the ESP32-P4, and it is a great deal more than a single synth — it is a multi-paradigm engine that covers most of the ground a dedicated workstation synth would. Its building blocks are individual oscillators (≈180 available at once by default, configurable), grouped into voices, managed by synths that handle polyphony, voice allocation, and note-stealing. A patch configures a synth, and AMY can run several synths at once, so it is inherently multitimbral.

Synthesis paradigms:

• Subtractive / analogue — ships with the Juno-106 patch set (presets 0–127): oscillators through resonant filters and envelopes
• FM — ships with the DX7 patch set (presets 128–255) and the full DX7 6-operator algorithms; you can also build your own operator algorithms (ALGO)
• Additive — explicit build-your-own partials mode (BYO_PARTIALS): a stack of sinusoids, each with its own amplitude envelope, plus interpolated-partials voices (the built-in piano, preset 256)
• Wavetable — plays 16,384-sample wavetable packs (e.g. waveeditonline.com) with interpolation across the table's 64 cycles
• Karplus–Strong physical-modelling pluck
• Core oscillators — band-limited saw, pulse/square (variable duty), triangle, sine, noise

Modulation and shaping:

• ControlCoefficients — a flexible modulation matrix: up to nine control signals (constant, note, velocity, two envelope generators, a modulating oscillator, pitch-bend, and two external/CV inputs) are scaled and summed to drive amplitude, frequency, filter frequency, duty cycle, or pan
• Any oscillator can modulate any other — LFOs and audio-rate FM both fall out of this, with additive modulation targets
• Two envelope generators per oscillator, each up to eight breakpoints — more capable than a plain ADSR
• Filters — low-pass, band-pass, high-pass with resonance, per oscillator; plus a per-synth EQ and volume

Sampling, audio-in, and effects:

• PCM sampler — 67 built-in drum/instrument samples, plus runtime load_sample of your own PCM, WAV-file playback from disk with pitching, looping, and stereo/multi-channel support
• Live resampling — sample AMY's own output bus or the audio input into a playable PCM preset on the fly
• Audio input as an oscillator — either channel of a stereo input (AUDIO_IN0/1) becomes an AMY oscillator, so audio can be filtered, enveloped, and effected like any synth voice. On the Ant64 the system audio input is captured at the codec by FireStorm and can be routed to Pulse, so AMY can process the live line/mic input directly — live effects, vocoder-style treatments, and resampling the input into a playable PCM preset
• Effects — echo / reverb
• Sample-accurate sequencer — AMY brings its own timestamped event scheduler and pattern sequencer (48 PPQ, tempo-locked, repeating patterns, MIDI-drum mode), clocked off the audio samples. This is the sequencer engine the Ant64's Pulse Sequencer is built on — Pulse needed a rock-solid timing foundation and AMY provides exactly that; the 16-track / parameter-lock / Euclidean feature set is the Pulse application layer on top of AMY's scheduler
• MIDI processing — AMY supplies the MIDI engine Pulse needs: it maps the 16 MIDI channels onto synths and handles note on/off and velocity, pitch-bend, program-change patch selection (reallocating voices), configurable CC → parameter mappings (per channel, with min/max/offset/log scaling — the mechanism behind MIDI Learn), MIDI drum note→preset translation, and All Notes Off. Pulse firmware wraps this with the physical ports (DIN In/Out/Thru, USB host), MIDI Out/Thru, clock master/slave + MMC, MPE translation, and the routing of MIDI to AMY, the FireStorm Tempest voices, and the DeMon SID engine

AMY voices count against Pulse's CPU budget rather than the chipset's voice count — the two synthesis paths are independent. A typical patch costs a few percent of the HP core per voice, with PIE-accelerated additive partials being the most expensive case. Full detail upstream: AMY synth documentation.

Tracker Engines

For MOD / S3M / IT-style sequencing. Pattern data is held in PSRAM; sample playback can be done either by AMY (PCM voices) or by Tempest voices via QSPI parameter writes — whichever fits the patch better.

Classic Synth Emulators

Specific emulations of well-known instruments outside the AMY model — for example, TB-303-style bassline emulators driven by the jog dials.

Speech Synthesis (on DeMon)

Speech synthesis is not hosted on Pulse — it lives on DeMon as the SAM engine, shared across all three CPUs. Pulse uses it musically by sending text or phoneme events over the bidirectional SPI link; DeMon renders the audio and streams it to FireStorm. From the sequencer's point of view, SAM is just another voice destination alongside Tempest voices, AMY, and SID.

Memory Model

The ESP32-P4 has 32MB of embedded PSRAM mapped into its address space, plus 16MB of external SPI flash. There is no separate FRAM (the earlier FRAM is gone — its role is taken by PSRAM for working data and flash for non-volatile storage).

A configurable region of PSRAM is reserved as the MIPI framebuffer, accessed by the MIPI engine via DMA. The remainder holds:

• AMY patch banks and per-voice state
• Sample libraries
• Sequencer / pattern data
• Working buffers for software synthesis (AMY audio output queues, etc.)
• Any large data structures the firmware needs

Communication Summary

Prototype

The prototype uses an Olimex ESP32-P4-PC board. The mipi dsi also goes to a hdmi adapter, io36 is pulled to 3v, io35 (boot) is pulled to 3v. The JTAG USB goes to the DeMon for debugging. The full USB goes to the Midi input. The HS USB is the FireStorm return link in production; on this standalone prototype it can be looped to a test USB device for bring-up.

| SPI2 SEL | Name | Behaviour | | 0 | DeMon | qspi | | 1 | FireStorm | opi |

LPSPI is slave to DeMon

Reference Links

• ESP32-P4 — espressif.com/en/products/socs/esp32-p4
• PicoADK Hardware — github.com
• ml_synth_organ_example — github.com
• Audcalc — github.com