Pulse Sequencer

The Pulse Sequencer is Ant64's real-time musical interface — MIDI, jog dials, audio synthesis, a sequencer UI rendered into the display, and a dedicated supervisor for everything performance-related.

Overview

Pulse is built around the ESP32-P4 — paired with 32MB of embedded PSRAM and a MIPI display output that feeds directly into the FireStorm FPGA as a display layer.

Why ESP32-P4 for Pulse?

The Pulse role is the real-time musical brain of the Ant64 — sequencer logic, MIDI processing, jog dial state, sample manipulation, AMY-based audio synthesis, software audio emulators, and now its own sequencer UI rendered into the display compositor.

Unlike DeMon, Pulse runs no user scripting and no general-purpose OS. It is a real-time RTOS (FreeRTOS / ESP-IDF) running a fixed set of dedicated tasks — sequencer, MIDI, synthesis, UI composition, FireStorm delivery — never user-launched programs. The focus is deliberate: with no OS load to time-slice against, Pulse can give its cycles over to pushing data to FireStorm at higher throughput and lower jitter than the busier, script-running DeMon (Pulse reaches FireStorm over OPI, DeMon over QSPI).

(Speech synthesis is handled separately by SAM on DeMon, accessible to Pulse over the bidirectional SPI link.) The earlier supervisor was capable but tight on memory (8MB) and limited for UI rendering. The ESP32-P4 brings:

• 400 MHz HP RISC-V dual core with PIE extensions (Espressif's SIMD-like instructions for AI/DSP) — runs the sequencer engine and UI rendering; 40 MHz LP core for low-power background tasks
• 32MB PSRAM — large enough for substantial sample libraries, multi-track arrangements, granular synthesis buffers
• MIPI display engine — Pulse renders its own sequencer / mixer / sample-browser UI and feeds it into FireStorm as a display layer
• Identical to DeMon — same toolchain, same hardware know-how, same recovery model

Pulse is ESP32-P4-based; DeMon is now a Raspberry Pi CM5. They differ in both silicon and role — Pulse is the real-time music / control MCU, DeMon the AntOS supervisor.

Roles & Responsibilities

Pulse (ESP32-P4)
 |
 ├── Audio sequencer (multi-track, MIDI, CV/trigger)
 ├── MIDI router (DIN + USB, MPE) — AMY MIDI engine + Pulse port/routing glue
 ├── I3C controller → [Audio Expansion Header](audio_more#audio-expansion-header) DAC daughterboard (DAC register setup, ID-EEPROM probe; I²C-legacy mode for non-I3C parts)
 ├── Expansion audio I²S — **moved to the [FPGA](fpga)** (a future-expansion I²S is reserved there); Pulse no longer drives an expansion audio I²S
 ├── AMY synth library — additive / FM / PCM / partials / filters / envelopes
 ├── Classic synth emulators in firmware
 ├── Triggers DeMon SID and SAM engines via SPI ↔ DeMon link
 ├── Jog dial / control surface state aggregation (via Cranky, see below)
 ├── Sticky interface (5V joypad I/O — joysticks, paddles, mice over SPI)
 ├── Cranky interface (8 RGB-illuminated jog dials with push buttons over SPI)
 ├── UPDI master for Sticky, Cranky and Clicky (serialUPDI, UART4) — firmware update + verify
 │      under DeMon's orchestration (see DeMon UPDI Orchestration)
 ├── Composes UI + sprite + tilemap layer descriptions → FireStorm RX #1 (sequencer, mixer, sample browser); FireStorm renders at scanout
 ├── Streams aggregated controller state (jog dials, joypads) to FireStorm over MIPI — lower priority than the display description
 ├── OPI → FireStorm (register access, audio parameter writes; small-packet path parallel to MIPI)
 ├── Bidirectional SPI ↔ DeMon (peer command/data link)
 ├── 5 Mbit/s asynchronous, GDMA-driven UART ↔ DeMon (additional peer link, complements the SPI channels)
 ├── Audio buffer streaming to Tempest for mixing
 └── Round TFT display under dome lens (jog dial labels / overlays — at-a-glance only, far too small for control panels)

Audio Path

The Ant64's audio architecture has multiple generation sources that meet at Tempest's mixer:

1. Tempest — the FireStorm chipset audio engine (in the FPGA) — the primary audio generator: up to 128+ voices of sample playback, FM, granular, physical modelling, hardware-rendered and hardware-mixed at the codec's sample rate. Tempest is a chipset block in the same FPGA as the FireStorm Execution Engine and is driven directly by whatever has access to its registers:

   • FireStorm EE itself — applications, games, demos, AntOS-launched programs write to Tempest via MMIO with single-cycle latency
   • Pulse — sends note on/off and parameter updates over its OPI register window (sequencer-driven voices)
   • DeMon — can use its QSPI window for system sounds, alerts, AntOS-side audio effects

   This is the high-voice-count workhorse for game audio, mixed scores, and any patch that maps cleanly to the chipset's sub-engines.

2. Pulse AMY voices (software-rendered on Pulse) — software synthesis on Pulse's 400 MHz ESP32-P4 core, accelerated by the PIE extensions. AMY (github.com/shorepine/amy) is an open-source additive / FM / PCM / partial / sample synth library that runs natively on the ESP32-P4. Pulse renders audio buffers into its 32 MB PSRAM and streams them to FireStorm over the MIPI bulk-data path; FireStorm's mixer treats them as another stereo voice source alongside the Tempest voices. Adds patch-style synthesis for timbres that don't map cleanly to the chipset (heavy partials, free-routing FM, filter-heavy patches).

3. DeMon Triple SID engine + SAM speech synthesizer — both shared audio resources hosted on DeMon (in software on the CM5), streamed to FireStorm over HDMI as the Accelerator's audio (PCIe for control). Pulse can trigger SID voices and SAM phrases directly via the bidirectional SPI link to DeMon — so the sequencer can route MIDI tracks to the SID engine alongside chipset and AMY voices, and dispatch text or phoneme events to SAM for sequencer-driven vocoder-style leads and dialogue tracks. FireStorm EE code can also trigger both (chipset event mailbox + IRQ to DeMon). See DeMon Triple SID and DeMon SAM.

4. FireStorm EE application audio — bare-metal or AntOS-launched programs on the FireStorm EE can also generate PCM directly (software synthesis, mod players, file-based audio playback) and route it through the same mixer.

5. Other Pulse-rendered audio — tracker engines and classic synth emulators on Pulse stream to FireStorm the same way AMY does.

All five meet at the Tempest mixer (also in the FPGA), which combines them with per-source gain / pan / send before driving the WM8960 / WM8962 codec.

So Pulse is both a controller and one synth source among several. A typical MIDI-driven workflow:

• Incoming MIDI (DIN or USB) arrives on Pulse and is processed by AMY's MIDI engine, which maps channels to synths and applies note/velocity/pitch-bend/CC/program-change events.
• For voices that suit the chipset engine (sample playback, FM operator chains, granular), Pulse issues per-voice parameter writes to FireStorm over OPI. FireStorm renders + mixes them in hardware.
• For voices that suit AMY (additive, partials-rich timbres, filter-heavy patches that don't map to the chipset's fixed sub-engines), Pulse synthesises them locally and streams the resulting audio to FireStorm via MIPI. FireStorm mixes them with the rest.
• Sample / wavetable / patch data shared by both paths is uploaded once to FireStorm DDR3 via the MIPI bulk path.
• Sequencer events are timed by Pulse's MIDI scheduler — the same scheduler dispatches events to Tempest voices (OPI writes) and to AMY voices (local AMY note-on).

Pulse's other audio sources — tracker engines and the classic synth emulators — coexist with AMY as additional Pulse-side voices. The 32 MB PSRAM is large enough to hold a respectable sample library and several AMY patch banks simultaneously. Speech synthesis is provided by SAM on DeMon — Pulse dispatches text or phoneme events over the SPI link and DeMon renders the audio, streaming it to FireStorm independently.

The codec (WM8960 / WM8962) is wired directly to FireStorm. Pulse does not drive the codec — it sends audio to FireStorm and FireStorm drives the codec. The architectural reason: Tempest, the AMY stream, any application-generated audio from FireStorm itself, and the audio side of HDMI output all meet at the Tempest mixer; the codec sees the final mix only.

UI Path

Pulse's MIPI TX drives the FPGA's MIPI RX hardcell (DeMon feeds over HDMI). FireStorm composites Pulse's MIPI feed as a display layer — typically a dedicated UI region for the sequencer, mixer, or sample browser. Pulse composes this UI as layer descriptions (text cell grid, tilemap tile-index map, sprite display list) and streams them to FireStorm, which renders the pixels at scanout; Pulse holds the layer state in its 32MB PSRAM but never produces a finished framebuffer (see Composer). Pulse also has a dedicated OPI (8 data line spi) port for requesting data from Temptest, with an interrupt if Tempest also needs to request Pulse asks for data.

Pulse's MIPI traffic is frame-locked to FireStorm's system vsync output — a dedicated signal from FireStorm to a Pulse GPIO that pulses once per display frame. A companion HBlank signal (GP39, from FireStorm) is now wired too, for per-scanline timing where a burst must land within a line rather than a frame. On each pulse Pulse bursts the next frame's layer descriptions (cell grid, tile-index map, sprite display list) plus that frame's AMY / emulator audio chunk over MIPI; the link is idle between pulses. The descriptions are tiny — index-per-cell grids and a sprite list, a small fraction of the per-frame MIPI budget — and FireStorm renders and scales them to the active output resolution at scanout.

Because the layer is a description rather than a flat surface, Pulse is a full sprite and tilemap layer source for FireStorm's compositor, not just a UI overlay. Tilemaps suit the sequencer's natural grid UIs especially well — the P4 streams a tile-index map and FireStorm renders the tiles from a tileset, so a step matrix or mixer strip is just per-cell tile indices:

• Music-performance sprites — VU meters, scope traces, level indicators, animated mixer faders (position + scale/flip/rotation in the display list)
• Sample / pattern browser sprites — icons, waveform thumbnails, animated track strips
• Live performance visuals — sequencer-synchronised graphics composited into the main display, driven by the same scheduler that drives the audio
• Jog-dial parameter overlays — rotated dial graphics, animated when knobs are turned, fed up into the main screen alongside the round display labels

Typical tilemap uses on Pulse: the step-sequencer grid (16–256-step matrix; per-step state by tile index), drum-machine pattern matrices, sample-browser tile grids, and mixer channel strips composed as tiles. Like DeMon, Pulse's supervisor-side tilemaps are for UI-scale grids — modest in size and update rate. Full-screen game-style tilemap backgrounds belong on FireStorm's native tilemap layers (graphics), driven by the running application, not here.

These descriptions are rendered by the same FireStorm tile and sprite hardware as its native layers; the difference is only the authoring path (a streamed description vs chipset-register writes). DeMon's feed (over HDMI) does the same — between the two supervisors, the Ant64 has two independent description-stream layer sources feeding the main display in addition to the FPGA's native sprite and tilemap engines. See Composer for the shared 4bpp texture format and the open render-split decisions.

This gives the user two concurrent UI surfaces above and beyond FireStorm's own output:

• The DeMon overlay for system status, recovery, and supervisor menus
• The Pulse overlay for music sequencing, mixing, and live performance

Both are composited by FireStorm alongside the running application's main display output.

Controls

• 8× RGB-illuminated jog dials with integrated push buttons — simulate analogue instruments such as the TB-303. The dials are managed by Cranky, a dedicated AVR128DB-family microcontroller next to Pulse on the board, talking to Pulse over SPI. See Cranky
• 4× 3.5 mm trigger / CV inputs (Kick · Snare · Sync · Foot)
• Round TFT display — small, dedicated, mounted on the far left of the case, just above the keyboard, for jog dial labels and per-knob state (SPI-driven from Pulse, separate from the main MIPI feed; always visible regardless of FPGA state)

Round TFT Display

A small round QSPI TFT — just over 1.5″ diameter, 360×360 (~340 ppi) under a dome lens, far left of the case above the keyboard — is mainly an ambient identity surface: a slow-animated Ant64 logo at idle, the occasional audio visual (a radial VU / spectrum ring), and the badge of whatever chipset the FireStorm core is currently emulating. A glanceable jog-dial readout — the parameter a dial controls, its value, the active bank — is a secondary use, not the point. At 1.5″ the high pixel count buys crispness, not density — a bold centred logo or a radial visual, rendered sharp under the magnifying dome rather than a dense UI. Centre-weighted content like a logo or a VU ring suits the dome naturally: the rim distortion becomes framing rather than a problem. It is deliberately not a control surface (the real sequencer/mixer UI is Pulse's MIPI overlay on the main display); it stays visible regardless of FPGA state, so the knobs are labelled even at boot or during recovery.

The dome lens. The panel most likely sits under a domed (cabochon) lens rising ~7 mm from rim to centre — a pronounced curve (radius of curvature ≈ 29 mm over the 1.5″ face) that gives the jewel-under-glass look and magnifies the readout, helping it read at a glance. A dome that steep also distorts toward the rim, so content stays central and large — value, arc or label in the flat-magnifying middle, the outer ring as framing rather than information. Because Pulse renders the frame, it could pre-warp it (an inverse-barrel map) so straight edges read true through the lens; for glanceable glyphs, central composition is simpler and enough.

Why Pulse drives it, not Cranky. A 360×360 QSPI panel is a ~253 KB framebuffer at 16 bpp. Cranky — the AVR128DB64 that owns the dials — has 16 KB of SRAM total, about a sixteenth of one frame, so it could only ever be a dumb pixel pipe fed pixel-by-pixel over the wire. Pulse has 32 MB of PSRAM and a rendering path already, so it renders the readout (dial arcs, values, labels) from the state Cranky reports. The screen sits with the chip that has the memory and the graphics; Cranky stays the real-time encoder/lighting controller it is good at.

Bus and pins. The panel is a 360×360 IPS LCD on an ST77916 QSPI controller — a QSPI device (SCL + IO0–3) on Pulse's SPI2 bus, selected by CS3 (GP43) alongside FireStorm (OPI), Sticky and Cranky, gated by chip-select. QSPI carries command and data in its own protocol, so there is no DC line. Pulse reads the panel's TE frame-sync (tearing effect) on GP40 and times its framebuffer writes to it, so a redraw never tears. Reset and backlight sit on Cranky — slow control lines that don't belong on the SPI master, commanded over Pulse's Cranky link. Moving those two off Pulse freed GP39/GP40, which now carry HBlank (GP39, from FireStorm) and the panel's TE (GP40).

Content sources. The idle logo animation and audio visuals are Pulse's own — it renders them from PSRAM (the audio ring driven by the same signal it already streams to FireStorm). The chipset badge is triggered by DeMon, which knows what the FireStorm core is running: DeMon tells Pulse "now emulating X" over the peer link and Pulse renders that badge. Any secondary jog-dial readout comes the same way it always did — Cranky → Pulse over SPI, Pulse composes and pushes the changed region over SPI2, timed to TE so animation and visuals never tear. The round TFT and the MIPI overlay are independent surfaces — the TFT is off the MIPI feed and survives any FPGA state, so the logo is showing before the main display is even up.

Built-in modes. The screen has a small set of built-in modes Pulse can render, so the behaviour is a defined subsystem rather than a grab-bag:

Modes are selectable (a user default for idle, an app or system override while active), and new ones drop in as render routines without touching the wiring. The DeMon status screen next to the keyboard is the supervisor's own surface — boot progress, FPGA configuration and reflash, recovery and errors — so that is where "what the system is doing" authoritatively lives, since DeMon does the work and owns the screen. The round screen defers to it: during boot it simply shows its identity (the logo), and it can mirror a working animation during the one operation where "don't power off" matters most — an FPGA reflash. Pulse boots its RTOS before FireStorm is up and drives the panel over QSPI directly, so it can animate independently of the FPGA; the authoritative progress still comes from DeMon over the same peer link that carries the chipset badge. Two of them layer on top of whatever mode is current rather than replacing it:

Transient overlays and app use. Two of these borrow the surface on top of the ambient content. When a jog dial that has multiple functions is pressed, Pulse flashes that dial's current-function icon on the screen; a quick second press cycles to the next function and the icon follows, reverting to idle after a moment. Pulse already has the button edges from Cranky and owns the dial-to-function mapping, so this is its to render. And a running application can claim the screen for its own use — a game driving it as a compass, say — by sending Pulse a heading (or a small image) through the supervisor/peer link; Pulse renders it. The surface therefore arbitrates by simple priority: an active app claim wins, a function-icon overlay interrupts briefly, and the ambient logo / audio visual / chipset badge is the fallback when nothing else is asking.

UPDI — Programming Sticky, Cranky and Clicky

Pulse hosts the UPDI programming service that programs all three AVR128DB-family microcontrollers in the system — Sticky (joystick / paddle I/O), Cranky (jog dials) and Clicky (keyboard). Sticky and Cranky sit on Pulse's side of the board; Clicky sits on DeMon's side, but its UPDI line runs across to Pulse too — so Pulse is the single UPDI programmer for the whole machine, and DeMon has no UPDI pin of its own. This UPDI service is the input-chip arm of the Ant64's firmware model — the supervisor paths (C5 and DeMon) live there.

DeMon owns the firmware files and the toolchain; Pulse owns the wire. The two divide work as follows:

UPDI runs at ~225 kbaud half-duplex over UART4 in serialUPDI wiring — the UART's TX and RX joined at the target's single UPDI pin through a series resistor (≈470 Ω to the pin, with the classic SerialUPDI 4.7 kΩ TX pull-up; a 1N5819 across it permits a larger series resistor for reverse-connection protection), so a hardware UART drives the line at full speed with no bit-banging. One UART4 serves all three AVRs, time-multiplexed by the ESP32-P4's GPIO matrix onto each target's UPDI pin — and the same UART4 also carries the CI1303 voice link and a MIDI Thru, so only one of those functions is live at a time. Each AVR is programmed independently, never two at once; a flash takes a few seconds, during which whatever else UART4 was doing pauses.

Scope is firmware update and possible debug only — not runtime control. UPDI also carries OCD-level debug (single-step, breakpoints, register inspection via pyupdi-style tooling), so any of the three AVRs can be put into a live debug session over UART4. Unlike a flash — a few-second blip — a debug session holds UART4 for as long as it is attached, so while you are debugging an AVR the shared CI1303 voice link and the extra MIDI Thru on UART4 are unavailable. Debugging is therefore a deliberate “take the machine out of live use” mode, not something to run mid-performance. Routine joystick data (Sticky → Pulse) and jog dial state (Cranky → Pulse) flows on the same SPI peer links that the AVRs use for normal operation; UPDI is reserved for the comparatively rare update events. The update sticky, update crank and update clicky AntOS commands are the user-facing entry points.

MIDI

• DIN MIDI (MIDI 1.0) — the classic 5-pin protocol; In / Out via UART1, software-driven Thru 1 via UART2, plus UART3 reserved for an optional 2nd MIDI Thru on a future studio-router daughterboard (UART4 now hosts the CI1303 voice board). Each Thru is independently filterable: drop channels, transpose, remap CCs, merge Pulse-generated MIDI in, gate on song state. The control surface for these Thru rules lives in the Workstation app's MIDI panel — rendered into Pulse's MIPI feed and composited on the main display by FireStorm (the round TFT next to the keyboard is far too small, and reserved for jog-dial labels and per-knob status). Per-Thru filter lists, transpose tables, CC remap tables, merge-source toggles, and song-state gating are configured there and saved per project, so a live setup (e.g. Thru 1 = drums routed to a hardware drum machine with cymbals dropped; Thru 2 = bass channel transposed to a second synth) loads with the song. Routed through the Audio Expansion Header daughterboard; populated on the Studio I/O variant that ships with the Ant64C, optional on other models (galvanically isolated optoisolators)
• MIDI 2.0 host (HS port) — a single hi-speed USB MIDI 2.0 host port for a keyboard / controller
• MIDI 2.0 (FS port) — the Full-Speed port is a dedicated, device-only class-compliant USB MIDI 2.0 device: the Ant64 plugs into a computer and appears in a DAW as a MIDI instrument / interface. It never acts as a host, so it stays in device mode permanently — no OTG role-switching, no ID-pin detection, no host/device arbitration

MIDI 2.0 and the DIN ports. MIDI 2.0 negotiates down to MIDI 1.0 automatically, so the USB ports interoperate with any older gear. The 5-pin DIN stays MIDI 1.0 because its 31.25 kbaud line can't carry a full MIDI 2.0 (Universal MIDI Packet) stream — the limit is the transport, not the protocol.

• CI1303 voice board — connects over UART4 (level-shifted 5 V), not USB: command/status + OTA firmware. Pulse also drives a PWM line (GP5 → RC filter → the CI1303's mic input, for injecting audio) and a power-control line (GP4)

Why the two USB ports moved. Pulse's High-Speed USB OTG used to carry FireStorm's mixer-output return link; that path now runs over the dedicated OPI port instead, which frees the 480 Mbps HS PHY for a far better use — a hi-speed USB MIDI 2.0 host port. MIDI 2.0's Universal MIDI Packet stream is bidirectional and high-bandwidth — 32-bit high-resolution controllers, per-note expression, and many logical channels — well beyond what the Full-Speed port or DIN MIDI can carry, so the HS PHY gives it room to breathe. The Full-Speed port, in turn, becomes a dedicated, device-only class-compliant USB MIDI device port — the Ant64 plugs into a computer and shows up in a DAW as a MIDI instrument / interface. It stays in device mode permanently (it never needs to be a host), so there's no OTG role-switching to arbitrate: the two ports have fixed, non-overlapping roles — HS = host (instruments plug into the Ant64), FS = device (the Ant64 plugs into a computer). The CI1303 voice board moved off USB entirely: it hangs off a level-shifted UART4 (command/status + OTA), with a PWM line for injecting audio into its mic and a power-control line (GP4). That power line matters because the CI1303 only checks for new firmware at power-on — so an update means staging the image over UART4, then power-cycling the chip via GP4 to make it pick up the new build.

USB PHY map. The ESP32-P4's three USB interfaces all sit on their default PHYs, so they run simultaneously with no USB_PHY_SEL eFuse change: the fixed USB-Serial-JTAG → DeMon for debugging, the Full-Speed OTG (GP26 / GP27) → the MIDI 2.0 device port, and the High-Speed OTG → the MIDI 2.0 host port.

CI1303 interface circuits

Two small circuits sit between Pulse and the CI1303. Both are provisional — component values want confirming on the bench against the CI1303's actual mic front-end and the chosen PWM carrier.

Power switch — GP4 switches the CI1303's 5 V rail. A 3.3 V GPIO can't switch a 5 V rail directly, so GP4 drives a high-side P-MOSFET through an N-MOSFET level-shifter:

   +5V ──┬──[R1 100k]──┬─────────────── Q1 source (+5V)
         │             │
         │             └── Q1 gate
         │                 Q1 drain ─────► CI1303 +5V   (+ 10µF decoupling)
         │
         └─ Q1 = P-MOSFET (e.g. AO3401), high-side switch

   GP4 ──[R2 10k]──┬── Q2 gate        Q2 = N-MOSFET (e.g. 2N7002)
   (3V3)           │   Q2 drain ────── Q1 gate node
                 [R3 100k]            Q2 source ── GND
                   │
                  GND

• GP4 high → Q2 on → Q1 gate pulled low → Q1 on → CI1303 powered
• GP4 low / floating → Q2 off → R1 holds Q1 gate at +5 V → Q1 off → CI1303 off (R3 guarantees the off-state at boot before GP4 is driven)
• Firmware update: stage the image over UART4, then pulse GP4 low→high — the CI1303 checks for new firmware on the resulting power-on
• One-chip alternative: a load-switch IC (e.g. TPS22918, active-high EN) — GP4 → EN, 5 V in/out, with soft-start built in

PWM audio into the mic — GP5 → RC filter → attenuator → mic input:

   GP5 ──[R1 1k6]──┬──[R2 100k]──┬──[C2 100n]──► MICP  (CI1303 mic +)
   (PWM, 3V3)      │             │
                 [C1 10n]      [R3 1k]           MICN ── mic reference
                   │             │                        (per CI1303 mic mode)
                  GND           GND

• R1 / C1 — low-pass, corner ≈ 10 kHz: removes the PWM carrier, keeps the speech band
• R2 / R3 — ≈ ÷100 attenuator: ~3.3 Vpp filtered signal → ~30 mVpp, i.e. mic level
• C2 — AC-couples the audio so it rides on MICBIAS rather than fighting the DC bias
• Use a PWM carrier well above the audio band (≈ 300 kHz+) so it filters cleanly; if you also want a live mic, put an analog switch (e.g. 74LVC1G3157) on MICP to select injected-PWM vs mic

Values are a starting point: the attenuation ratio and filter corner should be tuned to the CI1303's mic-input sensitivity/impedance once measured.

• MPE support — per-note expression

On-Pulse Synthesis

The 400 MHz HP core, accelerated by the PIE extensions, runs several audio engines whose output is streamed to FireStorm for mixing:

AMY — Primary Synth Library

AMY is an open-source synthesis engine designed to run on the ESP32-P4, and it is a great deal more than a single synth — it is a multi-paradigm engine that covers most of the ground a dedicated workstation synth would. Its building blocks are individual oscillators (≈180 available at once by default, configurable), grouped into voices, managed by synths that handle polyphony, voice allocation, and note-stealing. A patch configures a synth, and AMY can run several synths at once, so it is inherently multitimbral.

Synthesis paradigms:

• Subtractive / analogue — ships with the Juno-106 patch set (presets 0–127): oscillators through resonant filters and envelopes
• FM — ships with the DX7 patch set (presets 128–255) and the full DX7 6-operator algorithms; you can also build your own operator algorithms (ALGO)
• Additive — explicit build-your-own partials mode (BYO_PARTIALS): a stack of sinusoids, each with its own amplitude envelope, plus interpolated-partials voices (the built-in piano, preset 256)
• Wavetable — plays 16,384-sample wavetable packs (e.g. waveeditonline.com) with interpolation across the table's 64 cycles
• Karplus–Strong physical-modelling pluck
• Core oscillators — band-limited saw, pulse/square (variable duty), triangle, sine, noise

Modulation and shaping:

• ControlCoefficients — a flexible modulation matrix: up to nine control signals (constant, note, velocity, two envelope generators, a modulating oscillator, pitch-bend, and two external/CV inputs) are scaled and summed to drive amplitude, frequency, filter frequency, duty cycle, or pan
• Any oscillator can modulate any other — LFOs and audio-rate FM both fall out of this, with additive modulation targets
• Two envelope generators per oscillator, each up to eight breakpoints — more capable than a plain ADSR
• Filters — low-pass, band-pass, high-pass with resonance, per oscillator; plus a per-synth EQ and volume

Sampling, audio-in, and effects:

• PCM sampler — 67 built-in drum/instrument samples, plus runtime load_sample of your own PCM, WAV-file playback from disk with pitching, looping, and stereo/multi-channel support
• Live resampling — sample AMY's own output bus or the audio input into a playable PCM preset on the fly
• Audio input as an oscillator — either channel of a stereo input (AUDIO_IN0/1) becomes an AMY oscillator, so audio can be filtered, enveloped, and effected like any synth voice. On the Ant64 the system audio input is captured at the codec by FireStorm and can be routed to Pulse, so AMY can process the live line/mic input directly — live effects, vocoder-style treatments, and resampling the input into a playable PCM preset
• Effects — echo / reverb
• Sample-accurate sequencer — AMY brings its own timestamped event scheduler and pattern sequencer (48 PPQ, tempo-locked, repeating patterns, MIDI-drum mode), clocked off the audio samples. This is the sequencer engine the Ant64's Pulse Sequencer is built on — Pulse needed a rock-solid timing foundation and AMY provides exactly that; the 16-track / parameter-lock / Euclidean feature set is the Pulse application layer on top of AMY's scheduler
• MIDI processing — AMY supplies the MIDI engine Pulse needs: it maps the 16 MIDI channels onto synths and handles note on/off and velocity, pitch-bend, program-change patch selection (reallocating voices), configurable CC → parameter mappings (per channel, with min/max/offset/log scaling — the mechanism behind MIDI Learn), MIDI drum note→preset translation, and All Notes Off. Pulse firmware wraps this with the physical ports (DIN In/Out/Thru, USB host), MIDI Out/Thru, clock master/slave + MMC, MPE translation, and the routing of MIDI to AMY, the FireStorm Tempest voices, and the DeMon SID engine

AMY voices count against Pulse's CPU budget rather than the chipset's voice count — the two synthesis paths are independent. A typical patch costs a few percent of the HP core per voice, with PIE-accelerated additive partials being the most expensive case. Full detail upstream: AMY synth documentation.

Tracker Engines

For MOD / S3M / IT-style sequencing. Pattern data is held in PSRAM; sample playback can be done either by AMY (PCM voices) or by Tempest voices via OPI parameter writes — whichever fits the patch better.

Classic Synth Emulators

Specific emulations of well-known instruments outside the AMY model — for example, TB-303-style bassline emulators driven by the jog dials.

Speech Synthesis (on DeMon)

Speech synthesis is not hosted on Pulse — it lives on DeMon as the SAM engine, shared across all three CPUs. Pulse uses it musically by sending text or phoneme events over the bidirectional SPI link; DeMon renders the audio and streams it to FireStorm. From the sequencer's point of view, SAM is just another voice destination alongside Tempest voices, AMY, and SID.

Memory Model

The ESP32-P4 has 32MB of embedded PSRAM mapped into its address space, plus 16MB of external SPI flash. There is no separate FRAM (the earlier FRAM is gone — its role is taken by PSRAM for working data and flash for non-volatile storage).

A configurable region of PSRAM is reserved as the MIPI framebuffer, accessed by the MIPI engine via DMA. The remainder holds:

• AMY patch banks and per-voice state
• Sample libraries
• Sequencer / pattern data
• Working buffers for software synthesis (AMY audio output queues, etc.)
• Any large data structures the firmware needs

Communication Summary

Prototype

The prototype uses an Olimex ESP32-P4-PC board. The mipi dsi also goes to a hdmi adapter, io36 is pulled to 3v, io35 (boot) is pulled to 3v. The JTAG USB goes to the DeMon for debugging. The full USB is the USB-MIDI device port. The HS USB is the hi-speed MIDI 2.0 port in production; on this standalone prototype it can be looped to a test USB device for bring-up.

GP45 is SD_PWR_EN on the prototype board

Pins not available right now (past prototype use):

GP2 is the Green USER_LED on the prototype board

Boot control: GP35 is to a button (Boot1), and is tied via 10k to 3v with this set to 1, it's SPI boot mode and GP36, GP37, GP38 are don't care GP36 is tied via 10k to 3v GP37 GP38

The USB debug port (GP24/GP25) has USB1P1_0P and USB1P1_0N, which can be used to program and debug the ESP32 P4, presents as a serial port and jtag! Connected to a port on the DeMon USB2 hub. SPI2 is an OPI master to FireStorm, it's also a master to Sticky, Cranky, and a round screen.

Reference Links

• ESP32-P4 — espressif.com/en/products/socs/esp32-p4
• PicoADK Hardware — github.com
• ml_synth_organ_example — github.com
• Audcalc — github.com