Ant64 Sound Architecture

The Ant64 audio system is not designed merely to play sound — it is designed to invite musical creation.

The Lineage

Historically, several home computers unexpectedly became instruments:

In each case, the designers did not explicitly set out to build a "music workstation". Instead, they created systems where sound was immediate, programmable, constrained, and deeply tied to the machine itself.

Ant64 intentionally follows this lineage — not by emulation, but by evolution.

Sound on Ant64 is...

Hardware-first. Deterministic. Memory-mapped. Deeply programmable. Capable of misuse and experimentation.

These qualities are essential for a system to become a creative platform rather than a closed appliance.

Architectural Overview

Ant64 audio is generated in multiple places and mixed in one. The primary audio generator is Tempest — the FireStorm chipset audio engine, the umbrella for everything audio inside the FPGA (the 128+-voice synthesis engine, the audio DSP, and the hardware mixer) — sitting alongside the FireStorm Execution Engine. Pulse adds a second software-rendered synthesis path. The FireStorm EE can also generate audio directly. All of these meet at Tempest's mixer, which drives the on-board WM8960/62 codec (speakers, headphones, phono, HDMI audio, audio in) and a parallel I²S path to the Audio Expansion Header daughterboard for a clean dedicated line out, optical S/PDIF, and an optional I²S ADC line direct to Pulse.

Generation sources:

• Tempest — the FireStorm chipset audio engine (in the FPGA) — the primary, high-voice-count workhorse. Up to 128+ voices of hardware-rendered sample / FM / granular / physical-modelling synthesis. Generation and mixing both happen in FPGA fabric at the codec's sample rate. The engine is memory-mapped: the FireStorm EE writes voice parameters at MMIO speed (single-cycle latency, no IPC), and external chips with register windows (Pulse over OPI, DeMon over PCIe) can write the same registers from outside the FPGA.
• Pulse Sequencer (ESP32-P4 + 32MB PSRAM) — MIDI decode, sequencing, live control, the sequencer UI rendered as a MIPI display layer, and software synthesis on Pulse itself. The AMY open-source synth library runs natively on the ESP32-P4 (accelerated by its PIE extensions), providing additive, FM, PCM, partials, filters, and envelope generation. Audio rendered on Pulse is streamed to FireStorm via the MIPI bulk-data path, where it joins the Tempest voices at the mixer.
• DeMon Triple SID engine (software on DeMon, the CM5) — three software SID chip emulators (MOS 6581 / 8580) panned Left / Centre / Right. 9 SID voices total with the characteristic Commodore 64 timbre. The SID engine is a shared audio resource — although it lives on DeMon, it can be driven by any of the three CPUs in the system: AntOS (system sounds, scripts, SID file player), Pulse (sequencer-driven SID voices via the bidirectional SPI link to DeMon), and FireStorm EE code (game / demo / app sound effects, via a chipset event mailbox + IRQ to DeMon). Audio streams to FireStorm as standard HDMI audio over the Accelerator's HDMI link (DeMon's register and control writes use PCIe) and joins the mixer alongside everything else.
• FireStorm EE applications — games, demos, music players running on the FireStorm EE (in the FPGA) can generate PCM audio directly using software synthesis and route it through the same mixer.

Coordination:

• FireStorm EE (RV64GC in the FPGA) — composition, analysis, top-level synthesis setup, Xmath-accelerated DSP inner loops, MMIO-direct access to every Tempest register
• Copper & Update Units – real-time, timeline-driven audio manipulation, modifying audio registers at exact sample positions

Tempest is not a fixed "sound chip" — it is a reconfigurable, multi-paradigm audio fabric that generates and mixes. AMY on Pulse extends the palette with patch-style synthesis for timbres that don't map naturally to the chipset's fixed sub-engines.

Voice Architecture

Polyphony

The baseline Ant64 configuration supports 128 simultaneous hardware-mixed voices, with headroom for more depending on sample rate, effects usage, and synthesis mode.

Voices are not uniform; each voice can independently operate in one of several modes:

Voices are mixed in hardware, with per-voice spatialisation and effects routing.

Voice Components

Each voice may include:

• Amplitude Envelope — ADSR or multi-segment
• Pitch Envelope
• Per-voice filter — LP / HP / BP, resonant
• Per-voice LFOs — multiple, routable
• Stereo positioning
• Velocity and aftertouch modulation
• Copper-controlled parameter overrides

This mirrors — and in many cases exceeds — the per-voice control found in modern synthesizers.

Voice configurations are organised as patches — combinations of oscillator paradigm, filter routing, envelope assignments, mod-matrix bindings, and a small pool of opt-in character extras (currently the accent-RC accumulator and the mismatched-ladder mode). Classic sounds — TB-303 acid, 909 drum kits, Juno pads, M-86 hoover, DX7 patches, Minimoog leads — are all reference patches against this voice topology, not dedicated engines. See patches.md for the patch model, file format, reference library, and the unprecedented combinations the architecture makes possible.

Synthesis Modes

Sample-Based Synthesis

DDR3 / Flash → Sample Engine → Pitch / Rate Control → Mixer
                                      ↓
                               Loop / Crossfade / Granular Slice

• High-quality samples stored in DDR3 or flash
• Variable playback rate
• Independent pitch and duration control
• Looping, cross-fades, and granular slicing

This enables Fairlight-style instruments, MOD/XM-style trackers, and modern sampler workflows.

FM Synthesis (Enhanced)

Ant64 supports FM synthesis with sample-based operators — not limited to sine waves.

Operator A (sine / sample / procedural)
    |  modulates
Operator B (sine / sample / procedural)
    |  modulates
Operator C --> Output

• Configurable operator graphs
• Feedback paths
• Envelope and LFO modulation per operator

This allows:

• Classic DX-style FM
• Acid-style basslines (303-like)
• Hybrid FM/sample textures impossible on vintage hardware

Granular Synthesis

Granular synthesis treats sound as a stream of tiny grains — each independently controllable.

Implemented using the existing audio update units:

• Samples are divided into grains
• Grain position, duration, pitch, and density are hardware-controlled
• Real-time modulation via Copper or Pulse

Enables:

• Time-stretching without pitch change
• Textural soundscapes
• Evolving pads and ambiences

Phase Vocoder / Time-Domain Processing

Ant64 supports pitch-independent time manipulation using hardware-assisted resampling and phase-aligned processing:

While full FFT-based vocoding is handled in software (on the FireStorm EE using Xmath primitives, or offloaded to DeMon — the CM5 — for heavier loads), the FPGA provides the real-time primitives required for live playback.

Effects Architecture

Available Effects

Routing is flexible and fully programmable.

Reverb

Implemented using multi-tap delay lines, feedback networks, and damping filters:

• Room, hall, and plate-style reverbs
• Low-latency — suitable for live play
• Minimal FPGA resource usage

Reverb parameters can be Copper-controlled, LFO-modulated, or MIDI-automated.

Copper-Controlled Audio

One of Ant64's most distinctive features.

Just as the Copper can alter graphics registers per scanline, it can manipulate audio at sample-accurate precision:

• Change voice parameters at precise times
• Modulate filters rhythmically
• Trigger events sample-accurately
• Synchronize sound tightly with visuals

This enables effects that are extremely precise, impossible in software alone, and characteristic of the best classic hardware systems.

Pulse Sequencer Integration

The Pulse Sequencer (ESP32-P4 + 32MB PSRAM) acts as the real-time musical interface — and a second synthesis source alongside the chipset.

Connectivity

• USB MIDI
• DIN / DOUT / DTHRU MIDI (Creative Edition)
• Analog control pots
• Mouse and external controllers
• Live audio input

Responsibilities

• MIDI processing — handled by AMY's MIDI engine (channel→synth mapping, program change, CC mapping); the physical DIN/USB ports and routing live in Pulse firmware
• Sequencing & clocking — built on AMY's sample-accurate sequencer engine
• On-Pulse synthesis via the AMY library — a full multi-paradigm engine: subtractive (Juno-106 patches), FM (DX7 patches + algorithms), additive partials, wavetable, Karplus-Strong, and a complete PCM sampler with live resampling and audio-in-as-oscillator, all accelerated by ESP32-P4 PIE extensions
• Tracker engines and classic synth emulators (speech synthesis lives on DeMon as the SAM engine)
• Control data routing to FireStorm Tempest voices
• Audio buffer streaming to FireStorm for mixing

Pulse communicates with FireStorm via two paths: OPI for low-latency control writes (note on/off, parameter updates to Tempest voices), and MIPI (2-lane, 3 Gbps) for streaming AMY / emulator audio output and bulk sample uploads. Both paths converge at FireStorm's mixer, which combines them with the chipset's hardware-rendered voices before driving the codec.

Analysis & AI-Assisted Features

Audio analysis is handled in software on the FireStorm EE — its Xmath extension provides single-cycle saturating MACs, fast FP approximations, and trigonometric primitives well-suited to FFT, autocorrelation, and feature extraction at modest rates. For heavier ML-based analysis (transcription, instrument recognition, style transfer), it can be offloaded to DeMon (the CM5) — a quad-core ARM with a Linux ML toolchain — via the accelerator offload path.

Audio In --> FPGA Capture --> FireStorm EE (Xmath)  
                                  |                  
                       Note detection / Transcription
                       Pattern recognition           
                       Intelligent sequencing        
                                  |                  
                    (optional offload) --> DeMon (CM5)
                                          (heavier ML)

The FPGA does not perform heavy analysis — it focuses on deterministic sound generation.

Comparison to Modern Synthesizers

Creative Intent

Ant64 is designed so that unexpected uses of sound are not only possible, but encouraged.

Just as the C64, Atari ST, and Amiga were adopted by musicians for reasons their designers did not fully anticipate, Ant64 aims to create the same conditions:

1. Powerful but understandable hardware — you can reason about what it does
2. Constraints that inspire creativity — limits breed innovation
3. Direct access to sound generation — no abstraction layers between you and the hardware
4. Tight coupling between audio, visuals, and interaction — sound is part of the machine, not an addon

The goal is not merely to reproduce classic sound — but to create the environment in which the next musical subculture can emerge.

Click here for in-depth audio plans »