Accelerator — the always-available graphics chipset

Graphic output — sprites, tilemaps, particles — renders through one of two backends: the FireStorm FPGA chipset (hardware — fast, native to the loaded retro system) or the software chipset, rendered by DeMon at ImGui resolution and composited with the FPGA output — above it as an overlay and below it as a backdrop, using the CM5's two video outputs. The software chipset is always available, whatever FPGA image is loaded, and can enhance any simulated retro system from both sides — HUDs and effects on top, scenery and parallax behind. It is the graphics side of the Accelerator — the same layer that redirects a personality's peripherals to native hardware also provides its graphics.

Status: design. Not built yet.

Two backends, one graphics family

Exactly as the ee library runs the EE on real silicon or a software model behind one API, the graphics family renders on hardware or software behind one API:

• Hardware — the FireStorm chipset. Fast, FPGA-native sprites and layers, bounded by the loaded chipset (sprite count and size) and tied to the current FPGA image.
• Software — the standard software chipset. Rendered by DeMon at ImGui resolution on two planes — an over-plane above the FPGA output and an under-plane below it (see compositing). No FPGA image required; the same calls, a different renderer underneath.

A graphic object picks a backend (or takes the default); the API above it is identical, so code doesn't care which drew it.

Targeting a plane

Every OS graphic API except ImGui — sprites, tilemaps, particles, any software-rendered output — chooses its destination among three: the hardware chipset (FireStorm), the software top plane (over the FPGA), or the software bottom plane (under it). ImGui is the exception because it is the OS UI — it always lives in the top frame, top-most, so windows and menus sit above everything.

So the same libraries place things anywhere in the sandwich by a target argument: a sprite is a HUD element on top, a tile layer is parallax scenery on the bottom, a fast sprite is a hardware object in the FireStorm chipset — and the UI floats over all of it.

Always available — and it wraps anything

The software chipset is there whatever the FPGA is outputting — full FireStorm, a Spectrum personality, a bare test image, or nothing. And it composites on both sides of that output, so it can enhance a simulated retro system from above and below — on top, sprites, particles, a HUD, effects; behind, backgrounds, parallax, a skybox the original never had. The emulated machine is unmodified in the middle; the enhancement is software, wrapped around it.

A standard chipset

"Standard" means a fixed, defined capability set — sprite dimensions and counts, tilemap format, palette, layer count — that every Ant64 provides in software. Software written to it is portable and always runs, independent of whatever chipset the current FPGA image carries. The hardware backend may exceed the standard (more sprites, more bandwidth); the software chipset is the guaranteed baseline every machine has.

Measuring the standard — then mapping it

A real advantage of the software backend running on the CM and out over HDMI: its budget can be measured before the FPGA chipset exists. Render the over- and under-planes on the CM, push them out both HDMI links, and count what a frame actually affords at the target resolution and rate — how many sprites, how many tilemap layers, how large a particle budget. Those measured numbers define the standard, instead of being guessed and hoped into silicon later. The Accelerator is a classic-resolution chipset: its register map, sprite and tile coordinates, and framebuffer live in a retro resolution space — a base around 320×240, with the classic variants (320×200, 320×256, 256×224/240, 256×192…) — and the FPGA integer-upscales to the modern output for crisp, chunky pixels (4:3 content taking borders in a 16:9 frame, as it should: 240 lines → 720 is ×3, → 2160 is ×3 again). So "how many sprites per frame" is measured at classic resolution — ~1/12 the pixels of 720p, so cheap to render, landing the standard's budget squarely in, and above, classic-chipset territory. The CM↔FPGA HDMI link is just transport at a valid mode (the crisp ImGui UI rides it at 720p; the classic-res graphics ride within, integer-scaled), and the FPGA does the real scaling to the display — so the CM never renders at display resolution, which also keeps it inside the dual-HDMI ceiling (a CM4 does dual-1080p60, a CM5 far more). (It is the same "reference implementation first" move the EE simulator makes for the CPU — build the software version, learn the real numbers, then commit the hardware to them.)

Two things get pinned from it:

A register address map. The Accelerator is a chipset — from the FPGA's side, an accelerator chip — so it is programmed through an MMIO register map: a sprite table, tilemap descriptors, particle emitters, palettes, and global controls, exactly as a real chipset is. The software Accelerator implements that map now; the FPGA chipset implements the same map later — so code written to it runs on either, the way the ee library runs on the model or the soft core behind one interface. Measured on the software chipset, it runs on the FPGA unchanged.

The capability set the map exposes — the features a chipset of this lineage wants, several of them beyond what the vintage hardware could manage:

• Sprites — a bounded count (set by the benchmark), each with position, a palette, rotation, scale, flip, priority / z, and an optional parent (chains and trees — a child's transform is relative to its parent, composed up the chain).
• Tilemap layers — a bounded number, each a grid over a tileset, with per-layer scroll and palette.
• Particles — a bounded emitter and particle budget.
• Palettes — a defined, shared set, indexed by sprites and tiles.

The FPGA chipset is free to exceed the standard (more sprites, higher bandwidth); the software Accelerator is the guaranteed floor every Ant64 meets — and, because it is measured rather than assumed, a floor known to be real on the actual hardware. Once the numbers settle they deserve their own register-map spec (the graphics counterpart of the EE ISA reference), which the sprite / tilemap / particles libraries and the FPGA chipset both target.

From the FPGA side — an accelerator to the chipset

Everything above describes this from AntOS's side — a software graphics chipset. From the FPGA's side it is something more pointed: an enhanced accelerator chip wired to the image's own chipset. Whatever runs on the fabric — FireStorm's chipset, or a personality's native core — sees its output composited with the accelerator's over- and under-planes exactly as if an accelerator chip sat beside it, adding graphics the native chipset can't: extra sprite and tile planes, particles, effects, at ImGui resolution, above and below its own layers.

So this is not an OS overlay bolted on from outside — through the Accelerator module every image wires up to, it is, from the chipset's point of view, part of the graphics hardware: an accelerator that augments what the native chipset draws. It is the graphics counterpart of the peripheral redirection — the same Accelerator that stands in for a personality's Wi-Fi, clock, and storage also stands in as its graphics accelerator, and (as accelerator argues of the whole idea) vastly out-classes the board it replaces.

The sandwich — over and under

The display is a three-plane composite, and FireStorm is the compositor (it already ingests DeMon's video and drives the display):

under-plane (software, via the CM5's second HDMI) → FPGA chipset (the retro system, middle) → over-plane (software, DeMon's primary HDMI) → OS UI

• The over-plane is the overlay: HUD, extra sprites, particles on top of the guest.
• The under-plane is the backdrop: it shows through wherever the middle plane is transparent — for FireStorm's own chipset that's alpha; for an emulated guest it's a colour-key (key a Spectrum's border/paper colour, say, and the parallax layer beneath shows through exactly where the original was empty).

The under-plane repurposes the CM5's second HDMI, which today is the CM→FireStorm data-as-video link (cm45: HDMI1, the data channel). FireStorm already ingests it, so this is a reuse of an existing input, not a new one — the real cost is the three-plane compositor and reading HDMI1 as a video plane. The data role it gives up moves to the bidirectional PCIe — a better home for a data bus anyway. Mind the budget, though: PCIe is one CM lane (~400 MB/s on a CM4 Gen2, ~800–1000 MB/s on a CM5 Gen3), now carrying what the HDMI data link did plus the register comms — so the move is comfortable on a CM5, tight on a CM4. The three planes must also be frame-synced. In return the software chipset can put scenery behind a retro game, not just in front of it.

Plane resolutions — a fixed top, a flexible bottom

The two planes are not symmetric, because only one carries the UI:

• The over-plane stays a constant 720p. ImGui lives here and wants a stable resolution — a modeset would disrupt the UI — so the top link never changes mode. The Accelerator's top-plane graphics are authored at their classic resolution and scaled up to 720p to share the frame with the crisp UI; the FPGA then scales the 720p to the display.
• The under-plane can change resolution in hardware. It carries no UI, so the bottom link is free to modeset to whatever suits the content — a different classic resolution, an unusual aspect, a wide mode — a brief link retrain being harmless on a background layer. The FPGA scales whatever arrives.

A future the flexible bottom enables — layers-across packing. Because the bottom link's mode is free, it can run a wide, short frame — high horizontal resolution, low vertical — and lay two or four layers side by side across the width. The FPGA reads the one HDMI frame as several separate layers and inserts them between its own FPGA layers at chosen depths. Four 320×240 layers packed into a 1280×240 frame is ~307k pixels — a third of a 720p frame — so the software chipset hands the compositor several interleavable layers for less bandwidth than one full plane, turning the single under-plane into a small stack the FPGA threads through its native layers. (A later addition, not v1 — but the fixed-top / flexible-bottom split is what leaves room for it.)

Alpha on the top plane (over HDMI)

The top plane must be transparent where nothing is drawn, so the FPGA output and the bottom plane show through — but HDMI carries RGB and no alpha channel. (The bottom plane needs none: it is the opaque base.) Since both ends are ours, the pixel format is a private CM↔FireStorm agreement, selected by a mode number:

Mode 0 — chroma key. The key is RGB (0,0,0), which makes transparency free: clearing a framebuffer to zero is the cheapest fill the GPU / HVS can do, so the app clears to 0, draws opaque, and every untouched pixel is already the key — no alpha channel, no conversion pass, the fastest the CM can go. True black is rendered (0,0,1) — one unit on the least-perceptible blue LSB — so it never collides with the key. Full colour, 1-bit transparency; edges are hard (an anti-aliased edge fringes dark), ideal for retro sprites and hard-edged UI. The default.

Mode 1 — 24-bit 6:6:6:6. Reinterpret the 24-bit pixel as six bits of colour per channel plus a 6-bit alpha (64 levels). Smooth per-pixel alpha on the standard link with no extra bandwidth, at a slight colour-depth loss (262k colours).

Mode 2 — 30-bit 8:8:8:6. If the CM outputs 10-bit (RGB 10:10:10) deep colour, the extra six bits carry the alpha with no colour loss — full 8-bit colour and 6-bit alpha. The CM4 / CM5 can do 10-bit (the HDR10 output path, via a KMS 10:10:10 framebuffer format); the top link runs 30-bit (~+25% pixel bandwidth, comfortable at the top plane's resolution) and FireStorm's RX unpacks R8 G8 B8 A6.

Fast path. Mode 0 rides the CM with zero overhead — the clear colour plus opaque draws over the free HVS / DRM-KMS plane composite. Modes 1 and 2 add exactly one full-screen V3D pack pass over the composited ARGB8888 frame — cheap at the top plane's resolution, but a pass mode 0 avoids. So mode 0 is the default; step to mode 1 for smooth alpha on any link, or mode 2 when a 10-bit link is confirmed and you want smooth alpha with no colour loss. (HVS plane count, per-pixel alpha, and whether the pack can ride the HVS differ between the CM4's VideoCore VI and the CM5's VII — a DRM/KMS probe for bring-up.)

The mode field

The mode is a single byte, split into two nibbles so both planes' formats live in one register:

The top plane's three modes are those above; 3–F are held for future top-plane formats. The high nibble is reserved for the bottom plane — it carries no alpha today (it is the opaque base), but reserving its nibble means a future bottom-plane format (a different packing, a keyed backdrop, a lower-depth mode) slots in without disturbing the top-plane encoding.

On the stage

The family's targets are drawn in a render pass after React, through each object's backend; the software chipset advances and draws its sprites, tilemaps, and particles at ImGui resolution each frame. Rendering reads settled target state and writes pixels — it changes nothing the simulation depends on, so determinism is untouched.

Audio — over HDMI, into the mixer

The CM reaches FireStorm over HDMI, and HDMI carries audio as standard — up to eight channels of PCM in the stream's data islands, alongside the video. So the Accelerator delivers the CM's audio the same free, standard way it delivers graphics: AntOS sounds, the Triple SID engine, a script's or a game's audio rendered on the CM — all embedded in the over-plane HDMI stream, pulled out by FireStorm's HDMI RX (the same receiver already taking the video), and dropped into the FireStorm mixer beside the Tempest voices and Pulse.

The CM's HDMI audio is one source into the accelerator system, not the only one. Pulse — the ESP32-P4 audio processor — is another, contributing its own AMY synthesis and MIDI (decode, sequencing, live control) over its MIPI link. So the accelerator's sound aggregates three streams at the mixer — the CM over HDMI, Pulse (audio + MIDI), and FireStorm's own Tempest voices — the same "many sources, one mixer" shape the audio subsystem already has, the CM now joining it as a first-class source over the Accelerator's HDMI link.

No extra link, no custom protocol — it is ordinary HDMI audio, which every part of the chain already speaks. So from the FPGA's side the accelerator chip provides not just extra graphics planes but an audio source into the chipset's mixer: the CM's sound is as much part of the Accelerator as its sprites and particles.

Clients

• Luau — require("accelerator"): the default backend, capability query, and composite settings; the graphics libraries themselves expose the per-object backend choice.
• The OS composites the software-chipset layers over the FPGA output.

Related

• FireStorm — the hardware chipset it mirrors · sprite / tilemap / particles — what it renders · antos_ee — the same hardware/software backend pattern · AntOS — the compositor
• luau_accelerator — the Luau API