Composer — Display Compositing Architecture

Status: design record / forward direction. The supervisor side is firm; the FireStorm side (compositor, sprite unit, tile fetch, font ROM, genlock) is not built yet. This records the architecture and the data contract between the two.

The one-line idea:

The supervisors — DeMon (a CM5) and Pulse (an ESP32-P4) — keep the whole scene in their own RAM and do the high-level work. Each frame they hand FireStorm a display list of what's on screen. FireStorm draws it.

1. Division of labour

The supervisors are fast processors with room to hold a large, live scene — Pulse a 400 MHz dual-core RISC-V (PIE SIMD, 32 MB PSRAM), DeMon a quad-core Raspberry Pi CM5 (gigabytes of RAM). That is enough to hold a large, live scene — thousands of sprites with their animation and tweening state, multi-layer tilemaps far bigger than the screen, particle systems — and to advance it every frame. None of that lives in the FPGA (the GoWin GW5AST-138).

What the supervisor sends FireStorm is neither pixels nor the whole scene: it is a compact description of the current frame — a display list of the sprites and a tile-index map of the tiles that are actually visible. FireStorm renders glyphs, fetches tiles, samples sprite textures, applies palette / scale / flip, and composites the layers at scanout. The supervisor owns the scene; FireStorm owns the pixels.

This keeps any pixel framebuffer off the supervisor side — no software palette expand, no upscale, no per-pixel blit. The supervisor maintains state and emits descriptors; the FPGA does the drawing at output-pixel rate in the scanline mixer.

There are three kinds of description, one per content type:

Text — a character cell grid (§4).
Tilemap — a tile-index map over a tileset (§5).
Sprites — a display list over a texture pool (§6).

They ride the two supervisor feeds (DeMon over HDMI, Pulse over MIPI into FireStorm's RX), frame-locked to FireStorm's system vsync. Each feed can carry any mix, and FireStorm composites them with its own native layers (the running application, the native sprite and tilemap engines, the Copper).

The two feeds are equal in link rate but not in role: DeMon is the busy general-purpose host (AntOS, scripting, networking, storage), while Pulse runs a dedicated real-time RTOS with no user scripting. In practice Pulse is the heavier, faster data pump to FireStorm — it has the cycles to spare for that role. Pulse reaches FireStorm over a OPI side-bus; DeMon over PCIe.

2. Send only what's on screen

This is the whole trick. The supervisor may be tracking thousands of sprites and a tilemap far larger than the display, animating and tweening all of them — but FireStorm only ever needs the ones inside the current screen window. So each frame the supervisor culls to the visible window and sends just that subset: the sprites whose bounds intersect the view, and the tile rows and columns the view currently covers. FireStorm never learns about the rest.

The consequences are the point of the architecture:

FireStorm's per-frame cost is bounded by what's visible, not by the size of the scene. A 10,000-sprite world and a 100-sprite world cost the FPGA the same when the same number are on screen.
The visible working set fits on-chip. Because only the on-screen window is sent, the tile-index map and the tiles it references are small enough to be streamed into FireStorm's on-chip BSRAM — the fastest, lowest-latency memory on the FPGA — rather than DDR3. Scanout fetches them at single-cycle latency, and the DDR3 bus is left free for the application's framebuffer and bulk assets. How much of the window sits in BSRAM versus DDR3 is an FPGA-side budget call (§8).
The scene can be as big as the supervisor's RAM allows. Off-screen entities, spawn pools, the rest of a scrolling map, particle systems winding up before they enter — all live in supervisor RAM at no cost to the FPGA.
Scrolling is a window move. The map doesn't move, the visible window does; the supervisor streams only the rows and columns that scroll into view. A display list of the on-screen sprites plus a screen-window tile map is a few KB a frame.
Animation, tweening, physics and particle simulation are supervisor work — high-level, branchy, per-entity logic the processor is good at — and reduce each frame to a flat list of "this texture, here, this orientation" that the FPGA is good at.

3. Frame timing and delivery

Two things happen on different clocks and must not be confused. Delivery is the supervisor getting the next frame's description into FireStorm's reference RAM; render is FireStorm reading that resident data as the beam scans. They never touch the same frame's data at the same time, and that separation is what keeps the picture coherent.

Composite is per-scanline, from resident RAM. FireStorm holds no full composited framebuffer. As the beam walks each line, the scanline mixer samples every enabled layer at that pixel — palette-resolve, scale, colour-key, z-blend — and emits one output pixel per clock. The display list, tile-index map and visible-window tiles it reads must therefore be resident before the line is scanned, not fetched as the beam arrives. Render is just-in-time; delivery is ahead-of-time.

A frame ahead, promoted at vblank. The supervisor runs a frame in front of FireStorm: while FireStorm scans frame N out of one bank of reference RAM, the supervisor fills the next frame's description into a second (back) bank, and FireStorm's system vsync pulse promotes that bank to live for frame N+1. Filling spans the whole active period, not the narrow blanking gap — at 720p60 vblank is only ~3 % of the frame — so the transfer is never crammed into the blank; the swap is the only thing that lands at vblank. Each frame's data is thus whole and stable for that frame's entire scan: no tearing, no half-updated list, no sprite moved without its texture pointer.

The vsync pulse is also the genlock heartbeat. The two feeds are independent links, so the single shared pulse is what makes them agree on "frame N+1 starts now" — both supervisors fill their own back banks across the frame and both promote on the same pulse. (The back-channel protocol that carries this is still to specify — §8.)

Bandwidth — per supervisor, on its own link. Each supervisor drives its own link into FireStorm — Pulse a 2-lane MIPI link at 3 Gbps total (~375 MB/s, ~6 MB per 60 Hz frame) into its RX hardcell, DeMon over 2× HDMI into the fabric — completely separate paths with no sharing, so each supervisor has ample per-frame budget to itself. Against that, a frame's description is a rounding error: the sprite display list and tile-index map are a few KB, the visible-window texel set bound for BSRAM tens of KB. Delivery is never the bottleneck. The real limit is local and per-line: how many already-resident texels FireStorm's fetch unit can pull within one line's pixel-clock budget (the per-scanline sprite limit — §8). Delivery is bounded vblank-to-vblank, a whole frame wide; fetch is bounded by the scanline. They are separate budgets and should be reasoned about separately.

The link carries more than the frame, prioritised under it. The frame description has the hard vblank deadline, so it is the highest-priority traffic on each link and is always sent first; the wide spare bandwidth then carries lower-priority streams that have no per-frame deadline. On DeMon that is network and storage data for the FireStorm application — packets, file and DBFS reads, asset and OTA payloads; on Pulse it is controller data — aggregated jog-dial and joypad state — and the like. These ride beneath the display description and yield to it: they fill the gaps a frame leaves and never delay one. (Each supervisor also has a dedicated side-bus to FireStorm — OPI on Pulse, PCIe on DeMon — carrying latency-critical control and small packets in parallel with the display feed, and the primary path during bring-up; the display feed carries the bulk, prioritised under the frame.)

4. Text layer — the cell grid

The supervisor streams the character cell grid; FireStorm renders glyphs from its own font ROM during scanout — the classic hardware text-generator model, and the same one FireStorm's native text layers use. The grid is sys_text_cell_t × cols × rows (4 bytes/cell): a full 64×30 grid is ~7.5 KB, and dirty-cell streaming drops most frames to a few hundred bytes. Cell size, glyph tiling and the font become FPGA font-ROM concerns, not supervisor ones. See AntOS Text Architecture.

5. Tilemap layer — the tile-index map

A tilemap is the cell grid generalised: each cell holds a tile id into a tileset plus its palette bank and h/v flip, and the layer carries a coarse + fine scroll offset. The supervisor streams the index map for the visible window; FireStorm fetches each tile from the tileset and composites it — the standard hardware tilemap model (graphics), with the map authored by the supervisor rather than written to chipset registers. Tiles use the shared texture format (§7), so the tileset and the sprite texture pool are the same memory. A tile-index cell is a few bytes; scrolling moves the window, not the map.

The supervisor tilemap suits UI-scale and scene grids; full-screen game backgrounds at high update rates can equally stay on FireStorm's native tilemap engine, driven by the running application. Same renderer, two authoring paths.

6. Sprite layer — the display list

The supervisor computes the scene and ships a display list: per visible sprite, a descriptor of id, x, y, scale, flip, rotation, palette bank, z-order. Sprite art lives in FPGA texture memory; the hardware sprite unit composites from it at scanout. The supervisor never touches sprite pixels — it maintains state and emits descriptors, the standard console GPU model and the same hardware as FireStorm's native sprites. Transforms are power-of-2 scale, h/v flip and 90° quarter-turns; the classic type carries no arbitrary angle.

7. The shared texture format (the contract)

Sprites and tiles read the same thing from FPGA texture memory:

4bpp packed, 16×16 — 2 px/byte, 128 bytes per tile/sprite frame; halves source-read bandwidth, doubles cache density.
16 palette banks × 16 entries — texel nibble v, bank b → global index (b << 4) | v; the 256-entry table is exactly 16×16.
Index 0 is the colour key — per bank, per texel, at source resolution.

That format is the contract between the supervisors and FireStorm texture memory, and it does not depend on any of the open decisions below. So the software 4bpp blit is the executable spec: prove correct sprites and tiles from packed 4bpp + bank + key in C, then cast it into the FPGA. Prove the data format before committing logic to silicon.

8. Open decisions

Genlock back-channel. The vsync-paced delivery in §3 needs a real sync protocol between FireStorm and the two supervisors: what the pulse carries, how a supervisor recovers if it misses a frame, how the back-channel rides the MIPI link. The mechanism is settled — FireStorm is the genlock master and both feeds promote on the one shared pulse (§3) — but the protocol is not yet specified, and it shapes the MIPI link design. This is what makes or breaks two-feed compositing.
Sprite render split. How much geometry the supervisor pre-computes — a full per-sprite descriptor for FireStorm to transform, vs handing over pre-transformed coordinates for a simpler FPGA. Trades FPGA LUTs against supervisor cycles; decide once the sprite-unit budget is known.
Texture / tileset residency. Because the in-view subset is small, the strong direction is to stream the visible window into FireStorm's on-chip BSRAM for single-cycle scanout fetch and a free DDR3 bus, while the full asset store — every tile and sprite, on- and off-screen — stays in DDR3 and only the in-view art is promoted each frame. How much BSRAM to budget (and double-buffering it costs two copies — §3), when to fall back to DDR3, and the per-scanline sprite limit are FPGA-side.