AntOS Library — `imgui` (reference)

The imgui library's design and native core — the scripting interface that turns AntOS's shared Dear ImGui context into a multi-client windowing system. The script-facing API is luau_imgui.

Design-stage. AntOS already renders one ImGui context (the desktop overlay: top menu, settings, wiki reader, notifications). This library is the proposed layer that opens that context to scripts and system panels alike. The native pieces (sys_imgui, the netImgui transport) exist; the multi-client registration/dispatch layer is the new work.

What it is

One ImGui context, many clients. A client (a script or a system panel) registers a window with a per-frame draw callback; AntOS calls every registered callback inside its frame. The desktop becomes the window manager; the built-in panels are simply its first clients.

Why ImGui as the window manager

ImGui is immediate-mode: a window is redrawn every frame from the client's state, not created once and mutated. That is exactly why this is a small addition rather than a new subsystem — the window manager (layout, docking, z-order, focus, theming) is already written and running for the overlay. Opening it to clients means letting them contribute per-frame content, nothing more. The alternative (a retained-mode windowing API with widget objects, invalidation, and a scene graph) would be far more code and would fit AntOS's live-scripting ethos far worse.

Design notes

The callback is the window. There are no persistent widget handles; a client emits its window's contents each frame and keeps its own state between frames. This matches how Luau scripts already work (state in closures/tables) and keeps the API tiny.
Execution model — the terminal is the main window; parallel VMs get their own. A normal script runs in the main terminal window (the large window covering the background, owned by the primary VM) — text output, the default surface. To put a graphical window on the desktop, a script starts a parallel Lua VM task, and that task gets its own window(s): it holds the window's state, registers its draw callback, and lives as long as the window is open (imgui.run(), or alongside its other work). The console is therefore just the first client of the windowing system — even the terminal is an ImGui window.
One VM per task, drawn cooperatively. Each parallel task is its own Luau VM (not one shared VM), so tasks are isolated — a crash or runaway loop in one can't corrupt another's state. The scheduler multiplexes windowed tasks onto the render thread and resumes each one's draw per frame, so ImGui is only ever touched by one VM at a time. Exactly how a task's draw reaches the frame — and why it can't just call ImGui from its own thread — is the Concurrency section below.
Threading is the constraint to respect. ImGui has a single, non-thread-safe context; all widget calls must happen on the UI thread within one frame. So callbacks execute on the frame thread, and the library marshals a client's callback there. The rule this imposes on scripts: draw must be fast and non-blocking — heavy or blocking work lives on the client's own task, and draw only reads shared state.
Error isolation. Each callback is pcall-wrapped so one client can't crash the desktop; an erroring draw surfaces the error in its own window and the frame continues, with a per-callback time budget guarding against a callback that stalls the compositor.
Two contexts, one callback. A window can attach to two ImGui contexts: the local desktop (main display + keyboard TFT) and the netImgui context (a connected PC). They are separate contexts with separate frame loops — the remote one runs only while a PC is connected, and when it is, it calls the same registered callbacks to build the remote frame. A window declares which it wants (context = "local" / "remote" / "both"): a "both" window needs no remote-specific code, while a remote-only control panel or a local-only surface is just a flag. Each context keeps its own layout, so a window can sit in different places on the machine and on the PC; input from either drives the same script state. When netImgui isn't running, its callbacks simply aren't called — the registrations wait. Crucially, both frames are built on the one render thread — it does the local frame (SetCurrentContext → local), then the netImgui frame (SetCurrentContext → remote), in sequence — so a "both" window's draw runs twice on the render thread, never on two threads at once. That is what keeps the two-context model consistent with one VM per task: the VM is still only ever touched by the render thread. The library doesn't select a context for a window — it runs the draw in each targeted context that's active; a draw that must differ per display asks which it's in with imgui.current_context(). (gui reconciles one retained tree into each context and so rarely needs this; a raw immediate-mode draw runs per context and may want to vary.)
Window order is focus, not list position. A natural guess is that a window's front-to-back order follows its place in the callback list — but ImGui doesn't work that way. Window stacking is focus-ordered: clicking a window raises it, and ImGui keeps its own focus order independent of the order windows are submitted. So reordering the callback list does not restack windows; it only changes submission order (which affects keyboard nav/tab order and any shared background drawing, never window z). Raising a window is a focus call — imgui.focus(handle) — and "always on top" is a window flag. AntOS's window manager reads and writes focus for stacking; it never reorders callbacks to change z.
The docking branch — the desktop is a dockspace. AntOS uses Dear ImGui's docking branch, so the windowing system gets real dock management for free: windows dock, tab, and split into a central dockspace (the desktop), or stay floating. A window may request an initial dock position (opts.dock), but the user rearranges freely — drag to an edge to split, onto another window to tab. This is what makes "ImGui is the window manager" literal: tiling, tabbing, and splitting are the framework's, not something AntOS reimplements. Floating windows still stack by focus; docked windows are chosen by tab.
Docking locally, not multi-window pop-out. The docking branch also brings multi-viewport — windows becoming separate OS windows — but that needs a host window manager to create OS windows, which the local AntOS display doesn't have (it's one compositor surface). So locally, multi-viewport is off: windows dock or float within the one desktop. A connected PC does have a window manager, so pop-out could be offered on the netImgui side where the remote supports it — the two contexts differ here by nature, not by policy.
Shared look, automatically. Every window in a context shares that context's theme, font/glyph set, and input routing, so a script window is visually and behaviourally a system panel without doing anything.
Menus, and opening other windows. A window can carry a menu bar — the main console does, and its menu opens system windows like Settings and the wiki reader. Opening a window is just launching a windowed task (a parallel VM → its own window), so a menu item that opens Settings is os.spawn("settings") — the very same mechanism a script uses to open its own window. This is the per-window counterpart to the desktop's global slide-down top menu: the top menu is the desktop's menu bar, a window's menu bar is its own, and both open panels the same way. System panels (Settings, wiki, notifications) are themselves imgui clients — so "open Settings" and "a script opens a window" are one mechanism, which is the whole payoff of unifying system UI and script UI.
Textures live on the render thread. imgui.image draws a GPU texture, and texture uploads — like ImGui itself — are render-thread-only. So images are handles: a client allocates a texture (imgui.texture), draws the handle each frame (imgui.image), and pushes a copied snapshot into it when the pixels should change (imgui.set_texture) — the copy decouples the client's own buffer edits from the GPU, so it can draw at will and snapshot when ready (double-buffering). A texture is a GPU resource owned by the client that made it and freed automatically when that client exits (or via imgui.texture_free), the same lifecycle as its windows. For a "both" window, netImgui transfers the texture to the PC, so the image shows in both contexts.
Dogfooding. The existing overlay panels (Settings, wiki reader, notifications) can be reimplemented as imgui clients — the strongest test that the client API is capable, and it collapses "system UI" and "script UI" into one mechanism.

Concurrency — two lanes, chosen by whether a script touches ImGui

ImGui has one context that only one thread may touch — but that constraint only bites for scripts that actually draw. So AntOS runs scripts in two lanes, and most scripts never pay the UI cost:

Worker lane (parallel) — scripts that don't touch imgui. A text or compute script runs on a real worker thread, truly in parallel (the CM's other cores do the work). It never calls ImGui, so there is no context to race. It still gets a console window through two thread-safe channels: output is a shared text buffer it writes and the render thread draws each frame; input is a per-console key queue the render thread fills for the focused console and the script drains via its normal read calls (io.read, term.key — see terminal). Writing text and reading a key need no awareness of the frame — the two channels are the decoupling. This is the common case (a text tool, a network job, a build script) and it gets full parallelism for free.

UI lane (cooperative) — scripts that use imgui. These are multiplexed onto the single render thread (where ImGui lives) and scheduled cooperatively: each frame, between NewFrame and Render, the scheduler resumes each windowed task to run its draw — real ImGui calls, real immediate-mode results (a button that returns clicked this frame) — then it yields, and its non-UI work runs time-sliced in the gaps. So os.spawn-ing a drawing tool works because its imgui.* calls execute during its resumed slice on the render thread, exactly where they're legal. The price: a UI-lane task must yield rather than hard-loop (the scheduler enforces a per-slice time budget; the pcall guard catches the rest), and heavy CPU work is offloaded to a worker whose result the draw reads from shared state.

Routing — scanned, not guessed at runtime. Which lane a script takes is decided by whether its source does require("imgui"), detected when the script is catalogued into DBFS (the flag stored with its other catalogue metadata, so launch is instant) or at launch if it's new. Found → UI lane; absent → worker lane.

The safety net — wrong-thread ImGui is caught, not tolerated. A static scan can be evaded (a dynamic require("im".."gui"), an indirect load), which would land a drawing script in the worker lane by mistake. So the imgui binding checks the thread: if any imgui call — or the require itself — happens off the render thread, AntOS refuses it, closes the offending script, and reports why, naming the cause: "imgui used from a worker-lane task — make require(\"imgui\") statically visible so it's scheduled on the UI thread." A missed scan becomes a clear, actionable error instead of a context race or silent corruption. (Erroring rather than migrating is deliberate: a running VM with live state can't be moved threads mid-flight cleanly, and the fix — make the require visible — is trivial.)

Why the draw must be on the frame (and so why the UI lane exists): a separate thread calling ImGui races the single context; and recording widget calls to replay next frame loses immediate mode — clicked/hovered wouldn't be known when the script's code runs, breaking if imgui.button() then …. Preserving immediate mode is what pins draw to the frame.

Native (compiled) programs

A user's C/C++ program that links ImGui itself has its own ImGui context in its own process — it can't share AntOS's in-memory context (different address spaces). Two ways to put its windows on the desktop:

Over netImgui — the recommended path. This is what netImgui is for, pointed inward: AntOS runs a local netImgui server on loopback, a native program links the netImgui client and connects, and AntOS composites its windows into the desktop alongside script windows and system panels, streaming input back. Because it's a draw-data stream over a socket, not shared memory:

the program is a separate process — a crash doesn't take the desktop with it;
there's no ABI/version coupling to AntOS's ImGui build;
it runs fully in parallel in its own process, so none of the UI-lane cooperative rules apply — netImgui is its bridge to the frame, exactly as the text buffer is a worker-lane script's bridge.

That makes it the third client path to the one compositor: direct ImGui calls (a Luau UI-lane script), a text buffer (a worker-lane script), or a netImgui stream (a native process) — all feeding the render thread that owns the real context, none of them racing it. Docking works over netImgui, so a native program's windows dock into the desktop's dockspace like any other, and its lifecycle is the connection: launch → connect → windows appear; exit → disconnect → windows gone.

In-process native plugin — trusted extensions only. A native module compiled as a shared object and loaded into the AntOS process shares the address space, so it can call the real ImGui context directly. It maintains its window exactly as a script does: it hands AntOS a draw callback that the render thread invokes in-frame, so its ImGui calls run at the frame's draw point, on the render thread — fast, no serialization. The cost mirrors netImgui's benefits: no isolation (a crash takes down the desktop) and ABI lock-in (it must match AntOS's exact ImGui version and config). That trade is right for a vetted system extension and wrong for an arbitrary user program — so user programs take the netImgui path, and the in-process path is reserved for trusted ones.

Not by CPU affinity. A tempting shortcut is to spawn the plugin's own ImGui thread and pin its affinity to the render thread's core, hoping co-location makes it safe. It does not: same-core affinity stops two threads running at the same instant, but the OS still preempts between them, so a preemption mid-ImGui-call still corrupts the shared context. Safety comes from running the draw in-frame on the render thread (the callback above), not from co-locating threads. A plugin's own thread, if it has one, does compute only and never calls ImGui — the draw callback reads its results. (And for a separate process, affinity is moot regardless — it can't reach AntOS's context at all, which is why that case uses netImgui.)

Decided

Default context — a plain imgui.window(...) defaults to "local"; opt into the PC with "both" / "remote". Conservative: nothing leaves the machine unless asked.
Layout persistence — a user option, off by default. When enabled, per-window position/size/docking is stored in DBFS keyed by window ID and context (local and remote layouts differ), so windows reopen where the user left them on each display. Settings carries a "clear saved layout" button.
Modals — scoped to their owning window/context and non-blocking: message_box returns a result or nil, never stalls, so a script modal can't freeze the desktop.
Widget surface — v1 is the curated set in luau_imgui; native code has all of Dear ImGui, and the binding grows as needed.

Open questions

Focus & the top menu — exactly how a script window interacts with the slide-down top menu and global z-order (a UI-polish detail, not structural).

Native core — `libantos_imgui`

The windowing system is this native core — libantos_imgui, with libantos_gui layered over it — and the Luau binding is one consumer of it, not the system itself. AntOS's own UI proves the point: the desktop overlay, the slide-down top menu, the console, and the Settings window are native imgui clients, registering their windows and draw callbacks through the core directly, never through Luau. If the capability were Lua-only, AntOS couldn't draw its own desktop.

So native code gets the full API — window registration, both contexts and per-surface on_draw, focus, docking, textures, the frame-thread dispatch and the pcall/time-budget guard — plus direct access to raw Dear ImGui inside its draw (it is a C++ library, not limited to the binding's curated widget subset). A script window, a system panel, and a native in-process plugin are therefore the same kind of client at the native layer; the binding is only the thin marshalling on top. (This is the general two-layer rule — every libantos_* core is Luau-independent and linkable by anything — but it is most visible here, because the OS's own face is a first-class native client.)

Built on the existing sys_imgui context and frame loop, plus the registration/dispatch layer (the per-client callback list, the frame-thread marshalling, the pcall/time-budget guard). The netImgui transport is unchanged — it renders whatever the context produces.

luau_imgui — the script API · AntOS overlay · antos_netimgui · antos_terminal · antos_library_design · AntOS Libraries hub

AntOS Library — imgui (reference)