A rendering & non‑photorealistic
research engine, built on Rive

00What this is

At its root the project is an offline player for Rive's .riv vector-animation files. But almost everything interesting sits downstream of the rendered frame: once the scene is in a texture, the engine treats it as raw material for stylization, simulation, vectorization, and temporal stacking. Two questions organize the whole codebase:

• Can we reproduce what the official, closed-source tooling does? — leading to a hand-written GPU stencil renderer and a reverse-engineered .riv writer.
• What can we build on top that the official runtime has no concept of? — leading to the NPR engine, the spacetime volume, the vector beam, spline curves, and an effects-aware document format.

The sections below follow that arc: the pipeline, the replication work, then each extension, with deep-dive cards for the research-level pieces.

01Two backends, side by side

A frame can be drawn two ways, and they run in parallel so they can be compared pixel-for-pixel and FPS-for-FPS.

A live split-pane panel renders both into 360 px targets and reports a rolling-average FPS via glFinish timing; on typical scenes the two land within 0.5–2× of each other. The point of "ours" is not to win that race — it is to be a renderer we can instrument and extend.

02The frame pipeline

The scene is drawn once into sceneTex (already including any .rivx editor geometry). Everything after operates on textures, which is what makes the stack composable.

Each stage is a texture-to-texture operation, so they nest: the custom pipeline lets you put effects after a beam, a beam after a spacetime volume, and so on. Aux buffers are recomputed per stage — exact for scene-aligned stages, approximate after a spacetime warp.

03Replicating the official renderer

"Ours" is a stencil-then-cover vector renderer — the same family the official Rive GPU renderer uses. A path is flattened to device-space polylines, its winding is accumulated into the stencil buffer, then a bounding quad is drawn gated to the stencil test. Everything is composited in a 4× MSAA framebuffer and resolved at 1×.

The core mechanics

• Stencil-then-cover. The winding pass accumulates signed edge crossings into stencil bits 0–6 via GL_INCR_WRAP/GL_DECR_WRAP on front/back triangle fans; the cover quad tests even-odd (EQUAL 0x01) or non-zero (NOTEQUAL 0x00).
• Non-convex clipping packs an intersection bit (bit 7) alongside the 7 winding bits, so an arbitrary multi-layer clip mask is built once and AND-tested in the same cover pass — no redundant multipass compositing.
• Scale-aware flattening. Cubics subdivide to keep each chord ≤ 3 px on-screen; the step count comes from the control-polygon arc-length estimate, clamped to [1, 256]. Quadratics are degree-elevated to cubic first; near-duplicate points (< 0.05 px) are culled to avoid degenerate stroke normals.
• Strokes are the Minkowski sum of the path with a disc of radius thickness/2: a band quad per segment plus an 8-triangle radial fan per vertex, unioned via stencil so overlaps never double-blend.
• Gradients are evaluated per-fragment in object space (inverse 2×2 + translation), so radial gradients stay elliptical under non-uniform transforms.
• Blend modes (multiply/screen/darken/lighten) are approximated by snapshotting the destination into a background texture and seeding the MSAA buffer with it, then using non-standard blend equations.

The hard one — analytic feather via jump-flood SDF + erf

Feathering a hard-edged fill is the part the official renderer keeps proprietary, and the part that's genuinely difficult. Blurring the binary coverage mask works for small radii but loses crispness and develops radius-dependent artifacts. We instead compute, per fragment, the exact Gaussian integral of coverage to the nearest edge.

The trick is getting a signed distance field cheaply. A naïve transform is O(N) per pixel; instead we use the Jump Flood Algorithm (Rong & Tan 2006), which propagates nearest-seed coordinates in O(log N) ping-pong passes with a halving step. Crucially the cost scales with shape size, not screen size — every feather pass is scissored to a padded bounding box, and the scratch is explicitly cleared after each shape to maintain a zero-outside invariant (the subtle bug: JFA's ±1-texel seed kernel reads stale white from a prior shape otherwise).

The erf itself is the Abramowitz–Stegun 7.1.26 rational approximation evaluated inline — a 512-entry CPU reference table exists in the feather lab for validation (sub-0.5% delta).

The reverse-engineered .riv writer

The official runtime is open — but it only shows how to read a .riv. The editor that writes them is closed, with no published binary spec. writeSplatRiv() reverse-engineers the format well enough to emit valid RIVE v7.0 containers: a 'RIVE' magic, varuint version, a property table-of-contents with a 2-bit field-type bitmap, then a stream of core objects (Backboard, Artboard, then per-shape Shape + Ellipse/Rectangle + Fill + SolidColor / Linear&RadialGradient + GradientStop + Feather + Stroke), parented by artboard-local indices and emitted in reverse order so late shapes draw on top.

04The NPR effects engine

The stylization engine cleanly separates what an effect is (pure C++ data) from how it runs (a generic GL executor). An effect is a list of passes; the executor compiles shaders on demand and wires up the texture flow.

struct Effect   { id; label; params[]; passes[]; }
struct Pass     { shaderId; inputs[]; constUniforms[]; output; scratchSlot; prim; vertexRows; }
enum Src       { Prev, Original, Noise, Self, Aux, History, State }   // where a pass reads
enum Dst       { Scratch, Result, State }                            // where it writes
enum Prim      { FullscreenTri, LineStrip }                          // fragment vs. ribbon geometry
struct ChainEntry { effectIndex; enabled; params[]; }              // a live instance

Two details make the engine expressive:

• Persistent State — a 16-bit-float ping-pong buffer turns the post-processor into a substrate for multi-frame PDE and particle simulations. Ownership is per-effect; taking over mid-chain forces a re-seed so a new sim never reads another's garbage.
• The line-primitive ribbon pass — a particle chain stored in the State texture is drawn as geometry via vertex-texture-fetch (gl_VertexID indexes particles), emitting two verts per Catmull-Rom sample as a variable-width GL_TRIANGLE_STRIP. That's how the "one-line drawing" becomes a real, antialiased stroke instead of hardware lines.

The registry holds 122 effects across ~80 shaders, grouped for the UI into painterly (Kuwahara, oil, watercolor, painterly strokes), ink & line (ballpoint, Ciallo, sumi-e, crosshatch, FDoG, engraving), halftone & stipple, abstraction (toon, bilateral, comic), light (bloom, neon, iridescent, hologram, prism, matcap, tendrils), and temporal & sim — the research-level batch below.

05Auxiliary buffers & the silhouette vectorizer

Content-aware effects need to know about the geometry, not just the pixels. The aux producer re-renders the scene through the custom pipeline to manufacture eleven buffers, each lazily computed (a per-frame Done flag) and cached/deduped by name across the effect chain. A uHasAux executor signal lets every consumer fall back to a luma approximation when no producer is present (e.g. on the official backend).

The vectorizer — turning a silhouette into a parametric curve

The curvepts buffer is the hard one: it converts the rendered scene into an ordered, resampled polyline that effects can trace. It's a CPU pass (the only GPU→CPU readback, throttled to every 20 frames) running a five-stage classic computer-vision pipeline, run at both polarities (bright subject and dark subject) with the larger valid bounding box winning.

06Research-level algorithms

Each card states the problem, the technique as actually implemented (with the governing math), and the lineage. These are the pieces where the complexity lives — coherent line drawing, structure-tensor filtering, and a shelf of GPU PDE / agent simulations running entirely in fragment shaders on a 16-bit-float state buffer.

Line art & structure

State-buffer simulations

Each runs 1–8 fragment-shader substeps per frame on the persistent RGBA16F state, seeding from the artwork and biasing toward its geometry (edges, tangent flow, curvature) so the simulation describes the picture rather than ignoring it.

07The spacetime volume

The spacetime stage stacks recent frames into a 3D texture ring buffer (R×R×N, with GL_TEXTURE_WRAP_R for a seamless time axis) and raymarches it. A parallel R8 volume tracks paint-order depth so foreground strokes can be emphasized. At 512²×128 the ring is ~80 MiB and the orbit marches 192 steps per pixel.

The volume captures one slice per frame (with motion stored in alpha) or rebuilds all N layers for the bilateral and extrusion sources. Four modes share the marcher:

• Orbit — a camera flies through the time box; front-to-back accumulation with early-out.
• Slit-scan — screen-aligned, no camera: each pixel reads a per-pixel time layer driven by depth or age, so foreground reads "now" and laggy regions read older frames (the strange-time look).
• Real STM — a volumetric heightfield of the current frame with five gather modes (fill, soft shell, growable metaball spheres, iso-bands, point cloud), morphological dilation, and 14 scientific palettes.
• Isolines — keep only voxels near a luminance level, rendering topographic contour sheets that sweep through time.

08The vector beam

The beam stage renders the scene the way an oscilloscope would: it chains all the scene's contours into one continuous XY path and traces an electron beam onto a persistent phosphor buffer. The signature is the analytic beam integral — each segment is shaded by the exact intensity of a Gaussian spot sweeping along it, not a heuristic glow.

Because energy is inversely proportional to segment length, slow strokes burn bright and the fast jumps between disconnected shapes leave the authentic faint connector lines — all of it falling straight out of the physics, with no special-casing. The quad is inflated by 4σ to capture the Gaussian tails, and the erf is a fast 4th-order polynomial approximation.

09The curve system

Official Rive has no spline primitive of any kind, so the editor grows its own. Six families are evaluable as dense polylines, all C²-continuous where it matters, and each is unit-tested for continuity via finite differences.

The Yuksel interpolating splines (Cem Yuksel, ACM TOG 2020) pass through every control point with C² continuity and no degree elevation — each segment blends two overlapping local interpolants with trigonometric cos²/sin² weights whose value and first two derivatives vanish at the joins:

Only the cubic-pen exports as exact cubics to native .riv; every other family flattens losslessly to a polyline on save but stays fully editable in .rivx.

10The .rivx format

Since .riv can't hold splines or our runtime look, the editor has its own append-only container. Each version adds a section; older readers stop cleanly at the boundary.

Keying the chain by string-id (not registry index) keeps files robust to reordering; the POD blobs carry a sizeof tag so a struct that gains a field is skipped gracefully on old files. The loader is hardened against corrupt and truncated input — short reads are rejected via gcount() (not eof(), which is set together with the fail bit on a partial read), an over-cap length is consumed to stay aligned, and the v6 pipeline applies all-or-nothing: a corrupt count, size mismatch, or out-of-range stage slot drops the whole stack rather than restoring something the frame loop would index out of bounds. A GL-free self-test (--rivx-fx-test) round-trips a populated config plus seven crafted corruption cases.

The runtime GL objects are never serialized — on load the parallel vectors are resized to null pointers and the frame loop rebuilds each stage lazily (every Impl destructor is =default with no glDelete*, so this is safe with or without a current context).

11Results & capabilities

Official vs. ours

What we added on top

Honest limits

• Blend modes are approximated by background seeding, not a true framebuffer-fetch feedback loop.
• Aux effects fall back to screen-space luma on the official backend (no geometry re-render available).
• Only one State-owning simulation per chain (two back-to-back re-seed each other).
• The silhouette vectorizer's figure/ground split grabs the frame on full-background-rect scenes.
• Spacetime trades resolution against history (512²×128 typical); bilateral slicing has a luma bandwidth.
• The greedy contour chain is O(n²) — fine for typical scenes, suboptimal for tangled paths.