Computational imaging · RadianceLab
Once a renderer can follow a photon exactly, it can answer questions ordinary rendering never asks. When did the light arrive. What phase was it. What would a sensor that measures those actually see — and is the answer it gives right.
This is the one thread on the site with no photographs. Everything on it is built, compiles and runs, and — as far as the repository records — none of it has been deliberately photographed, so every figure below is a capture instruction with the expected result written into it. The instrument does not need the camera to know its own answers: the holography bench walks the entire inverse problem in fourteen displayable stages, scores its output in wraps of object phase against ground truth, and names which of ten setup faults you have made.
src/spectral/01 — What is actually here
What is here, in one breath. A camera-mounted structured-light projector that is a real next-event light, an amplitude-modulation sensor model with four independent phase terms, four-shot continuous-wave time-of-flight capture, three holographic reconstructions, and a weighted-least-squares phase unwrapper solved by preconditioned conjugate gradient on the GPU. All of it builds and runs. None of it appears in a single figure anywhere in the repository — the project's own capture inventory lists it as the subsystem its earlier sweeps missed, and the one to shoot early.
Unphotographed is a fact about the photography, not about the instrument. What this one does when you run it is the reason the page exists.
UIManagerRender.cpp:4376-4391). Every intermediate a textbook would draw is a texture you can put on the viewport.UIManagerRender.cpp:4749-4755; the FFT panel prints the same four numbers without the solver line at :4411-4422).:4657-4703). Nothing else in this codebase diagnoses its own setup.So each section does two things. It states the capability, points at where it lives and says what it produces — and then carries the capture instruction that would put a photograph behind it. A reader who wants to know whether this works has a shooting list with the expected result already written in. A reader who wants to know whether it is worth shooting has the fourteen stages, the residual readout and the advisor, each with an anchor.
| Section | What it claims | Grade |
|---|---|---|
| The lamp itself | Four delta geometries with four falloff laws, seven patterns, a beam angle solved from a quad you size, an emission disk that appears in mirrors, and a plan view of the whole rig | never viewed — regraded from sighted 2026-09-06 |
| Modulation inside the transport | Phase rides on the traced path length, not on the depth buffer | seen in source; unphotographed |
| Light in flight | A path-length window that isolates a wavefront | seen — the one row the catalogue grades sighted |
| Four-shot capture | atan2 recovery from four accumulated buckets | never viewed |
| Sensor simulators | Correct algorithms fed the ground-truth depth buffer | never viewed |
| The fourteen-stage bench | Sideband isolation, IFFT, WLS-PCG unwrap, every stage displayable | never viewed |
| The residual readout | A pipeline that prints its own numbers, in wraps, gauge-cancelled | never viewed |
| The setup advisor | Ten rows that name the misconfiguration you have chosen | seen — graded sighted, never framed |
| The rest of the panel | The ground-truth depth read-out, the third time-of-flight implementation, the noise models, the amplitude mask and the frustum overlay — four of them carrying a defect | mixed — graded row by row in §09 |
| Branch-cut unwrapping | Nothing. The enum exists, the function body is empty — see the last section | not built |
| Structured-light decoding | Nothing. There is no decoder — see the last section | not built |
One useful consequence. Because nothing here has been looked at, anything odd in the first captures should be treated as a real candidate bug rather than an expected gap. The project's own triage document records the opposite habit failing repeatedly: reasoning about why an estimator looks dark produced three confident wrong conclusions in one session, each settled by a single instrumented run.
02 — The lamp, before the measurement
The projector appears forty-odd times on this page and every one of those mentions is about phase: what it modulates, what it carries, what a decoder recovers from it. That is a real use and it is not the whole object. Underneath the modulation is a next-event light source with four distinct delta geometries, four falloff laws, seven patterns, a beam angle that is solved rather than typed, and a plan view of the entire rig — none of which any figure on this site has shown, and none of which the rest of this page needs in order to make its argument.
The four shapes are not four presets on one light. Each is a different delta geometry with its own shadow-ray construction and its own distance law, which is exactly what makes the set useful: put the same power through all four and the differences in the image are differences in the physics, not in the artwork.
| Shape | Geometry | Falloff | In a mirror? |
|---|---|---|---|
| Point source | A cone from the camera position; UV is perspective, so a pattern projects the way a slide projector's does | 1/r², inverse square | Yes — a 0.05 m aperture disk |
| Quad laser panel | A collimated plane wave covering everything the camera sees; the wavefront is flat | None. Collimated light does not spread | No disk exists |
| Spinning slit laser | A line emitter through the camera position, oriented by a sweep angle from −90° to +90°; light reaches a surface from the nearest point on the line | 1/r, cylindrical spreading about the line | No disk exists |
| Flashlight | A cone with a smoothstep halo and a physical emission-disk radius you set | 1/r², with a soft edge | Yes — at the radius you set |
That last column is the one a reader would not guess. At any bounce past the first, the ray marches against an aperture disk sitting at the camera and facing backwards, and if it lands inside, the projector's own emission is returned and the path terminates — absorbed by the light source, which is what a real lamp does to a ray that finds it. So the projector appears in mirrors and in glass, correctly, as a bright disk. Two of the four shapes have such a disk and two do not, and the asymmetry is not arbitrary: a collimated panel and an infinite line have no aperture to hit.
There is no “cone angle” slider. You size a quad, place it at a depth, set how far behind it the emitter sits, and the half-angle falls out — clamped to between 0.5° and 89° so it can never be degenerate. The arithmetic is shape-dependent, and one branch of it is worth reading closely:
On top of the derivation sit three fitting aids, and the third is the one that makes the four shapes comparable at all. Match Image Plane puts the quad at the camera's focus distance spanning the horizontal field of view; Match Aperture puts the emitter at zero and sets the cone equal to the camera's field of view. Size Matching goes further: it solves the beam angle from a desired half-width at a reference depth, so “point versus panel versus line” becomes a controlled comparison at a fixed illuminated area rather than three differently-sized beams. It is also, incidentally, one of two controls that survived this project's own inert-control audit by reaching the renderer through a derived write rather than a direct one.
Beside all of this is a draggable top-down window that draws the whole rig in the XZ plane: the scene's primitives as circles, the camera frustum, the image and focal planes, the effective quad, the emitter's position behind it, a translucent beam cone with seven fan rays — and, when the path-length gate is on, the time-of-flight wavefront itself, drawn as an expanding arc for the point and line sources and as a straight advancing line for the panel.
That last detail is this page's own argument, drawn. §04 spends several paragraphs, and a whole clip, on the difference between a curved wavefront and a flat one and what each does to a recovered depth. The renderer already draws that difference, in a plan view, live, next to the controls that cause it. It has never been photographed, and it is the first shot on this page's list for that reason.
Grid, Checker, Rings, Bars, Gobo/Star, Gradient and Plain, evaluated in projector UV with a repeat-scale slider. Plain is not a null option: it is uniform emission, which is what makes interference fringes readable with no pattern printed on top of them. Beside the pattern list sit two controls that look like debug toggles and are really experiment design. Ignore Lambert bypasses the cosine and the light's directional term so every surface is equally bright — built to verify a wavefront's shape without brightness confounding it. And Path Offset shifts the effective path length uniformly, so the curvature of a point, panel and line wavefront can be compared at one time offset instead of three.
The node editor carries an unrelated one. Wire a Camera node — which snapshots the live view-projection and basis at one button press — and a Texture node into a DataMosh Projector, and a photograph is cast into the scene from that frozen pose. The photograph need not come from a file: the frame loop snapshots the clean, pre-mosh display texture into any Texture node that has asked for a capture (EngineRender.cpp:4127-4130), so the renderer can photograph itself and then project the result back in. Two of the six modes are the ones a reader will want first: Light cone and Light plane turn the image into emitted light — a perspective cone from the captured apex, or an orthographic plane of a given world size, with an intensity slider. A photograph, used as a luminaire, inside a path tracer.
The other four are not variations on that one idea. They enter the renderer at four different places, and two of them never enter the path tracer at all: the tracer's copy of the projector is switched on only for modes 1, 3 and 4 (EngineRender.cpp:1709-1711). So mode 2 — Forward is a rasteriser-only mode, and mode 0 — Reproject never reaches either renderer at all. Mode 1 — Project is the one that runs in both, and its two sub-branches do opposite things to the surface they land on. The gate is easy to miss, because the combo offers all six under one node with no indication that they run in different places.
| Mode | Where it actually runs | Grade |
|---|---|---|
| 0 — Reproject | Neither renderer. A post pass on the display texture, sequenced after data moshing and before glow (EngineRender.cpp:4146-4150), re-projecting frozen pixels through the frozen depth buffer | never viewed |
| 1 — Project | Both renderers. Per hit in the megakernel (pathTrace.comp:982-998), per fragment in the rasteriser (basic.frag:437, :447). A Re-lit checkbox switches the branch, and the two branches do very different things — see below | never viewed |
| 2 — Forward | Rasteriser only. Replaces the surface albedo with the photo and lets the raster lighting run over it. Selecting it while the tracer is active does nothing at all | never viewed |
| 3 / 4 — Light cone / plane | Both, deliberately: the tracer helper's own header says it “mirrors basic.frag's projectorLight() so raster and tracer match”. Additive, scaled by N·L, with no shadow ray — so it is a luminaire that casts no shadows | never viewed |
| 5 — Bake to UV | The rasteriser renders the projection in texture space into the tracer's per-entity baked-albedo array; the tracer then samples it as an ordinary texture. 64–4096 px, three choices of which surface side receives it, explicit Re-bake button | never viewed |
| Optional fixes | Three, under their own separator in the node panel: fit-aspect letterbox against the viewport aspect, flip-V, and an edge fade smoothstepped in projector UV. Blend, glitch and decay sit above that separator and apply to every mode (NodeEditor.cpp:1277-1288) | never viewed |
The two halves of mode 1 are worth separating, because only one of them is a photograph. Re-lit overwrites the surface albedo and then zeroes that surface's emission, so projecting onto an emitter silences it. Unlit adds the photo colour to the radiance and immediately breaks the path — the surface stops being a surface and becomes the picture. And both light modes and the unlit branch are skipped whenever the light strategy is 3, 4 or 7, the two pure light-tracing strategies and Light Sample Only, because each adds its contribution at the hit rather than through a connection. The re-lit branch is the exception: it changes an albedo, so it survives every strategy.
A grade corrected. The landing page graded the projector seen until 2026-09-06, on the strength of everything else on this page. But everything else on this page uses it as a carrier for phase; not one figure, anywhere in the repository, shows what it looks like as a lamp. The catalogue's projector block runs to thirteen rows and grades never viewed every one of the ten that is a capability rather than a caveat — the lamp itself, the four shapes, the seven patterns, the derived beam geometry, the two fitting buttons, Size Matching, visibility in reflections, the plan view, Ignore Lambert and Path Offset. The other three are the two limitations named above and a debug frustum. It is right, and the ledger row now agrees with it.
03 — Where the measurement enters
The projector is not a post-process. It is a delta light rigidly attached to the camera, sampled by next-event estimation inside the megakernel (pathTrace.comp:2050-2062), and the amplitude modulation is applied at that connection as a function of the path length the tracer actually walked — including, optionally, the whole multi-bounce eye path.
A continuous-wave time-of-flight camera emits amplitude-modulated light and correlates the return against a reference. What a pixel measures is a cosine of a sum of phases, and the interesting design decision here is that those phases are kept separate rather than collapsed into one offset. There are four of them, entering at four different physical places.
shaders/modules/pt_sky.glsl:246-307uCamPos. The baseline is exactly zero, so triangulation is not merely unimplemented, it is geometrically impossible in this configuration. Depth here comes from phase, never from disparity.uProjector* uniforms are uploaded from one place, the projector block at ComputePathTracer.cpp:1939, and from nowhere else in src/render/. The rasteriser's uProjector* set is an unrelated system that casts a captured photo. Press 2 before any capture on this page.pathTrace.comp:2061-2062 skips strategies 0, 3 and 4 — a delta light cannot be hit by BSDF sampling, and pure light tracing does no eye-side next event. The panel's health check flags exactly this.pathTrace.comp:2229), as does the light-tracer splat (:3038). Three secondary sites reach the projector through evalProjectorTOF instead (:904, :939, :3511): the projector seen head-on through the aperture, its reflection off an escaped ray, and light-tracer vertices. That helper calls the two-argument overload, which substitutes vec2(0.5) for the pixel and uCamPos + uCamForward·L for the surface point (pt_sky.glsl:313-315).A finding from writing this page, sharpened on the second pass. The first draft said the three secondary sites carry no rolling shutter. They carry no emitter ramp either, and that follows by construction rather than by inspection: the substituted position lies on the optical axis, so dot(toPoint, uCamRight) and dot(toPoint, uCamUp) are identically zero in the projector-UV block at pt_sky.glsl:273-291, and φemit collapses to zero along with φsensor. Both spatial carriers vanish at those three sites; only φpath and φdemod survive. It is small — it affects the source's appearance in a mirror, not the illumination — but it is exactly the sort of thing a reader who opens the shader will notice, so it belongs here rather than in a commit message.
04 — Gating on arrival time
Independently of the modulation, every contribution in the kernel knows its own accumulated path length, and a filter can keep only the contributions whose length falls inside a window. Sweeping that window forward is light in flight.
This is one of the few parts of the subsystem the feature catalogue grades seen rather than never viewed: it is item B2 in the capture inventory, and the top-down setup view draws the wavefront position whenever the filter is active. It also works at volume scattering vertices, not only at surfaces, so in a foggy scene the window shows an expanding shell rather than a moving contour on the walls.
Two qualifications, both checkable in one file.
UIManagerRender.cpp:1144-1161The gate is not one gate. Eight sites in the kernel apply the path-length window and they do not agree on its edges. Six feather it identically — volume next-event (pathTrace.comp:823), surface next-event (:1966), point-light next-event (:2033), the projector's own direct lighting (:2264), and two light-tracer connections (:3173, :3489) — admitting out to 1.5 window half-widths and attenuating smoothly. A seventh, the light-tracer projector splat, tests a hard interval first (:3018) and only then feathers inside it (:3042), so its window is effectively narrower than everyone else's. The eighth is a path that lands on an emitter by chance rather than by connection (:1164): hard interval test, no attenuation at all.
So a scene lit by an emissive quad shows a soft-edged wavefront through its next-event contributions and a hard-edged one through its BSDF-sampled contributions, at the same window setting. For the illustrative use this filter is put to, that is harmless. For anything quantitative it is not, and it is the first thing to reconcile if the gate is ever promoted from a display filter to an estimator.
The gate is a display filter, not an estimator. Contributions outside the window are zeroed or attenuated and nothing is renormalised, so a gated frame is not a correctly-normalised transient image. It is a picture of which light paths fall in a length band. That is the right tool for showing wavefront geometry and the wrong tool for measuring transient radiance, and the distinction should be stated wherever the images are used.
05 — Recovery
A real continuous-wave sensor takes four correlation measurements per depth frame. This one does the same thing across four consecutive rendered frames, accumulating each into its own layer of a four-layer floating-point image array, and then demodulates.
The CPU cycles the demodulation phase through 0, π/2, π, 3π/2 on the frame index and tells the kernel which bucket it is writing (ComputePathTracer.cpp:2002-2014). The kernel accumulates into that layer with the same running-mean weight as the main accumulator but at a quarter rate, so after 4N frames every bucket holds N samples (pathTrace.comp:3767-3776). A post-process then reads all four and applies the standard formula.
shaders/tofRecovery.frag, 271 linesThe recovery pass offers the four raw shots tiled, recovered phase as a hue wheel, wrapped recovered depth, recovered amplitude, ground-truth depth, a wrapped-versus-truth error view, a six-panel pipeline overview, a GT-guided unwrapped depth, and an unwrapped-versus-truth error view (UIManagerPanels.cpp:1041-1051). Three caveats travel with them, all in the shader source.
k = floor((dGT − drec)/wrap + 0.5) — they pick the integer wrap count by rounding against the depth buffer (tofRecovery.frag:217-238). That is a legitimate diagnostic, because it isolates within-wrap accuracy from wrap ambiguity, but it is not a blind reconstruction and it must not be captioned as one. Blind multi-frequency unwrapping exists elsewhere in the stack and is described in the next section.:82-89 notes is exact only for a cone or point source in the paraxial limit. Any error there shows up as a slow spatial ramp in the recovered phase.06 — The distinction that matters
This is the section a reader will check first, so it is stated plainly. The modules in this subsystem split into two families that measure different things — one the render, one the depth buffer — and the page keeps them apart because the instrument does: the same recovery code can be pointed at either, which is what makes the bench a controlled experiment rather than a demo.
One family reads the renderer's own transport. The projector modulation rides on the traced path length, the four buckets accumulate rendered radiance, and the recovery pass demodulates what came out. If multipath interference corrupts the phase — and in a Cornell box it should — that corruption is present, because it is a property of the light that was actually traced.
The other family reads the ground-truth depth buffer, synthesises the measurements an ideal sensor would have made from it, and then runs the recovery algorithm on that. Those modules are correct implementations of published methods, and their error readouts are real, but what they measure is the numerics, not the renderer. The CW time-of-flight shader says so itself in its header: it synthesises four phase-shifted intensities at up to three frequencies from the z-buffer (tofCW.frag:69-70).
The holography bench is the interesting case, because it can be pointed at either ladder from a combo box. Its Source setting chooses between synthesising the hologram from depth and taking the luminance of the rendered frame (HoloDepthV2.cpp:168-190), which is what makes it usable as a controlled test of the numerics and as a test of the whole chain.
| Module | What it consumes | Where |
|---|---|---|
| Projector and ToF modulation | traced path length, inside the megakernel | modules/pt_sky.glsl:102-339 |
| Four-shot bucket accumulator | rendered radiance, four layers | pathTrace.comp:3767-3776 |
| Four-phase recovery | the four buckets | tofRecovery.frag |
| HoloDepthV2, Source = Rendered frame | rendered luminance, resampled to N×N | holo_v2/renderedToHolo.comp |
| CW time-of-flight cascade | the depth buffer | tofCW.frag:69 |
| ToF tracer, geometric path | the depth buffer, unprojected | tofTracer.frag |
| Off-axis holography, direct | the depth buffer | offAxisHolography.frag |
| Off-axis holography, FFT | the depth buffer | offAxisHoloFFT.frag + 5 compute |
| HoloDepthV2, Source = Synthetic | the depth buffer | holo_v2/synth.comp, chosen at HoloDepthV2.cpp:178 |
| Phase unwrap, weighted least squares | wrapped phase from either ladder | PhaseUnwrap.cpp + 11 compute |
| Epipolar geometry, stereo correspondence | one depth buffer, warped to a hypothetical second camera | epipolar.frag, stereoCorrespondence.frag |
The blind unwrapping lives on the lower ladder. The multi-frequency cascade — low frequency for unambiguous range, high frequency for precision, the coarse estimate choosing the wrap index for the fine one, with a third frequency for redundancy — is implemented properly in tofCW.frag, including the Chinese-remainder framing in its header at :47-66. But it runs on synthesised measurements. Porting the cascade onto the four-bucket path, so that the wrap index comes from a second modulation frequency rather than from the depth buffer, is the single change that would move the strongest algorithm in this subsystem onto the ladder that measures the render.
07 — The whole inverse problem, one stage at a time
Off-axis holography adds a tilted reference wave to an object wave and records the intensity. The interference term separates in the Fourier domain into two sidebands displaced from DC by the carrier, so isolating one sideband and inverting recovers the complex object field — amplitude and phase — from a single real-valued image. That is four or five textbook diagrams. Here it is a combo box, and each entry is one of those diagrams rendered from the live data.
There are two holography panels and they are not the same size. Off-Axis Holography [FFT] carries the fourteen-entry display combo tabulated below (UIManagerRender.cpp:4376-4391) — that is the teaching instrument. Off-Axis Depth V2 (clean) is the later rewrite: nine display stages of its own (:4729-4740), but it is the one that carries the residual readout, the setup advisor and the two presets, so it is the panel every capture instruction on this page names. Internally V2 runs the same reconstruction as fourteen lettered stages, (a) through (n) — produce the hologram, extract its real part, forward FFT over rows then columns, log-magnitude for display, crop and re-centre the sideband, log-magnitude again, inverse FFT twice, extract amplitude and wrapped phase, unwrap, then three diagnostic reductions (HoloDepthV2.cpp:168-348).
What follows is the FFT panel's combo, entry by entry. This is the difference between a filter and a teaching instrument: a filter hands you the output, this hands you every intermediate that the output depended on, in the order the algorithm produced them.
| # | Stage | What it shows, and what a failure looks like |
|---|---|---|
| 0 | Ground Truth Depth | The linearised z-buffer, turbo-coloured. The reference every later error view is scored against |
| 1 | Hologram (raw fringes) | H = |O + R|². Straight uniform fringes everywhere mean the object phase is not modulating the carrier |
| 2 | FFT Magnitude (centered, log) | DC blob plus two symmetric sidebands at ±K. Sidebands touching DC means the carrier is too low; clipped at the edge means too high |
| 3 | Crop Mask (on FFT) | The band-pass window drawn over the spectrum, so you can see whether it clears DC and the twin before you crop |
| 4 | FFT After Crop | One sideband surviving, recentred to DC. If two survive, the mask radius is too large |
| 5 | IFFT Magnitude (|U|) | The object amplitude. Dark halos here are the ones that wreck the weighted solver two stages later |
| 6 | Recovered Phase | arg(U), wrapped, as a hue wheel. Smooth ramps broken by clean 2π seams that follow depth |
| 7 | Recovered Depth | Phase converted to depth, still wrapped inside one wavelength period |
| 8 | Error vs Ground Truth | Wrapped recovery against truth, before any unwrapping |
| 9 | Pipeline Overview (6 panels) | Hologram, spectrum and cropped spectrum over IFFT magnitude, phase and depth — the whole method in one frame |
| 10 | Comparison (IFFT | Phase | GT | Recon) | Four panels putting reconstruction beside truth for a direct visual read |
| 11 | Amplitude Mask (unwrap) | The valid-pixel mask the solver will actually use. If this is mostly black, every number below it is computed on almost nothing |
| 12 | Unwrapped Phase (WLS-PCG) | The solved continuous phase. Surviving wrap seams mean the solve did not converge |
| 16 | Phase Error vs GT (WLS-PCG) | The signed residual with the gauge subtracted, four colormaps, a labelled legend strip in wraps — and the four-line stats block described in the next section |
The gap in the numbering is itself a record. Indices 13, 14 and 15 are missing, so the combo carries an index map rather than using its position directly (UIManagerRender.cpp:4374). The comment above it says why: 13 and 14 are "reserved for Task 6 Goldstein residues/cuts" and 15 for a depth error view (:4371-4373). Those stages were planned and never built. The combo routes around them rather than offering entries that would do nothing — the correct behaviour, and the visible evidence for the correction in the last section of this page.
There are three holographic reconstructions in the tree. The first synthesises the hologram, demodulates by multiplying by the conjugate carrier, and low-passes with a Gaussian; no FFT at all. The second does it properly, with a real GPU FFT: a radix decomposition in fft1d.comp, recompiled when the transform size changes, run over rows then columns. The third, HoloDepthV2, is a clean-slate rewrite whose header carries the whole derivation, and it is the interesting one because of what the derivation is for.
demodulateCarrier = false. Setting the tilt to zero would not be enough, because that pass conjugates its input unconditionally and would flip the sign of the recovered phase. src/spectral/HoloDepthV2.h:9-30That comment is the reason to show this module. It is a sign-convention trap written down at the point where someone would fall into it, with the specific wrong fix — zero the tilt — named and rejected. The same codebase records a second finding of the same kind a few files away, and this one is a numerical result rather than a convention.
Phase unwrapping here is a weighted least-squares problem solved by preconditioned conjugate gradient across eleven compute shaders, with an FFT-based Poisson solve as the preconditioner. The natural weighting is by amplitude — trust the bright pixels — using Wij = min(Ai², Aj²). It stalls.
The recorded cause, in an eleven-line comment on the config field itself (PhaseUnwrap.h:26-36): when |U| varies by roughly a hundredfold across the mask — dark halos around depth discontinuities — the condition number of the weighted operator explodes, and the unweighted Poisson preconditioner is then nothing like the operator it is meant to precondition, so it stops accelerating anything. Setting every valid neighbour pair to weight one makes the operator match the preconditioner exactly, and PCG converges in a handful of iterations. The unweighted form is exposed as a checkbox labelled Unweighted Laplacian, with that explanation in its tooltip (UIManagerRender.cpp:4722-4727).
The cost is real and the source states it in the same comment: unweighted least squares trusts a dark, noisy pixel exactly as much as a bright one, so where the fringe contrast is genuinely poor the solution is smoothed rather than downweighted. This is a conditioning trade, not a free win. Note also that the config field defaults to false — the weighted operator is the default, and the fast path is the checkbox.
08 — The numbers, and the advice
Two things sit under the V2 display combo, and between them they are the reason this subsystem is on the site. Above the fold: a live per-frame readout of the reconstruction residual in wraps of object phase, with the expected value printed beside it. Below: a block that recomputes ten conditions every frame and says, in plain language with the remedy attached, which one you have got wrong.
Both panels print the readout, and they differ by exactly one line. The FFT panel prints four numbers (UIManagerRender.cpp:4411-4422); V2 prints the same four and adds the solver’s iteration count and residual (:4749-4755). Both blocks appear only when the display is set to the phase-error stage, which is why the capture instructions below pass through it.
A numerical pipeline that reports on itself every frame is a stronger artefact than a picture of its output, and one that also diagnoses the scene feeding it is stronger still. This read-out is the whole of V2's quantitative self-validation, and it is worth being exact about what that quantitative claim covers: it is a convergence and range monitor rather than a scalar accuracy figure, and the paragraphs below say exactly which number means what — because this page's own source material once read them wrongly, and the correction is part of the record.
mean(diff) is a gauge, not an error. It is computed by a GPU block reduction of φu − φgt over the valid mask (holo_v2/errMean.comp), and the same value is then passed to the display shader as uErrorMeanRef and subtracted from every pixel before colouring (holo_v2/display.frag:100). This is correct — least-squares unwrapping recovers phase only up to an additive constant, so the constant must be removed before the residual means anything — but it means the printed number quantifies the ambiguity that was cancelled, not the accuracy that remains.
RMS(phi_u) is the spread of the signal, not of the error. The readback loop takes the standard deviation of the unwrapped phase about its own mean (HoloDepthV2.cpp:370-384), and min/max are reported the same way, as offsets from that mean. The panel is honest about this in the line underneath, which states the expected value as wavelength / sqrt(12) on a uniform-depth ground truth: the standard deviation of a uniform distribution spanning one wavelength's worth of wraps. At the shipped wavelength of 2.0 that is 0.577 wraps. It is a range check, and a useful one, and it is not an accuracy figure.
So there is no scalar accuracy number yet. After the gauge is removed, the accuracy view is the error image itself — the signed residual against ground truth, in wraps, with the gauge already cancelled. The missing number is one reduction away: reduce the squared residual after gauge subtraction in the same shader that already computes the mean, and print an RMS error alongside the RMS spread. Until that exists, no number from this panel should be quoted as an accuracy.
Correction recorded, and the code is not the culprit. This page's own source material described the panel as reporting RMS(φu) as though it were an error metric. The engine says otherwise in a comment three lines above the assignment — "RMS/min/max = stats of the unwrapped phase over the valid mask" (EngineVFX2.cpp:5936-5938) — and the UI label is literally correct, because it says phi_u and not error. The mistake was made downstream, by summaries that read the label and not the line. The readout is a convergence and range monitor; the scoring against ground truth is the residual image, in wraps, gauge-cancelled. The distinction matters precisely because this subsystem's claim is that it checks itself, and a self-check only counts if it is read correctly.
Underneath the sliders sits a setup-health block (UIManagerRender.cpp:4657-4703) that recomputes its conditions from live UI state every frame and prints each as a green OK or a red WARN with the remedy in the same line. Nine of the rows are two-state; a tenth appears only as a warning, when the rendered-frame source is selected but the projector cannot produce fringes at all; and two amber NOTE rows handle the ambiguous cases.
| Row | What it is checking | What it tells you to do |
|---|---|---|
| Projector ON | projectorOn | Turn it on or no light reaches the scene |
| Power sufficient | power ≥ 10 | Fringes may be too dim to read |
| Strategy can see it | lightStrategy != 0 | Naive PT will not hit a delta emitter — switch to NEE or Light Tracer |
| Environment light off | !envEnabled | Env light adds DC haze that washes out fringe contrast |
| Projector ToF on | projectorTofOn | No modulation, so no fringes at all |
| Reference carrier present | emit ramp or sensor roll non-zero | Without one the sideband sits at DC and cannot be isolated |
| Frequency matches | |fToF − λV2| < 5 % | Object phase will not read correctly |
| Tilt matches the carrier | within 0.5 of whichever carrier is live | Names which of the two carriers it compared against |
| Carrier uniform in screen space | quad panel, or a sensor-side carrier | A cone projector's carrier direction varies with angle |
| Source vs projector (warn-only) | rendered source with no usable projector | The one row with no green state |
| Dual carrier (note) | both emit and roll active | Effective carrier is their sum — not an error, but say so |
| Scene lights (note) | always shown | Point, area and directional lights are not auto-disabled; turn them off by hand |
What makes this more than a checklist is that the two buttons above it are the fix. Apply plane-wave off-axis preset (:4543-4581) does not set a flag: it turns the projector on, switches it to a quad panel, derives the panel width and height from the camera's vertical FOV, aspect ratio and focus distance, raises the power to at least 50, matches the ToF frequency to the reconstruction wavelength, copies the tilt into the emitter phase ramp, switches the sensor rolling shutter off so exactly one carrier is active, moves the light strategy off Naive if it was there, disables the environment light, and points the source at the rendered frame. Apply sensor-rolling-shutter preset (:4584-4614) does the mirror-image thing: emitter ramp to zero, carrier on the sensor, any projector shape allowed — because a sensor-side carrier does not care about the beam geometry, which is why the ninth health row accepts it unconditionally.
The presets and the advisor are one mechanism: the ten rows enumerate the failure modes, and the two buttons are the constructive proof that a valid configuration exists. That is a small piece of engineering with an unusually high ratio of usefulness to lines, and it costs one screenshot to show.
09 — The rows the sections above did not reach
The sections above take one capability at a time. What is left of this subsystem's inventory fits in one table, on the site's own rule that a feature earns prose only where there is a number, a mechanism or a defect to state. Four rows have one, and they are written out underneath the table.
| Row | What it is, and where | Grade |
|---|---|---|
| Ground-Truth Linear Depth | The reference every error figure on this page is scored against, exposed as an operator of its own: three sources (Auto, force-linearise from NDC with its own near and far, assume linear) × three displays (turbo, greyscale, contour bands with a 4–64 band count), plus an overlay mix. Its tooltip states the point — verify the z-buffer before trusting it as ground truth (UIManagerRender.cpp:4363-4382) | measured |
| CW ToF — the two noise sliders | Ambient 0–0.3 and “Shot Noise” 0–0.1, added to all four synthesised shots before the arctangent (tofCW.frag:168-169; bound at EngineVFX2.cpp:5500-5501). Neither is what its label says — see below. The inventory files these under sensor noise models and grades them never viewed; the reading below is a source read, and the capture at the end of this section is what would turn it into a measurement | never viewed |
| Tracer ToF, geometric path | The third time-of-flight implementation, and the largest: five light shapes (plane behind the camera, horizontal and vertical line scanners, point scanner, camera-collocated), collimated or omni, rolling light delay and rolling sensor shutter as separate X/Y ramps at ±50 wraps each, and a tilted plane-wave carrier at up to 200 wraps that turns the whole thing into an off-axis holography rig. Ten display modes, ending on the light's own emission coordinate (UIManagerRender.cpp:4901-4955) | measured |
| Projector visible in reflections | Confirmed in the kernel: at any bounce past the first, the ray is tested against an aperture disk at the camera facing backwards, sized 0.05 m for the point source and at the flashlight's own disk radius for the flashlight, and a hit returns the projector's emission and terminates the path (pathTrace.comp:875-911). It carries a guard the rest of this page does not mention: under Light Sample Only the disk is still intersected and returns nothing | never viewed |
| Projector on the CPU integrators | Invisible on every one of them. The catalogue grades this superseded rather than absent, and that is the right call under this project's own reachability rule: the feature is reachable, on the GPU, so the row records where it lives rather than a missing capability | superseded |
| Debug frustum overlay | The projector cone drawn into the foreground draw list in FPS capture, tinted by the projector colour, with a labelled end square and cross (EngineRenderViz.cpp:2505-2554). Two conditions gate it that the panel does not say: it draws only while the viewport is captured, and Clean View suppresses it. It also draws two shapes where the light has four — see below | seen |
| Amplitude mask and erosion | Threshold on |U| plus a neighbourhood pass that zeroes any pixel with a sub-threshold neighbour, producing the valid-pixel mask every number in §08 is computed over. The threshold is absolute, not relative to the frame's peak amplitude, so it does not track exposure. The erode control is a two-state switch wearing a slider — see below | seen |
Both time-of-flight simulators build their four measurements the same way: Ik = clamp(cleank + ambient + noise, 0, 1.5), where cleank spans [0, 1] and the noise term is (hash(…) − 0.5)·2·uNoise. Two consequences follow from the arithmetic, and neither is stated in the panel.
Ambient is a constant added to all four shots, and every recovered quantity is built from differences of them — I3−I1 and I0−I2, for the phase and for the amplitude alike. A constant cancels identically in both. The only route by which it could survive is the clamp, and at the sliders' own maxima the sum reaches 1.4 against an upper clamp of 1.5, so the top never bites; the bottom bites only when Ambient is set below Shot Noise. So over essentially its whole range the Ambient slider changes the raw-shot display and nothing else — not the phase, not the depth, not the amplitude, not any error figure. tofTracer.frag goes further and deliberately holds ambient out of the illumination scaling (:343), which preserves the cancellation exactly.
“Shot Noise” is uniform additive noise. The term is drawn from a hash and is independent of the intensity it is added to; real shot noise scales with the square root of the signal, which is the whole reason a time-of-flight paper models it. It is also seeded from the pixel and the shot index alone, with no frame or time input, so it is frozen: it does not vary between frames and cannot be averaged away by letting the render sit. That makes it a fixed fault pattern rather than a noise process, which is a perfectly good thing to have — it is repeatable — but it is not the thing the label promises, and a robustness claim made from that slider would not carry.
Where this leaves the shader's own sanity note. tofCW.frag:117-121 tells you that in the noiseless case the recovery is bit-for-bit identical to the input depth, because the four-phase formula is an exact inverse of the synthesis, and invites you to raise Shot Noise “to see the recovery fail realistically”. The first half is exactly right and it is the reason this ladder is worth having. The second half is the sentence this section is qualifying: the failure you get is real and it is not realistic.
UIManagerRender.cpp:4834), and the mask shader's uniform is documented in its own header as “0 = no erosion, 1 = one-pass 3×3 erosion” and used as a boolean gate around a single fixed 3×3 pass. Settings 1, 2 and 3 produce byte-identical masks. The field behind it is separately misnamed — it is called maskDilate and passed as uErodeRadius, which shrinks the valid region rather than growing it — and the header flags that in a four-line comment while the slider says nothing.EngineRenderViz.cpp:2509, :2551-2552). The line source, whose whole point is that it has no vertical extent, is the one shape the overlay can never draw correctly.tiltRemove, enableInpaint and inpaintIters are declared in PhaseUnwrap.h:23-25 and appear nowhere else in the tree — not in the solver, not in a caller, not in the UI. They are not inert controls, because no control was ever built for them; they are the shape of two features that were planned beside the ones that shipped.10 — The other half of the ledger
The name of this subsystem in the codebase is structured light, and a reader who knows the field will ask the obvious question: where is the decoder. There is not one, and the reason is more interesting than the absence. One row below is a correction to this project's own reachability record, and one is a correction to an earlier version of this page.
| Not here | Why, precisely | Grade |
|---|---|---|
| Branch-cut phase unwrapping | The record said the phase is unwrapped two ways, by a weighted-least-squares solver and a Goldstein branch-cut method. It is not. PhaseUnwrap.cpp:400 is void PhaseUnwrap::runGoldstein(...) {} — an empty body, with runPostAndDepth empty beside it. The enum, the two output textures and the else branch that would call it all exist, but the selector holoUnwrapAlgorithm has exactly two references in the whole tree: its declaration at UIManager.h:1674, initialised to 0, and the read at EngineVFX2.cpp:5828. No writer, so the branch could not be taken even if it had a body | not built |
| Structured-light decoding | Nothing in the repository decodes a projected pattern. No Gray-code sequence, no phase-shifted stripe decode, no correspondence search from projector to camera. The projector emits seven patterns and they are illumination, not code. What this stack decodes is phase, never pattern index | not built |
| Triangulation | Structurally impossible in this configuration, not merely unimplemented. The projector has no position of its own; every evaluation is relative to the camera. Zero baseline means zero disparity for any pattern at any depth | excluded by design |
| Transient rendering | The path-length filter is a gate on contributions, not a binned time axis, and nothing renormalises the gated frame. There is no transient histogram and no time-resolved film format | not built |
| A real second camera | The epipolar and stereo-correspondence passes both warp one depth buffer into a hypothetical second view — their own headers say hypothetical and simulates. The anaglyph output is a good screenshot and it is not a stereo rig | seen, correctly labelled in source |
On the branch-cut row. The claim it replaces was not invented; it was read off a declaration in a header without opening the definition three hundred lines away. That is a specific failure mode worth naming, because a declared method, an allocated output texture and a live else branch are exactly the evidence a real implementation leaves behind. The falsifying detail is in the UI: the display combo skips the two indices reserved for branch-cut residues and cuts, so the feature's own display stages were never wired either. The reachability record has been corrected in place; this page states the corrected version and the route by which the wrong one was reached.
The triangulation row is the one worth dwelling on, because it is a design consequence rather than an omission. Collocating the projector with the camera is what makes the time-of-flight path clean: the outgoing and return legs share a geometry, the round-trip length is twice the camera-to-surface distance for a point source, and the shape-specific corrections for a collimated panel or a line source are one branch each in evalProjectorTOF. A projector with its own transform would buy triangulation and would cost that. The stack chose phase over disparity, and every capability and every gap on this page follows from that one choice.
An earlier version of this page put a fourth item in the list above: camera-side sweeping primitives, graded design only, on the grounds that they would need a splat-buffer film path and this renderer does not have one. That was wrong, and it was wrong in the specific way this project has a written rule against — it answered is this file live when the question was is this feature reachable.
The estimator research elsewhere on this site sweeps sampling dimensions of the light subpath into a surface. The construction dualises to the camera subpath, and the sphere case is built and reachable: it is strategy 17 in the PP estimator, a live entry in the strategy combo (UIManagerRender.cpp:249, and again in the full PP tab at :7793), implemented as ppCameraSphereSplatPass (PPIntegrator.cpp:7719-7804) and dispatched as its own branch of the film loop (:8935-8941). It samples a photon, free-flights it to a scatter point in fog, projects that point to a pixel, and splats a single-scatter contribution weighted by the pinhole camera importance We = (w·h)/(4·halfW·halfH·cos³θ). The splat buffer is real: sixteen per-thread images summed and blended into the running-mean film, which is how it avoids atomics.
It is a first version, and its limits are legible in its own first eleven lines. It takes the PP config by reference and never reads a single field of it — not one cfg. in its eighty-six lines — so every setting in the panel is ignored. It requires the photon to start inside the fog (:7741), which restricts it to a light-in-fog scene. It applies fog transmittance on the camera leg but performs no occlusion test, so it casts no shadows. And the area-emitter cosine is unhandled, named as a follow-up in the header comment. It was validated at a whole-image ratio of 0.9992 against a reference — a converged number from the renderer, not from a derivation.
sampleOnePixel.When this page carries results instead of instructions, the captures have a dependency order, and the early ones are the ones that can fail.
Two operational notes that will otherwise cost a session. There is no screenshot function in the application — F12 logs "Snapshot requested" and writes nothing (Engine.cpp:388-391), and File › Export Image is a stub whose menu item calls notImplementedBorder() (UIManager.cpp:359) — so these are operating-system captures. And the window size is read from prism.json relative to the working directory, so the copy that matters is the one beside the executable; both the build script and a CMake post-build step copy the repository root's copy over it, which means a rebuild silently reverts whatever you set for capture. Both are recorded in docs/CAPTURE-SHOOTING-LIST.md rather than discovered again.