LightField / Neural

A photograph records directional information at one viewpoint. A calibrated collection records it at many positions. The workbench places those samples back into space and lets a virtual camera move through them: recorded images become the material from which another image is rendered.

The virtual-source interpretation makes the geometry visible. Each sample position carries many directional colors, rather than one uniform light intensity. Moving the viewer, blending neighboring samples or changing the synthetic aperture produces parallax and focus effects directly from this field. NeRF and Gaussian splatting extend the study with learned representations; the raw light-field renderer is a complete experiment in its own right.

A sparse field does more than blur. Nearest-view selection breaks it into territories; a wider matching kernel lets images overlap; a proxy-depth sweep stretches them into layers. These modes expose different visual structures in the same observations, before any neural model is fitted.

I treat those artifacts as material to explore, much as in the photon-primitive studies. The artistic question is where to preserve a recognizable view and where to let the representation become strange. Missing observations still limit reconstruction: a striking extension is not evidence that an unseen view is physically correct.

Two captured scenes take the experiment beyond the Cornell box. Copper and glass pairs moving reflections with a brushed-metal plinth; Cloud on brushed metal replaces the central surface with an actual heterogeneous participating medium. The native renderer supplies the observations. The learned models reproduce their appearance with different compromises.

The copper rig records 224 views around a full orbit. The cloud rig uses 147 overlapping views across a front arc from −75° to +75°. Both capture 192 × 192 images at 128 samples per pixel. These modest input images keep training practical; a larger output does not create missing source detail.

The saved Gaussian, Instant-NGP and NeRF checkpoints can open directly in Model Viewer. For recording, use the Gaussian model for smoother camera movement; the PyTorch neural renderers suit slow moves and settled views. The native reference remains available alongside each approximation. Reflection blur, splat artifacts and uncaptured viewpoints are part of the comparison.

Capture generates images from a configured family of cameras. The implementation supports planar grids, look-at planar grids, rings, spheres, spherical caps and parabolic arrangements. A rig carries positions, orientation rules, field of view, resolution and sampling settings. Capture metadata records the resulting views and sidedness so the inspection tools can interpret the array.

A planar array gives the field a clear spatial grid; ring and sphere layouts distribute directional views around a region. The scene map and camera markers make this arrangement part of the rendered experience. Changing the baseline or orientation changes the ray samples available to a moving viewer, connecting acquisition choices to parallax, overlap and disocclusion.

Scene presets coordinate acquisition and training settings, but capture and training remain explicit operations. This is useful when revisiting an experiment: the rig, dataset and model configuration can be examined separately instead of assuming that one preset label defines every part of the run.

Coverage and density solve different problems. A front-facing grid can be locally dense but says little about the back of an object; a coarse sphere sees more sides but leaves large jumps between observations. Increasing blur joins sparse contributions at the cost of detail. More overlapping views address the sampling gap, while lower per-view resolution keeps acquisition and storage practical. Neither interpolation nor blur recovers an unseen disocclusion.

The raw field has its own renderer. A free camera queries the recorded directional images, while Sharp and Smooth matching change how tightly a ray selects nearby capture positions. The scene map, sample markers and ray-coverage display connect the synthesized image to the data that contributes to it.

Refocusing turns the camera array into a synthetic aperture. Views are reprojected toward a chosen focus plane and combined: features aligned by that plane stay sharper, while other depths spread across the image. Aperture changes the contributing region of the array. Epipolar slices provide a complementary view of the same geometry by laying image rows alongside their camera positions.

For a rectified pair of cameras, a useful approximation is residual disparity Δx ≈ f B (1/z − 1/z₍focus₎), with focal length f in pixels and baseline B in scene units. A wider baseline makes wrong-depth features separate more strongly; denser camera sampling fills that blur more smoothly. Moving focus changes alignment, while increasing the ray-matching kernel merely mixes less similar observations. They are different controls.

The explicit light field stores captured images and their calibrated positions. Learned representations reorganize those observations: NeRF into neural density and directional color, Gaussian splatting into spatial primitives. They can be explored under a corresponding inspection camera, making the choice of representation visible rather than hiding it behind one final render.

The NeRF route represents density and view-dependent color with neural functions and renders by accumulating samples along camera rays. The project includes coarse/fine sampling and training orchestration around this model. A representation can fit observations while still blurring edges or leaving unsupported regions ambiguous; the saved test renders make those effects inspectable.

Gaussian splatting uses an explicit collection of spatial Gaussian primitives. Their projected shapes, appearance and opacity contribute through a differentiable renderer. The reference backend uses gsplat rasterization and its density-control strategy; the project also includes my own experimental renderer.

Both methods can reproduce appearance from a calibrated set of observations. Neither image quality nor a completed training cycle alone establishes correct geometry, relightable material, or generalization outside the captured region. The workbench keeps those questions separate from the basic ability to synthesize a view.

The C++ viewer launches Python jobs with explicit dataset, method, backend and profile settings. Status files, subprocess output, previews and checkpoints connect the running training loop back to the application. A queue can retain later jobs while the current job continues. The application reports failed stages explicitly, so a stalled job can be distinguished from a slowly updating preview.

Fast profiles are explicit budgets for each registered method, not a promise that every representation converges equally quickly. The standard NeRF preview uses 1,200 iterations and a 96-pixel evaluation image; Instant-NGP uses 600 iterations at the same preview size. Six experimental methods use smaller 500-iteration configurations. Their previews help inspect whether training is making progress, while longer runs remain necessary for difficult geometry and held-out views.

Training progress and final evaluation also mean different things. A high training PSNR can coexist with a weaker held-out view. The interface can show the process, while a report must identify the dataset partition and the quantity actually measured.

Some representations need more than images. Neural textures requires a mesh that agrees with the capture; its fallback Cornell mesh is a proxy, not recovered geometry. Fast mode now reports coverage and rejects an empty training mask, instead of reporting a deceptively perfect score for an all-black result. A successful short run validates the pipeline, not convergence.

• Select a scene and acquisition rig; render and inspect the capture before starting training.
• Choose method, backend and profile explicitly; retain the dataset and parameter files with the run.
• Watch training status, preview updates and checkpoint creation as separate stages.
• Render saved test cameras and inspect errors before selecting a checkpoint for live exploration.

The supplied narrow Cornell capture contains 64 cameras: 56 training views and eight test views, with 400×400 images. Saved NeRF and reference-GS runs include test renders and per-view metrics. The images below use the first test camera, paired through the dataset frame and the producer's index convention rather than through matching filenames.

NeRF was recorded at 3,000 steps and reference GS at 5,000. Mean PSNR across the decoded PNG test images is approximately 29.37 and 23.41 dB respectively, using the corrected evaluator. The runs used different budgets and not all settings were saved, so these numbers describe the examples shown rather than rank the methods.

The comparison is most useful visually. Inspect surface softness, wall boundaries and the ceiling emitter. The captured target itself contains rendering noise, so the reference is a sampled synthetic observation rather than an exact noise-free image.

The inspection camera moves through a directly rendered light field, a supported learned checkpoint and a source-scene reference. These views expose different ways of carrying the same observed appearance. The model registry distinguishes the project’s Gaussian, NeRF and Instant-NGP routes from the more exploratory methods; a method appearing in the registry is not a claim of equal preview speed or reconstruction quality.

The native application also projects the inspection camera into the raw light-field parameterization and can render a scene reference. This creates a useful three-way inspection of captured-ray interpolation, learned representation and known source scene. Synchronizing the camera does not automatically make the information budgets equal: the saved raw array includes observations that were excluded from model training.

A controlled interpolation experiment should restrict the raw-LF baseline to the same training observations and evaluate a newly held-out pose. The live mode remains valuable for finding coverage gaps, but those gaps should be interpreted with the permitted input views and the rig's parameterization in view.