ADR 0044: AI-assisted Gaussian visual repair remains a candidate pipeline

Status

Proposed for a Worker 006 ROI experiment on 2026-08-29. The existing Gaussian optimization and collision paths remain accepted and unchanged. No repair model is admitted for production use by this decision.

Context

The current Simulation Worlds vertical accepts one XGRIDS LCC/LCC2 source, builds preview and streamed SOG artifacts through DC Gaussian Pipeline, and publishes source-mesh collision independently. The two current ready projects prove that path. AutoCap repairs only conservative holes in the source collision mesh; it does not repair the visual Gaussian representation.

The retained MAROSEYKA archive contains an LCC Quality scene with 86,471,152 splats and a 44,787-sample 10 Hz device trajectory. It does not contain the captured camera images, camera intrinsics or camera extrinsics. The pose records do not identify RGB frames and cannot be treated as calibrated camera poses.

AI repair methods that permanently improve a 3DGS model need rendered novel views, real reference images and known cameras. Plausible generation alone cannot recover the factual appearance or geometry of a surface that was never observed.

XGRIDS LCC Studio Creator Data provides the missing supported interchange boundary:

perspective/images/ contains undistorted perspective images;
perspective/masks/ contains invalid-region masks;
perspective/sparse/ contains COLMAP camera intrinsics and extrinsics aligned with the optimized LiDAR point cloud;
poses.csv and high_frequency_poses.csv retain device trajectories when they are useful for selecting a repair corridor.

Decision

The immutable original LCC/LCC2 bundle is the visual source of record. SOG is a compressed web delivery artifact and is never the input to AI repair.
A repair candidate starts from full-quality LCC/LCC2 -> standard 3DGS PLY conversion. It may operate only on bounded spatial tiles or route-derived regions of interest; loading or training the complete 86M-splat scene as one model is not an accepted Worker 006 profile.
A repair request additionally admits XGRIDS Creator Data. The minimum input is undistorted images plus a complete COLMAP sparse model. Masks are strongly preferred. Device poses alone do not satisfy camera admission.
The first experiment uses FreeFix at an exact source revision with the SDXL refinement path. It was selected because it is fine-tuning-free at the diffusion-model level, uses per-pixel confidence to preserve reliable regions, reports outdoor/Waymo evaluation, and publishes MIT code. An adapter must import standard XGRIDS/PlayCanvas PLY attributes into the gsplat checkpoint layout and export a standard PLY after refinement.
Difix3D+ is the mandatory comparison baseline for the same ROI. It directly targets artifacts in under-constrained views and supports progressive distillation into gsplat, but its combined NVIDIA/Stability licensing needs a separate commercial-use review.
The experiment runs in an adjacent ndc- prefixed Docker composition on Worker 006. It shares neither Python/CUDA environments nor writable model directories with DC Gaussian Pipeline. The existing pipeline is called only after a candidate PLY is complete and immutable.
Every result is a candidate generation. Mission Core keeps the active visual generation until an operator compares the baseline and candidate and explicitly promotes the candidate. Failure or cancellation cannot modify the active scene.
Generated visual content never becomes collision, navigation, traversability or ground-truth evidence. Existing source-mesh collision and AutoCap remain independent. Each candidate retains an uncertainty/hallucination mask and exact model/config provenance.

Candidate contract

A quality-enhancement request must bind all inputs by digest:

project ID and original source-bundle SHA-256;
LCC/LCC2 entrypoint and full-quality PLY conversion revision;
Creator Data bundle SHA-256;
COLMAP cameras, images and points model plus their coordinate-alignment report;
selected route interval and/or world-space ROI;
algorithm, source revision, model IDs/digests, prompt policy and numerical parameters.

The adjacent provider returns:

a standard enhanced PLY for the selected tile or composed candidate;
preview and streamed SOG artifacts created by the unchanged optimization pipeline;
baseline/candidate camera-path renders and objective image metrics where held-out real views exist;
changed-region, confidence and generated-content masks;
wall time, peak VRAM, source revision, image digest and terminal job state.

The provider states are transport-neutral and bounded: queued, validating_source, aligning_cameras, extracting_roi, importing_splats, refining, building_delivery, evaluating, ready, failed, and cancelled.

Product placement

The action belongs in the existing project edit window as Улучшить качество, because it acts on one durable world and does not create a new workspace. The action opens a canonical bounded Window that owns Creator Data admission, ROI choice, actual provider state and baseline/candidate review. It is shown only for a ready outdoor/interior project and becomes an executable action only when the enhancement provider publishes the exact accepted capability.

Alternatives considered:

Add controls to the live PlayCanvas scene. Rejected because source admission and a long-running candidate lifecycle are project operations, not per-frame runtime controls.
Add a new top-level workspace. Rejected because repair has no independent identity outside one Simulation World project.
Add an always-enabled button before provider/source admission exists. Rejected because it would be placeholder product UI and would misrepresent the current system state.

Experiment acceptance

The first ROI experiment is accepted only when:

the adjacent composition starts and stops without changing the current Gaussian Pipeline containers or native SplatTransform spool;
one Creator Data bundle passes COLMAP/image/alignment validation;
one bounded road-and-facade ROI fits the RTX 4090 24 GB profile without host OOM or impact on the active optimization queue;
FreeFix and Difix3D+ run on identical admitted cameras and ROI;
candidate output round-trips through standard PLY and the existing SOG build;
held-out views and an operator review show improvement without unacceptable changes in reliable regions;
active SOG and collision artifacts remain byte-identical until explicit promotion.

Consequences

The first useful input request is XGRIDS Creator Data, not a raw device track and not SOG.
Existing ready RAR archives cannot start factual image-conditioned repair by themselves.
Large scenes require ROI/tile scheduling, overlap blending and candidate composition.
Visually plausible fill may be valuable for rendering while remaining inadmissible as physical truth.
Worker deployment is intentionally blocked until a Creator Data sample and restored Worker 006 provider connectivity are available.

7.5 KiB Raw Blame History