NODEDC_MISSION_CORE/docs/adr/0044-ai-assisted-gaussian-visual-repair.md

# ADR 0044: AI-assisted Gaussian visual repair remains a candidate pipeline

## Status

Proposed for a Worker 006 ROI experiment on 2026-08-29. The existing Gaussian optimization and
collision paths remain accepted and unchanged. No repair model is admitted for production use by
this decision.

## Context

The current Simulation Worlds vertical accepts one XGRIDS LCC/LCC2 source, builds preview and
streamed SOG artifacts through DC Gaussian Pipeline, and publishes source-mesh collision
independently. The two current ready projects prove that path. AutoCap repairs only conservative
holes in the source collision mesh; it does not repair the visual Gaussian representation.

The retained MAROSEYKA archive contains an LCC Quality scene with 86,471,152 splats and a
44,787-sample 10 Hz device trajectory. It does not contain the captured camera images, camera
intrinsics or camera extrinsics. The pose records do not identify RGB frames and cannot be treated
as calibrated camera poses.

AI repair methods that permanently improve a 3DGS model need rendered novel views, real reference
images and known cameras. Plausible generation alone cannot recover the factual appearance or
geometry of a surface that was never observed.

XGRIDS LCC Studio Creator Data provides the missing supported interchange boundary:

- `perspective/images/` contains undistorted perspective images;
- `perspective/masks/` contains invalid-region masks;
- `perspective/sparse/` contains COLMAP camera intrinsics and extrinsics aligned with the optimized
  LiDAR point cloud;
- `poses.csv` and `high_frequency_poses.csv` retain device trajectories when they are useful for
  selecting a repair corridor.

## Decision

1. The immutable original LCC/LCC2 bundle is the visual source of record. SOG is a compressed web
   delivery artifact and is never the input to AI repair.
2. A repair candidate starts from full-quality `LCC/LCC2 -> standard 3DGS PLY` conversion. It may
   operate only on bounded spatial tiles or route-derived regions of interest; loading or training
   the complete 86M-splat scene as one model is not an accepted Worker 006 profile.
3. A repair request additionally admits XGRIDS Creator Data. The minimum input is undistorted
   images plus a complete COLMAP sparse model. Masks are strongly preferred. Device poses alone do
   not satisfy camera admission.
4. The first experiment uses FreeFix at an exact source revision with the SDXL refinement path. It
   was selected because it is fine-tuning-free at the diffusion-model level, uses per-pixel
   confidence to preserve reliable regions, reports outdoor/Waymo evaluation, and publishes MIT
   code. An adapter must import standard XGRIDS/PlayCanvas PLY attributes into the gsplat checkpoint
   layout and export a standard PLY after refinement.
5. Difix3D+ is the mandatory comparison baseline for the same ROI. It directly targets artifacts in
   under-constrained views and supports progressive distillation into gsplat, but its combined
   NVIDIA/Stability licensing needs a separate commercial-use review.
6. The experiment runs in an adjacent `ndc-` prefixed Docker composition on Worker 006. It shares
   neither Python/CUDA environments nor writable model directories with DC Gaussian Pipeline. The
   existing pipeline is called only after a candidate PLY is complete and immutable.
7. Every result is a candidate generation. Mission Core keeps the active visual generation until
   an operator compares the baseline and candidate and explicitly promotes the candidate. Failure
   or cancellation cannot modify the active scene.
8. Generated visual content never becomes collision, navigation, traversability or ground-truth
   evidence. Existing source-mesh collision and AutoCap remain independent. Each candidate retains
   an uncertainty/hallucination mask and exact model/config provenance.

## Candidate contract

A quality-enhancement request must bind all inputs by digest:

- project ID and original source-bundle SHA-256;
- LCC/LCC2 entrypoint and full-quality PLY conversion revision;
- Creator Data bundle SHA-256;
- COLMAP cameras, images and points model plus their coordinate-alignment report;
- selected route interval and/or world-space ROI;
- algorithm, source revision, model IDs/digests, prompt policy and numerical parameters.

The adjacent provider returns:

- a standard enhanced PLY for the selected tile or composed candidate;
- preview and streamed SOG artifacts created by the unchanged optimization pipeline;
- baseline/candidate camera-path renders and objective image metrics where held-out real views
  exist;
- changed-region, confidence and generated-content masks;
- wall time, peak VRAM, source revision, image digest and terminal job state.

The provider states are transport-neutral and bounded: `queued`, `validating_source`,
`aligning_cameras`, `extracting_roi`, `importing_splats`, `refining`, `building_delivery`,
`evaluating`, `ready`, `failed`, and `cancelled`.

## Product placement

The action belongs in the existing project edit window as `Улучшить качество`, because it acts on
one durable world and does not create a new workspace. The action opens a canonical bounded Window
that owns Creator Data admission, ROI choice, actual provider state and baseline/candidate review.
It is shown only for a ready outdoor/interior project and becomes an executable action only when
the enhancement provider publishes the exact accepted capability.

Alternatives considered:

1. Add controls to the live PlayCanvas scene. Rejected because source admission and a long-running
   candidate lifecycle are project operations, not per-frame runtime controls.
2. Add a new top-level workspace. Rejected because repair has no independent identity outside one
   Simulation World project.
3. Add an always-enabled button before provider/source admission exists. Rejected because it would
   be placeholder product UI and would misrepresent the current system state.

## Experiment acceptance

The first ROI experiment is accepted only when:

- the adjacent composition starts and stops without changing the current Gaussian Pipeline
  containers or native SplatTransform spool;
- one Creator Data bundle passes COLMAP/image/alignment validation;
- one bounded road-and-facade ROI fits the RTX 4090 24 GB profile without host OOM or impact on the
  active optimization queue;
- FreeFix and Difix3D+ run on identical admitted cameras and ROI;
- candidate output round-trips through standard PLY and the existing SOG build;
- held-out views and an operator review show improvement without unacceptable changes in reliable
  regions;
- active SOG and collision artifacts remain byte-identical until explicit promotion.

## Consequences

- The first useful input request is XGRIDS Creator Data, not a raw device track and not SOG.
- Existing ready RAR archives cannot start factual image-conditioned repair by themselves.
- Large scenes require ROI/tile scheduling, overlap blending and candidate composition.
- Visually plausible fill may be valuable for rendering while remaining inadmissible as physical
  truth.
- Worker deployment is intentionally blocked until a Creator Data sample and restored Worker 006
  provider connectivity are available.

## Primary references

- [XGRIDS Creator Data](https://docs.xgrids.com/en-us/06-lixel-cybercolor/01-lcc-studio/v2.3.0/06-model-reconstruction.html#creator-data-and-nvidia-ncore-data)
- [PlayCanvas SplatTransform](https://github.com/playcanvas/splat-transform)
- [FreeFix](https://github.com/hyzhou404/FreeFix)
- [Difix3D+](https://github.com/nv-tlabs/Difix3D)
- [GSFix3D](https://github.com/GSFix3D/GSFix3D)
- [ArtifactWorld](https://github.com/fyting/ArtifactWorld)