SILVER/NODEDC_MISSION_CORE
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

docs(sim): зафиксировать контур AI-улучшения гауссов

Browse Source
This commit is contained in:
DCCONSTRUCTIONS
2026-08-30 11:57:49 +03:00
parent 757d368c86
commit e9ffb829c9
2 changed files with 156 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/20_SIMULATION_WORLD_SURFACE_BRIEF.md
+20
View File
@@ -122,3 +122,23 @@ Primary implementation references:
- a ready project mounts direct PlayCanvas Engine and loads Streamed SOG with preview fallback;
- visual, collision and combined remain distinct runtime modes, and collision is loaded lazily;
- viewer quality, layer-axis correction and camera inversion survive navigation and reload.
## AI-assisted visual quality extension
AI-assisted visual repair is a candidate lifecycle of one ready Simulation World project. It does
not replace or modify the accepted ingest/optimization path. The original LCC/LCC2 bundle remains
the source of record, while an explicitly promoted enhanced generation may replace only the active
visual SOG references.
The product entry point is `Улучшить качество` in the existing project edit window. It opens a
canonical bounded Window for XGRIDS Creator Data admission, region-of-interest selection, actual
provider state and baseline/candidate review. It is not placed in the live scene controls because
source admission and a long-running candidate build are project operations. It does not receive a
separate workspace because the candidate has no identity outside its parent project.
The control is not shipped as a placeholder. It becomes available only with a ready project and an
enhancement provider publishing the accepted capability. The provider consumes full-quality PLY
derived from the immutable LCC/LCC2 source plus aligned Creator Data images/COLMAP cameras; SOG is
delivery output only. Generated visual regions remain forbidden as collision, navigation or
qualification evidence. The detailed boundary and first experiment are defined by
[ADR 0044](adr/0044-ai-assisted-gaussian-visual-repair.md).
docs/adr/0044-ai-assisted-gaussian-visual-repair.md
+136
View File
@@ -0,0 +1,136 @@
# ADR 0044: AI-assisted Gaussian visual repair remains a candidate pipeline
## Status
Proposed for a Worker 006 ROI experiment on 2026-08-29. The existing Gaussian optimization and
collision paths remain accepted and unchanged. No repair model is admitted for production use by
this decision.
## Context
The current Simulation Worlds vertical accepts one XGRIDS LCC/LCC2 source, builds preview and
streamed SOG artifacts through DC Gaussian Pipeline, and publishes source-mesh collision
independently. The two current ready projects prove that path. AutoCap repairs only conservative
holes in the source collision mesh; it does not repair the visual Gaussian representation.
The retained MAROSEYKA archive contains an LCC Quality scene with 86,471,152 splats and a
44,787-sample 10 Hz device trajectory. It does not contain the captured camera images, camera
intrinsics or camera extrinsics. The pose records do not identify RGB frames and cannot be treated
as calibrated camera poses.
AI repair methods that permanently improve a 3DGS model need rendered novel views, real reference
images and known cameras. Plausible generation alone cannot recover the factual appearance or
geometry of a surface that was never observed.
XGRIDS LCC Studio Creator Data provides the missing supported interchange boundary:
- `perspective/images/` contains undistorted perspective images;
- `perspective/masks/` contains invalid-region masks;
- `perspective/sparse/` contains COLMAP camera intrinsics and extrinsics aligned with the optimized
LiDAR point cloud;
- `poses.csv` and `high_frequency_poses.csv` retain device trajectories when they are useful for
selecting a repair corridor.
## Decision
1. The immutable original LCC/LCC2 bundle is the visual source of record. SOG is a compressed web
delivery artifact and is never the input to AI repair.
2. A repair candidate starts from full-quality `LCC/LCC2 -> standard 3DGS PLY` conversion. It may
operate only on bounded spatial tiles or route-derived regions of interest; loading or training
the complete 86M-splat scene as one model is not an accepted Worker 006 profile.
3. A repair request additionally admits XGRIDS Creator Data. The minimum input is undistorted
images plus a complete COLMAP sparse model. Masks are strongly preferred. Device poses alone do
not satisfy camera admission.
4. The first experiment uses FreeFix at an exact source revision with the SDXL refinement path. It
was selected because it is fine-tuning-free at the diffusion-model level, uses per-pixel
confidence to preserve reliable regions, reports outdoor/Waymo evaluation, and publishes MIT
code. An adapter must import standard XGRIDS/PlayCanvas PLY attributes into the gsplat checkpoint
layout and export a standard PLY after refinement.
5. Difix3D+ is the mandatory comparison baseline for the same ROI. It directly targets artifacts in
under-constrained views and supports progressive distillation into gsplat, but its combined
NVIDIA/Stability licensing needs a separate commercial-use review.
6. The experiment runs in an adjacent `ndc-` prefixed Docker composition on Worker 006. It shares
neither Python/CUDA environments nor writable model directories with DC Gaussian Pipeline. The
existing pipeline is called only after a candidate PLY is complete and immutable.
7. Every result is a candidate generation. Mission Core keeps the active visual generation until
an operator compares the baseline and candidate and explicitly promotes the candidate. Failure
or cancellation cannot modify the active scene.
8. Generated visual content never becomes collision, navigation, traversability or ground-truth
evidence. Existing source-mesh collision and AutoCap remain independent. Each candidate retains
an uncertainty/hallucination mask and exact model/config provenance.
## Candidate contract
A quality-enhancement request must bind all inputs by digest:
- project ID and original source-bundle SHA-256;
- LCC/LCC2 entrypoint and full-quality PLY conversion revision;
- Creator Data bundle SHA-256;
- COLMAP cameras, images and points model plus their coordinate-alignment report;
- selected route interval and/or world-space ROI;
- algorithm, source revision, model IDs/digests, prompt policy and numerical parameters.
The adjacent provider returns:
- a standard enhanced PLY for the selected tile or composed candidate;
- preview and streamed SOG artifacts created by the unchanged optimization pipeline;
- baseline/candidate camera-path renders and objective image metrics where held-out real views
exist;
- changed-region, confidence and generated-content masks;
- wall time, peak VRAM, source revision, image digest and terminal job state.
The provider states are transport-neutral and bounded: `queued`, `validating_source`,
`aligning_cameras`, `extracting_roi`, `importing_splats`, `refining`, `building_delivery`,
`evaluating`, `ready`, `failed`, and `cancelled`.
## Product placement
The action belongs in the existing project edit window as `Улучшить качество`, because it acts on
one durable world and does not create a new workspace. The action opens a canonical bounded Window
that owns Creator Data admission, ROI choice, actual provider state and baseline/candidate review.
It is shown only for a ready outdoor/interior project and becomes an executable action only when
the enhancement provider publishes the exact accepted capability.
Alternatives considered:
1. Add controls to the live PlayCanvas scene. Rejected because source admission and a long-running
candidate lifecycle are project operations, not per-frame runtime controls.
2. Add a new top-level workspace. Rejected because repair has no independent identity outside one
Simulation World project.
3. Add an always-enabled button before provider/source admission exists. Rejected because it would
be placeholder product UI and would misrepresent the current system state.
## Experiment acceptance
The first ROI experiment is accepted only when:
- the adjacent composition starts and stops without changing the current Gaussian Pipeline
containers or native SplatTransform spool;
- one Creator Data bundle passes COLMAP/image/alignment validation;
- one bounded road-and-facade ROI fits the RTX 4090 24 GB profile without host OOM or impact on the
active optimization queue;
- FreeFix and Difix3D+ run on identical admitted cameras and ROI;
- candidate output round-trips through standard PLY and the existing SOG build;
- held-out views and an operator review show improvement without unacceptable changes in reliable
regions;
- active SOG and collision artifacts remain byte-identical until explicit promotion.
## Consequences
- The first useful input request is XGRIDS Creator Data, not a raw device track and not SOG.
- Existing ready RAR archives cannot start factual image-conditioned repair by themselves.
- Large scenes require ROI/tile scheduling, overlap blending and candidate composition.
- Visually plausible fill may be valuable for rendering while remaining inadmissible as physical
truth.
- Worker deployment is intentionally blocked until a Creator Data sample and restored Worker 006
provider connectivity are available.
## Primary references
- [XGRIDS Creator Data](https://docs.xgrids.com/en-us/06-lixel-cybercolor/01-lcc-studio/v2.3.0/06-model-reconstruction.html#creator-data-and-nvidia-ncore-data)
- [PlayCanvas SplatTransform](https://github.com/playcanvas/splat-transform)
- [FreeFix](https://github.com/hyzhou404/FreeFix)
- [Difix3D+](https://github.com/nv-tlabs/Difix3D)
- [GSFix3D](https://github.com/GSFix3D/GSFix3D)
- [ArtifactWorld](https://github.com/fyting/ArtifactWorld)