Object-local meshes
Complete assets supplied by a replaceable image-to-3D provider, independent of its pose or layout predictions.
Easy-3Dfy Object-Level Geometric Composition for Single-Image 3D Scene Construction
Qualitative comparisons across multi-object examples, showing object layout and scene assembly.
Principle
A complete mesh lives in an object-local coordinate system. A scene pointmap captures visible surfaces in a shared scene frame. Easy-3Dfy connects them through object-level point cloud registration with a Rectified Point Flow-Matching (RPF) aligner.
A scene pointmap
Pixel-aligned scene geometry provides the common reference frame and an object-specific partial observation.
Local-to-pointmap-scene transforms
An explicit uniform-scale similarity transform maps each complete mesh into the shared pointmap scene frame.
Method
Replaceable asset and scene-geometry providers are linked by an RPF-based geometric aligner.
01Extract
Identify the visible object
Accept masks from an upstream automatic or user-prompted segmentation model. Each mask defines the visible object region and crop; mask estimation is outside this work.
Image + object masks
02Complete & generate
Build an object-local asset
Occlusion-aware image editing completes the object using crop, mask, and scene context. A plug-and-play image-to-3D provider converts the completed image into a complete object-local mesh.
Completed image → local mesh
03Observe geometry
Construct query and anchor
A scene pointmap and normals provide a masked, filtered partial query for each object. Sample anchor points and normals from the corresponding complete local mesh.
Partial query + mesh anchor
04Align & assemble
Recover the scene transform
RPF uses XYZ + normal features to predict each query point’s correspondence position in the anchor frame. Unweighted Umeyama/SVD fitting recovers a uniform-scale Sim(3) transform, applied to the complete mesh.
Correspondences → Sim(3) → scene
RPF training
The RPF aligner learns correspondence positions from individual object meshes and rendered partial observations, without complete scene layouts or explicit inter-object relationship supervision.
Object-level geometry supplies the partial query, complete anchor, and ground-truth correspondence positions.
Render & observe
Sample mesh anchors and normals. Render pixel-aligned RGB and depth from sampled viewpoints; MogeV2 estimates the partial query and normals from the rendered RGB.
Match through depth
Random occlusions remove query observations. At each retained pixel, rendered depth supplies the matching ground-truth position on the complete object surface.
Augment independently
Apply independent rotations, scales, and translations to query and anchor-side groups, with correspondence targets kept in the anchor group. Axis-wise scaling perturbs object shape.
Conditioned on anchor and query geometry, RPF learns a rectified-flow velocity field that transports noise to query-to-anchor correspondence positions.
Qualitative results
The main comparison shares identical SAM-3Dfy meshes between SAM-3Dfy and Easy-3Dfy, isolating layout prediction. Easy-3Dfy registers each asset to its pointmap-derived partial query.
Matched inputs and views reveal differences in scale, orientation, ordering, spacing, and object contact.
Full-scene views and close-ups separate geometric compatibility from visual detail. Dimensions and proportions that match the partial observation help alignment; richer texture alone does not guarantee a better layout.
Full scenes show extent and placement; close-ups expose texture and fine surface detail.
Additional visualizations on 3D-FUTURE from the appendix extend the qualitative comparison beyond the main examples.
Additional 3D-FUTURE scene-construction visualizations from the appendix.