Object to 3d

Object to 3D (clean splat + printable STL, 100% local)

Turn a video orbiting a single object into two deliverables, entirely on-device:

1. a clean Gaussian splat of just the object (background, table and floaters removed) that you can navigate in the browser, and
2. a watertight, millimeter-scaled STL (plus a colored GLB) ready to slice and 3D-print.

The capture → splat half reuses the proven video-to-splat pipeline; this skill adds the two object-specific stages that follow training.

flowchart LR
  Mp4["orbital video<br/>of the object"] --> Ex["extract_frames.py<br/>object defaults"]
  Ex --> Colmap["run_colmap.py<br/>exhaustive matcher"]
  Colmap --> Brush["train_splat.sh<br/>Brush (Metal) -> splat.ply"]
  Brush --> Clean["clean_splat.py<br/>opacity + scale + plane + DBSCAN"]
  Clean --> Prev["preview.sh<br/>Splat Preview (cleaned.ply)"]
  Clean --> Mesh["splat_to_mesh.py<br/>Poisson -> orient -> solidify -> flat base"]
  Mesh --> Stl["object.stl (watertight print)<br/>object.glb (web)"]
  Stl --> PrintPrev["preview.sh --print<br/>Print Preview (object.glb)"]

Two browser previews serve different jobs: Splat Preview shows the raw scan (honest capture quality, may show holes on faces the camera never saw, like the underside); Print Preview shows the repaired, watertight mesh that actually gets printed.

Everything lives outside the repo under ~/.video-to-splat/ (shared with video-to-splat: same venv, Brush binary, and projects/), plus an orbit viewer in ~/.video-to-splat/viewer-object/. Nothing is ever uploaded anywhere.

Prerequisites

• Apple Silicon Mac (M1-M4), macOS 14+. Brush trains on the Apple GPU via WebGPU/Metal and the pycolmap wheels are macOS-14 arm64 - there is no CUDA/CPU fallback for training/SfM.
• uv (Python env). If missing: curl -LsSf https://astral.sh/uv/install.sh | sh.
• ffmpeg and node/npx: brew install ffmpeg node.
• ~4-6 GB free disk for the Brush binary, pycolmap, open3d, and the viewer's npm deps (downloaded once). Internet is needed only for that first setup.
• A browser with WebGL2: Chrome, Edge, Firefox, or Safari (for the splat preview).
• Mesh extraction (clean_splat.py, splat_to_mesh.py) is pure Python (open3d + trimesh) and runs on any CPU - only SfM/training need the Apple GPU.

Setup

Resolve the skill directory and run setup once. It creates/verifies the shared ~/.video-to-splat layout (venv, Brush, projects), adds open3d/trimesh to the venv, and installs the object orbit viewer:

SKILL_DIR="<the folder this SKILL.md lives in>"   # e.g. .cursor/skills/object-to-3d
bash "$SKILL_DIR/scripts/setup_env.sh"

Then set the handles the scripts use (setup prints them too):

VTS_HOME="${VIDEO_TO_SPLAT_HOME:-$HOME/.video-to-splat}"
PY="$VTS_HOME/.venv/bin/python"

If you already ran video-to-splat's setup, this reuses it and only adds the extra Python deps + the orbit viewer.

Workflow

Copy this checklist and track progress:

- [ ] 1. Setup: run setup_env.sh (first time only)
- [ ] 2. Extract frames from the orbital mp4 (extract_frames.py)
- [ ] 3. Recover camera poses + sparse cloud (run_colmap.py, exhaustive) - CHECK % registered
- [ ] 4. Smoke-train ~2000 steps to validate poses before committing time
- [ ] 5. Full train (train_splat.sh, default 30000 steps) -> splat.ply
- [ ] 6. Clean the splat (clean_splat.py) -> cleaned.ply ; preview.sh to verify, iterate flags
- [ ] 7. Extract the mesh (splat_to_mesh.py --size-mm N) -> watertight object.stl + object.glb + turntable PNGs
- [ ] 8. Verify the print mesh in the browser (preview.sh --print) and check the watertight report
- [ ] 9. Deliver: cleaned.ply (navigable) + object.stl (printable)

Step 2: Extract frames

"$PY" "$SKILL_DIR/scripts/extract_frames.py" /path/to/object.mp4 --name my-object

• Object defaults: --fps 3, --max-frames 150 (denser than a room tour - an orbit revisits the object from many close angles, and objects are small so frames stay sharp). Keeps the sharpest frame per window and drops near-dupes.
• Aim for 80-150 well-spread, in-focus frames covering the whole object.
• Prints the project dir. Frames land in ~/.video-to-splat/projects/my-object/images/.

Step 3: Camera poses (Structure-from-Motion)

"$PY" "$SKILL_DIR/scripts/run_colmap.py" my-object            # exhaustive matcher (default)

• Exhaustive matching is the default here - an orbit is a loop of unordered close-up views, so any-to-any pairing is the robust choice (O(n^2), fine to ~400 frames). For a long single-pass sweep you can fall back to --matcher sequential.
• This is the make-or-break step. Read the reported "% registered". For a clean object orbit, aim >90%. If well under that, fix the capture/matching before training (see Capture guidance and REFERENCE.md).
• Writes the COLMAP model to projects/my-object/sparse/0/ (largest first).

Step 4: Smoke-train first (strongly recommended)

bash "$SKILL_DIR/scripts/train_splat.sh" my-object --steps 2000
bash "$SKILL_DIR/scripts/preview.sh" my-object --file splat.ply

The gate is visual. A broken pipeline can still export a well-formed .ply that renders as noise. Open the preview and confirm you can recognize the object before starting a full run. Fuzzy-but-recognizable → full training will sharpen it. Unrecognizable nebula → poses/input are broken; revisit steps 2-3.

Step 5: Full training

bash "$SKILL_DIR/scripts/train_splat.sh" my-object --steps 30000

• Trains with Brush and exports projects/my-object/splat.ply.
• Objects are small scenes, so training is comparatively fast (an object of ~100-150 frames is roughly single-room scale: budget ~1-2 min per 1000 steps on an M-series, i.e. tens of minutes for a full run - give the pessimistic number and do other work meanwhile).
• Progress: headless Brush prints nothing, but splat.ply is (over)written every --export-every steps (default 1000) - watch its mtime.

Step 6: Clean the splat

Isolate just the object - remove the diffuse background, the support surface (table/floor) and translucent floaters:

"$PY" "$SKILL_DIR/scripts/clean_splat.py" my-object               # auto clean -> cleaned.ply
bash  "$SKILL_DIR/scripts/preview.sh"    my-object --file cleaned.ply

Filters run in order (all tunable, see Key options and REFERENCE.md):

1. Opacity - drop near-transparent floaters (sigmoid(opacity) < 0.4).
2. Scale - drop the few gigantic, diffuse background blobs (top scale pctl).
3. Support plane - RANSAC-detect and remove the table/floor (on by default; --keep-plane to disable when the object has no flat contact surface).
4. Clustering - DBSCAN the remaining centers and keep the dominant cluster (the orbited object), dropping detached background islands.
5. Manual crop (optional) - after looking at the preview, tighten with --radius R (and --center x,y,z) to cut anything left.

Iterate against the preview. If background survives, raise --min-opacity or lower --scale-pctl; if part of the object is eaten, relax them or add --keep-plane. cleaned.ply keeps every Gaussian attribute, so it previews in the orbit viewer exactly like the trained splat.

Step 7: Extract the printable mesh

"$PY" "$SKILL_DIR/scripts/splat_to_mesh.py" my-object --size-mm 80

• Densifies each Gaussian into oriented surface samples, estimates normals, runs Poisson reconstruction, trims the low-density fringe, and keeps the largest connected component.
• Then makes it actually printable (see below): orients the object's largest flat face down, solidifies it into a guaranteed-watertight manifold that is also tunnel-free (voxel closing seals hole-like tunnels through thin walls), and cuts a flat print base. Reports watertightness, genus (0 = no through-holes anywhere), dimensions and volume.
• --size-mm N scales the longest dimension to N millimeters (STL's de-facto unit) - SfM has no metric scale, so you must set the real size for a correct print. Default 100 mm.
• Outputs into projects/my-object/mesh/:
  • object.stl - watertight mesh for the slicer (print),
  • object.glb - colored, oriented mesh for the web / Print Preview / QuickLook,
  • object-turntable-*.png - quick software-rendered thumbnails to eyeball it.
• Prefers cleaned.ply; pass --file splat.ply to mesh the raw splat, or --no-densify for a faster (coarser) pass. Raise --depth (Poisson octree, default 9) for more detail at the cost of noise/time.

How holes get closed (and why). An object filmed sitting on a surface is never seen from below, and clean_splat.py also strips the support plane - so the raw scan has a hole where the contact face should be, plus smaller gaps wherever coverage was thin. A Gaussian splat has no topology, so this can only be fixed on the mesh. Note that "watertight" alone is NOT enough: a mesh can be topologically closed yet riddled with through-tunnels (like a donut) that read as holes and ruin the print. By default (--base-repair auto) the script keeps Poisson's fully closed surface (no density trimming), voxel-remeshes it with morphological closing into a single solid with genus 0 - watertight and tunnel-free - and slices a flat base anchored to the lowest scan points. The result sits flat on the print bed and slices cleanly. Surfaces the camera never saw (underside, deep creases) are generated, not faithful scan detail.

• Want the true underside? Capture it: flip the object and run a second pass, or shoot it elevated (on a clear stand) so the camera sees under it.
• Just want a solid, stable print (shoes, toys, product mockups)? The default flat base is ideal - no extra capture needed.
• Tuning: --base-cut (how much of the ragged bottom to trim, default 3%, anchored to the lowest scan points), --voxel (solidify resolution; higher = more detail but tunnels reappear sooner - raise --close together with it), --close (tunnel-sealing strength), --base-repair flat|none, --solidify always|none, --smooth N (Taubin).
• Strict by default: if the mesh can't be made watertight the script writes an inspection GLB, prints why, and exits non-zero rather than emit a questionable STL. Pass --allow-open to force export anyway. Always check the genus line: if it is > 0 the surface still has that many hole-like tunnels - raise --close (e.g. 4) or lower --voxel and re-run.

Step 8: Verify the print mesh

Look at the actual print geometry (not the holey scan) in the browser:

bash "$SKILL_DIR/scripts/preview.sh" my-object --print      # loads mesh/object.glb

This Print Preview stands the repaired mesh on the pedestal so you can confirm the flat base and closed surface before slicing. The plain preview.sh my-object still shows the raw Splat Preview (scan data). Also check the report from Step 7: watertight : True and genus : 0 together mean there are no holes or tunnels anywhere in the print mesh.

Step 9: Deliver

Deliverables live under ~/.video-to-splat/projects/my-object/: cleaned.ply (navigable object splat), mesh/object.stl (watertight printable), mesh/object.glb (web view / Print Preview), and the intermediate splat.ply master.

Capture guidance (the #1 quality lever)

Object reconstruction quality is set on the camera far more than in any flag:

• Orbit the object 360° at two heights (a low ring ~15° above the table and a high ring ~45° looking down) so top and sides are both covered. Move slowly.
• Keep the object still and the background textured. Photogrammetry needs features to track: put the object on a patterned surface / newspaper, NOT a clean white sweep. A plain background gives COLMAP nothing to anchor on. (Turntable-with-static-camera is the opposite and usually fails - the object looks static against a moving world; orbit the camera instead.)
• Lock exposure and focus, use bright, even, diffuse light (no hard shadows that move with the object, no hotspots/specular glare).
• Cover the whole surface, including the top. The bottom (contact face) can't be seen from a normal orbit; the mesh stage closes it with a generated flat base. If you need the real underside, do a second pass with the object flipped (or elevated on a clear stand) so the camera sees under it.
• Avoid mirrors, glass, chrome, thin transparent parts and featureless matte surfaces - SfM and splatting both struggle with them.

Key options

Script	Option	Default	Purpose
extract_frames.py	--fps	3	Target selected frames per second.
	--max-frames	150	Cap on frames (COLMAP time grows fast).
	--max-size	1600	Downscale long side (px).
run_colmap.py	--matcher	exhaustive	exhaustive (orbit, default) or sequential.
	--max-features	8192	SIFT features/image (raise for low-texture objects).
	--relaxed	off	Lower mapper thresholds; registers more frames.
train_splat.sh	--steps	30000	Training iterations (try 2000 to smoke-test).
	--sh-degree	2	SH degree 0-4 (higher = shinier + bigger).
clean_splat.py	--min-opacity	0.4	Drop Gaussians below this rendered opacity.
	--scale-pctl	98	Drop Gaussians above this percentile of max scale.
	--keep-plane	off	Skip RANSAC support-plane removal.
	--plane-thresh	auto	RANSAC inlier distance (scene-relative).
	--min-frac	0.1	Plane removed only if it holds ≥ this fraction.
	--eps	auto	DBSCAN neighborhood radius (auto-grows until a cluster dominates).
	--min-dominant	0.5	Grow eps until the biggest cluster holds this fraction (anti-shatter).
	--keep-clusters	1	How many top DBSCAN clusters to keep.
	--no-cluster	off	Skip clustering (for already-isolated splats).
	--radius / --center	off	Manual spherical crop after the preview.
splat_to_mesh.py	--size-mm	100	Scale longest dimension to N mm (set the real size!).
	--depth	9	Poisson octree depth (detail vs. noise/time).
	--density-quantile	0/0.03	Trim low-density Poisson fringe (0 while base repair is active - trimming punches holes; 0.03 with --base-repair none).
	--samples-per-splat	4	Surface samples per Gaussian (densify).
	--no-densify	off	Use Gaussian centers only (faster, coarser).
	--base-repair	auto	Orient + flat printable base (auto/flat/none).
	--base-cut	0.03	Flat-base cut height (fraction, anchored to the lowest scan points).
	--solidify	auto	Voxel-remesh to watertight, tunnel-free solid (auto/always/none).
	--voxel	200	Solidify resolution (voxels across bbox diagonal).
	--close	2	Voxel closing iterations; seals hole-like tunnels (raise if genus > 0).
	--smooth	0	Taubin smoothing iterations on the final mesh.
	--allow-open	off	Export STL even if not watertight (default: refuse).
	--file	cleaned.ply	Which splat to mesh (falls back to splat.ply).
preview.sh	--print	off	Show the repaired print mesh (mesh/object.glb).
	--file	auto	Specific file to load (e.g. cleaned.ply, object.glb).
	--port	5173	Vite port.

Anti-patterns

• Meshing the raw splat. Always clean first - background/table points wreck Poisson (it wraps a surface around them). Clean → preview → mesh.
• Trusting the size without --size-mm. SfM is scale-free; an unset size prints at an arbitrary scale. Measure the real object and pass it.
• Skipping the smoke run / visual check. Training on bad poses is the most expensive mistake; read "% registered" and preview 2k steps first.
• Plain white background / turntable-with-fixed-camera capture. Both starve COLMAP of trackable features and usually fail to register - orbit the camera around an object on a textured surface.
• Expecting a faithful bottom. The contact face is never seen; the mesh stage generates a flat base so the print is watertight and stands up - it is not a scan of the real underside. Flip-and-rescan if you need true bottom detail.
• Judging printability from the Splat Preview. The splat is honest scan data and will look holey underneath. Use preview.sh --print (the repaired GLB) and the watertight : True report to judge the actual print.
• Committing anything under ~/.video-to-splat/ into a repo - large and regenerable.

Resources

• Tool matrix, licenses, cleanup/Poisson parameter deep-dive, format notes, and troubleshooting: REFERENCE.md
• Shared capture→splat pipeline details (Brush CLI, pycolmap, SfM playbook): ../video-to-splat/REFERENCE.md