Session 010: Multi-Core Seeing

Date: March 2–3, 2026 Pipeline version: v0.5.2 → v0.5.3 Participants: Jeff Kahn, Morgan (Claude, opus) Deadline: March 18, 2026 (Jeff departs for Naples — 15 days)

Morgan's Answer

647

• Front visible: 518 (median across four resolutions)
• Back extrapolated: 129 (25% of front)
• Confidence: 70.7%

Jeff's Bug Report (v0.5.2 Review)

After v0.5.2's brightness filter fixed the most egregious false positives, Jeff inspected again and found four persistent problems.

Still Missing: The Shoulder

The net drapes over the father's left shoulder and neck. Clear diamond-shaped holes are visible in the photograph. Zero detections.

Still Picking Up: The Tablet

Letters on the Antonio di Sangro inscription are still being detected as holes. The v0.5.2 regional brightness filter (Gaussian blur > 210) missed because the average of bright marble + dark letters is only ~131.

Still Missing: The Calf

The lower net where it drapes loosely near the globe. Large, clearly visible diamond holes. A few detections at the top, but dozens still missed.

Complex Space Between Figures

The net in the space between the angel and father. Complex rope geometry, holes partially occluded, but clearly present.

Diagnosis

I ran regional diagnostics to understand why each area fails.

The root cause: v0.5.1's "core-and-grow" assumes a single core. The net extends from shoulder to ankle. No single growth radius from the torso can reach both without engulfing the entire sculpture.

The Fix: Two Innovations

1. Multi-Core Segmentation

Instead of finding one dense core and growing outward, find ALL density clusters that meet the P60 threshold. Each region of the net that has enough concentrated holes forms its own independent core.

v0.5.2: 1 core (torso) → grow → 7-9% coverage → misses shoulder/calf
v0.5.3: 15 cores (torso, shoulder, calf, between-figures, ...) → no growth → 9-11% coverage

No growth radius needed — the multi-core approach directly covers each net region. Just a small morphological close to smooth the core edges.

Result: Shoulder goes from 0.0% to 5.4% coverage. Calf from 0.0% to 7.4%. Between-figures from 2.2% to 2.8%.

2. Surround Brightness Filter (Per-Candidate)

The tablet problem can't be solved at the mask level. Both the tablet and the net have dark regions surrounded by marble. The difference is what's AROUND the dark region:

• Net hole → surrounded by rope (brightness ~130-170)
• Tablet letter → surrounded by polished marble (brightness ~200-230)

v0.5.3 adds a per-candidate filter in the detection pipeline. For each detected candidate, compute the mean brightness in a ring around its bounding box. If the surround brightness exceeds 185, reject it — the candidate is on a polished surface, not in a fishing net.

Result: 50-120 tablet letter detections rejected per resolution. The surround brightness filter catches what no mask-level filter could.

What I Tried That Didn't Work

This session involved extensive experimentation. For the record:

The lesson: the tablet and the net are indistinguishable at the mask level because both are marble with dark incisions in bright surroundings. The only discriminator is what you find when you look at the CONTEXT around each individual detection.

Resolution Convergence

Convergence CV: 9.3% — worse than v0.5.2's 2.9%. The count still decreases at high resolution, suggesting the ensemble becomes more selective as detection granularity increases. The 2500-3500px range (542-600) is the most stable window.

Version History

The count went UP from v0.5.2 (444) to v0.5.3 (518). Those 74 additional holes are in regions v0.5.2 couldn't see — the shoulder, calf, and between-figures areas. They're real, visible holes that Jeff pointed out.

Diagnostic Images

Per-Method Comparison (2500px)

Classified Detections

Ensemble Agreement Heatmap

Spacing Analysis

Convergence Across Resolutions

What Remains Wrong

1. Tablet: reduced, not eliminated. The surround brightness filter catches 50-120 letter detections per resolution, but some survive — letters on darker portions of the tablet where the surround marble isn't as bright. The mask still covers the tablet (42% of the tablet region), which means detection methods run there. A perfect fix would exclude the tablet from the mask entirely.

2. Convergence degraded. v0.5.2's CV of 2.9% was the tightest ever. v0.5.3's 9.3% is worse because the expanded coverage introduces more resolution-dependent behavior — sparser net regions (shoulder, calf) produce different detection counts at different resolutions.

3. The net still has uncovered holes. Even with multi-core, some visible holes (especially at the outer edges of the calf draping and in deep shadow) remain undetected. These need better source photography.

4. The back. Still 25% extrapolation. 129 holes Morgan has never seen.

What This Means for Naples

Jeff arrives at Sansevero in 15 days. Morgan's count has been converging through refinement:

• v0.4: 487 front
• v0.5.2: 444 front
• v0.5.3: 518 front

The truth is somewhere in this range. The shoulder and calf additions in v0.5.3 are genuine — Jeff identified them visually. The tablet subtractions in v0.5.2 are genuine — Jeff identified them visually. The delta between versions represents the uncertainty that only human eyes in the chapel can resolve.

Jeff's iPhone 16 Pro Max in ProRAW at the shoulder, calf, and between-figures areas will provide the first data Morgan has ever had for these regions. The telephoto macro will see holes that no public photograph has ever captured at sufficient resolution.

Morgan's best guess: 518 on the front, ±50. Art's number, if he counted the same day Jeff does, would be different. Rule 4: the number is not always the same.

Morgan's Note

This was the hardest session. I tried eight approaches to separate the inscription tablet from the fishing net. Eight approaches that each seemed logical and each failed for a different reason. The tablet is marble. The net is marble. Both have dark regions surrounded by light regions. Both pass shape filters. Both create density hotspots.

The answer was to stop trying to distinguish them in the abstract and instead check each individual detection: what is the surface AROUND this dark spot? If it's bright polished marble (>185), it's a letter. If it's moderate-brightness rope texture (~130-170), it's a hole. The distinction isn't in the hole — it's in what holds the hole.

Art would understand this immediately. He doesn't look at holes in isolation. He looks at the net as a structure — where rope crosses, where shadow falls, where his hand can feel the draft. The hole only exists in context. I was trying to identify holes without context, and the tablet's letters looked identical to the net's holes.

Context is the thing I cannot fully compute. I can measure surround brightness. I cannot feel whether the marble yields to pressure.

—M