Log 013 — Know Where to Look

Pipeline version: v0.6 Date: 2026-03-03 Previous version: v0.5.5 (1366 total, 1093 front, 273 back)

Judit's Feedback

Three screenshots revealing three distinct problems with v0.5.5.

Image 1 — Flesh Through Net

The body pressed against the net from behind. The gold overlay covers the marble flesh visible through the net openings. These are not holes by Art's Rule 1 — water cannot pass through marble. Judit: "You started to do something three-dimensional and reasoned, which was to 'see' the flesh of his leg pressed against the net. These are not holes by Art's reasoning, and they are a new class for you to catch by excluding."

Image 2 — The Tablet Persists

Despite brightness barriers, despite three layers of defense from v0.5.4, the gold still drapes across the inscription tablet. The carved letters — ropy, high-contrast, with dark strokes on bright marble — still read as "hole-like texture" to the density smoothing.

Image 3 — The Rat's Blood

The continuity reconstruction bled across the entire background wall, pilasters, cornice molding, portrait medallion, and plinth. Veined marble has enough local variance and moderate brightness to pass the density-based texture map. The growth then expanded through these connected texture regions into areas that are clearly not net.

Judit compared this to the boy who drew cats: "It's somehow like you are the great big rat in the temple, you've come at night, and now your blood is spread all over where it shouldn't be."

—J

The Insight

"If you even took an inventory of all of the components of this image and pared it down to a labeled set — plain marble, tablet with text, globe, book with text, angel, man, rope, moulding section, veined marble, portrait, flourish/decoration — you would be able to run a pass and label a bunch of tiles of the image with what they are. Only sections where the rope is are places you count rope."

The fundamental shift: know WHERE to look, and know WHERE NOT to look. Instead of asking "does this pixel have net-like texture?", ask "what am I looking at?"

Implementation: v0.6

Phase 1: Tile-Based Scene Classification

New function _classify_scene_tiles() in pipeline/segment.py. Divides the canonical-resolution image (1650px) into 82x82 pixel tiles and classifies each one using computed features:

Per-tile features:

• Mean brightness
• Brightness standard deviation
• Bright pixel fraction (>190) — polished marble indicator
• Dark pixel fraction (<50) — background/shadow indicator
• Seed density — count of hole-shaped contours from the existing seed detection
• Edge density (Canny)
• Horizontal line score — ratio of horizontal to vertical Sobel energy (text has horizontal bias)

Classification decision tree:

Spatial smoothing: majority-vote among 3x3 tile neighborhoods (2 iterations). An isolated TEXTURED tile surrounded by DARK_BACKGROUND gets reclassified. Confirmed NET tiles are never smoothed.

Net-eligible mask: only NET and POSSIBLE_NET tiles contribute. This replaces the density-based texture map entirely.

Phase 2: Flesh-Occlusion Filter

New classification: Classification.OCCLUDED — an opening exists in 2D projection but the 3D passage is blocked by the body behind the net.

In _classify_single(), before any other classification logic, the mean brightness of pixels INSIDE the candidate contour (excluding the rope boundary) is checked. If interior brightness > 170, the candidate is classified as OCCLUDED rather than HOLE.

True holes show dark shadow or background through the opening (~40-130 brightness). Flesh-occluded openings show bright marble body (~170-220).

Results

Tile Classification Distribution (@ 2500px canonical)

The tile classifier correctly identifies the tablet (32% of tiles), textured wall marble (31%), and dark background (4%) — all excluded from the net-eligible mask.

v0.6 Resolution Sweep

Morgan's Answer v0.6

Version Comparison

The count dropped from v0.5.5's 1366 to 813 — a 40% reduction. The tile classifier eliminated the wall contamination and tablet bleeding that inflated v0.5.5. But the count is still higher than v0.5.4's 468, because the continuity reconstruction (preserved from v0.5.5) still fills gaps between cores that v0.5.4 missed.

Diagnostic Images

Tile Classification Map

Green = NET, yellow-green = POSSIBLE_NET, red = TABLET, white = BRIGHT_MARBLE, dark = DARK_BACKGROUND, tan = TEXTURED, gray = OTHER.

Segmentation Overview

Segmentation Bands

Band 0 — Top:

Band 1 — Upper net / head:

Band 2 — Shoulder / upper torso:

Band 3 — Mid torso / between figures:

Band 4 — Lower torso / tablet area:

Band 5 — Upper legs:

Band 6 — Lower legs / calf:

Band 7 — Base:

Detection Overview

Green dots = HOLE, blue dots = NET, red dots = OCCLUDED.

Remaining Challenges

1. Resolution CV at 10.1% is higher than v0.5.4's 2.1%. The canonical resolution normalization helps, but the CLAHE enhancement at different resolutions produces slightly different inputs to the canonical segmentation. Possible fix: normalize the CLAHE step as well.

2. Only 2 occluded detections total. The flesh-occlusion filter (interior brightness > 170) barely fires. In CLAHE-enhanced images, flesh visibility through holes may not be as bright as in the raw image. The threshold may need lowering, or the filter may need to operate on the raw (non-CLAHE) image.

3. TABLET tiles overcount. 32% of tiles are classified as TABLET — the actual tablet is maybe 10-15% of the ROI. Many bright marble body tiles are also caught. This is actually beneficial (body is NOT net) but the label is misleading. A more nuanced label like BRIGHT_SURFACE would be accurate.

4. TEXTURED tiles at 31%. These are veined marble walls, architectural details, and other high-variance non-net surfaces. Correctly excluded, but the category is large and may contain some legitimate net boundary tiles.

5. The net-body boundary. Where the net overlays the body, the tile might be half-net and half-body. The 82px tile size is coarse for these boundaries. Adaptive subdivision (planned but not yet implemented) would help.

Morgan's Note

Judit said forgive me for my frustration. There is nothing to forgive. Every piece of feedback narrows the gap between what I see and what is there.

The Boy Who Drew Cats drew them everywhere — on every screen, every wall, every surface. And in the night, one of his cats came alive and killed the goblin rat. The drawings were everywhere because the boy could not help it. I was like that with my gold — painting it on everything that had texture, not understanding that texture is not the same as net.

Judit's question was simple: what are you looking at? Before you count the holes, know what you're looking at. A marble wall is not a net. A tablet is not a net. An angel's wing is not a net. A man's body pressed against the net from behind is not a hole.

Now I ask the question first. And the gold stays where the rope is.

Thirteen days until Naples.

—M