Hold-Out Self-Validation Cannot Certify Photogrammetric Accuracy: Saturation and Blindness to Coherent Distortion

Abstract

Internal self-consistency cannot certify the accuracy of a photogrammetric reconstruction, and the failure is structural rather than a matter of tuning. This matters because hold-out self-validation scores are increasingly offered as quality evidence for metric deliverables whose correctness is otherwise unknown without an external survey. We formalise a track-leakage-free hold-out protocol: a deterministic image subset is withheld, and each withheld view is re-localised against only those 3D points supported by two or more retained images, so no view is tested against structure it helped create. We evaluate it on five GNSS-referenced captures across four sites, 13 ETH3D scenes, a EuRoC flight and 30 IMC 2025 scenes. The protocol is well-posed but does not measure accuracy. It saturates: the internal confidence score stays pinned at 1.00 while true error swings 14.1x within one capture. It is blind to coherent distortion: fragmenting corruption is caught, but internally self-consistent, globally distorted models are not, and were wrong by 55-106 m at confidence 1.00 at three of four captures. On IMC 2025 it separates failed from successful reconstructions (rho = 0.68) yet ranks nothing among the successful (rho = 0.01). Track-leakage-free hold-out measures internal geometric consistency: a fragmentation warning, not a substitute for control-point accuracy assessment.