Md. Asif Uddin

Proposition 4III.6.P0469 of 76 in the corpus

One photograph cannot fix scale.

A small object nearby and a large one far away project to identical pixels. Monocular depth is therefore predicted up to an unknown factor, and a metric claim needs information the image does not contain.

Two scenes, one projectionA camera centre with two objects along the same ray: a small one nearby and a large one far away. Both fill the same region of the image, so a single photograph cannot separate size from distance. Monocular depth is therefore predicted only up to an unknown scale.the same pixelscameranear, smallfar, largeimageBoth objects subtend the same angle, so both occupy the same pixels.Monocular depth models predict relative or scale-invariant depth — a metric claim needs stereo,a known object size, or a sensor that measures distance directly.
Fig. 4 — Two scenes projecting to identical pixels. One photograph cannot separate size from distance.

Demonstration

Perspective projection maps a 3D point to an image point by dividing by depth. Double every distance in a scene and double every object’s size and the projection is unchanged, pixel for pixel. The scene and its scaled copy are indistinguishable from one viewpoint.

So a network predicting metres from a single image is not solving a well-posed problem. What it can do is exploit the regularities of the world it was trained on — doors are about two metres, corridors have known proportions, this camera has this focal length — and those regularities are priors about the training distribution, not geometry.

The literature is unusually honest about this and the terminology records it. Relative depth predicts ordering. Scale-invariant losses score a prediction after fitting the best scale, so the model is never penalised for the part it cannot know. Metric depth models exist and are explicitly restricted to a domain and camera, because that is the extra information that resolves the ambiguity.

MiDaS’s contribution was to train across many datasets whose depth annotations are in incompatible units — some metric, some stereo disparity, some structure from motion — by using a loss invariant to scale and shift. Giving up on the unknowable is what let the data be pooled.

Genuine resolution comes from adding information: a second camera at a known baseline, motion between frames, an object of known size, or a sensor that measures distance directly.

Corollary

When a monocular depth model reports metres, ask what fixed the scale. If the answer is the training set, the number is a prediction about how similar your scene is to that dataset, and it will be wrong in exactly the way that assumption is wrong.

Sources