Md. Asif Uddin

Proposition 2III.5.P0263 of 76 in the corpus

The augmentation list is a list of things the model is told not to care about.

Two views differ by whatever you applied, and the objective says those differences are noise. Copy a recipe from natural images into a medical domain and you may have declared the finding irrelevant.

Each augmentation declares an invarianceFour augmentations, each naming the factor the representation is being told to ignore. Colour jitter and greyscale discard hue, which is correct for natural images and destructive for a retinal photograph, where the colour of a lesion is the diagnosis.what you augment is what you discardrandom cropignore scale and framingflipignore left–rightcolour jitterignore illuminationgreyscaleignore hueon a fundus photographHaemorrhages are dark red.Exudates are yellow-white.Jitter the colour hard enoughand you have trained a modelto ignore the finding.The recipe that works on ImageNet encodes assumptions about photographs of objects. Every one of them isa claim about your domain, and they are rarely checked before being copied.
Fig. 2 — Each augmentation names a factor the representation is told to ignore. On a fundus photograph, colour is the finding.

Demonstration

SimCLR’s ablation is the clearest evidence in this literature: the augmentation composition mattered more than the architecture or the loss, and the strongest single pair was random cropping with colour distortion. Not because those are magic, but because of what they declare.

Read each one as a sentence about the world.

  • Random crop — scale and framing are irrelevant.
  • Horizontal flip — left and right are interchangeable.
  • Colour jitter — illumination and white balance are irrelevant.
  • Greyscale — hue is irrelevant.

For photographs of objects on the internet, all four are true, which is why the recipe works there.

Now transplant it. On a fundus photograph, haemorrhages are dark red and hard exudates are yellow-white; the colour is the finding. Colour jitter strong enough to help on ImageNet trains a representation to ignore the difference between the two lesion classes. On a chest radiograph, horizontal flip declares that situs inversus is the same as a normal heart. On a histology slide, scale is diagnostic and aggressive cropping across magnifications discards it.

The reason colour distortion is needed at all in the natural-image case is instructive: without it, two crops of the same photograph share a colour histogram, and the model can match them on that alone. The augmentation exists to close a shortcut, which is the same lesson as the pretext task in Chapter III.

Corollary

Before adopting an augmentation pipeline, write out what each transform asserts and check it against a domain expert. It is a ten-minute conversation, and it is the difference between a representation that ignores nuisance and one that ignores the diagnosis.

Sources

Depends on

Used by

Nothing yet.