Md. Asif Uddin
    I.8.X09

    An augmentation that makes perfect classification impossible

    counterexample▲▲△

    A balanced dataset contains just two inputs: x=−1x=-1 has label zero and x=1x=1 has label one. During training, independently negate the input with probability 1/21/2 but keep its original label. Compute the conditional label distribution after augmentation. What classification error is unavoidable, and which assumption was broken?

    Hint

    List the original input and the transformation as separate random choices.

    Solution

    The four equally likely transformed pairs are (−1,0)(-1,0), (1,0)(1,0), (1,1)(1,1) and (−1,1)(-1,1). At either observed input, each label now has conditional probability 1/21/2. Thus any deterministic classifier has error 1/21/2 on this augmented distribution. A random classifier cannot do better in expectation.

    The original distribution had zero achievable error: return whether xx is positive. Negation changes that target, so treating it as label-preserving creates contradictory supervision.

    The failure has nothing to do with the optimiser’s ability to fit the model. The training distribution was changed so that the input no longer identifies the label. If negation were required, transforming the label at the same time would preserve the relationship; that would be a different augmentation.

    Draws on