Md. Asif Uddin

    Chapter 1 III.1

    Images as Data

    An image is a tensor whose axes carry meaning, and the spatial structure is information a model can use or throw away.

    An image is a tensor whose axes are the picture.

    How this chapter is built

    M2Substantive

    The derivations are the chapter. A reader who skips the algebra has not learned it.

    basics2/9what the words mean
    concept2/2what to picture
    theory0/2why it works, and when it does not
    mathematics0/9derive it, then compute it
    practice0/6build it, break it, read the papers

    Five strands, not one. Mathematics is the spine; the other four are the body. A chapter cannot pay its way out of teaching with problems, nor out of problems with teaching.

    Before you start

    The problem

    Before any vision architecture, the reader needs to see why an image is not simply a long vector. The arrangement of the numbers is the content, and destroying it is a measurable loss.

    An image as a stack of channel planesThree planes of the same height and width, one per colour channel, drawn separated so the axes are visible. Together they are a single tensor of shape height by width by channels; each entry is one number.shape (H, W, C)RGBH — rows of pixelsW — columns of pixelsC — channels, three hereone number per (row, column, channel)A greyscale scan has C = 1.A CT volume adds a depth axis.Nothing else about the model changes.The numbers are only numbers. The shape is the claim about what they are.
    Fig. 1 — An image as a stack of channel planes. The numbers are only numbers; the shape is the claim about what they are.

    What this chapter covers

    • pixels
    • channels
    • RGB
    • resolution
    • normalisation
    • convolution
    • receptive fields

    Apparatus

    The mathematics this chapter leans on, held in Book 0 so it can be assumed here without being taught here. Not a gate — follow a link when a step stops making sense.

    Vectors, matrices and the row-major convention 0.LA.01 · Expectation 0.PR.02

    Notation

    • HSpatial height in pixels
    • W (spatial)Spatial width in pixels
    • CChannels in vision; compute in FLOPs in the scaling chapters
    • XA batch of token representations, T×d, rows are tokens
    • ℝThe reals

    Propositions

    1. Prop. 1An image is a tensor whose axes are the picture.A photograph reaches a model as a block of numbers indexed by row, column and channel. Nothing about it is visual; the axes are what make it an image rather than a list.
    2. Prop. 2The arrangement is the information, not the values.Two images with identical pixel statistics can be a photograph and a jumble. Everything a vision architecture is built to exploit is in the arrangement, and the arrangement is exactly what a dense layer discards.
    3. Prop. 3Normalisation is a promise about the numbers the model will meet.Subtracting a mean and dividing by a deviation fixes what counts as a typical intensity. The statistics must come from training data and be applied unchanged afterwards, or the promise is broken silently.

    Worked problems

    0/3 problems0/3 variants0/6 exercisesowes 9 more

    Not yet written. At M2 this chapter owes 3 worked problems across 3 distinct variants, and 6 exercises, every one with a published solution. The build enforces that from the day the chapter is marked published.