Md. Asif Uddin

    Chapter 8 VIII.8

    Reproducibility

    A result is reproducible when someone else can obtain it without asking you a question.

    Someone else can obtain it without asking you a question.

    How this chapter is built

    M0Narrative

    No mathematical load. Notation may appear; nothing is derived.

    basics1/4what the words mean
    concept2/2what to picture
    theory—why it works, and when it does not
    mathematics—derive it, then compute it
    practice0/3build it, break it, read the papers

    Five strands, not one. Mathematics is the spine; the other four are the body. A chapter cannot pay its way out of teaching with problems, nor out of problems with teaching.

    Before you start

    The problem

    Reproducibility is not a virtue added at the end; it is a property of how the work was recorded while it happened. The test is operational and unforgiving.

    Warm-up followed by cosine decayLearning rate against training step. The rate rises linearly from zero over a short warm-up, reaches a peak, then decays along a cosine curve to near zero by the end of training.learning rate against stepwarm-updecaypeak ratethe rise exists because the first steps are takenby an optimiser with no statistics yetthe fall exists because a large step near the endthrows away what the previous ones foundChanging the step count changes the whole curve. A schedule tuned for one budget is not valid for another.
    Fig. 8 — Warm-up then decay. The rise exists because the optimiser has no statistics yet; the fall exists because a large step near the end discards what the earlier ones found.

    What this chapter covers

    • Code organisation
    • Configuration
    • Environments
    • Seeds
    • Logging
    • Experiment tracking
    • Version control
    • Dataset versions
    • Model checkpoints

    Propositions

    Not yet written. The topics above are the plan for this chapter; each will become a proposition with its own figure.