Md. Asif Uddin

    Chapter 6 VI.6

    Experiments

    Randomisation buys identification, and the price is stated in advance as a sample size.

    Randomisation buys identification; the price is stated in advance as a sample size.

    How this chapter is built

    M3Load-bearing

    The content is mathematics. Understanding is demonstrated by computation, not recall.

    basics2/11what the words mean
    concept2/2what to picture
    theory0/4why it works, and when it does not
    mathematics0/15derive it, then compute it
    practice0/9build it, break it, read the papers

    Five strands, not one. Mathematics is the spine; the other four are the body. A chapter cannot pay its way out of teaching with problems, nor out of problems with teaching.

    Before you start

    The problem

    The cleanest way to identify an effect is to assign the treatment yourself. What that buys is exact; what it costs is a power calculation that has to be done before the experiment rather than after it.

    The batch gradient as an estimateTwo panels, each showing the true gradient direction as a solid arrow and several batch estimates around it. With a small batch the estimates scatter widely; with a large batch they cluster near the true direction.batch of 8noisy direction, many cheap stepsbatch of 512clean direction, few costly stepsNoise in the estimate is not purely a defect: it is also what lets a small batch escape a shallowbasin that a large one would settle into.
    Fig. 6 — The batch gradient as an estimate of the true one. A small batch scatters widely and steps often; a large batch points true and steps rarely.

    What this chapter covers

    • Randomisation
    • Treatment and control
    • A/B testing
    • Experimental design
    • Power
    • Bias

    Apparatus

    The mathematics this chapter leans on, held in Book 0 so it can be assumed here without being taught here. Not a gate — follow a link when a step stops making sense.

    Estimators, bias and variance 0.ST.01 · Confidence intervals 0.ST.02 · Hypothesis tests 0.ST.03 · Variance and covariance 0.PR.03

    Notation

    • 𝔼Expectation
    • VarVariance
    • θ̂An estimate, as against the quantity it estimates
    • BBatch size

    Propositions

    Not yet written. The topics above are the plan for this chapter; each will become a proposition with its own figure.

    Worked problems

    0/5 problems0/4 variants0/10 exercisesowes 15 more

    Not yet written. At M3 this chapter owes 5 worked problems across 4 distinct variants, and 10 exercises, every one with a published solution. The build enforces that from the day the chapter is marked published.