Md. Asif Uddin

    Chapter 5 V.5

    VLM Architectures

    The architectures differ in where fusion happens, and where it happens decides what can be trained separately.

    Where fusion happens decides what can be trained separately.

    How this chapter is built

    M1Definitional

    Mathematics defines, and stops there. At most three displayed equations, no derivations.

    basics2/6what the words mean
    concept2/2what to picture
    theory0/1why it works, and when it does not
    mathematics0/4derive it, then compute it
    practice0/5build it, break it, read the papers

    Five strands, not one. Mathematics is the spine; the other four are the body. A chapter cannot pay its way out of teaching with problems, nor out of problems with teaching.

    Before you start

    The problem

    The same two encoders and one decoder can be wired several ways. The choice is not aesthetic: it fixes which components can be frozen, which must be trained jointly, and what a new modality would cost to add.

    The two checkpoints in one releaseA comparison of the 2.4 trillion parameter flagship against the 27 billion parameter model released beside it. The headline belongs to the first; the second is the one most people can actually run.Qwen3.8-2.4T-A95BQwen3.8-27Bparameters2.4T, 95B active27.8B denseinputtext onlymultimodalthinkingalways onswitchablelicencebespoke, revenue clausesApache 2.0you can host itnoyesThe open checkpoint is not the API model, and the licence is not Apache whatever the headlines said.
    Fig. 5 — The two checkpoints in one release. The headline belongs to the 2.4 trillion; the 27B is the one that fits on hardware you can buy, and it is the Apache one.

    What this chapter covers

    • CLIP-style encoders
    • BLIP and BLIP-2
    • LLaVA-style systems
    • Flamingo-style systems
    • Modern multimodal LLMs

    Notation

    • dModel width
    • LNumber of layers

    Propositions

    Not yet written. The topics above are the plan for this chapter; each will become a proposition with its own figure.

    Worked problems

    0/1 problems0/1 variants0/3 exercisesowes 4 more

    Not yet written. At M1 this chapter owes 1 worked problems across 1 distinct variants, and 3 exercises, every one with a published solution. The build enforces that from the day the chapter is marked published.