AlexNet Deep Dive

Series · 2 posts

← Back to Paper Reading

  • AlexNet Part 1: Reading the Evidence Behind an ImageNet Turning Point

    Introductory Build the foundations first
    AlexNet Deep Dive · Part 1 · Note · Mar 18, 2026 · Paper · 2012 · CV

    A source-grounded rereading of AlexNet’s problem, evaluation, historical result, and evidence boundary.

    Understand it in 90 seconds
    Problem
    in 2012, training a deep CNN on millions of high-resolution images was constrained by optimization speed, GPU memory, and overfitting.
    Core insight
    AlexNet is not one “big network” trick: convolutional locality, ReLU, a constrained two-GPU split, and an eight-layer architecture form a trainable system.
    Strongest evidence
    ILSVRC-2010 top-1/top-5 error is 37.5%/17.0%; the 2012 competition top-5 error is 15.3% versus 26.2% for second place (Section 6; Table 1).
    Main boundary
    LRN, the two-GPU split, and some kernel choices are hardware-era tradeoffs, not claims of modern optimality.
    Read the full deep dive

  • AlexNet Part 2: Turning the Training Recipe into Testable Design Choices

    Intermediate Build the foundations first
    AlexNet Deep Dive · Part 2 · Note · Mar 19, 2026 · Paper · 2012 · CV

    A source-grounded reading of ReLU, multi-GPU splitting, overlapping pooling, augmentation, and dropout in Figure 1–3 and Sections 3–6.

    Understand it in 90 seconds
    Problem
    a 60M-parameter CNN can overfit even with 1.2M images, and its training recipe must be separated from the competition result.
    Core insight
    random crop/flip, RGB PCA lighting jitter, and dropout change or regularize the effective training distribution; SGD, momentum, weight decay, and a learning-rate schedule make Part 1's architecture converge.
    Strongest evidence
    color augmentation reduces top-1 error by over 1%, overlapping pooling by 0.4/0.3 points, and the full system reaches 37.5/17.0 on ILSVRC-2010 and 15.3 top-5 in 2012 (Sections 4–6; Table 1).
    Main boundary
    these ablations belong to the era's architecture, data, and compute; they do not show every modern vision model needs ten-crop, LRN, or the same schedule.
    Read the full deep dive

For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.

Speaking & contact