ResNet deep reading

Series · 1 posts

← Back to Paper Reading

  • ResNet: Residuals Make Depth Trainable, but ImageNet 2015 Is Not a Ready-Made Detection or ViT Contract

    Intermediate Build the foundations first
    ResNet deep reading: Part 1 , Note: Aug 28, 2026 , Paper: 2016 , CV

    A source-grounded reading of He et al., CVPR 2016 / arXiv:1512.03385: identity shortcuts let stacked layers learn residual F(x)+x and fix plain-net degradation. ResNet-152 reaches 4.49% top-5 validation error on ImageNet; this is 2015 classification evidence, not a YOLO, ViT, or modern ConvNet leaderboard contract.

    Understand it in 90 seconds
    Problem
    After batch normalization and good initialization made tens of layers trainable, stacking more plain conv layers still triggers degradation—deeper models show higher training error (Figure 1, Figure 4 left), which is not ordinary overfitting.
    Core insight
    Recast the target mapping. Instead of asking stacked nonlinear layers to fit $\mathcal{H}(\mathbf{x})$ directly, let them fit $\mathcal{F}(\mathbf{x}):=\mathcal{H}(\mathbf{x})-\mathbf{x}$ and output $\mathbf{y}=\mathcal{F}(\mathbf{x})+\mathbf{x}$ through identity shortcuts (Equation 1). The control point is whether the solver must fit $H(x)$ from scratch or learn a correction $F(x)$ on top of identity.
    Strongest evidence
    On ImageNet with matched parameter counts, plain-34 top-1 error is 28.54% versus plain-18 27.94%, while ResNet-34 is 25.03% and beats ResNet-18 27.88% (Table 2, 10-crop validation). On CIFAR-10, plain-56 training error exceeds 60% and is omitted from Figure 6 left, while ResNet depth scans down to ResNet-110 6.43% (Table 6, Figure 6). ResNet-152 single-model top-5 validation error is 4.49%; a six-model ensemble reaches 3.57% top-5 on test (Tables 4–5).
    Main boundary
    The headline contract is 2012 ImageNet classification plus CIFAR-10 depth diagnostics; PASCAL/COCO detection is a Faster R-CNN backbone transfer table (Tables 7–8), not a YOLO contract, not ViT, and not a modern ConvNet leaderboard.
    Read the full deep dive

For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.

Speaking & contact