Original Transformer deep reading
Series · 1 posts
-
Transformer: Drop Recurrence for Self-Attention, but WMT 2017 BLEU Does Not Represent Later LLMs
Intermediate Build the foundations firstA source-grounded reading of Vaswani et al., NeurIPS 2017 / arXiv:1706.03762: stacked encoder-decoder with multi-head self-attention and positional encodings replaces RNNs and convolutions for machine translation. On WMT 2014, the big model reaches 28.4 BLEU EN-DE and 41.8 BLEU EN-FR; this is 2017 sequence-transduction evidence, not a BERT, GPT-3, or ViT contract.
Understand it in 90 seconds
For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.
Speaking & contact