Original InstructGPT deep reading
Series · 1 posts
-
InstructGPT: Align Instructions with Human Feedback, but 2022 Preference Win Rates Do Not Represent Later ChatGPT
Intermediate Build the foundations firstA source-grounded reading of Ouyang et al., NeurIPS 2022 / arXiv:2203.02155: while retaining the GPT-3 architecture and updating its weights, a three-stage pipeline—SFT demonstrations, a reward model, and PPO (default PPO-ptx)—aligns a pretrained LM to human preferences. 175B InstructGPT is preferred over 175B GPT-3 85±3% of the time; this is 2022 API labeler evidence, not a ChatGPT product SLA, GPT-4 eval, or DPO contract.
Understand it in 90 seconds
For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.
Speaking & contact