RLHF (Overview)

Summary

Reinforcement Learning from Human Feedback — training a model using human preference rankings to better align its outputs with what people actually want — the technique behind ChatGPT-style alignment.

• 1Read a high-level explainer of the three RLHF stages: SFT, reward model, RL fine-tuning.
• 2Understand why RLHF is expensive and requires substantial human-labeled preference data.
• 3Learn how DPO emerged as a simpler alternative to full RLHF.