Fine-tuning ruGPT-3 XL 1.3B to a Modern LLM Level and Comparison with Gemma3 1B
Сбербанк
Google/DeepMind
The ancient ruGPT-3 XL 1.3B model from 2020 was fine-tuned using SFT and DPO methods, turning it into a modern instruct-following LLM. The result was compared with the 2025 Gemma3 1B model, showing that the fine-tuned ruGPT often outperforms Gemma3 in logic and creativity, though it struggles with negative instructions and role-playing.
The ruGPT-3 XL 1.3B model, originally a pretrain model from 2020 that could only continue text, was fine-tuned using SFT (Supervised Fine-Tuning) and DPO (Direct Preference Optimization) to create a modern instruct-following LLM. After SFT on a 42M-token dataset derived from saiga_preferences, the model learned to follow instructions, answer questions, and maintain multi-turn dialogues. It was then compared with Gemma3 1B, a 2025 model trained on 2T tokens. The fine-tuned ruGPT performed better on tasks like Russian rock band knowledge, multi-step conversations, quantum physics questions, and a spatial riddle. It also handled negative instructions (e.g., avoiding a specific word) after several regeneration attempts, while Gemma3 failed on most tasks except role-playing and terminal simulation, where ruGPT struggled. DPO further improved alignment, though the article notes that the small SFT dataset limited generalization.
- Сокращения
- SFT = Supervised Fine-Tuning — Обучение с учителем
- DPO = Direct Preference Optimization — Прямая оптимизация предпочтений
- LLM = Large Language Model — Большая языковая модель
- RLHF = Reinforcement Learning from Human Feedback — Обучение с подкреплением на основе человеческой обратной связи
- VRAM = Video Random Access Memory — Видеопамять
- LoRA = Low-Rank Adaptation — Адаптация низкого ранга
- CoT = Chain-of-Thought — Цепочка размышлений
- EOS = End of Sequence — Конец последовательности
Source: Habr — хаб ML —
original