Accepted

Curating multi-mode chain of thought for efficient math reasoning with GPT-OSS

Fewer, cleaner reasoning traces make a better student model.

Hai-Au Trinh, Tue-Anh Vu, Dai-Nhan Tran, Uyen Khoi-Minh Huynh, Anh-Khoi Nguyen

Role: Co-author
Venue: ICISN 2026
When: 2026

Overview

We distilled math reasoning from a GPT-OSS teacher into Llama 3.2 3B. The teacher wrote solutions at low, medium and high reasoning budgets; we kept only traces with a verified final answer and a sensible length.

Distillation datasets are noisy: wrong answers with confident reasoning, and traces that ramble for pages. We bet on fewer, cleaner samples rather than more of them.

I built the curation pipeline — generating multi-budget traces from the teacher, verifying final answers, filtering by length around the median — and ran the SFT and GRPO workflows on LLaMA-Factory and MS-SWIFT.

The student trained on the shortest, low-budget traces came out best after GRPO: GSM8K 0.8006 and MATH500 0.4760 zero-shot, up from 0.7043 and 0.3960.

Compare teacher budgets

Teacher reasoning budget used to write the training traces

GSM8K0.8006
MATH5000.4760

Llama 3.2 3B after SFT and GRPO, zero-shot accuracy. The tick marks the same model before distillation: GSM8K 0.7043, MATH500 0.3960.

Results, Llama 3.2 3B, 0-shot

Student	GSM8K	MATH500
Before distillation	0.7043	0.3960
SFT on low-budget traces, then GRPO	0.8006	0.4760
SFT on medium-budget traces, then GRPO	0.7771	0.4480
SFT on high-budget traces, then GRPO	0.7559	0.4740

Built with

LLaMA-Factory
MS-SWIFT
GPT-OSS
Llama 3.2 3B
SFT
GRPO

Cite

@inproceedings{trinh2026multimode,
  title     = {Curating Multi-Mode CoT for Efficient Math Reasoning with GPT-OSS},
  author    = {Trinh, Hai-Au and Vu, Tue-Anh and Tran, Dai-Nhan and Huynh, Uyen Khoi-Minh and Nguyen, Anh-Khoi},
  booktitle = {ICISN 2026},
  year      = {2026},
  note      = {Accepted},
  url       = {https://github.com/hugebenevolence/LLaMA-OSS}
}