An empty shopping cart returns null. The API specifies 0. Give the same bug report and repository revision to GPT or Claude. A successful run yields a patch. If we keep the tool log, it also yields the observations that led to that patch. These are two different training sets even when the patches agree.
In both cases the teacher is a black box. We can sample its responses, and we can record the tools we execute on its behalf. We train a separate model with accessible weights. The useful question is what each of these records lets that model learn.
1. The objective when all we have are samples
Fix a task $x$. Let $q(y)$ be the teacher's distribution over complete responses and $p_\theta(y)$ the student's distribution for the same task. Suppose $p_\theta(y)>0$ wherever $q(y)>0$. Classical distillation can use the teacher's soft probabilities [1]. A closed API typically gives us sampled responses $Y_1,\ldots,Y_n$. With independent samples, supervised fine-tuning uses
\[ \widehat L_n(\theta) =-\frac{1}{n}\sum_{i=1}^n\log p_\theta(Y_i). \]Taking expectations term by term gives $\mathbb E[\widehat L_n]=L$. To identify $L$, add and subtract $\sum_y q(y)\log q(y)$ in the sum.
\[ \begin{aligned} L(\theta) &=-\sum_y q(y)\log p_\theta(y)\\ &=H(q)+\sum_y q(y)\log\frac{q(y)}{p_\theta(y)}\\ &=H(q)+D_{\mathrm{KL}}(q\Vert p_\theta). \end{aligned} \]$H(q)$ is constant in $\theta$. If the student family contains $q$, matching $q$ minimizes the population loss. In training, the student evaluates $p_\theta(Y_i)$ by multiplying the probabilities of its tokens in sequence. The teacher only supplies $Y_i$.
2. Passing a test changes the target distribution
For the cart bug, reserve one acceptance test for the empty cart and another for a cart with items. Give the teacher the bug report, the repository, and the tools to work on a fresh copy. Run the reserved tests on the resulting patch. The tests define a binary verifier $V(y)$. Let $s=\Pr_{Y\sim q}(V(Y)=1)$ for this task.
Draw at most $n$ independent candidates from the same initial state and keep the first that passes. The probability of keeping the task is
\[ \Pr(\mathrm{keep})=1-(1-s)^n. \]To keep a particular answer $y$ at attempt $j$, the first $j-1$ candidates must fail, and candidate $j$ must be $y$ and pass. Summing over $j$ gives the joint probability of keeping the task and keeping $y$.
\[ q(y)V(y)\sum_{j=0}^{n-1}(1-s)^j. \]Divide by $\Pr(\mathrm{keep})$. For $s>0$, the geometric sum cancels, leaving
\[ q_V(y)=\Pr(Y_{\mathrm{kept}}=y\mid\mathrm{keep}) =\frac{q(y)V(y)}{s}. \]More attempts increase task coverage. At a fixed task, the first passing answer still follows $q_V$. If tasks originally have distribution $\mu(x)$, Bayes' rule weights each task by its probability of leaving a sample. Write $s_x$ for the teacher's pass rate on task $x$.
\[ \mu_V(x)\propto\mu(x)\big[1-(1-s_x)^n\big]. \]A common attempt budget therefore overrepresents tasks with high teacher pass rates. Count how many examples survive in each task category before deciding where more sampling will help.
3. A trajectory adds supervision at each decision
Let a run have $m$ turns. At turn $t$, the model sees history $h_t$, chooses action $a_t$, and receives observation $o_t$. Suppose the teacher and student use the same action space and environment. Denote their action policies by $\pi_T$ and $\pi_\theta$, and the environment's response distribution by $K(o_t\mid h_t,a_t)$. For $j\in\{T,\theta\}$, the unfiltered trajectory distribution factorizes as follows. The task $x$ is fixed and implicit.
\[ P_j(\tau)=\prod_{t=1}^{m} \big[\pi_j(a_t\mid h_t)K(o_t\mid h_t,a_t)\big]. \]In the log ratio of the two trajectory probabilities, the $K$ factors cancel. The remaining terms are action log ratios. Taking the teacher expectation and conditioning each term on $h_t$ gives
\[ \begin{aligned} d_t(h)&=D_{\mathrm{KL}}\big( \pi_T(\cdot\mid h)\Vert\pi_\theta(\cdot\mid h)\big),\\ D_{\mathrm{KL}}(P_T\Vert P_\theta) &=\sum_{t=1}^m\mathbb E_{h_t\sim P_T}[d_t(h_t)]. \end{aligned} \]This is why the training loss goes on the teacher's actions. Tool observations belong in the history that conditions those actions. After selecting successful runs, the action targets come from the retained histories. The joint KL identity above concerns unfiltered runs, since conditioning on success can also change the distribution of observations.
4. How much can one observation tell the student?
Stop the cart log just after python -m pytest is called. Let $B$ be the visible history at that point, and consider two equally likely tool results $O$. The first says No module named pytest, in which case the next action is to check the Python environment. The second says expected 0, got None, in which case the next action is to inspect the total calculation. The command and the history before its result are identical in this example.
For a predictor of the next action $A$, the smallest expected log loss given input $U$ is $H(A\mid U)$. This follows from the first derivation applied at each value of $U$. The optimal predictor uses the true conditional action distribution. The reduction from showing the observation is therefore
\[ H(A\mid B)-H(A\mid B,O) = I(A;O\mid B). \]Here the best predictor with only $B$ assigns probability $1/2$ to each action. Its loss is $\log 2$. With $O$, the action is determined, so the ideal loss is zero. The tool return supplies exactly one bit about the next action in this example. A final patch can be identical in both runs while leaving that decision out of the training record.
Public GPT and Claude APIs expose responses and, for supported settings, reasoning summaries. The underlying reasoning tokens remain on the provider side [3, 4]. A worked explanation can be a useful target. For the example above, the evidence that determines the action is the result returned by the test runner.
5. A controlled comparison
Use the same accepted tasks and patches to train two students from the same base model. One sees only the patches. The other sees the recorded actions and observations as well. Evaluate on held-out repositories, with identical tools and inference budgets. Report task-matched results, then repeat with comparable training token budgets. Measure both pass rate and recovery after a failed command. These distinguish learning to predict the demonstrated actions from learning to complete new tasks.
One published result illustrates the scale of collection. SWE-smith attempted 17,906 runs over 8,686 distinct tasks and retained 5,016 expert trajectories. Fine-tuning Qwen2.5-Coder-Instruct-32B on those trajectories gave 40.2% single-attempt resolution on SWE-bench Verified, compared with 58.2% for the Claude 3.7 Sonnet teacher in the same SWE-agent framework [2]. This compares the trained student with its teacher. The patch-versus-trajectory comparison above tests the additional value of keeping the process.
For commercial teachers, the agreement determines which training uses are permitted. OpenAI and Anthropic publish restrictions on training competing models, with exceptions or approvals specified in their terms [5, 6].
References
- Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network.
- Yang, J., et al. (2025). SWE-smith: Scaling Data for Software Engineering Agents, Section 4.
- OpenAI. Reasoning summaries.
- Anthropic. Summarized thinking.
- OpenAI. Services Agreement, Section 3.3.
- Anthropic. Commercial Terms of Service, Section D.4.
Citation
If you need to cite this article, please use the BibTeX below.
@misc{vann2026Distillation1,
title = {How Do You Distill Closed-Source Models Like GPT and Claude?},
author = {Zhou, Yufan},
year = {2026},
month = November,
howpublished = {\url{https://yufanzhou142.github.io/posts/vann-distillation1.html}},
note = {Blog post}
}