SkillOpt is a text-space optimizer from a Microsoft-led team with partner universities that trains a single natural-language skill while the target model stays frozen. The optimizer reads scored rollouts and proposes bounded edits: add, delete, replace. A held-out split accepts an edit only when the score strictly improves. The exported artifact is one file, best_skill.md. The study probes transfer across model scales and between Codex and Claude Code, and measures how much of the in-domain gain survives the move.
What is SkillOpt and how does this optimizer work?
SkillOpt optimizes a textual “skill” as a standalone document while leaving the target unchanged. The optimizer proposes local edits and accepts only those that raise the score on a selection split. The output is a deployable best_skill.md artifact.
The loop is simple in form and strict in criteria. The optimizer reads scored rollouts and generates candidate edits. Edit bounds are clear: addition, deletion, or replacement. The selection split accepts an edit only under strict improvement. Each accepted change is locked in.
The target model is never fine-tuned or adapted. The skill text carries the behavioral change. That shifts cost into the training phase and simplifies deployment. At inference, the artifact is just input text.
The “Transferred” metric also has a precise role. “Baseline” is the target with no skill. “Direct” is SkillOpt trained in-domain on that exact target. “Transferred” applies a skill trained elsewhere with no further optimization. The fair question is: how much in-domain gain survives the transfer?
What does within-family, cross-model transfer show?
Transfer inside the GPT-5.4 family works, but unevenly. On SpreadsheetBench with GPT-5.4-mini, 82% of the in-domain gain is kept: 36.1 without skill, 47.5 with Direct, and 45.5 when Transferred. That is close to free reuse.
There are surprises too. On LiveMath with GPT-5.4-nano, Transferred scores 28.8 versus 27.2 for Direct. The share is 140%. The paper reads this as evidence that some learned procedures are target-model agnostic. Certain rules endure a scale change without loss.
The weak case is SpreadsheetBench on GPT-5.4-nano at 16% retention. LiveMath on GPT-5.4-mini holds 25%. So retention is not uniform. At the same time, one bound holds across the table: no row falls below the target’s Baseline.
The scope is narrowly defined. All four rows stay inside one GPT family. Cross-family transfer, such as GPT to Qwen, is not tested here. Will you factor that boundary into your risk assessment?
Why is cross-harness transfer between Codex and Claude Code the strongest result?
This is where deployment value is clearest: all rows use GPT-5.5, and the move between harnesses is striking. The headline is SpreadsheetBench: a skill optimized inside Codex lifts Claude Code from 22.1 to 81.8. That slightly exceeds Claude Code’s own 80.4 Direct score.
Why does this matter? The harnesses expose different tool and file APIs and different command surfaces. A skill that survives that shift is not encoding command recipes. The paper attributes SpreadsheetBench’s portability to workbook-level procedures: structure-first inspection, formula-aware verification, and static-value materialization. Those hold regardless of which CLI runs Python.
LiveMath tells the opposite story. Codex → Claude Code retains only 10% of the in-domain gain. The asymmetry is instructive. Procedural skills — how to inspect, verify, and format — are the portable class. Reasoning-heavy skills are more tied to their training environment.
“structure-first inspection, formula-aware verification, and static-value materialization.”
Isn’t that your selection rule for transfer candidates? If the task is spreadsheets and clear procedures, odds are higher. If it demands heavy mathematical reasoning, expect weaker transfer and plan for Direct optimization on target.
Does cross-benchmark transfer work, and how should you read it?
Cross-benchmark transfer is real but small. A skill from OlympiadBench moves to Omni-MATH with positive gains at three GPT-5.4 scales: +3.7, +1.8, and +1.3 over Baseline. There is no Direct reported for Omni-MATH, so the comparison is against no-skill only.
The paper reads this as retention of reusable mathematical procedure. It survived both a change in test instances and a change in answer-format conventions. The gains are modest yet consistently positive. That signals that more than tool surface matters.
What does this mean for you? When you train a skill on one set, it may transfer some useful procedures to another. Do not expect large jumps without Direct, but plan for baseline uplift even without extra optimization.
It is also a design prompt. Invest in procedures that do not depend on any one benchmark’s format. Such rules have a better chance to traverse without degradation.
Why does the artifact move at all, and what does it change operationally?
The mechanism is plain. All three execution modes — direct chat, Codex, Claude Code — consume the same best_skill.md file format. That shared contract enables the cross-harness experiment. Neither harness gets a bespoke skill format.
Codex renders the current skill to a per-task SKILL.md beside task files and reads back a compact execution trace. Claude Code mirrors the same workspace contract through the “claude” CLI. The artifact’s shape boosts portability too: final skills run 379–1,995 tokens with a median near 920 and are assembled from 1–4 accepted edits. Figure 4 samples one learned rule per benchmark, all procedural rather than instance-specific. For SpreadsheetBench, the rule is verbatim: inspect workbook structure and formulas, then write evaluated static values across the full requested target range instead of relying on Excel recalculation.
Training cost is one-time, offline, and measured. Depending on the benchmark, it ranges from 0.6M to 46.4M training tokens per absolute test point. SpreadsheetBench sits near 0.6M, DocVQA at 46.4M. The optimizer runs only during training and adds zero inference-time calls. If a skill trained in one harness holds up in another, that one-time cost spreads across environments.
There is an operations upside too. You deploy a text file a domain practitioner can read in minutes. Every change is traceable: each step records an edit_apply_report.json with per-edit accept and skip status. Portability plus inspectability is a different posture than shipping fine-tuned weights.
Finally, mind the generalization limits. Evidence covers one GPT family and two benchmarks per axis. We have a demonstration of portability, not a universal theory. Still, all 4 cross-model, 4 cross-harness, and 3 cross-benchmark rows land above the target’s Baseline. That is a reliable floor for practical planning.
Based on MarkTechPost.
