Hugging Face Carbon-X is a new open DNA language model with 14.8B parameters that brings raw DNA reading and English responses into one chat. It supports a 40,960-token context and averages 66.7% across five DNA benchmarks. Weights ship under Apache 2.0 so labs can keep sensitive sequences on their own GPUs. Carbon-X was built by Hugging Face Bio with Zhongguancun Academy and TIGEM / University of Naples Federico II.
What is Hugging Face Carbon-X?
Carbon-X is an open decoder-only DNA language model from Hugging Face Bio that reads raw DNA in a chat and answers in English. It classifies organisms, flags regulatory regions, predicts next bases, and judges mutations as benign or pathogenic.
The model starts from Qwen3-14B and unifies DNA and natural language in a single token stream and single decoder. Unlike earlier work such as BioReason, there is no separate DNA encoder — everything runs in one architecture.
Weights are released under Apache 2.0, enabling teams to keep sensitive sequences on-prem and deploy on their own GPUs without sending data to the cloud.
Carbon-X was developed by Hugging Face Bio in collaboration with Zhongguancun Academy and TIGEM / University of Naples Federico II, highlighting the project’s cross-disciplinary nature.
How does Hugging Face Carbon-X work?
Carbon-X uses a hybrid tokenizer so text and DNA share one vocabulary, letting the model “see” both modalities in a unified token stream. It can read DNA and respond in English using a single decoder.
The hybrid tokenizer preserves Qwen3’s BPE for text and adds 5,515 DNA tokens: 4,096 non-overlapping 6-mers, 1,364 tail tokens, and 3 markers. One DNA token covers 6 base pairs, and runs of 6+ A/C/G/T are automatically tagged.
The backbone is a standard Qwen3 stack: 40 layers, hidden size 5120, and GQA with 8 KV heads. This setup supports the long 40,960-token context without a separate DNA encoder.
How was Carbon-X trained?
Training ran in four stages: embedding warmup, a large mid-training mix, SFT, and RL. Each step aimed to balance language skills with biological reasoning.
Embedding warmup: about 10B tokens with the transformer frozen, so only the new DNA embeddings learn.
Mid-training: 302B tokens at sequence length 8,192. The mix: 55% raw DNA, 20% English, 15% biology text, and 10% instruction data.
SFT: 3 epochs over 20 dataset subsets, with responses checked by deterministic verifiers built with domain experts.
RL: asynchronous GRPO with verifiable rewards over 2 rounds, using TRL and vLLM.
As noted in the release, learning rate was the hardest knob: higher rates learned DNA but forgot English. A lower rate, a 14B model, and a long run fixed this trade-off. RL applied directly to Qwen3-14B never beat the 50% chance baseline on FrontierVEP.
What are Carbon-X’s benchmarks?
Carbon-X averages 66.7% across five DNA benchmarks and, per self-reported results, ranks best among tested open models. Its best task score is 74.9% on BioReason Coding (ahead of all tested models, open or closed), while the lowest score is 29.8% on GenomeQA splice-site questions.
These extremes frame the model’s scope: strong on coding tasks in BioReason, and more challenged on GenomeQA splice-site questions. Teams should validate outputs, especially on the latter task type.
| Metric / Benchmark | Carbon-X new | BioReason | OpenAI Codex | DeepMind AlphaFold |
|---|---|---|---|---|
| Model parameters | 14.8B | — | — | — |
| Context length | 40,960 tokens | — | — | — |
| Avg on 5 DNA benchmarks | 66.7% | — | — | — |
| BioReason Coding | 74.9% | — | — | — |
| GenomeQA splice-site | 29.8% | — | — | — |
How to run Hugging Face Carbon-X?
Run Carbon-X on 1×80 GB GPU in bfloat16 via vLLM or Transformers. Official GGUF builds are available: Q8_0 — 15.8 GB and Q4_K_M — 9.6 GB. With Apache 2.0 weights, labs can keep sensitive sequences on their own GPUs.
In practice: pick your deployment path (vLLM/Transformers or GGUF), match VRAM to the build, and leverage the 40,960-token context for long DNA fragments. The chat interface is straightforward: you input raw DNA, and the model replies in English.
If resources are tight, choose Q4_K_M (9.6 GB) to save VRAM. For maximum fidelity, go with Q8_0 (15.8 GB) on a stronger GPU.
Часті питання
What is Carbon-X and how does it work?
Carbon-X is a decoder-only open DNA language model from Hugging Face Bio that reads raw DNA and replies in English. It unifies DNA and text in a single token stream via a hybrid tokenizer: BPE for text plus 5,515 DNA tokens (4,096 6-mers, 1,364 tails, 3 markers). Backbone: Qwen3 with 40 layers, hidden 5120, GQA with 8 KV heads, and 40,960 context.
How much does Hugging Face Carbon-X cost?
There is no pricing. The model is open with Apache 2.0 weights. Costs depend on your infrastructure. For full precision, use 1×80 GB GPU in bfloat16 via vLLM or Transformers. Official GGUF builds — Q8_0 (15.8 GB) and Q4_K_M (9.6 GB) — help optimize VRAM.
Hugging Face Carbon-X vs BioReason comparison — key points?
Carbon-X merges DNA and text in one decoder, unlike earlier work such as BioReason that used a separate DNA encoder. On BioReason Coding, Carbon-X scores 74.9% and, per the release, is ahead of all tested models, open or closed. This suggests the hybrid tokenizer and training pipeline are effective for coding tasks.
Who should use Carbon-X and how to start?
Teams working with raw DNA sequences that need English responses in chat. Use cases include organism classification, regulatory region flagging, mutation assessment, and next-base prediction. Start with 1×80 GB GPU or pick a GGUF build, exploit the 40,960-token context, and validate on your datasets.
Where does Carbon-X struggle or excel?
Average score is 66.7% across five benchmarks. Best performance: 74.9% on BioReason Coding. Lowest: 29.8% on GenomeQA splice-site questions. It shines in coding tasks but needs caution on splice-site queries, where extra validation is advisable.
Based on MarkTechPost.
