SchemaForge v1

• A 1.08B student that matches its 31B teacher on the in-domain suite — 0.0% JSON syntax error rate and 1.000 extraction F1 on both, at 16x smaller VRAM (≈2.4 GB vs ≈38.5 GB) and 5.0x throughput (61.91 vs 12.40 tok/s). At 36 workers per 96 GB GPU against the teacher's 2, that is roughly 110x aggregate system throughput.
• Teacher scale conferred no measurable quality advantage. A gemma-4-E4B-it teacher and a gemma-4-31B teacher produced identical error rate and F1 on this task, which makes the small teacher the cost-effective choice — a result worth more than the headline speedup.
• A documented prompt-brittleness failure. The canonical template is a versioned API contract, not a suggestion: changing only the instruction header collapsed zero-shot validity from 70.0% to 0.0% — worse than the untrained base model. Both failing iterations are published alongside the working one.
• Limitations stated in full, including the ones that undercut the headline: n=5 evaluation documents, n=5 training samples, a single seed, and no SFT control.

openbmb/MiniCPM5-1B (1.08B parameters, 24 layers, GQA 16/2, LlamaForCausalLM) is distilled from Gemma-4 teachers using a multi-task objective: hard cross-entropy blended with temperature-scaled soft-logit KL in log space, alpha 0.5, tau 2.0. Teacher and student vocabularies are bridged by shared-subspace truncation, 256,000 down to 130,560.

Training: AdamW at 2e-5, cosine schedule, 5% warmup, 3 epochs early-stopped on validation loss, bfloat16, 2,048 max sequence length, on a single NVIDIA RTX PRO 6000 Blackwell (96 GB) via Nebius AI Cloud.

Inference binds to one exact prompt string — Extract structured JSON from the text:\n{doc}\nJSON Output: — with no chat tokens, no system persona, no trailing newline. In production the recommendation is to layer FSM-guided constrained decoding on top: distillation supplies semantics, the grammar guarantees syntax.

Five-domain enterprise suite (in-domain, n = 5 documents): base MiniCPM5-1B scores 64.2–67.1% across finance, supply chain, IT procurement, biomedical, and cloud-ops documents. The distilled model scores 100.0% on all five, F1 1.000, at ~62 tok/s.

Out-of-domain (suneeldk/text-json): 70.0% validity against the base model's 34.2% — and 0.0% for the two iterations that differ only in prompt header.

Read the in-domain numbers with the CI, not the point estimate. 5/5 has a Wilson 95% interval of [56.6%, 100.0%]. The honest claim from this release is the efficiency result — 16x VRAM, 5x throughput, teacher scale not mattering — not the accuracy result.

Evaluation scale is n = 5 documents, one per domain. The 100% / 1.000 figures are exact matches on a small curated set, not population estimates.

Training scale is n = 5 samples, and no SFT control was run (alpha = 1.0, no teacher logits). That omission is the serious one: without it, the contribution of knowledge distillation cannot be separated from that of prompt-format conditioning. Given that changing the prompt header alone swings validity between 0% and 70%, format conditioning is a live alternative explanation for the whole result.

Single seed, no variance estimates. Sub-2B models vary substantially run-to-run on small datasets.

Prompt-template brittleness is the headline failure mode — deviation drops accuracy to ~0, not to degraded-but-usable.

Out-of-domain ceiling ≈ 70%: roughly 30% of unseen real documents produce unparseable output.

In-domain documents are synthetic — clean ASCII, consistent labels, no OCR noise, English only. Targets are teacher outputs, so wherever the teacher was wrong the student learned the error; no human-annotated gold exists for this checkpoint. No comparison to Qwen2.5-1.5B, Phi-3-mini, rule-based extractors, or commercial document-AI APIs.

The result sits in the practical distillation literature on task-specific small students: where the task is narrow and the output format fixed, a 1B student can match a much larger teacher, and the interesting question becomes what is actually being transferred.

That question is unresolved here. Prompt-format conditioning and knowledge distillation are confounded in this release, and the prompt-header ablation — 70% to 0% on a header change — suggests format may be doing more of the work than the KD objective. The successor version abandons this objective entirely on separate grounds: cross-tokenizer logit KL assumes vocabulary alignment that Gemma and MiniCPM do not share.