Arjhine Ty

Every gradient is a hypothesis. Every checkpoint is evidence.

I’m Arjhine Ty. I study what post-training actually changes in small open-weight language models, and what it quietly breaks.

Get in touchSee the studies

Three studies, each published with its cost.

OpenGrad and Small-Mind each publish a claims audit and errata beside their frozen reports. OpenPapers keeps its evaluation in the repository.

beforeafterthe number that complicates the result

  • Call F1, When2Call, 1,277 held-out examples

    0.6264 to 0.7548

  • GSM8K, 8-shot, same 1,319 questions

    70.4% to 55.5%

  • IFEval, strict, prompt level

    67.8% to 45.8%

OpenGrad: measured results, before and after.

OpenGrad

“We improved the metric. Then the model stopped answering.”

Can post-training teach a 2B model when to call a tool, and what does it break?

Call F1 rose from 0.6264 to 0.7548, and the same model refused all 1,319 zero-shot GSM8K questions.

  • Qwen3.5-2B
  • SFT + DPO
  • Regression audit
  • GGUF ladder
What went wrong on the way
  • The jump from “not promoted” to “promoted” was a change of gate, not of model. DPO moved call F1 by +0.0078 over the SFT checkpoint, 7 more of 453 calls on one seed, which is inside the noise.
  • Of nine GGUF quantizations, only Q6_K and Q8_0 pass the preservation gate, and a rerun of the unquantized model fails it too. The line sits inside run-to-run noise, and the report says so.
Study site (opens in a new tab)Code

  • Correct abstention, 80 unanswerable probes

    13.75% to 71.25%

  • Recall, 608 answerable probes

    0.16% to 18.59%

Small-Mind: measured results, before and after.

Small-Mind

“Better recall and worse abstention arrived together.”

How much of a small model’s memory gap can retrieval and LoRA post-training close?

Correct abstention on 80 unanswerable probes rose from 13.75% (the raw model, no memory) to 71.25%, while recall on the 608 answerable probes reached only 18.59%.

  • gemma-4-E2B
  • SFT → DPO → distillation
  • Hybrid retrieval
  • 688-probe benchmark
What went wrong on the way
  • A baseline with no memory access scored about 94% on personalized recall. That is impossible by definition, so I went looking: an operator-precedence bug was dropping the gold answer from the judge’s rubric.
  • Scaling the training data 10× made abstention look worse. Two bugs: a dedup step had collapsed about 227 abstention examples into 1, and the eval’s detector missed the model’s newly correct phrasing.
Study site (opens in a new tab)Code

  • Answer correctness, frozen holdout, 40 tasks

    1.00

  • Candidate-fact precision, same holdout, every candidate

    0.563

  • Candidate-fact precision, v4 development set

    0.134

OpenPapers: measured results, before and after.

OpenPapers

Can a research tool return sources with locators, and say “unknown” instead of guessing?

Zero fabricated answers on a frozen 40-task holdout, but only 0.563 of the candidate facts it extracted were correct.

  • MCP server
  • Provenance
  • Refusal path
What went wrong on the way
  • The low precision is a design choice that is measured, not hidden: extraction over-emits candidate facts so answer assembly has material to work with, and the answer layer is scored separately.
Code

Two labs: one measures the hardware, one measures the models.

Co-Founder and ML Research Engineer at Experimental Machines, and co-founder of Experimental Intelligence.

Experimental Machines

Co-Founder and ML Research Engineer

Benchmarks of AI accelerators at datacenter, laptop and phone scale, with published logs.

experimentalmachines.org (opens in a new tab)

Experimental Intelligence

Co-founder

A research lab that trains and post-trains small open models and measures what each step changes, including what gets worse.

experimentalintelligence.org (opens in a new tab)

Four threads that run between the studies.

Four lines of work outside the three studies, each with what came out of it, including when that was a failure or nothing yet.

Long context on one GPU

A 27.8B hybrid model holds its native 262K context on a single GPU. Stretched to 512K and 1M, the servers loaded but never produced a first token, and a separate 512K pilot failed on quality.

brainwaves-longwriter-bench (opens in a new tab)

Serving

DeepSeek V4 Flash on one GPU with K=7 speculative drafting: 200.6 tokens/s cold and 280.6 warm on 600-token generations, with 20 of 20 required tool calls parsed.

Serving benchmark (opens in a new tab)

Distillation for structure

Rules take the fields they can, and a 1B model distilled from gemma-4-31B takes the rest: field F1 0.6827, against 0.291 for rules alone and about 0.484 for the model alone. Omission still causes 52 to 59% of failures.

SchemaForge (opens in a new tab)

Routing after the evidence

A mixture of experts drafts a sentence, a critic judges it in named error categories, and a router accepts, repasses or reroutes it. English↔Filipino first. No model results yet.

DiaGrad (opens in a new tab)

Smaller results, and the systems they run on.

Write-ups from SchemaForge and the agent-skills work, and the software built alongside the studies. Where a run has not happened yet, the entry says so.

All research

Notes from inside the runs.

Long-form write-ups of what broke, what was fixed, and what the numbers mean once the bugs are out.

All writing

Aug 2026

I scaled my training data 10x. My model got worse at saying it didn't know something.

Scaling small-mind-companion's training data 10x looked like it made abstention worse — until I found a dedup bug that had collapsed 227 abstention examples down to 1, and an eval detector that could not recognize the model's own newly-correct phrasing. Fixing both uncovered a real third problem, over-hedging, which needed a separate fix. Where it landed: UAR at 70.0%, false-abstention on answerable questions down to 32.1%, and a reminder that a good-looking number is a reason to look harder, not stop.

Code (opens in a new tab)Model (opens in a new tab)

Aug 2026

My hybrid extraction system beats both its halves. It still goes silent on half its failures.

V2 closed with a hybrid system that beats rules alone and the model alone on every metric — and one dominant failure, silent omission, that four separate fixes could not move below half of everything that goes wrong. V3 has spent four iterations against it since. The best result so far: three points narrower. Not closed. Here is the whole account, including the labelling bug that could have invalidated the headline number if an audit had not caught it.

Aug 2026

My model card says 1.000 F1. Do not believe it.

My first model card reports 1.000 F1. Here is why you should not believe it, what happened when I rebuilt the benchmark to stop flattering the model — scores fell, on purpose — and what the retraining running right now is meant to settle.

Awards and certifications.

All of them

Certifications

  • Nebius AI LeaderNebius · Aug 2026
  • Nebius Agentic AI LeaderNebius · Aug 2026
  • Python Data AssociateDataCamp · Aug 2026

Hiring for applied ML or research engineering?

arjhine@experimentalmachines.org

Open to roles · Quezon City, Philippines

Email meResume