Work · Arjhine Ty

Work

AI systems engineering and low-level ML.

Shown as I actually split my time — nothing here is a flagship, everything is a peer.

result

Do agentic skills beat retrieval?

Do agentic skills beat plain web search? 120 controlled runs across four arms says no on five of six tasks, and skills cost 2.6x the tokens. A negative result, reported as one.

result

Arjhine Ty

I design AI systems that behave credibly under real use — multi-agent orchestration, retrieval, and the evaluation and serving layers that decide whether any of it survives contact with users.

github linkedin email x

About Systems Research Writing Credentials Resume Contact Evidence RSS

result

Do agentic skills beat retrieval?

Do agentic skills beat plain web search? 120 controlled runs across four arms says no on five of six tasks, and skills cost 2.6x the tokens. A negative result, reported as one.

result

The hybrid wins on every metric. It still leaves half its failures unexplained.

The hybrid system beats both a tuned deterministic pre-pass and the model alone, on every metric, on every checkpoint tested — field F1 0.6432–0.6827 against 0.2911 for rules alone and 0.4461 for the model alone. It also has a dominant, unsolved failure: silent omission, 55–62% of every failure breakdown, immune to four independent fixes. And a non-deterministic teacher labeller quietly invalidated part of the project's own early evidence before an audit caught it.

exploratory

Four iterations into closing the omission gap. It is 3 points narrower, not closed.

V3 targets the one number V2 could not move: omission at 55–62% of failures. Four iterations in, the best configuration reaches 52.3% — a new low, and still not below half. No V3 checkpoint beats the V2 release candidate on the comparable eval. This is an interim account of work in progress, not a result.

exploratory

Distilling a 1B model for what regex cannot do

A ~1B student distilled from a 31B teacher, measured only on the fields a tuned regex pre-pass cannot reach. Hallucination down 37%, schema validity up 9 points — and field F1 still 0.005 below the un-distilled base model.