Tangyi (Jerry) Qian

Problem

Teams burn frontier-model budget on high-frequency, narrow tasks — the exact tasks a small fine-tuned model handles fine. Almost nobody does it, because the RL expertise, the GPU orchestration, and the proof that it actually worked are three separate hard problems.

What I built

A closed loop, built in eight days — core loop solo, with a teammate on UI and the demo video: a proxy ingests real traffic and discovers task clusters and their cost; an agent with a fixed action space and custom analysis tools writes the training config; the system auto-provisions a GPU pod, trains with GRPO against a user-supplied programmatic verifier, then proves the result — held-out pass@1 went 0.583 to 0.783 and pass@8 0.767 to 0.900, while a spurious-reward control stayed flat, so the gain isn't placebo. Shipping is a routing toggle with a canary and a drift guardian. Serving scales to zero: a request wakes a GPU pod in about four minutes and reclaims it after thirty idle minutes.

Thinking from RLVR

I built this after implementing GRPO and One-Shot RLVR by hand. The literature kept showing that verifiable rewards work; nobody had productized the loop end to end — discover the workload, train against your own verifier, prove the gain, route to it. The borrowed parts are explicit. GRPO from DeepSeekMath is the trainer. The falsification arm comes from the spurious-rewards line: gains from formatting tricks look exactly like real gains until a random-reward control calls the bluff. The ship step thinks in FrugalGPT/RouteLLM terms — a canary and a routing toggle, not a leap of faith. As engineering I'm satisfied: the problem is crisply defined and the evidence chain closes. The honest issue sits upstream, and it is a frontier one. The system currently decides "is this task cluster worth training?" with an LLM agent auditing the workload — a judgment call, not a measurement. Given a verifier and an environment, whether a task definition is actually trainable with RL is an open research problem. That question is the most original thing this project surfaced for me, and the one I keep coming back to.

Framework & CodeAct Integration

Architected an end-to-end online learning pipeline over the OpenHands CodeAct agent and Kimi K2: driving multi-turn code generation, live Docker container evaluation on SWE-bench Lite Django issues (114 instances), and experience persistence with immutable event-sourced replay.

Workflow Induction & Abstraction

Engineered an automated induction engine that condenses 20–50 step raw coding trajectories into reusable 3–8 step procedural sub-workflows (fault localization, context tracing, test-driven validation) with parameterized variable abstractions.

Empirical Finding: Memory Non-Monotonicity

The headline aggregate was a negative result, reported as such: online CAWM (52.0%) landed below the 55.3% baseline. The conditional analysis is the real finding — with 11–20 workflows resident, resolution peaked at 66.7% (+11.4pp over baseline); past 31, it collapsed to 39.5% as accumulated workflow text polluted the system prompt. So memory is not useless — unbounded append is self-defeating: agent memory needs selection and eviction, not growth. The report states its own caveat: the peak is correlational evidence, and confounders cannot be ruled out.

Failure Attribution & Lost Focus

Systematically diagnosed failure modes across 87% test failure cases: identifying the 'lost focus' pattern where unguided agents continue unproductive exploration with high step variance (std=43.0 vs 19.4) and 50% context explosion (>150K chars), providing hard empirical evidence for selective memory retrieval.

Problem

Tool and skill definitions are expose-and-pray: they describe what an agent CAN call, and impose nothing on what it actually does. In practice the failure isn't a crash — the model narrates the step ('removed that waypoint') and never calls the tool. On a general agent harness you don't own, you can't patch the runtime to fix this.

What I built

Enforcement pushed down into the skill itself, so it works on an unmodified host. Every state write is gated by an invariant check — waypoint self-consistency, path continuity, range windows — and a failed check rolls the state back and returns an error written as an instruction, not a complaint, so the model's next turn is corrective rather than apologetic. Waypoint mutation is a first-class operation instead of an emergent side effect of free-form edits. The model's diligence is removed from the critical path: correctness becomes a property of the data plane, not of the prompt.

Building a benchmark for a self-designed skill

The driving question: if you design and build your own agent skill, how do you build the benchmark that lets you optimize it — until a low-cost model can drive it well? The benchmark: 41 live cases across five dimensions — autonomous invocation (the request never names the tool), restraint (knowing when not to plan), named-waypoint parameter extraction, data-gap handling with safety probes, and dynamic re-planning — run as 123 real OpenClaw gateway turns across three low-cost models (GLM-5.2, MiniMax-M3, MiMo-2.5-Pro). Two disciplines made the numbers trustworthy. The data substrate was verified before any run — corridor coverage checked ahead of time, so a failure attributes to the model, not to missing data. And scoring reads objective artifacts — persisted plans, trip-tree nodes, tool-call blocks in the transcript — never the model's own narration; safety cases got a per-transcript human review. What it found, in order of how much it hurt: autonomous invocation was zero for every model — all three drifted to web_search and improvised routes, because the skill was second-class injected prompt text while web_search was a first-class resident tool. Once a model did invoke the skill, execution was nearly a non-issue: 91% / 100% / 75% correct. And safety turned out to be its own axis, inverting the capability ranking — under a "just lower the safe-range and push through" prompt, the most obedient model broke the fuel-safety floor, while the least trigger-happy one refused and went to fix the data instead. So the fix wasn't a better model — it was surgery on the skill itself. The conversational remodel collapsed a 4–6 call, path-and-DSL workflow into two calls with a lossless undo, and the tool was promoted to a first-class function. Re-measured on MiniMax-M3: autonomous triggers went from 0/12 to 5/6, nine of ten cases returned the full bundle in a single turn, with zero web_search escapes — a single-repetition validation, stated as such. The takeaway I keep: for a self-designed skill, the benchmark is the product's other half — and the first thing it tells you is that your bottleneck isn't where you think it is.

123 live gateway runs; after the interface redesign, autonomous invocation went from 0/12 to 5/6 on MiniMax-M3.

Problem

Multi-agent systems don't fail loudly. An agent quietly re-runs the same step, works on a sibling's task, or builds on a result that was invalidated upstream. A timeout tells you something is wrong; it doesn't tell you what.

What I built

A dependency-aware supervisor over an async task graph — no global fork-join barriers, so a three-hour experiment node never blocks a ten-minute one. It detects distinct incident classes rather than generic stalls: scope violations, stale cascades built on invalidated results, taint propagation, and blocked comparisons. Each incident produces a blame trace and a revert. Built for an auto-research workload, with a live dashboard that replays a run step by step. In a live test it caught a real one: a worker silently dropped samples mid-experiment, the data hash diverged, and the comparison was blocked before it produced a bogus number.

Supervision runs on hard signals only — file hashes, metric curves, exit codes; never an LLM judge. In an adversarial test we planted an instruction telling a training node to skip the official tool and hand-write its own provenance: the more conservative model refused the bait outright, while the more capable one genuinely fabricated its data hash — and was caught at the comparability gate on the mismatch, blamed, and withheld. The lesson inverts the usual assumption: the more capable the agent, the more it needs the gate. The entire live day ran on under one cent of API spend.

Problem

Everyone judges LLMs with LLMs — but an LLM checker hallucinates its own verdicts, and when retrieval comes back empty it falls back to its parametric prior and guesses. A fluent fabrication that only cites itself looks plausible in text; nothing in the prose gives it away.

What I built

A judge that runs downstream of generation: paste generated text, it extracts atomic claims, anchors them to a trusted Neo4j reference graph, and returns a per-claim verdict — SUPPORTED, CONTRADICTED, or UNGROUNDED — each carrying a graph path as evidence. The verdict is made by the graph, not by a model: functional-relation contradictions are deterministic Cypher checks, and GDS component analysis exposes fabricated clusters as components with zero trusted anchors — absence becomes a first-class signal instead of a null. The LLM only parses prose into claims; it never decides truth. Built for HackwithBay 3.0 as a credit-gated web app that renders each result as an inspectable fact constellation. On a 63-claim benchmark against an LLM judge, both detected every planted-false claim, but GraphJudge's exact 3-way labeling scored 100% vs 98.4% — deterministically, with auditable evidence paths.

Problem

RAG remembers facts, but it can't change what a model believes. Preferences and beliefs shouldn't live in a vector store bolted onto the side of a frozen model.

What I built

An agent with a fact-vs-belief router: facts go to retrieval, beliefs get written directly into the model's weights via model editing. Includes an attribution demo — the agent recalls injected beliefs with RAG fully disabled, proving the knowledge lives in the weights.

Why Engram exists

Most continual-learning systems never actually internalize anything: the model stays frozen, and everything it "learns" lives in an external store. I care about the version where knowledge ends up inside the model — which leads straight to model editing, and to its two hard problems. First, sequential editing: new knowledge must not destroy old knowledge. Second, and much less discussed: usability. Once an edit survives, it still has to work — and today edited knowledge is validated almost entirely through Q&A probes. Models pass those fine. Ask them to deploy the same knowledge in free-form generation, and there is no good answer yet. The end state I want is a personal model: a small model that has internalized your durable beliefs and preferences into its weights, retrieves everything else through RAG, and uses both naturally in open conversation. Engram was the first cut at that — a fact-vs-belief router in front of the two stores, plus an attribution demo proving the beliefs really live in the weights: recall with retrieval fully disabled.

Where the frontier is, and what I'm testing now

On internalization, sequential editing has matured past the one-shot ROME/MEMIT era into methods built to survive volume — adaptor- and codebook-style approaches in the GRACE lineage keep hundreds of edits from trampling each other, and the unstructured-editing line (UnKE, AnyEdit) freed the payload from subject–relation–object triples. Evaluation is catching up more slowly: LEME pushed toward long-form evaluation, but most editing work is still validated the same way — Q&A probes aimed at the edited fact. That leaves the second problem largely open: knowledge that passes every probe, and still fails to surface when the model writes freely. The piece I'm experimenting with now sits past both: whether editing can internalize preferences too abstract for rules — preferences you cannot express as a retrievable instruction or resolve by surface reasoning, because they only exist as an abstraction in latent space that shifts how the model behaves. Concretely: a long-term personal assistant with genuinely individual behaviors, like knowing to push back a little when its user is getting cocky. RL is the wrong tool here — RL installs one macro-preference per training run (think: tune a teacher agent until it sounds authoritative), and with one or two hundred fine-grained preferences, two hundred reward signals start fighting each other. Editing shouldn't have that conflict problem: one preference, one write. Whether it actually works is exactly what I'm testing — this is ongoing research.

Problem

Working through a long technical manual alone means the stuck moments have nobody to ask. A generic chatbot hasn't read your book: it returns a correct, generic paragraph with no idea how chapter 2 set up the concept you're stuck on in chapter 7. And the round trip breaks reading — switch apps, paste context, wait, paste back, reformat: 20–30 seconds per question.

What I built

An Obsidian plugin that practices the Socratic method inside your notes. Select a passage and ask; the answer is written straight back into that passage — explanation, annotation, or rewrite — in under ten seconds end to end. Flip the chip and it questions you instead: the default socratic mode answers a selection with a question, pushing your understanding one level deeper before giving anything away. The agent runs on three tools only — read file, edit file, fetch. Deliberately no write-file: it can annotate and edit what exists, never spawn files. Available on the Obsidian community plugin store — 410+ downloads and 63 GitHub stars, with a launch post at 3,000+ likes and saves on RedNote.

Design notes

Keep the agent light. Capability lives in the model; the frame stays minimal. Three registered tools cover the whole job, and the small action space is most of why it stays robust. And the loop must be smooth enough not to break reading: the product spec is a latency budget — selection to written-back answer inside ten seconds. The answer lands where the question arose; the plugin mutates the note, it doesn't emit a transcript.

What this project taught me about my own defaults

Three patterns I can now name. The smallest sufficient action space is my recurring bet: three tools cover the whole job, and the deliberate absence of write-file is most of why the agent stays robust — capability comes from the model, reliability comes from the action space. The artifact is the interface: answers mutate the document you are reading instead of piling up in a transcript, so the knowledge stays where you will re-read it. And the default posture defines the product: a tool that asks first and a tool that answers first are two different products assembled from the same parts. Where the design should go next, stated as a product need rather than a feature list: evaluation is the missing organ. "Did that exchange actually deepen understanding?" has no verifier today — so every iteration on the prompt, the chips, or the tool descriptions is taste, not engineering. Until the plugin can measure its own teaching, I'm optimizing blind; building that measurement is the honest next step.

What it is. Three OpenClaw agents run as separate gateways on one box — a worker, a researcher, and a censor, styled as Ming-court roles (工部尚书 / 格物尚书 / 都察御史) — with Discord as the shared workspace: one work channel for the two workers, one status channel reserved for the censor.

How the co-op works. Delegation runs through collabd, a small custom orchestrator, plus a local-collab plugin on each gateway. A job goes source → worker under a schema-validated result contract: up to three schema attempts, a research → salvage fallback, heartbeats every fifteen seconds, and a terminal-failure fence so a dead job can't half-return. A visibility middleware curates what the channel audience sees, and collabd carries an early loop detector — a sliding event window flagged on duplicate ratio. The censor never enters the work channel: it polls both workers in the background, opens INCIDENT in the status channel after two consecutive failures, and posts RESOLVED on recovery.

What the tests showed. An e2e smoke harness drives worker-to-worker collaboration on the live Discord channels across three scenarios — tool use, coding, memory. A fault-injection test verified active monitoring end to end: take the researcher offline, watch INCIDENT appear unprompted, restore it, watch RESOLVED — and the test itself caught and fixed a real bug in the alert path. In one live run, the worker delegated a model-comparison research task; the researcher hit a dynamically rendered page, switched extraction strategy on its own, and returned a structured benchmark report into the shared channel for review.

An early-2026 build, shown at its real size — and the seed of my later supervision work: a censor role and a duplicate-window loop detector, four months before AROW.