← Back to Red Deer Investments  ·  AI Library Home

Research Note · June 5, 2026

The HRM Fine-Tuning Journey: Failing, Debugging, and Finally Getting Modest Improvement

The headline: Over four days and 13 training runs, I fine-tuned a 1-billion-parameter hierarchical reasoning model on an 8GB Mac Mini using QLoRA. I discovered a silent bug that zeroed out all learning, fought catastrophic forgetting across four iteration cycles, and ultimately achieved a net-positive result: +3.4% weighted score over the base model, with agent tool-calling jumping from 10% to 60%.

The improvement is modest. But the journey — the specific failures, the debugging, the architectural constraints that boxed me in — is worth documenting because it reveals the real ceiling of local fine-tuning on consumer hardware and the specific shape of the problems you will encounter when you try it yourself.

The Model: HRM-Text-1B

HRM-Text-1B is not a conventional transformer. It is a Hierarchical Reasoning Model — a recurrent architecture where the model processes each token through 8 passes (H=2 hidden cycles × L=3 loop cycles, plus the initial forward pass) before moving to the next token. The idea is that the model gets "thinking time" inside its hidden states without needing to emit explicit chain-of-thought tokens.

The result is a model that punches above its weight on reasoning tasks but has two major liabilities:

  • Zero pre-alignment. It is a base model — no instruction tuning, no RLHF, no code-specific pretraining. Ask it a question and it might autocomplete the prompt instead of answering.
  • Strict 4096-token context window. Hard architecture limit, not a config setting. Go over it and the model crashes.

The base model scored 58.3% on our 30-prompt v3 benchmark — competitive for a 1B model, but miles behind even the smallest model in our main benchmark (Ministral 3B at 68.3%, Qwen 4B at 74.4%).

The question was: could QLoRA fine-tuning on an 8GB Mac Mini close any of that gap?

The Journey: 13 Runs in 4 Days

What follows is every significant iteration, what changed, what broke, and what we learned. The full history is in the supplementary ranking file with per-run caveats; this is the narrative version.

Phase 0: Baseline (58.3%)

The base HRM-Text-1B model, loaded as-is from MLX 4-bit weights. On the benchmark, it scored:

  • AGENT: 10% (near-zero tool calling — it would echo the function schema instead of executing a call)
  • CODE: 70% (surprisingly competent — pre-training on code data left traces)
  • HALL: 62% (mixed — sometimes refused, sometimes fabricated)
  • Overall: 58.3% weighted

The base model has raw capability but no instruction-following discipline. SFT should help — in theory.

Phase 1: The Silent Gradient Bug (56.7%, 51.7%)

The first training runs looked like they worked. The script printed loss numbers that decreased. Memory was consumed. Adapter files were saved. Everything looked normal.

Everything was a lie.

When I scored the fine-tuned model, it scored lower than the base — sometimes substantially. I assumed the SFT data was noisy or the model was overfitting. I ran more iterations with different datasets. They all scored the same as or worse than the base.

Then I looked at the training code. The LoRA adapter was computing updates in the quantized weight dtype — but quantized weights in MLX 4-bit format use uint32 integers, not floats. The adapter was multiplying its gradient updates by integer zero. The model was not learning anything. It was writing saved adapter files that contained random initialization noise.

Fix: Cast the LoRA computation to the activation dtype (x.dtype, a float type) instead of the weight dtype. Suddenly, loss started dropping in real time. The second attempt — now with actual gradient updates — scored 53.3%, still worse than the base. But at least the training was real this time.

Phase 2: The Breakthrough That Broke Everything (53.3%)

With the gradient bug fixed, I implemented two major upgrades:

  • Masked response loss: Only the answer tokens contributed to the loss — not the prompt template. This stopped the model from wasting parameter capacity on formatting.
  • MLP target layers: In addition to attention projections, I hooked adapters to the SwiGLU MLP blocks (gate_up_proj, down_proj). This gave the adapters access to the model's core logic pathways.

The result was mixed in exactly the pattern that matters:

  • HALL soared to 88% — the highest score ever on reasoning and hallucination resistance. The MLP targets worked.
  • CODE collapsed to 40% — the model overfitted on the narrow set of coding templates in the SFT data and could not generalize.
  • AGENT collapsed to 10% — same story: the SFT agent data used simple sentences, the benchmark used complex JSON schemas, and the model could not bridge the gap.

The technical term for this is catastrophic forgetting: the model's limited parameter capacity got overwritten by the narrow SFT data, washing out the broad coding and agent knowledge from pre-training. The validation loss hit 0.0020 (near-perfect fit), but that just meant the model had memorized the training set — not that it had learned generalizable skills.

Phase 3: Nothing Works (45.0%, 43.3%, 51.7%)

The obvious next step was to make the training data more diverse. I expanded the SFT dataset to include multi-language code, complex JSON tool schemas, and varied instruction formats. I added best-validation early stopping to prevent overfitting.

It got worse.

Three successive runs — v4a, v4b, v5 — scored 45.0%, 43.3%, and 51.7%. The more diverse I made the training data, the more the model seemed to thrash. The MLP-targeted adapters were amplifying the noise: every gradient update through the recurrent MLPs modified parameters that would be applied 8 times per token, turning small changes into large disruptions.

The stop token was also wrong. During inference, the model would keep generating past the ideal stopping point because it was not recognizing the correct end-of-turn signal (token ID 11, <|box_end|>). Fixing this recovered some formatting quality but did not fix the core overfitting problem.

I was going backward. Five iterations in and still below the base model.

Phase 4: The v6 Breakthrough (61.7%)

Three things changed in v6:

  1. Removed MLP target layers. After the oscillation caused by recurrent MLP amplification, I went back to attention-projection-only adapters. This immediately stabilized training.
  2. Balanced the dataset carefully. The new split: 30% code (600 samples), 20% agentic Glaive (400 samples), 15% GSM8K math (300), 25% Alpaca instruction (500), and — critically — 10% Cosmopedia textbook replay (200 samples). The replay samples were formatted as pretraining data with empty prompts, which acted as a regularization anchor to prevent the model from forgetting its pre-training knowledge.
  3. Fixed the stop token. Proper evaluation with token ID 11 as EOS.

The result:

  • Overall: 61.7% — the first net-positive result (+3.4% over the base)
  • AGENT: 60% — up from 10%, a 6x improvement. The Glaive function-calling data worked.
  • CODE: 60% — recovered from the 40% collapse in v3 (though still below the base's 70%)
  • HALL: 75% — solid, though below the 88% peak from v3
  • CTX: 75% — matching the base model, no forgetting
  • SUMM: 67% — matching the base
  • INST: 60% — above the base's 40% baseline (the model now follows instructions instead of autocompleting them)

The improvements were uneven but real. Agent tool-calling went from non-functional to actually useful. Instruction following emerged where it did not exist before. The fine-tuning was no longer a net negative — it was a net positive.

Phase 5: The v7 Tradeoff (61.7%)

With the v6 baseline established, I tried one more iteration: replace the Alpaca instruction data (25%) with OpenHermes-2.5 (35%), and reduce GSM8K math from 15% to 5%. The idea was that OpenHermes's more diverse, higher-quality instruction data might improve coding and instruction following without the regression that Alpaca caused.

The result was a perfect tie on overall score (61.7%) with a shift in internal distribution:

  • CODE: 50% — improved from v6's 40% baseline code score (+10%)
  • All other categories unchanged

This confirmed that the OpenHermes dataset improved code generation — but the improvement was not large enough to move the overall score. The v7 model is marginally better at coding and the v6 model is just as good everywhere else. The tradeoff is real but small.

What Did We Actually Achieve?

Model Overall AGENT CODE HALL INST CTX
HRM-Text-1B BASE 58.3% 10% 70% 62% 40% 75%
HRM-1B SFT v6 Balanced 61.7% 60% 60% 75% 60% 75%
Delta +3.4% +50pp -10pp +13pp +20pp 0

The headline numbers are honest but uncomfortable:

  • +3.4% overall improvement after 13 training runs and 4 days of work
  • Agent tool-calling: from non-functional to functional. That is a real win. The Glaive SFT data taught the model a capability it did not have at all.
  • Instruction following: from non-functional to functional. Another real win.
  • Code: regressed. The base model was better at coding before we fine-tuned it. That is the alignment tax in action.
  • Still behind every other model in the benchmark. Ministral 3B (68.3%), Qwen 4B (74.4%), and the rest all significantly outperform the fine-tuned 1B model.

What Constrained Us

Six hard constraints limited what was achievable, and they are worth naming explicitly so anyone attempting similar work knows what to expect:

  1. 8GB memory ceiling. The Mac Mini has 8GB of unified memory. Training a 1B model with AdamW optimizer states requires approximately 12GB. The only way to fit is 4-bit quantization for the base model, LoRA adapters (not full fine-tuning), and a batch size of 1. This caps the effective learning capacity of any training run.
  2. Recurrent gradient amplification. Because HRM's H and L modules are called 8 times per token, any parameter change in the MLP blocks is applied recursively. Gradient scales can explode. The only safe target layers are attention projections, which are stabilized by softmax bounds. This limits what you can teach the model.
  3. 4096-token context ceiling. Hard architecture limit. Long prompts combined with verbose model outputs hit this wall frequently, truncating the response and reducing usable signal.
  4. Narrow SFT data availability. High-quality supervised fine-tuning datasets that are small enough to fit in a 2,000-sample budget and diverse enough to prevent overfitting are hard to find. OpenHermes-2.5 helped, but the diversity/relevance tradeoff is real.
  5. No RLHF or DPO capability on-device. Preference optimization algorithms like DPO require 2x the memory (reference model + training model) and are not feasible on 8GB. ORPO would work but requires paired chosen/rejected data that we did not have.
  6. No cloud offload. All training was local. For the cost of a few hours of GPU time, a cloud instance could have completed 10x the training in 1/10th the time. But that was not the point of this exercise.

The Honest Conclusion

This project worked — in the sense that we improved the model, we understand why and how, and the gains are real. Agent tool-calling went from an embarrassing 10% to a respectable 60%. The model now follows instructions. It hallucinates less.

But the improvement is modest, and the ceiling is real. A 1B-parameter model trained via QLoRA on 8GB of memory with 2,000 samples cannot catch up to a 3B or 4B dense model trained at scale. The architectural innovations that make HRM efficient also make it harder to fine-tune. The recurrent structure that gives it reasoning depth also amplifies gradient noise when you try to change its behavior.

If I were optimizing purely for results, I would use a cloud GPU instance — an L4 at $0.20/hour would have completed this entire journey (13 runs) in under an hour. But the point was to understand the constraints, not to bypass them. And the answer is clear: local QLoRA fine-tuning on consumer hardware can produce real, measurable improvements, but it is slow, iterative, and bumpy. The gains are real but small. The bottlenecks are architectural and financial, not procedural.

The model went from "broken in interesting ways" to "usable on specific tasks." That is not a headline result. It is an honest outcome of working within real constraints. And that is worth documenting.

Appendix: Training Details

Hardware: Apple Mac Mini M2, 8GB unified memory, macOS.

Software: MLX (Apple's ML framework), mlx_hrm_text (custom runtime), HuggingFace datasets, Transformers tokenizer.

Training config: QLoRA with rank=16, alpha=32. Attention projections only (gqkv_proj, o_proj) after v3. Learning rate 2e-5. Batch size 1, gradient accumulation. 2,000 iterations per run. AdamW optimizer.

Dataset composition (v6/v7): 2,000 total samples. 20% Glaive function-calling, 30% code instructions, 10% Cosmopedia replay, 15%/5% GSM8K math, 25%/35% instruction (Alpaca/OpenHermes). Masked response loss.

Adapter size: ~22 MB (adapters.npz), loaded from best_adapters.npz at inference time.

Scoring: 30-prompt v3 benchmark, rubric-based independent scorer, single temperature (0.3), single repeat. Scores are directional — true variance is unknown at n=30, r=1. The main benchmark runs used n=90, r=3 for statistical confidence.

Code and data: All training scripts, benchmark data, and fine-tuned adapters are available at github.com/blackdeerbits/sid-local-llm-bench (code + data) and huggingface.co/theblackdeer/hrm-1b-sft-qlora (adapters).

— the_red_deer