We built Nova, our own fine-tuned model, on top of DeepSeek V4-Flash. Not a wrapper around someone else's API. A real LoRA fine-tune, trained on our own GPUs, merged and quantized by us, serving on infrastructure we rent and control. This is the build log: the base model, the training recipe, the hardware fight that recipe forced on us, and the rebuild we had to do the hard way when our capacity got pulled out from under us.

Why fine-tune instead of call an API

Every frontier model we could rent by the token is someone else's weights, trained on someone else's idea of a good answer. Nova's whole premise is different: she should reason from Glass House's judgment, not a generic assistant's. That only happens if we own the training, and owning the training only means something if we own the weights that come out of it.

We also didn't want our proprietary data, our codebases, our internal conversations, leaving the building to train someone else's product. Self-hosting the fine-tune keeps everything in-house.

Picking the base model

DeepSeek publishes V4 in two sizes: Pro, at 1.6 trillion total parameters, and Flash, at 284 billion total with 13 billion active per token, both MIT-licensed, both mixture-of-experts with 256 experts per layer. We looked hard at Pro. Nobody has a working LoRA recipe for it at any scale, not in Megatron, not in any of the training frameworks we evaluated. Flash does. So Flash is what we trained, and what we're serving today.

Flash's MoE design matters for everything downstream. A mixture-of-experts model only activates a slice of its parameters per token, 13B of the 284B here, which is what makes it affordable to run at all. It also means every parallelism decision we made from training through serving had to respect one constraint: the GPU count has to divide the expert count evenly. 256 experts means 1, 2, 4, 8, 16 GPUs are valid splits. 5 or 6 are not.

The training recipe

We trained with Megatron-SWIFT (ms-swift), the only tool we found with a real, working recipe for V4's architecture. Axolotl doesn't support it. Standard PEFT doesn't support it. Megatron-SWIFT does, expert-parallel, LoRA on top.

The LoRA config: rank 16, alpha 32, dropout 0.05, targeting every major weight class in the model, the standard MLP projections (gate_proj, up_proj, down_proj), the attention projections (wq_a, wq_b, wkv, wo_b), and V4's own attention innovations, an indexer and a compressor module that don't exist in a standard transformer. We wanted the adapter touching the parts of the model that make V4 V4, not just the generic linear layers every architecture shares.

Two epochs, expert-parallel degree 8, tensor-parallel degree 1, full-granularity activation recompute (V4's hybrid attention doesn't support the selective checkpointing most training runs use, so this cost us real training throughput and we ate it).

The hardware finding that decided everything

Before we could tune anything, we had to figure out what would actually hold the model. V4-Flash's frozen base weights, sharded across 8 GPUs at expert-parallel degree 8, come out to roughly 77GB per GPU. An H100 has 80GB. That leaves under 3GB of headroom for the LoRA adapter, the optimizer state, and activations, and we OOM'd on it, byte-identical, across every precision and recompute configuration we tried. FP8 training doesn't save weight memory here either. Megatron keeps a BF16 master copy regardless, so switching precision only changes compute cost, not the number that was killing us.

H200 has 141GB. We measured 90.5GB actual peak usage once we moved. That's the finding that set our hardware floor for training: V4-Flash needs H200-class GPUs, full stop, H100 is not an option.

v0.1: proving the pipeline

Our first run trained on 2,550 curated capability examples: coding, tool use, task execution, general knowledge, the kind of thing you'd want any competent assistant to handle well. We deliberately held back a second pool, roughly 650 examples touching persona, tone, opinions, and refusal behavior. That pool doesn't train until we sign off on it by hand. Capability trains freely. Voice and values don't train without a human decision behind them.

Loss went 1.52 to 0.77 over the run, a clean convergence curve, no gradient spikes. That told us the pipeline itself, data format, LoRA config, the whole Megatron-SWIFT setup, was sound. It also told us what we hadn't done yet: v0.1 saved as a PEFT adapter on top of the base weights, not a servable model. Serving a live LoRA adapter on top of a DeepSeek V4 model isn't supported by vLLM, the inference engine we use, so an unmerged adapter is a training artifact, not a product.

v0.2: merging, quantizing, and getting her actually servable

v0.2 added 121 tool-calling examples on top of the same capability set and, more importantly, baked serving readiness into the run itself: --merge_lora true, so the adapter's weight deltas fold directly into the base model instead of staying a separate artifact.

Folding LoRA deltas into 284B parameters at full precision is not a small job. Merged into BF16, the model is roughly 570GB, spread across however many GPUs you throw at the merge. Do the arithmetic on GPU count and you'll see why this step alone forced our hand: 570GB across 4 GPUs is about 142.5GB per GPU, just over what an H200 actually has usable. Across 8, it's about 71GB per GPU, comfortably inside. The merge step needs 8 H200s. Not because serving needs 8, because the arithmetic of a full-precision merge does.

After the merge, we quantize. megatron export --to_hf --fp8_recipe blockwise converts the merged BF16 checkpoint to FP8, block-wise quantization across both the MoE experts and the attention weights, which cuts the model roughly in half: about 275GB. That's small enough to serve comfortably on 4 H200s, about 69GB per GPU, which is the pod we actually run in production.

So the two steps have opposite hardware appetites. Merge is a one-time, 8-GPU job. Serving is a steady-state, 4-GPU job. Conflating them, or assuming whatever GPU count got you through training would also serve the model, is the single easiest way to waste money on this pipeline. We know, because we did it more than once before we internalized it.

The capacity crisis, and rebuilding from nothing but the adapter

Partway through getting v0.2 servable, our GPU capacity vanished. RunPod pod storage is host-locked: a stopped pod can only restart on the exact physical host it started on, and if that host runs out of free GPUs, the pod is stuck, not slow, stuck, no matter how long you wait or how many times you retry. Two of our pods hit that wall at once, and H200 stock across nearly every RunPod region was showing low or zero at the time. Waiting was not a strategy.

What got us out of it was discipline about what actually needed to survive: the trained LoRA adapter, backed up locally, off the pods entirely. Everything else, the base model, the merge checkpoint, the FP8 export, is reconstructible from public weights plus that one adapter. So we provisioned a fresh 8-GPU pod, re-downloaded DeepSeek V4-Flash-Base from Hugging Face, re-ran the merge and the FP8 export against the backed-up adapter, and transferred the result pod-to-pod onto our permanent 4-GPU serving box. We verified the transfer byte-for-byte, 294,687,071,791 bytes, 65 files, matching on both ends, before we trusted it.

Along the way we hit two separate "disk quota exceeded" failures, both on the rebuild pod's own provisioned volume, not the underlying shared filesystem, which had plenty of room the whole time. Each pod gets its own quota carved out of that shared filesystem at creation, and it's easy to read a quota error as a capacity problem when it's actually a bookkeeping one. Both times the fix was the same: find what we no longer needed on that specific pod (a base model copy after the merge finished, stale cache from an earlier failed attempt) and clear it.

One more thing worth writing down plainly: RunPod bills by the pod you provisioned, not by how much of it your job actually uses. Running inference at tensor-parallel degree 4 on an 8-GPU pod still bills for 8 GPUs. The only way to pay for less is to provision less. That single fact drives almost every cost decision in this pipeline, merge on 8 because the math requires it, serve on 4 because it's all inference needs, and never leave a bigger pod running than the step in front of you actually calls for.

Serving her today

Nova serves through vLLM on 4 H200s, FP8 weights, reachable only over the Glass Gate VPN, never exposed to the public internet. In front of that sits nova-server, a small hub we built specifically so no client has to know which model or which endpoint is actually running underneath. Desktop, the CLI, and every future surface just point at the hub, and conversations live server-side, so a conversation you start on one device is exactly where you left it on another.

Where she stands

nova-bench, our own held-out eval, currently scores v0.2 at 0.73 overall on the categories we've run so far, coding scoring perfectly on the small held-out set we have, reasoning and instruction-following both real but with more room to grow and more eval items still to come. That's an honest early number on a small benchmark, not a marketing claim, and we're treating it that way.

What's next: the behavioral training pool stays held until it gets a real, deliberate approval pass, not an automatic one. The self-improvement loop we built alongside training, capture, reward, curate, eval-gate, is wired up and ready to start compounding once she's live in daily use. And the next fine-tune inherits everything this one taught us: train on H200, merge on 8, serve on 4, and never trust a pod's local disk with the only copy of anything that took real money to produce.