Reproducible recipe: serve abliterated Gemma-4-12B (gemma4_unified) at 50-118 tok/s on no-NVLink Blackwell (SM120) via vLLM nightly + ModelOpt FP8/NVFP4 + MTP spec-decode.
Patches + recipe to deploy festr2/MiMo-V2.5-Pro-NVFP4-MXFP8-attn-TP8 on 8-node DGX Spark sm_121 (Ray + vLLM, TP=8). Fixes the fused-qkv loader bug that mis-slotted Q values as K/V on 7 of 8 ranks.
A ~800-line PyTorch implementation of Megatron-LM's TP + PP + DP + AMP. 1.6-2x faster than Megatron-Core on 125M models.
Will a Hugging Face LLM fit your GPU or DGX? Plan weights, KV cache or recurrent state, context, TP/DP, and measured calibration
Serve an abliterated Gemma-4-12B at high speeds on Blackwell GPUs without NVLink using vLLM, FP8 quantization, and MTP speculative decoding.
Add a description, image, and links to the tensor-parallel topic page so that developers can more easily learn about it.
To associate your repository with the tensor-parallel topic, visit your repo's landing page and select "manage topics."