SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + Control Center GUI.

TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs

Lightweight Modular AI Routing Engine for Local LLMs — Run specialised experts efficiently on consumer GPUs using smart Mixture-of-Experts routing.

Verified AI infrastructure for regulated deployment. UltraCompress (our wedge): near-lossless 5-bit compression with SHA-256-reproducible reconstruction - prove the model in production is the one you validated. 23 architectures (0.6B-405B), Hermes-3-405B @ 1.0066x. OpenAI-compatible API. pip install ultracompress

🦁 Local AI for Consumer GPUs — Run powerful LLMs on GTX 1060/1080. No cloud. No subscriptions. Built on llama.cpp + CUDA.

RAM-Backed MCP Memory Architecture for Consumer LLM Inference — 900K token context on 16GB VRAM

Arbitrary Numbers

Dynamic GPU Layer Swapping: Train large models on consumer GPUs with intelligent memory management

Optimize FAISS-compatible vector quantization for fast, accurate vector search with TurboQuant

Surgical reasoning on consumer silicon. Hybrid SSM + causal memory architecture with entropy-gated System 1/2 dispatch, O(1) inference memory, and continual learning — designed for 16 GB VRAM.

ismail is a from-scratch Turkish language model implementation designed for low-end hardware, built and trained on a single RTX 5070 (12GB).

RAMP: RL-guided Adaptive Mixed-Precision quantization for GGUF models. Data-free sensitivity analysis, evolutionary search, per-tensor type optimization. Produces hardware-optimized GGUF for consumer GPUs.

Self-hosted LLM chat client with streaming UI for vLLM servers. Run Mistral-24B locally on RTX 4090/3090. Privacy-focused ChatGPT alternative for homelab/gaming PCs. Python/Rich terminal UI.

Reproducible local inference for FLUX.1 [schnell] on 8 GB Turing GPUs (RTX 2070). Sequential CPU offload, fp16 compute, mock-tested pipeline init - portfolio piece for a neuroscience to ML transition.

Adaptive hot/cold MoE expert cache for llama.cpp: keep the hottest experts in VRAM to speed up decode when offloading to RAM on a consumer GPU. Measured +26% (Qwen3-Coder-Next 80B) and +31.6% (gpt-oss-120B) on one RTX 5090.

Kernel-level profiling of batch-1 decode on consumer GPU to prove that decode is memory-bandwidth-bound against the hardware roofline, then beating the baseline with a fused dequant+GEMV CUDA kernel. (regime-aware attribution at the end)

A comprehensive, modular framework for fine-tuning Stable Diffusion 3.5 models using LoRA (Low-Rank Adaptation). Create custom AI image generators tailored to your artistic style, objects, or concepts with memory-efficient training on consumer GPUs.

Researching local AI image editing on affordable consumer hardware: models, ComfyUI workflows, benchmarks, and real outputs on an RTX 5060 Ti 16GB.

Independent reconstruction of Memory Layers at Scale (Berges et al., 2024) on a single 24GB consumer GPU — Qwen2.5-7B, frozen backbone, with honest lessons learned.

Empirical study of speculative decoding with Qwen2-0.5B (draft) and Qwen2-1.5B (target) on a single RTX 2070. Measures token acceptance rate, speedup, KV sync overhead, and validates the Leviathan et al. (2023) analytical model. Shows why cost_ratio=1.18x (not 3.1x as expected) causes failure and what hardware conditions make it viable.