news
312

Delta Inference Engine, a Windows port of the Linux engine Xolotl, runs all 552B parameters of DeepSeek-V4.1-Flash-1017 with 2-bit experts on a single consumer PC. It serves an OpenAI- and Anthropic-compatible API with up to 1M tokens of context. It needs an NVIDIA RTX card with 12 GB+ VRAM, 128 GB+ RAM and about 385 GB on a fast NVMe SSD. On an RTX 3090 with 128 GB RAM it writes about 33 tokens per second.

Comments

0

Only full members can comment

Log in to express your opinion.

No comments yet.