kindi , A custom LLM inference engine for the NVIDIA Quadro K420...

kindi is a custom LLM inference engine for NVIDIA Quadro K420. It delivers 20+ tokens per second with an OpenAI‑compatible server, running in Python on CUDA Kepler GPUs. The repo has 13 stars and supports BitNet models, enabling local LLM inference on modest hardware.

Opening Kapyn…