On-device LLM Inference Powered by X-Bit Quantization
Universal LLM Deployment Engine with ML Compilation
High-performance In-browser LLM Inference Engine
Optimizing inference proxy for LLMs
AirLLM 2.8T LLM inference with single 4GB GPU
A high-throughput and memory-efficient inference and serving engine for LLMs
A Home Assistant integration & Model to control your smart home using a Local LLM