Skip to content
TechIndexSubmit
Back to directory

Colibri

A hierarchical inference engine designed for frontier Mixture-of-Experts (MoE) models, optimizing expert placement across GPU, RAM, and NVMe for efficient execution on existing hardware.

Similar tools

  • Optimized GPU compiler for LLM inference. Choose from a list of optimized recipes or optimize your own model via kernel fusion, autotuning, and advanced scheduling. Run benchmarks across different GPU types and configurations, track results and share experiments with the community.

    Dev Tools
  • A high-performance inference engine designed for Jev-like decision models, offering significant speedups over existing solutions. It supports over 30 popular local decision models for AI developers.

    Dev Tools
  • A high-throughput and memory-efficient inference and serving engine for LLMs

    Models
  • JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

    Dev Tools
  • Mechanical Efficiency Engine for the Spectrum Ecosystem. Hardware-accelerated execution powered by NVIDIA NIM and OpenRouter.

    Models
  • On-device LLM Inference Powered by X-Bit Quantization

    Models