Skip to content
TechIndexSubmit
Back to directory

emmy

Optimized GPU compiler for LLM inference. Choose from a list of optimized recipes or optimize your own model via kernel fusion, autotuning, and advanced scheduling. Run benchmarks across different GPU types and configurations, track results and share experiments with the community.

Similar tools

  • Open-source LLM inference benchmarks — TTFT, TPOT, Throughput, Latency & Cost-per-token for models like Llama, Qwen, Gemma, DeepSeek, Gpt-Oss etc. deployed on different dedicated GPUs.

    Models
  • Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application

    Dev Tools
  • EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU

    Models
  • LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

    Models
  • A high-throughput and memory-efficient inference and serving engine for LLMs

    Models
  • Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Agents