Skip to content
TechIndexSubmit

Similar tools

  • High-performance In-browser LLM Inference Engine

    Models
  • A high-throughput and memory-efficient inference and serving engine for LLMs

    Models
  • On-device LLM Inference Powered by X-Bit Quantization

    Models
  • Universal LLM Deployment Engine with ML Compilation

    Models
  • Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

    Dev Tools
  • LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

    Models