Skip to content
TechIndexSubmit
Back to directory

paddler

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale πŸ“πŸ¦™ Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

Similar tools

  • Selfhost modern LLM stacks. Run the whole fleet from your terminal

    Dev Tools
  • Self-hosted multi-model LLM and image platform that runs on free Kaggle and Colab GPUs. Chat UI, streaming, live metrics.

    Models
  • Self-hosted AI/LLM gateway built with Java & Spring Boot for running open-source LLMs locally via llama.cpp, with RAG, AI agents, vision, embeddings, web search, RBAC, and a React playground.

    Models
  • Deploy open-source LLMs on AWS in minutes β€” with OpenAI-compatible APIs and a powerful CLI/SDK toolkit.

    Models
  • This is a PHP library for Ollama. Ollama is an open-source project that serves as a powerful and user-friendly platform for running LLMs on your local machine. It acts as a bridge between the complexities of LLM technology and the desire for an accessible and customizable AI experience.

    Models
  • A high-throughput and memory-efficient inference and serving engine for LLMs

    Models