Skip to content
TechIndexSubmit

Similar tools

  • XCrawl gives developers one API for web data. Scrape URLs into Markdown, JSON, HTML or screenshots; crawl sites; map domains; and fetch structured SERP results. Rendering, proxies and retries run server-side. Python/Node SDKs, CLI, MCP server and n8n node are available; new accounts get 1,000 free credits.

    Dev Tools
  • Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data β€” all from Rust. CLI, REST API, and MCP server.

    MCP Servers
  • πŸ”₯ Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.

    MCP Servers
  • Open-source multi-server Discord channel scraper. Configure many servers in one TOML file, pick channels by name or ID, export to JSONL for offline search and LLM pipelines.

    Datasets
  • Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.

    Models
  • A multithreaded πŸ•ΈοΈ web crawler that recursively crawls a website and creates a πŸ”½ markdown file for each page, designed for LLM RAG

    Dev Tools