llm-dataset-pipeline
Lightweight, containerized data ingestion and sanitization pipeline formatting raw web/API extractions into deduplicated JSONL instruction datasets.
Lightweight, containerized data ingestion and sanitization pipeline formatting raw web/API extractions into deduplicated JSONL instruction datasets.