Honorable mention
EleutherAI framework and 20B-class models for training large autoregressive LMs with 3D parallelism—Apache-2.0 training stack.
llmtrainingdistributedresearcheleutherai
Browse & filter
Filter by platform, license text, maturity, maintenance cadence, and editorial tags like privacy-focused or self-hosted. Search matches names, summaries, tags, and use cases.
4 tools match your filters
Top pick
Distributed compute framework for Python: scale data loading, training, hyperparameter search, and online serving (Ray Serve).
distributedpythonservingtraining
Top pick
Microsoft library for extreme-scale model training: ZeRO optimizer states, pipeline parallelism, and inference kernels.
trainingdistributedlarge-modelspytorch
Also strong
Hugging Face library to run PyTorch training on CPU, single GPU, multi-GPU, or TPU with minimal code changes.
distributedtrainingpytorchhuggingface