ML Engineering

Stas Bekman; Sam Foreman

February 20, 2024

Launchers with SLURM

The following are complete SLURM scripts that demonstrate how to integrate various launchers:

torchrun - to be used with PyTorch distributed.
accelerate - to be used with HF Accelerate.
lightning - to be used with Lightning (“PyTorch Lightning” and “Lightning Fabric”).

Back to top

Citation

BibTeX citation:

@online{bekman2024,
  author = {Bekman, Stas and Foreman, Sam},
  title = {ML {Engineering}},
  date = {2024-02-20},
  url = {https://saforem2.github.io/ml-engineering},
  langid = {en}
}

For attribution, please cite this work as:

Bekman, Stas, and Sam Foreman. 2024. “ML Engineering.” February 20, 2024. https://saforem2.github.io/ml-engineering.