PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Zhao, Yanli, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, et al. 2023. “PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel.” arXiv:2304.11277 [cs.DC]. arxiv.org/abs/2304.11277
BibTeX
@misc{zhao2023pytorch,
title = {{{PyTorch FSDP}}: {{Experiences}} on Scaling Fully Sharded Data Parallel},
author = {Zhao, Yanli and Gu, Andrew and Varma, Rohan and Luo, Liang and Huang, Chien-Chin and Xu, Min and Wright, Less and Shojanazeri, Hamid and Ott, Myle and Shleifer, Sam and Desmaison, Alban and Balioglu, Can and Nguyen, Bernard and Chauhan, Geeta and Hao, Yuchen and Li, Shen},
year = 2023,
eprint = {2304.11277},
primaryclass = {cs.DC},
archiveprefix = {arXiv}
}