Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Parallelism
Submitted by admin on May 9, 2024 - 3:16 pm
Title | Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Parallelism |
Publication Type | Conference Paper |
Year of Publication | 2024 |
Authors | Yuan T, Liu Y, Ye X, Zhang S, Tan J, Chen B, Song C, Zhang D |
Conference Name | 2024 USENIX Annual Technical Conference (USENIX ATC 24) |
Date Published | 07/2024 |
Publisher | USENIX Association |
Conference Location | Santa Clara, CA |
ISBN Number | 978-1-939133-41-0 |
URL | https://www.usenix.org/conference/atc24/presentation/yuan |