Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Parallelism

TitleAccelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Parallelism
Publication TypeConference Paper
Year of Publication2024
AuthorsYuan T, Liu Y, Ye X, Zhang S, Tan J, Chen B, Song C, Zhang D
Conference Name2024 USENIX Annual Technical Conference (USENIX ATC 24)
Date Published07/2024
PublisherUSENIX Association
Conference LocationSanta Clara, CA
ISBN Number978-1-939133-41-0
URLhttps://www.usenix.org/conference/atc24/presentation/yuan