Viewing a single comment thread. View all comments

itsyourboiirow t1_jecqjqd wrote on March 31, 2023 at 1:34 AM

Reply to comment by Evening_Ad6637 in [D] Training a 65b LLaMA model by Business-Lead2679

Training requires a significant more amount of memory as it it has to keep track of the gradient for every parameter. I would check to see how much memory it takes up on your computer.