Submitted by Zondartul t3_zrbfcr in MachineLearning
avialex t1_j14p22o wrote
Reply to comment by sayoonarachu in [D] Running large language models on a home PC? by Zondartul
There's a VRAM memory leak in that code btw. I haven't tracked it down yet, but it's easy to solve with a torch cache clear in the forward method.
Viewing a single comment thread. View all comments