Submitted by head_robotics t3_1172jrs in MachineLearning
catch23 t1_j9dxlze wrote
Reply to comment by smallfried in [D] Large Language Models feasible to run on 32GB RAM / 8 GB VRAM / 24GB VRAM by head_robotics
Their benchmark was done on a 16GB T4 which is anything but beefy. The T4 maxes out at 80W power consumption, and was primarily marketed toward model inference. The T4 is the cheapest GPU offered by google cloud.
Viewing a single comment thread. View all comments