GPUaccelerated OP t1_iu4umuw wrote on October 28, 2022 at 3:45 PM

Reply to comment by ShadowStormDrift in Do companies actually care about their model's training/inference speed? by GPUaccelerated

Yeah, see in your use case, speed makes so much sense. Thank you for sharing.

Mind sharing that site with us here?

I'm always interested in taking a look at cool projects.

Also what kind of hardware is currently tasked with your project's inference?

ShadowStormDrift t1_iu53ih6 wrote on October 28, 2022 at 4:44 PM

Of course!

www.sasdghub.up.ac.za

The semantic search as well as a few other key features haven't made it up yet. We're aiming to have them up end of November, mid December.

We've got a two server setup with the second being our "Work-horse" intended for GPU related jobs. It's an RTX 3090 with 32GB VRAM, 64GB DDR4 RAM and a 8 core CPU (I forget it's exact setup)

GPUaccelerated OP t1_iuim3cp wrote on October 31, 2022 at 4:42 PM

Very cool! But I think you mean 24GB of VRAM for the 3090?

Issues loading the web page, btw.

ShadowStormDrift t1_iuivphh wrote on October 31, 2022 at 5:45 PM

https://sasdghub.up.ac.za/en/

Let's try this lol.

GPUaccelerated OP t1_iuixkzj wrote on October 31, 2022 at 5:57 PM

So cool! Good for you!