Would be another 24G of vram allowing me to almost fully keep GPT-OSS-120B in vram probably speeding it up 200%.
You are viewing a single comment's thread from:
Would be another 24G of vram allowing me to almost fully keep GPT-OSS-120B in vram probably speeding it up 200%.
How long would it take to load the gguf from flash though, I mean I guess if it only has to do it once.
Probably 10-20 minutes instead of 10-20 seconds. I do it once every 3 months or so. I do want to try 5090 on this setup, that would be a massive boost.