I'd be interested to know more about the options. Some people run local system, but you need the hardware for that.
You are viewing a single comment's thread from:
I'd be interested to know more about the options. Some people run local system, but you need the hardware for that.
The hardware you need is becoming less as time goes by and the models get better. Just access any free online AI like copilot, tell it what hardware you have and what you want to do (coding for example) and it will tell you the best open source model to get and how to set it up. Took me about a half hour to do that.
A good person on here to follow for info on local AIs is @tydynrain who is making great progress running local AIs and agents.
Wishing you all the best!
I need to have a look. My new PC has a GPU that may do something.
You may find the following video useful to understand what various hardware can run when it comes to local AI.
Source: YouTube
I understand a bit better now. I bought my PC this year, but the high cost of memory limited what I got. I also need to work out what I would actually do with these tools. Generating images, video and music doesn't really interest me, but I might use them for coding.
How much VRAM does your system have, my friend? Using llama.cpp, or one of its forks like ik_llama.cpp, llama-cpp-turboquant, or buun-llama-cpp, it's feasible to run much larger MOE (Mixture of Experts) models than would normally be expected, since part of the experts are kept in RAM/CPU. Models like Qwen3.5 35B-A3B or Gemma-4 26B-A44B are very doable even with 6-8GB of VRAM. And as to coding, that's what I'm doing. I've been working on an autonomous liquidity-pool agent, to assist me in my asset-managing tasks. 😁🙏💚✨🤙
Yeah, I think it would be cool to do that, but I just don't have the capacity or know how for it.
You could write a bit about what you use and the limit of a free account if that's what you do. I've seen people say the prices could go up a lot as the companies try to turn a profit.
I've been thinking about that. I was going to do a follow up post, I just haven't gotten around to it yet.
I’d love to see more posts on this topic.