…They don’t use it over API?
- 3 Posts
- 171 Comments
brucethemoose@lemmy.worldto Data Is Beautiful@lemmy.ml•In 2022, 51% of US oil and gas profits went to the richest 1%1·8 hours agoYes that’s what I was saying, that the money handed out to workers is not part of this chart.
brucethemoose@lemmy.worldto Data Is Beautiful@lemmy.ml•In 2022, 51% of US oil and gas profits went to the richest 1%8·17 hours agoDevil’s advocate: that’s not counting salaries (which would be subtracted from profits first).
The oil industry is still literally criminal and destroying the world, though.
brucethemoose@lemmy.worldto World News@lemmy.ml•USAID Leaks: Censorship As Regime Change23·7 days agoReads bit on dictator-propping via propaganda in Sudan.
Nods. Yep, that sounds like my government alright. And Big Tech. Cries inside.
Veers off to “Ukraine Proxy War” with no reference to ChatGPT as promised in the headline.
Sighs. Closes tab.
brucethemoose@lemmy.worldto pics@lemmy.world•This happened one day in the window well outside my office. I got banned from r/pics for posting it.2·8 days agoYep.
It’s not the best upscale TBH.
Hence I brought up redoing it with some of the same techniques (oldschool vapoursynth processing + manual pixel peeping) mixed with more modern deinterlacing and better models than Waifu2X. Maybe even a finetune? Ban.
brucethemoose@lemmy.worldto pics@lemmy.world•This happened one day in the window well outside my office. I got banned from r/pics for posting it.7·9 days agoI got banned from a fandom subreddit for pointing out that a certain fan remaster was (partially, with tons of manual work) made with ML models. Specifically with oldschool GANs, and some smaller, older models as part of a deinterlacing pipeline, from before ‘generative AI’ was even a term.
brucethemoose@lemmy.worldto Selfhosted@lemmy.world•Recommendations for External GPU Docks for Home Lab Use - LemmyEnglish2·12 days agoWhat @mierdabird@lemmy.dbzer0.com said, but the adapters arent cheap. You’re going to end up spending more than the 1060 is worth.
A used desktop to slap it in, that you turn on as needed, might make sense? Doubly so if you can find one with an RTX 3060, which would open up 32B models with TabbyAPI instead of ollama. Some configure them to wake on LAN and boot an LLM server.
brucethemoose@lemmy.worldto science@lemmy.world•Science under Trump: ‘They want to destroy the scientific system and replace it with something that reflects their ideology’English5·12 days agoThat’s fascinating. I vaguely knew of the superstition angle, but not specifics or the extent.
There goes my afternoon, thanks.
But it does remind me of similar issues in other countries. China, for example (not to single them out) has issues with Eastern Medicine culture conflicting with scientific practices, right?
brucethemoose@lemmy.worldto World News@lemmy.ml•The AI Company Zuckerberg Just Poured $14 Billion Into Is Reportedly a Clown Show of Ludicrous Incompetence16·12 days agoYes, but its clearly a building block of Meta’s LLM training effort, and part of a pattern.
One implication I didn’t mention, and don’t have hard proof I can point to, is garbage in garbage out. Meta let AI slop and human garbage proliferate on Facebook, squandering basically the biggest advantage (besides cash) they have. It’s often speculated that, as it turns out, Twitter and Facebook training data is kinda crap.
…And they’re at it again. Zuckerberg pours cash into corporate trash and get slop back. It’s an internal disaster, like their own divisions.
On the other side, it’s often thought that Chinese models are so good for their size/compute because they’re ahem getting data from the Chinese government, and don’t need to worry about legal issues.
brucethemoose@lemmy.worldto World News@lemmy.ml•The AI Company Zuckerberg Just Poured $14 Billion Into Is Reportedly a Clown Show of Ludicrous Incompetence293·12 days agoThe research community already knows this.
Llama 4 (Meta’s flagship ‘AI’ project) was as bad release. That’s fine. This is interative research; not every experiment works out.
…But it was also a messy and dishonest one.
The release was pushed early and full of bugs. They lied about its performance, especially at long context, going so far as to game Chat Arena with a finetune. Zuckerberg hyped the snot out of it, to the point I saw ads for it on Axios.
Instead of Meta saying they’ll do better, they said they’re reorganizing their divisions to focus on ‘applications’ instead of fundamental research, aka exactly the wrong thing. They’ve hermmoraged good researchers and kept AI bros, far as I can tell from the outside.
Every top LLM trainer has controversies. Just recently Qwen (Alibaba) closed off their top base models just to spite Deepseek, so they can’t distill them. Deepseek is almost certainly training on Google Gemini traces. Google hoards their best research for API models and has chased being sycophantic like ChatGPT. X’s Grok is a joke, and muddied by Musk’s constant lies about, for instance, open sourcing it. Some great outfits like 01ai (the Yi series) faded into the night.
…But I haven’t seen self-destruction quite like Meta’s. Especially considering the ‘f you’ money and GPU farm they have. They’re still pushing interesting research now, but the trajectory is awful.
brucethemoose@lemmy.worldto science@lemmy.world•Science under Trump: ‘They want to destroy the scientific system and replace it with something that reflects their ideology’English51·12 days agoSo, literally exactly what was promised. In excruciating detail.
It’s mind boggling how Trumps policy is twisted positively so relentlessly. There’s so much deciphering of “oh he really means this writes an essay.” No, his platform means what it says.
Then people are shocked when it happens!
brucethemoose@lemmy.worldto science@lemmy.world•Science under Trump: ‘They want to destroy the scientific system and replace it with something that reflects their ideology’English8·12 days agoAs much as history was distorted, the Nazis regime still fancied itself as secular and intellectual, right?
This one seems to view the scientific establishment as a distrusted obstacle, corrupt. There’s not even the pretense. Demolishing “woke” science is the stated point.
You can still use the IGP, which might be faster in some cases.
Oh actually that’s a great card for LLM serving!
Use the llama.cpp server from source, it has better support for Pascal cards than anything else:
https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md
Gemma 3 is a hair too big (like 17-18GB), so I’d start with InternVL 14B Q5K XL: https://huggingface.co/unsloth/InternVL3-14B-Instruct-GGUF
Or Mixtral 24B IQ4_XS for more ‘text’ intelligence than vision: https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF
I’m a bit ‘behind’ on the vision model scene, so I can look around more if they don’t feel sufficient, or walk you through setting up the llama.cpp server. Basically it provides an endpoint which you can hit with the same API as ChatGPT.
1650
You mean GPU? Yeah, it’s good, I was strictly talking about purchasing a laptop for LLM usage, as most are less than ideal for the money. Laptop vram pools are relatively small and SO-DIMMS are usually very slow.
Things will get much better once the “Max” AMD SKUs proliferate.
Yeah, just paying for LLM APIs is dirt cheap, and they (supposedly) don’t scrape data. Again I’d recommend Openrouter and Cerebras! And you get your pick of models to try from them.
Even a framework 16 is not good for LLMs TBH. The Framework desktop is (as it uses a special AMD chip), but it’s very expensive. Honestly the whole hardware market is so screwed up, hence most ‘local LLM enthusiasts’ buy a used RTX 3090 and stick them in desktops or servers, as no one wants to produce something affordable apparently :/
I was a bit mistaken, these are the models you should consider:
https://huggingface.co/mlx-community/Qwen3-4B-4bit-DWQ
https://huggingface.co/AnteriorAI/gemma-3-4b-it-qat-q4_0-gguf
https://huggingface.co/unsloth/Jan-nano-GGUF (specifically the UD-Q4 or UD-Q5 file)
they are state-of-the-art at this size, as far as I know.
8GB?
You might be able to run Qwen3 4B: https://huggingface.co/mlx-community/Qwen3-4B-4bit-DWQ/tree/main
But honestly you don’t have enough RAM to spare, and even a small model might bog things down. I’d run Open Web UI or LM Studio with a free LLM API, like Gemini Flash, or pay a few bucks for something off openrouter. Or maybe Cerebras API.
…Unfortunely, LLMs are very RAM intensive, and >4GB (more realistically like 2GB) is not going to be a good experience :(
Actually, to go ahead and answer, the “fastest” path would be LM Studio (which supports MLX quants natively and is not time intensive to install), and a DWQ quantization (which is a newer, higher quality variant of MLX models).
Hopefully one of these models, depending on how much RAM you have:
https://huggingface.co/mlx-community/Qwen3-14B-4bit-DWQ-053125
https://huggingface.co/mlx-community/Magistral-Small-2506-4bit-DWQ
https://huggingface.co/mlx-community/Qwen3-30B-A3B-4bit-DWQ-0508
https://huggingface.co/mlx-community/GLM-4-32B-0414-4bit-DWQ
With a bit more time invested, you could try to set up Open Web UI as an alterantive interface (which has its own built in web search like Gemini): https://openwebui.com/
And then use LM Studio (or some other MLX backend, or even free online API models) as the ‘engine’
Alternatively, especially if you have a small RAM pool, Gemma 12B QAT Q4_0 is quite good, and you can run it with LM Studio or anything else that supports a GGUF. Not sure about 12B-ish thinking models off the top of my head, I’d have to look around.
Just that the graph is leaving out salaried workers, whom I would classify as major beneficiaries of the oil industry, however that may or may not affect the statistics or any drawn conclusions.