If you are using CPU only, you need to look at very small models or the 2-bit quants.
Everything will be extremely slow otherwise:
GPU:
Loaded Power: 465W
Speed: 18.5 tokens/second
CPU: Loaded Power: 115W
Speed: 1.60 tokens/second
GPUs are at least 3 times faster for the same power draw.
I don’t know what GPU you’ve got, but Lexi V2 is the best “small model” I’ve seen with emotions, that I can just cite from the top of my head.
It tends to skew male and can be a little dark at times, but it’s more complex than expected for the size (8B feels like 48-70B).
Do Q8_0 if you’ve got the VRAM, Q5_KL for speed, IQ4_XS if you’ve got a potato.
My answer to this is yes.
I’m an AI Developer and my only option was to self host because I didn’t want my training data leaking out onto the web and by extension, China and the rest (I trained on my own data, writing, and notes, along with Wikipedia).
Self-hosting gives you complete freedom but also as one other user cautioned, don’t fall down the well/rabbit hole.
Use an executable like LM Studio, and then an off the shelf pre-trained model from Huggingface.
VRAM × 0.8 for max size.
Experiment until you find one you like.
I think the grand irony about this thing is that if there really was a bacteria that could eat away it plastic there would be a mass panic – "new dangerous bacteria found eating away at plastic containers, all packaging rotting on store shelves!"
I thought this was a direct to video/streaming release.
Now that I know it’s in theaters, I’m gonna go watch it.
TLDR: ROCM is AMD’s variant for general purpose compute on GPUs, like CUDA is for NVIDIA.
OpenAI being for profit.
I use ROCM extensively (I write, design and train my own AIs/LLMs) and it’s annoying of an acronym, especially in command lines:
--rocm-6.6.7.2-dev1 --cuda
But I figure it’s a four letter acronym to match “CUDA”.
I also pronounce it “Rock’em” in verbal versus “Koo’dah” (Barracuda).
Everything is about marketing and goodwill. You can just say something has “blast processing” and fail to elaborate.
The bottom right “art” in green is pure schizoposting, the “DMs from the divine” and then walls of text are typical of degenerative schizophrenia.
I’m sorry OP, they might be an art thief, they might be insane, maybe both.
You now own two cats.
Your great grandpa looks like my great grandpa
Wtf OP
That scissor style used to cut portraits is pretty distinctive to the USSR
… Are we related?
Put four pots over the squares over the ground.
Shoot the dragon head statues, the pedestals raise.
The pedestals make stone grinding sounds and…
Only one pedestal has raised, the pots have caused the animation to bug out and the game engine to assume that the pedestal is in the final position on the floor.
The floor position has the lever locked.
The game developer never anticipated what a massive idiot I was
OP casually walks by pelicans and hears the Mortal Kombat music begin playing
I’m a big believer in the Pareto Principle where 20% of the users are responsible for 80% of the content.
Lemmy is Reddit’s Top 20%.
Try changing your dynamic. The human brain thrives on novelty.
Play genres you normally wouldn’t – e.g: Hate horror games? Good - try out Silent Hill or some classics you skipped over.
Get a VR headset, go from 2D screen to proper full body 3D - fly a plane over the Andes, feel like you’re actually flying. Pilot a huge mech. Build a colony on Mars with your own two hands.
Burnout happens and it’s worth examining other areas in your life, but gaming’s always been the wondrous thing it always has.
That’s a hell of a vacuous argument. Would you rather get seen outside of a window or let into the store?
Nobody gives a shit about the non-voting numbers or third party ballots, if anything, their response is negative, not positive: They claim people aren’t exercising their right to vote, or that someone has “spoiled the election”.
Meanwhile, those aligned to parties are forming orderly queues and voting consistently.
Which strategy do you think is more effective, from a political science and historical perspective?
Apparently the article states that the river failed E. Coli testing after heavy rainfalls, with levels detected 10 times the acceptable count.
This honestly just seems like a failure of the administration to properly take care of the athletes and wanting to go ahead with “a historic river race”.
They could have delayed or re-tested.
Olympics or not, they swam through a river of shit.
shaves the sphere down with a sculptor’s knife
There. 3.1416. Not perfectly round but it’ll bake in the oven just fine.
I’m sorry; AI was trained on the sole sum of human knowledge… if the perfect human being is by nature some variant of a psychopath, then perhaps the bias exists in the training data, and not the machine?
How can we create a perfect, moral human being out of the soup we currently have? I personally think it’s a miracle that sociopathy is the lowest of the neurological disorders our thinking machines have developed.
Powerful quote, also true