#localllm

Qwen3.8 27B FP8 is the breakthrough coding model marking the arrival of local AI capable of producing commit-worthy code. Previously, it was the exception rather than the rule. This one is finally worth adopting for daily workflows. The only issue is speed on older-generation CUDA and MPS hardware. The next substantial improvement of the same size will be the Opus 4.5 moment of local AI. Who's excited for Christmas? :) #qwen #qwen38 #ai #llm #localai #localllm

Ran Qwen3.6-27B locally on my Mac for a few weeks. Every database it created came out broken in one way or another. Opus 5 has been cleaning up after it, and doing a great job. Not a fair comparison - one of these is on my desk runs for free. What I keep coming back to: I'd have gotten a lot more out of the local model if I'd accepted from the start how much more work my prompting needed. I handed it frontier-model instructions and expected it to fill in the gaps. It couldn't. #AI #LocalLLM

Local LLM Arena #5 Pierwszy prawdziwy test GPT-OSS jako lokalnego agenta programistycznego przyniósł więcej informacji, niż się spodziewałem — tylko niekoniecznie o samym modelu. Okazało się, że pierwsza wersja testu miała problem z izolacją środowiska Codex CLI. Do kontekstu lokalnego modelu trafiały informacje o wtyczkach i narzędziach, które nie miały nic wspólnego z zadaniem programistycznym. Dlatego wyniku tego testu nie traktuję jako miarodajnego wyniku GPT-OSS. Zacząłem więc poprawiać środowisko: czyste sesje, brak chmurowego fallbacku, izolacja hidden testów, kontrola retry, timeouty i dodatkowa diagnostyka. I tutaj pojawił się kolejny problem. Sam system testujący zrobił się zbyt skomplikowany. Kolejne preflighty potrafiły zawieszać procesy na wiele godzin, a podczas ostatniej próby Antigravity doszedł do około 48 GB (?!?!?) zajętej pamięci na MacBooku Air M4 z 16 GB RAM. System w końcu przestał odpowiadać i potrzebny był twardy restart. Na szczęście właściwy Trial #2 nigdy nie został uruchomiony, a po restarcie Mac wrócił do normalnego stanu: 0 MB swapu i około 87% wolnej pamięci. Wniosek jest prosty: nie ma sensu dokładać kolejnych warstw do wadliwego harnessu. Teraz robię krok wstecz. Codex ma przeprowadzić audyt całej obecnej infrastruktury i pomóc zaprojektować prostszą wersję V4. Założenia są już inne: – każdy proces ma twardy timeout, – żadnego czekania godzinami, – test ma własny niezależny System Guard, – w razie problemów zatrzymywana jest tylko grupa procesów trialu, – hidden testy są całkowicie poza zasięgiem modelu podczas kodowania, – iCloud nie jest częścią krytycznej ścieżki, – Antigravity ma tylko przygotować i uruchomić test, a nie czekać na niego godzinami. Cel się nie zmienił. Tym razem jednak najpierw trzeba mieć pewność, że sam test nie jest groźniejszy dla Maca niż model, który ma sprawdzać. #LocalLLM #LLM #AI #AppleSilicon #MacBookAir #M4 #LMStudio #Codex #GPTOSS #Coding #OpenSourceAI

@RnDanger@infosec.exchange There's so many problems with these claims. Banning generated code? How do you even tell? Code quality? 'Claude' committed it? Has a certain "smell"? Or is it the US supreme court obscenity standard of " I'll know it when I see it."? Cause that last one is what Codeberg admins said they'd use. For obvious reasons, this is a terrible precedent. As for environmental impact, everyone who howls about that ignores that you can run #localllm on your own machines. And with Chinese open source/weight models freely available, means I can own my means of computation. And Chinese energy is some of the most environmentally sound given TW's of their energy is solar, and rewilding the edge of the Gobi while they're at it. I guess it does use a mug of water an hour.... Cause I'm drinking that mug of coffee. And 'submitting slop to the stack', so most FLOSS projects are just 1 human. So 1 human+1 bot is now somehow troubling?? I reject that assessment. The bot could be doing test writing, documentation, validating specifications, attacking the code in a container, etc. Now the big projects that are inundated - there are simple ways to impede that. Account age. Minimum viable commits. Required tests. Proofs of concept. No refactoring. Most of those would block the deluge.

Industrial GPU Adapted for the Desktop https://hackaday.com/2026/07/22/industrial-gpu-adapted-for-the-desktop/ #Computerhacks #Computer #Geforce #Gpu #LocalLLM #NVIDIA #PCIe #Server #Tensorsplitting #Vram

@ainmosni@social.ainmosni.eu @ErikJonker@mastodon.social Running #LocalLLM IS seizing the means of production. The american LLMs aren't sharing, but the Chinese are sharing. Guess what models I run?

UPDATED, pinged the problem children. @mozilla@mastodon.social @thunderbird@mastodon.online Sigh. I was hoping that #Thundermail #Mozilla #email would be good. NOOOOOOOPE. I had my #localLLM review their Terms of Service #TOS and #PrivacyPolicy . Its general Big Tech nonsense. My words, not the LLM. Terms of Service fuckups The "Arbitrary Dictator" Clause (Section 9c). WE'S DONT LIIKE THE CUT OF YOUR JIB. No due process. Fuck right outta here, no refunds. The "We Don't Care If You Lose Everything" Clause (Sections 11 & 12). You pay for a service but they promise absolutely nothing. They could shut down tomorrow, and youre SOL. The "Pay Our Lawyers" Clause (Section 13). If you fuck up, or not even fuck up and someone comes after them and they mention you, YOU have to pay their legal bills. The "We Can Change the Rules Whenever We Want" Clause (Section 14). Usual Big Tech bullshittery. (LACK OF) Privacy Policy The "We Use Your Real Emails to Test Stuff" Clause (Section: Processing Purposes). These fuckers actually use YOUR personal emails on their testing systems. The "Manual Review" of Private Emails (Section: Private Information: MZLA Access). If they THINK you might violate their ToS or their authoritarian feels, they read everything of yours. The "We Know You Opened This" Trap (Section: Cookies/Tracking) They pixel-tag their emails that you cant bloody turn off. The "Data Retention Black Hole" (Section: Security and Retention) Theres no time limit how long they'll feed your REAL EMAILS in their testing systems, or how long they retain anything. The "Third-Party Data Sharing" Loophole (Section: Sharing) Oooooh, like, uhhh, DATA BROKERS? And who needs a court order? We'll just call it "fraud and abuse". They want to pretend they're an upstart, 'To The Power of the People' or some bullshit. In reality this is yet another Big Tech 'Fuck you I got Mine' terms of service and privacy policy.

Frand just got smarter—now it teaches you how AI works, too. Dive into interactive courses on LLMs, agents, and more, all inside the app. No accounts, no cloud, just pure on-device learning. Try it today. Download Frand: your private AI companion, running right on your device. #Frand #OnDeviceAI #OfflineAI #PrivateAI #PrivacyFirst #Gemma #LocalLLM #AIAssistant #EdgeAI #Codrlabs

👋 Hello Fediverse! We're Snippbot — self-hosted, powerful AI Platform, you run on your own hardware & models. Snippbot is the host body for your AI model. Multi-agent chat, memory, and goal-seeking Loops that pursue a goal until it's met, then audit their own work. Bring your own models, keep your data. Glad to be here and learn from the #selfhosted / #FOSS-adjacent crowd 🙌 snippbot.com #introduction #AI #LocalLLM #selfhosted #sourceavailable

Have pushed 0.9.5-dev branch to codeberg of foxing ( https://codeberg.org/aenertia/foxing/src/branch/0.9.5-dev ) in preparation for release tagging. A LOT of features and a couple of bug-fixes now the packet/file processing engine has stabilized ; including Semantic Routing to Parsers for Metadata Extraction and in-path Binary analysis using local ORT/BERT models ; letting you get semantic search powers for free when you copy something with foxingd/fxcp #linux #filesystem #bert #vectordb #postgres #xfs #stratis #blake3 #localllm

I've been playing with #LMStudio for #localLLM with mediocre results. However #gemma4 really changed that. It's faster and is more capable then the other models I could try on my hardware. It has recent data and is able to use a fetch tool(among others) to get info on stuff it doesn't know! So I installed #ollama and now it runs even faster, to the point where delay(waiting) is not that noticeable. Since I am a lightweight user, I can see myself using it as mainly source.