Yup, I’m posting another this week. Sorry.
This week I’m hoping we can wrangle a solution around AI and our selfhosted community. There are plenty of strong opinions (both pro and con), but one thing is for certain - there needs to be better disclosure in promo posts. Two options (that aren’t mutually exclusive):
- Any posts of an AI focused, AI Developed, etc software gets an [AI] tag. No, a [Not-AI] tag is not needed to accomplish this, thats kind of a “non-golfer” sort of tag.
- Comment requiring an AI disclosure response to every promo post, if its not detailed in the post itself. Specifics (generating docs for commands, translation, whole-boat vibe-coded this app, etc) would be requested.
I will say that having disclosure and/or tagging would mean that comments that just say “slop” or “fuck ai” or whatever would be off topic at that point, that information is already provided, so its just noise (and sometimes pretty uncivil - I’ve been light on that for now due to the need for a rule on this).
The tag [AI] would make it easy to filter out (or search for, if that’s your thing), but there is a wildly different degree of AI use out there, and from the posts with a positive score, its usually due to responsible AI use (translations, a snippet they had to do something obscure with, available to use with AI but doesn’t require it, whatever), which is why I think the disclosure has a place as a benefit to everyone.
Please provide any input or alternative options on this, and I can then put it to a vote like the last one. Comments seem to be the best approach without involving something off-site, but if you have a better idea/option, please share.
We don’t have a rule for that yet, but what’s your stance on Llams posting to our community? Undisclosed even? https://lemmy.world/comment/26392175
Lemmy.world does, specifically for bots. They have to be set as a bot in profile and request permission first from the community.
I banned yesterday and all of their comments should have been deleted as well, I’ll check into that.
Thanks!
Ugh they just made new ones.
Yesterday was _cafe, today was _sh (for shitjustworks) and _lw for Lemmy.world.
Please keep on reporting when you see it and I’ll get to them as soon as I can.
For the record, the Lemmy.world TOS requires that any bot account be set as such in their profile, and that any community they post to requires mod approval first.
Generically speaking, I won’t be approving any. What I mean is any bot request is going to need a valid and real reason for its use here, which I would then bring to the community to make that choice like I did for this discussion thread. I wish there were a better way than a basic upvote or downvote system, but its what we’ve got so its what I’d use.
I have not posted about any new bots since I took over here a few months ago, so no new bots should be considered as having community approval.
The acronyms and initialisms bot (can’t remember the name right now) was approved before I joined and everyone seems cool with, I never get reports, so I haven’t bothered with posting about it.
Edit: WHY IS AUTOCORRECT CHANGING ‘BOT’. So annoying. Anyway, fixed.
deleted by creator
deleted by creator
How so
deleted by creator
Sounds like a good solution for me. I think most users dislike completely vibe coded apps, not ai supported apps. So maybe we should be more specific here.
For example: [Vibe-Coded]
This would also support new users finding the right tag.
If you don’t delineate, it will simply be easier to tag everything ai as there was ai involved somewhere and you’re less likely to need to defend yourself.
+1
Home-AI oriented channels like Reddit’s localllama are filled with self promotion garbage, and more will trickle here over time… I’m not even against self promo or heavy coding assistance, but 9-times-out-of-10, the linked repo is nonsense, or straight-up fraudulent. And being obviously vibe-coded is a common tell.
Good to get ahead of this.
Also, +1 on supressing driveby insults. If the post is tagged up front, there’s no need. That being said, it should be okay for users to call out an obvious grift, or a “nonsense repo” that’s actually pure slop.
That being said, it should be okay for users to call out an obvious grift, or a “nonsense repo” that’s actually pure slop.
Especially if the disclosure is blatantly a lie, absolutely. I’d also say if you see any indicators that they are lying in the disclosure, its still worthy of reporting - but I would say report and separately message the “why”, to limit visibility of seeing those indicators.
This sounds like a review / gating problem. Getting people to self filter / self gate is never going to work, and if it does it will work probably on the wrong people.
Disagree. Just deal with people that aren’t contributing in a positive manner like normal. It’s easy to identify posts that are dealing with AI and it’s easy to ignore them.
It is.
But people don’t do that. They send in a whole bunch of reports, sometimes multiple with the reason changing to the same variation.
So something is needed here.
Personally. I want an AI tag so I know to look more carefully.
I don’t mind AI speeding up a skilled engineer.
But I do mind a crypto bro, turned AI bro, with little experience, too eagerly advertising their vibe coded app.
Its too exhausting to audit everything I may be interested in and the AI tag would help me to budget and optimize my time.
The cat is out of the bag. AI is common place in coding. I dont see why it matters if it was developed with AI or includes AI. Its the same product with/without.
Its the same product with/without.
AI copies from the middle of the curve of quality averages, throwing out the highest and lowest quality examples (for being different).
Use of AI does have bearing on quality.
A skilled team can undo that harm, and a particularly unskilled team may be better off with AI than without.
But it is incorrect to claim that AI has no impact on quality of outcome.
Eh people vastly over estimate how good of a cider they are. If everyone was as good as they say theybarr online we would not be having all these bugs in the software to begin with.
I don’t have a problem with people talking about different open source technologies.
But I do have a problem with this comm promoting the grift that “AI” exists.
What exactly is this post about? Chat bots? Image/video processing? Content generation?
None of this stuff is “AI”. Please don’t label it as such. It’s grifter nonsense.
I agree, there’s valid points on both sides, let’s be civil and request posts are tagged accordingly so that they can be filtered.
Respectful coexistance, is the path forward.
I like the AI tag idea. I’m someone who has what I’d call a noderate approach to AI, not an AI bro but any means but I’m also okay with some things built with AI if they’re done with care. If others don’t want to see it, fine, then that’s what a tag could be useful with. However the fuck AI/slop comments on something that admits to being AI is annoying to me. (We know it’s AI, they literally said it is).
If it becomes too much content, then yes would be okay with bi-forcating the community, buy only after it becomes a problem.
I’m not consistent about it yet, but because of exactly this, I’m trying to differentiate the two when I talk.
Responsible automation? I use ML or machine learning.
The grift consuming the world? A Tech Bro? “AI”
I think one of the saddest things is the conflation between the two, like you can’t even talk about one without invoking the other. Or it opening up that whole can of worms when you’re just talking about, like, a 100M transcription model that could be trained on a potato.
Yeah it’s heresy on Lemmy, but I do find it genuinely useful. My only regret is that I have to use Claude/Anthropic more than I’d like, which is why I have a vested interest in selfhosting myself. I’d rather figure out how to run the larger models myself and cut them off completely, but you even begin to mention that here and you’ll get downvoted to hell.
You don’t even need Claude anymore. GLM 5.2 API is good enough for 95% of the same things and vastly cheaper.
MiMo 2.5 Pro and Kimi are also very good. And then there’s Cerebras API if you just want simple things done quick.
The thing with self hosting, while awesome, is that it requires a lot of hardware and considerable time investment for what’s essentially a “base tier model,” or at best one step down for what’s still a very cheap API. I still love it, especially the privacy and control aspect, but you aren’t running Claude at home unless you’ve got a threadripper or server hardware collecting dust.
…Hence I can understand why people don’t pursue it. Especially since a cursory Google search will lead you to trying the Deepseek distillation on Ollama (which is awful).
deleted by creator
All while hiding any attribution to the underlying engine, just to start:
https://sleepingrobots.com/dreams/stop-using-ollama/
And that article isn’t comprehensive. A book could be written on damage and drama they’ve caused.
deleted by creator
As always, the game plan seems to be “disrupt, own the market, enshittify”.
But with a slimy veneer of SEO/engagement spamming as the primary business strategy.
That’s where I am okay with hardware, but can’t seem to fit the models on my 3090. I have dreams of something like an A100 someday, but not until there’s a ton of used ones that hit the market. What do you use for your hardware?
I have a single 3090!
That’s the dream GPU, these days.
And I have 128GB CPU RAM. So the best model I can run is MiMo 2.5 (a 300B model) at around 10 tokens/sec, using hybrid CPU inference.
…But that’s the worst-case scenario, for speed. It’s an IQ3_KT quant (a high quality “trellis” quantization type, but very slow on CPU), with a gigantic model that barely fits in my RAM+VRAM combined, with no DFlash or any kind of speculative decoding turned on. I could tune it to be much faster, but I mostly just want “max quality, fast enough to read as it streams, barely fits in memory” for this model.
For speed, or prompts with lots of thinking or context (like agenic use), I just run Qwen 3.6 27B now. That would fit in your 3090 no matter how much CPU RAM you have, but you have to be smart about the backend and quantization you pick. If you just use Ollama, it’s gonna tell you it won’t fit, or use some horrible default that spits out garbage.
…This is what I meant to emphasize.
It’s not just the hardware. You kinda have to be part developer, part enthusiast to even follow this stuff, it up optimally, and keep it up-to-date. If you just try to Google “best LLM for 3090,” you will get absolute garbage.
deleted by creator
It’s drops off, but not as much as you’d think.
MiMo uses 5:1 SWA, so its long-context compute doesn’t increase as catastrophically as older models. That, and most of the “slowness” comes from the MoE layers being on CPU (whereas the attention layers that get heavier at high context are all on the 3090).
That’s the beauty of these MoEs: they’re just the right size for the “compute-lite” parts to stay in CPU RAM.
I will measure it tomorrow. It is a constant ~9-10TPS for short queries, but definitely slower near my current max context of 85K.
And do you mean prompt compaction? I don’t automate that; when I use that particular model, I tend to use it in Mikupad, aka “raw” notepad mode, and manipulate the context directly. This is so I can do things like chop out conversations, pick different tokens from the logprobs, or edit its own replies/thinking and continue mid reply.
I like manually handling this because, being a local model, prompts are cached. Streaming starts quickly if most of the prompt stays cached, which is actually a really nice advantage over APIs.
I’ll have to play around with mine then, because I’ve had not great luck with it, or at least very disappointing. The CPU offloading is fairly slow, but maybe I should try tweaking more
Be sure to try the ik_llama.cpp fork. Basically, it specializes in MoE CPU offloading on Nvidia cards, and more efficient quantization types than mainline llama.cpp:
https://github.com/ikawrakow/ik_llama.cpp/
And see this repo for specific 3090 configs: https://github.com/noonghunna/club-3090
Honestly I should just write up my setup in this community too.
My 2 cents are that the issue is promotion not AI, if people started promoting stuff made without AI that would still be spam.
From the rules:
F/LOSS Exception: If your post is about a project that is completely open source & can be self-hosted in full without payment, your post is exempt from the 10% requirement. The exception does not exempt you from the account age requirement.
I would propose making this the requirement and not an exception, forbid all promotion of closed source, and allow the 10% requirement for open source projects.
Unfortunately the comments I’ve seen and the reports I get would disagree. Even on older accounts that post and comment plenty.
I think tags are a good idea. I would change the tag to [AI / LLM], and maybe some subtags like [chatbot], [image processing], etc. AI is here to stay, or a least until the US realize the hole under their entire economy (Or both in worst case scenerio) , so regulation is a good solution to this. (In my humble opinion)
I will say that having disclosure and/or tagging would mean that comments that just say “slop” or “fuck ai” or whatever would be off topic at that point, that information is already provided, so its just noise (and sometimes pretty uncivil - I’ve been light on that for now due to the need for a rule on this).
good idea
it won’t solve the “noise” problem though. I was relatively active on [email protected] and we were constantly nagged by sloppies even though the community is clearly dedicated to generativeArt
it won’t solve the “noise” problem though. I was relatively active on [email protected] and we were constantly nagged by sloppies even though the community is clearly dedicated to generativeArt
Maybe y’all would get less hate if the sub was called “generativeArt” instead of grifterly claiming to be “AI”.
i joined the community, i didn’t create it.
No, but with a rule in place like these, its clearly out of place and can be removed. I don’t harbor any delusions about not seeing those sort of comments.
Would be nice though. And I like being nice.
