In the last weeks Lemmy has seen a lot of growth, with thousands of new users. To welcome them we are holding this AMA to answer questions from the community. You can ask about the beginnings of Lemmy, how we see the future of Lemmy, our long-term goals, what makes Lemmy different from Reddit, about internet and social media in general, as well as personal questions.
We’d also like to hear your overall feedback on Lemmy: What are its greatest strengths and weaknesses? How would you improve it? What’s something you wish it had? What can our community do to ensure that we keep pulling users away from US tech companies, and into the fediverse?
Lemmy and Reddit may look similar at first glance, but there is a major difference. While Reddit is a corporation with thousands of employees and billionaire investors, Lemmy is nothing but an open source project run by volunteers. It was started in 2019 by @dessalines and @nutomic, turning into a fulltime job since 2020. For our income we are dependent on your donations, so please contribute if you can. We’d like to be able to add more full-time contributors to our co-op.
We will start answering questions from tomorrow (Wednesday). Besides @dessalines and @nutomic, other Lemmy contributors may also chime in to answer questions:
Here are our previous AMAs for those interested.
What have been the biggest challenges with the project over the years, both in terms of technical and non technical aspects. I’d be interesting to hear a bit of retrospective on how has the stack’s been working out, and what surprises you might’ve run into in terms of scaling and federation. What recommendations you’d make based on that and what you would’ve done differently knowing what you know now.
The stack is great, I wouldnt want to change anything. Postgres is very mature and performant, with a high focus on correctness. It can sometimes be difficult to optimize queries, but there are wizards like @[email protected] who know how to do that. Anyway there is no better alternative that I know of. Rust is also great, just like Postgres it is very performant and has a focus on correctness. Unlike most programming languages it is almost impossible to get any runtime crashes, which is very valuable for a webservice.
The high performance means that less hardware is required to host a given number of users, compared to something like NodeJS or PHP. For example when kbin.social was popular, I remember it had to run on multiple beefy servers. Meanwhile lemmy.ml is still running on a single dedicated server, with much more active users. Or Mastodon having to handle incoming federation activities in background tasks which makes the code more complicated, while Lemmy can process them directly in the HTTP handler.
Nevertheless, scaling for more users always has its surprises. I remember very early in development, Lemmy wasnt able to handle more than a dozen requests per second. Turns out we only used a single database connection instead of a connection pool, so each db query was running after that last one was finished, which of course is very slow. It seems obvious in retrospect, but you never notice this problem until there are a dozen or so users active at the same time.
With the Reddit migration two years ago a lot of performance problems came up, as active users on Lemmy suddenly grew around 70 times. You can see some of that in the 0.18.x release announcements. One part of the solution was to add missing database indexes. Another was to remove websocket support, which was keeping a connection open for each user. That works fine with 100 users, but completely breaks down with 1000 or more.
After all there is nothing I would do different really. It would have been good to know about these scaling problems earlier, but thats impossible. In fact for my project Ibis (federated wiki) Im using the exact same architecture as Lemmy.
It’s great to hear things mostly worked out. Stuff like scaling bottlenecks is definitely tricky to catch until you have serious loads on the system, but sounds like the fixes very mostly trivial validating overall design. It also looks like you managed to get a way with a fairly simple stack by leveraging Postgres and Rust. I’ve had really good experience with using pg myself, and really don’t see a point in using anything else now. You can use it both as a relation db and a document store, so it’s extremely flexible on top of being highly performant. Keeping the stack simple tends to be underappreciated, and projects often just keep adding moving pieces which end up adding a lot of overhead and complexity in the end.
2nding @nutomic, that I’m really happy with the stack.
The one that seems really magical to me, is diesel. With it we get a compile-time-checked database, that’s tightly integrated to the rust objects / code.
Every single join, select, insert, etc is checked before lemmy is even run, and it eliminates a whole category of errors resulting from mismaps.
Its made adding columns, and changing our data structures so much less error-prone than when I lived in the java-world.
Whenever we find that we’d want to do things differently, we usually do a refactor ASAP so as not to keep rolling spaghetti code. We’ve had to do this many times for the federation and DB code, and even have 2 major refactors that also add features ongoing. But luckily we’ve been able to stay in the rust eco-system for that.
As for UI, leptos didn’t exist when I built lemmy-ui, so I went with a fast react-like alternative, inferno. Its showing its age now, so @sleepless1917 is working on lemmy-ui-leptos, which hopefully will supercede lemmy-ui.
I briefly worked with Hasura which does this sort of magic to produce GraphQL API on top of Postgres. Incidentally, also written in Rust. I do like Leptos approach, it sounds similar to HTMX approach where you just treat the DOM as a dumb terminal.
Communities should be more unified across servers, especially for niche ones. I want to see an active Metroid community, I don’t give a crap what instance is hosting it (or if it’s a mostly-opaque medley of instances) so long as I’m federated with it. This is probably the biggest UX misunderstanding new users have.
This should be among the first priorities. It would really help kick things off. Not only niche communities, but bigger ones as well. They represent topics of interest. I think I’ve seen a thing like macro community in one of the clients?! Could that be it?
Eternity Android client allows grouping communities into a multi community, but it only helps on getting consolidated feed, not necessarily reaching the same people
the Summit app does it too
Being able to view “all” communities with the same name across all instances would be so nice.
this post provides a good overview of this issue and possible solution. https://popcar.bearblog.dev/lemmy-needs-to-fix-its-community-separation-problem/
Proposed solution 3: Communities following communities
Really, really like that idea!
Having distinct communities is a feature, not a bug. If two cities set up their own lemmy instances, say
lemmy.sao_luis.br, andlemmy.lagos.ng, they can each have anewscommunity, without them overlapping.Do a search for metroid, and subscribe to whichever ones you like.
It would still be a huge benefit, especially for more niche topics, if we had something like a federation-wide comm like
/f/niche_hobbythat you could subscribe to instead of 20 different/c/niche_hobbycommunities.Maybe comms could opt in/out of behavior to avoid the issue you described.
This would also benefit smaller instances because few people will subscribe to their comms because they are too inactive, making it so their content never gets traction.
My biggest complaint with Lemmy is that it is too hard to group & categorize content. Sometimes I want politics, sometimes I want nerd shit, but my only three options are subscribed, local, and all, which doesn’t have any categorization unless you are on an active, niche server.
Multireddits are pretty much the only thing I miss from reddit.
Piefed has exactly that: https://lemmy.dbzer0.com/post/38733273?scrollToComments=true
Interesting, is this all manually curated like multireddits? Would also be nice to have automatic ones (with include/exclude overrides)
The problem with it just being Piefed is that Lemmy clients probably won’t bother to support it unless it becomes standard.
Is this a frontend specific thing or does it also require the Piefed backend on your instance too? If it is just frontend, I would definitely use it for desktop browsing.
Dope seeing implementation diversity resulting in experimentation and innovation. Would love to see this adopted in other Lemmy implementations too
Interesting, is this all manually curated like multireddits? Would also be nice to have automatic ones (with include/exclude overrides)
They have both
- user defined feeds, public or private
- admin defined “topics”
It’s a whole different software, backend and frontend
Consolidation isn’t always a good thing, communities on different instances will have different styles and trends, and that’s a good thing. The benefit of federated social media is just as much in local instances as it is in federation, unique niches are going to have unique comments even if the post is the exact same.
I think the benefit of federation is that nobody controls the whole ecosystem. The downside of federation is splintering.
It does not have to be something mandatory…
I mean, there could be some form of “metacommunities”, something like being able to group multiple communities together in a “view” that shows them to you visually as if they were a single community despite being separated. Bonus points if everyone can make their own custom groupings (but others can subscribe to them… so there can be some community-managed groupings).
In theory you could have multiple “metacommunities” for the same topic still… but at least they could be sharing the same posts if they share communities. I feel grouping like this would be helpful because small communities feel even smaller when they are split.
I think reddit has something similar to that, multireddits or something I think they are called.
Piefed has feeds that achieve exactly that: https://lemmy.dbzer0.com/post/38733273?scrollToComments=true
What exactly is it you’re asking for, though? A change in user behaviour towards consolidation? Some new feature of the platform similar to multi-reddits? How exactly do you suggest that should work?
Not a change in user behaviour. How about: communities on different instances with the same name appear as one community essentially. As in, all instances’ version of that community appear in your feed if subscribed, and when viewing posts in a community, all instances versions of that community are visible.
Perhaps the user can restrict to just one instance’s community or just the local instance’s community with a button (like local/all), if that’s their preference.
Good luck with [email protected] and [email protected]
I think the greatest strength is that it is so compatible with other Threadyverse software like PieFed and Mbin. This brings a lot of freedom to the users.
Yes I’m very excited about the growth of other fediverse software, and a lot of the cool new features they’re adding. Its a great eco-system where we can experiment, be creative, and learn from each other.
Yes this is a major benefit of an open network. Lemmy is a very large project already, so it takes a lot of effort to implement new features, because they have to meet high standards for quality and performance and also work together with all the existing features. A project like Piefed is much smaller and can implement new features more quickly. This allows for more experimentation, and successful features can later be added to Lemmy.
Also users who are not happy with Lemmy for any reason can switch to a different platform while still interacting with those on Lemmy. So if Piefed and Mbin grow that is also a benefit for Lemmy.
+1 on registration experience being the #1 issue.
Would also be cool if we could stop 404/500ing deleted posts and instead display some indication it has been deleted. See https://en.m.wikipedia.org/wiki/Principle_of_least_astonishment.
Thanks for Lemmy! 💙
We have gone back and forth a few times on how deleted content is returned by the API, its very tricky to get right.
I’ve read the Github issues. While I could agree there’s some nuance to it, black-holing the entire thing as though it never existed is a bummer.
When will users be able to frictionlessly migrate between instances without losing their posts, their comments, their history, their relationships, their reputation etc? (Without requiring the consent of the exiting instance owner, or that this server still even exists, as they sometimes don’t)
From my perspective we need better Mod and Admin tools. Forum software has a lot of them but Lemmy is lacking in this department.
The key important one is being able to move posts to different communities. You’ll often get reports of posts not being appropriate for a community but there is no way to actually move it.
Which tools specifically? I ask because this is a common complaint, but 99% of the time its something we already have, that most ppl are unaware of.
The key important one is being able to move posts to different communities.
Lemmy like all federated services, can’t rewrite history, but you can already cross-post (although it would be the mod cross-posting, as we don’t let mods alter user data except to remove it). It would just take someone adding that as an issue to lemmy-ui and working on it.
Which tools specifically? I ask because this is a common complaint, but 99% of the time its something we already have, that most ppl are unaware of.
- mod mail, so that users can reach out to the whole mod team at once, and the team can come back to them
- a more structured mod queue, allowing to filter by community. The Reddit one on old.reddit was good to help keep an overview on the mod actions to take
more structured mod queue, allowing to filter by community
The upcoming combined modlog has this, as well as other more detailed filters.
You can read through these issues related to modmail, but the short version is that it’s way out of scope for us, and not something we have time to do. Replicating private group chats is better done by other services like matrix, or using a shared email inbox.
It would be nice to have way for admins and mods to make notes on people. E.g. If you are giving someone a warning, you currently need to use an external tool to log the warning.
Which tools specifically?
Standard Web forum tools include:
- Editing posts - the main issue is misleading titles
- Moving posts to different communities
- Merging posts
- Splitting comments into separate posts
- IP check
This post makes some good points about reports federating (being worked on, I believe) but also about the lack of what we’ll call a “moderation panel” where you can access tools for the community, like seeing a list of banned users and being able to add to it there or unban someone.
There are other “nice to have” tools like post approval
I am curious to see what moderation tools PieFed, has and NodeBB now they are federated, but the documentation is skimpy on that front.
Editing posts - the main issue is misleading titles
Moving posts to different communities
You can read over the discussion here, but we will never allow mods or admins to act as / impersonate users, or edit their content.
We also can’t rewrite history in the fediverse (unlike a forum) so “moving” a post would also entail deleting and recreating content other people made.
Splitting comments into separate posts
Merging posts
These ones sound really strange, but its similar, I don’t want mods to be able to rewrite user history or move it.
IP check
We don’t store IPs so that’d be impossible.
You can read over the discussion here, but we will never allow mods or admins to act as / impersonate users, or edit their content.
I really don’t get this. Why is editing user content with slur_filter or modifying URLs accepted but allowing mods/admins to change the NSFW toggle isn’t? It also ignores that savvy-enough admins can edit user content with SQL queries.
I think slur filters, tracking param removals, and local link rewriting are acceptable, because (with the exception of the slur filter) they’re non-moderation actions, and also applied uniformly regardless of who made them.
It also ignores that savvy-enough admins can edit user content with SQL queries.
That’s unavoidable of course, anyone with DB access ultimately can edit things. But if people catch on, I doubt your server would gain many users or last that long. Most importantly, we shouldn’t allow that to happen via the API.
You’re free to start a “Should mods be able to edit user’s data?” discussion, but I doubt it would get much support, especially from reddit allowing this and it souring everyone to it.
Most importantly, we shouldn’t allow that to happen via the API.
My view is that not adding this to the API will only encourage admins who want this to do it through less transparent means, like injecting fake activities into the
sent_activitytable. Most admins are reasonable people, and have good relations with their users, so if admins explained themselves then I think most users would be pretty accepting.You’re free to start a “Should mods be able to edit user’s data?” discussion, but I doubt it would get much support, especially from reddit allowing this and it souring everyone to it.
I mean there’s been like 3 or 4 GitHub issues opened about this, so there’s clearly some demand for it. Should I make a post in [email protected]? So users not on GitHub can chime in.
Sure, and be sure to link that closed github issue.
Some Lemmy clients offer the option to auto-hide posts and comments which contain certain keywords of the choice of the user. Are there any plans to implement this feature into the stock Lemmy experience?
I know it is possible to do some hacky stuff with UblockOrigin to do the same, but that is not something most know about and are willing to do.
This is currently work in progress.
- From a code architecture perspective, how close is Lemmy/ActivityPub to reaching its maximum capacity for posts/comments per second? Are there any ways to 10x the load ActivityPub can handle?
- With Nicole in everyone’s DMs, what does the future of spam filtering look like on Lemmy?
- There is no specific maximum capacity, in theory it can scale indefinitely with horizontal scaling. Also see my reply here regarding scaling.
- 0.19.10 already includes a fix to remove private messages when a user gets banned which should help a lot. There is an issue about disabling private messages by default, but Im not sure if that will be necessary. Also 1.0 will include a plugin system, so other devs and instance admins can write their own checks. That way spam waves can be fought in a more flexible way, without having to get a change merged into Lemmy and then waiting for a new release.
Hi, I think that Lemmy is great thank you for your hard work
I actually think that given the ads and other distorsions, and thanks to federation, Lemmy is overall actually better than reddit!
Some features I miss are:
- tags
- direct messages outside Lemmy (even if not encrypted)
- better rendering of posts on mastodon (something beyond the title only). Not sure what side is responsible for this, tho!
Keep up the good work guys!
- Tags are work in progress
- Not exactly sure what you mean by “direct messages outside Lemmy”, but in version 1.0 they will be compatible with Mastodon and other platforms
- Its a known problem with Mastodon because it only renders
Noteobjects properly, which are meant for short texts less than a paragraph. Lemmy usesPagewhich is meant for longer text. Some platforms like Wordpress (iirc) have an option to federate even long posts asNoteso that it gets rendered fully in Mastodon, but that seems like a bad idea to me. In the end its up to Mastodon how to render different types of federated content on their frontend, so it needs to be fixed by them. Here is an entire discussion about this by developers of different Fediverse platforms (including a Mastodon dev).
Thank you, great to hear all my points are being addressed! The thing about post rendering, well, I just hope a common solution is found 😊
I like Lemmy a lot, but when you share a URL it’s just an ID number. Compare that to Reddit, where you can get a lot more information on what you’re about to look at just from the URL alone.
https://lemmy.ml/post/27659153 vs
https://old.reddit.com/r/AsahiLinux/comments/1jg3vlk/progress_report_linux_614/Are there any plans to make Lemmy URLs more meaningful?
There’s some discussion of this here. It’s not something I care too much about, but anyone is free to add more detailed URLs to either lemmy-ui or other front-ends.
Its been years but no one has found it important enough to them to work on.
Another issue is that post links are instance-specific, since the post ID isn’t the same across instances.
ex: https://lemmy.ml/post/27659153 is https://lemmy.ca/post/41237641 on Lemmy.ca
There are external tools like https://lemmyverse.link/ and some browser addons to alleviate those issues, but it’d be nice if this could be addressed at the source if doable.
And I dream of a
lemmy:\\protocol handler one day.Potentially, using some sort of predictable hashing to get the same id across instances might also help in the detection of duplicate links so that they can be aggregated in a single place (sort of what was suggested at point 2 here).
I fear this could be too much of a breaking change though.
We won’t add UUIDs or any “universal” sort of identifier, but universal links are still possible without them.
It will be a feature for Lemmy 1.0: https://feddit.org/post/5390705
Until then https://lemmyverse.link/feddit.org/post/5390705 ;)
Hello,
Thank you for organizing this AMA!
Starting with a quite expected question: when do you think you’ll be able to release Lemmy 1.0?
Its hard to say because these things always take longer than expected. Now we are finally getting to the point where all the breaking database and api changes are almost finished. After that it will take some months to update lemmy-ui for all the backend changes and new features, and the same for all other apps. Then a testing period to fix all the problems that come up. So maybe around autumn for the final release, although lemmy.ml and some other instances may upgrade some months before already.
With the rate ppl are adding issues (and we’re finding more), is sometimes feels like it keeps getting farther away than nearer, but we’ll get there in some months.
Are you disappointed with the way things are growing with people trying to marginalise the likes of ML and Grad?
It seems some people simply need some target to hate on. Hopefully they will learn to accept different opinions when they arent being manipulated by for-profit social media anymore.
Communities that go against hegemonic capitalist/imperialist discourse will always get marginalised. Not being able to take down those communities easily like on Reddit is a huge win by itself for Lemmy. The software offers a valuable savehaven for e.g ex r/chapotraphouse, r/genzedong etc.
Yep, the fact that Communists can build their own platform and networks free from any outside censorship on corporatized platforms is itself the strategy for building leftist spaces. The goal isn’t hurt by more non-Communists being on the overall Lemmy platform because these non-Communists can’t actually do much to shut the Communists out.
That’s a good thing, as a Communist I’m happy we have spaces.
I get a chuckle out of the “Tankie Triad” talking point some people keep using. It sounds like a villain organization from a Saturday morning cartoon.
The anti-communist witch-hunters are extremely peeved that they can’t remove our communities like they can on reddit. Overall it doesn’t bother me because I don’t work for them, and they can always go back to reddit where their views are already dominant.
Anyone trying to make the world a better place, will always be hated and hunted by some people; it’s a fact of life, and the sooner we accept it, the better.
Dunno if I’m too late, but here goes. My question is about federation between instances.
On PeerTube an instance follows another instance and then federates every channel and videos available.
On Lemmy, the user can follow a specific community and then that community will federate with the users instance.
How about being able to, either as the instance itself or a user, to follow an entire instance and have it federate everything?
An example. I have a user on Lemmy.wtf, but I am also very interested in the communities at Feddit.dk. I never know when new communities have been created on Feddit.dk, unless I go directly to Feddit.dk and look. If I could subscribe my instance to Feddit.dk, then all future communities would be visible to me automatically.
If something like that isn’t possible, then what about being able to browse other instance’s communities from my own instance?
No questions right now. Just wanted to say thank you for your hard work.
I know y’all catch a lot of shit and get hammered with requests/demands, so I wanted to let you know that your work is greatly appreciated.
Thanks for dedicating your time and energy to making a non-corporate, federated social environment possible.
Being on Lemmy has been a breath of fresh air.
Thx! Really appreciate it, and I’m glad someone thinks its worthwhile work we’re doing.
Old user, haven’t been active recently. Where’d all this growth come from?? Another reddit refugee situation?
[email protected] started to ban people based on upvotes
[email protected] movement has motivated people to look around for European alternatives to Reddit
Blaze means the website Reddit, not the community they linked
Thought you meant pre-2023 by old user
