Backfilling Conversations: Two Major Approaches
-
Hi robz@toot.robzazueta.com! This could be related to some better support for non-
Notetypes introduced by Mastodon in later versions. Your instance is running v4.1.18 which is 11 months behind the latest version.That isn't necessarily cause for concern, but I think that might be why you're seeing the HTML tags?
-
@julian Ah... that actually may make more sense - thanks.
I'm working on my own AP implementation and hadn't yet run into this issue, so assumed.
Time to upgrade!
-
neither approach conflicts with the other
I don't fully agree with this statement, because these "threading paradigms" suggest two different solutions to the problem of moderation. If the OP is the single source of truth, they can moderate the entire conversation (represented by
contextcollection: Streams). If not, then each reply is independent and authors moderate only the direct replies (represented byrepliescollections: GoToSocial).In theory two solutions can be combined, but at the cost of significantly increased complexity.
-
We're using both here. There's an icon to let you know that you're looking at an actual conversation -- vs. a collection of microblog posts that once had a common ancestor.
The differences in signal/noise ratios between the two styles are quite dramatic. Neither is better or worse than the other. They are different. And they can both co-exist.
Also, conversation containers has the ability to "reply to all" as well as "reply to sender". Microblogs don't have this concept, and instead "reply to all" means "send to all your followers, instead of a reply directed to the actual conversation audience.Additionally, consumers are also able to query the context owner for an index without needing to crawl the entire reply tree.
While this is certainly true, when conversation containers are working correctly, you never need to backfill a conversation. It is all delivered to you. -
N.B. I am not certain whether the service would crawl up the inReplyTo chain first, before expanding downwards, or whether context is set in intermediate and leaf nodes that point to the root-level object.
Current impl starts at the expanded post and goes down - one can start a crawl at any point in a tree. If one starts at a lower point in the tree and then triggers a crawl higher up in the tree, lower part only gets crawled once within a configurable cooldown period to avoid double crawling.
-
hen conversation containers are working correctly, you never need to backfill a conversation. It is all delivered to you.
I think there is one case where backfill is necessary: public conversations that are not discovered through following (e.g. by retrieving an object by its ID).
-
silverpill@mitra.social mikedev@fediversity.site correct. Backfill is important even when you have good synchronization systems in place.
One example I use is Lemmy's use of 1b12. It is exceedingly good at keeping subscribers in sync, but if you discover a new node or leaf, then backfill is required to get you the conversation up to that point.
-
True, but fetch one collection and you've got it all. Might be paged, and with Mastodon that means another fetch for every ten activities (seriously?), but those are just implementation details. -
silverpill@mitra.social said:
> If the OP is the single source of truth, they can moderate the entire conversation (represented by context collection: Streams). If not, then each reply is independent and authors moderate only the direct replies (represented by replies collections: GoToSocial).That is a good point. The approaches are broadly compatible when top-down moderation by the context owner is not assumed.
In a moderated scenario, crawling the reply tree would not be useful unless paired with some sort of "is member of" validation with the context owner... at which point the served collection would be more performant.
It could be useful for discovery by the context owner itself though.
-
I think a couple of folks have mentioned trying to consolidate both of these approaches into one. I once used something that resembled 1b12 (long before there was a "threadiverse"), but as I recall it didn't really work well with private groups and aspects/circles - where you're often dealing with third-party permissions. You can only relay public activities to third parties via an Announce, and so conversations with restricted audiences don't work out very well for viewers on Mastodon. The third party does not have permission to access the activity from its author, only from the conversation owner. Once you've run into this issue, you are likely to more fully understand the advantages and disadvantages of these two approaches. Container operations are pure relays and work correctly with third-party access control, assuming you're using signed objects (which everybody should be using, but that's a hill to die on another day). -
but as I recall it didn't really work well with private groups and aspects/circles
Last time I heard about 1b12 private groups, the proposed solution was to use a "collection inclusion endpoint" to verify that actor is a member of a group
-
silverpill@mitra.social do you still need to if you're not using a shared inbox?
-
@julian It should be noted that a platform receiving a moderated conversation thread does not have to honor it for its own local users. Whether this is desired or not is another discussion.
In this case, the owner of the thread (either the forum or the person who started the thread) tells you what comments are part of the thread. Some comments may be removed due to moderator actions or user-initiated blocks.
But as a remote platform importing the thread, you may be aware of other replies that are part of the reply tree, but not in the official moderated version of the conversation according to the thread owner.
As a remote platform, you have an option. You can honor the thread owner's official version of the thread and only display the moderated version, or you can modify it. You may remove replies from actors blocked on your server, for example. But you could also add comments from the reply tree that are not part of the moderated version of the conversation. -
-
Just thought of something interesting. In the case of moderated threads, it may be useful to tell other platforms that you know about a particular comment, but have removed it on purpose from the official moderated version of the thread. Because there is a difference between "I didn't know about that reply due to a technical issue" and "this content was removed by a moderator." -
That'd be accomplished with a
Removeactivity, most likely.For those expressing the context collection as a set of objects, then removal from the set should suffice. There are probably better signals to send.
-
@julian Wouldn't a remove would remove it from everywhere, including the server of the person who posted it. That may be desired, but also could lead to confusion, since on many platforms like Mastodon, they can't see threads and don't realize their comment can be deleted everywhere (including their own copy). -
scott@loves.tech not necessarily, a remove merely represents that it has been removed from a collection. A Delete would instruct the recipient servers to purge the object, and that can't be done unless the actor is same-origin.
-
@julian Okay, that makes sense. -
julian@community.nodebb.org said in Backfilling Conversations: Two Major Approaches:
> A number of implementors follow this approach to backfill, including NodeBB, Discourse, WordPress, Frequency, Mitra, and Streams. Additional implementors like Lemmy and Piefed have expressed interest.Is this implemented currently? One weakness I have noticed in NodeBB's current federation is that posts which are in reply to a topic (e.g. a Lemmy comment) show up as individual threads until (or if) the root post of that topic shows up in the local NodeBB. It's a bit confusing of a UX, I think. Because you think:
- (Before root post) Why is this a post that seems to be just a random comment?
- (After root post) Why did that other thread disappear and why am I seeing the same comment again?
