Missing messages (unproven user reports)

I've had a protracted discussion with some of our users about this. Because some of them use the system (as contributors) quite a bit more heavily than I do, I have struggled to understand the nature of the issue; I could always find the missing messages. In fact, I still don't think that messages actually go missing, but the system is causing confusion. I'd appreciate feedback from others to see whether this is a common issue.

If a link (e.g. in an email notification) is to a message in a thread, that message is shown in context with its parent and (I believe) any children. Any siblings of the reply may be hidden; this looks like a deliberate decision to show relevant context, by having a sequence in which the parent is followed immediately by the reply to which the link was made. The "missing" siblings can be seen using "Load Next", but they will appear below the sibling that is already being displayed.

By contrast, if the link is to the thread, a parent message is followed by its children in date order, including any grandchildren, thus producing a different sequence. This difference in sequencing of the same siblings is definitely confusing our users. I think, at minimum, I'd like to see some indication that messages have been omitted. The problem is exacerbated because we're tending to get quite complex threads with multiple levels of indentation, which don't display well in any forum, nested or linear. I'm trying to encourage more of a "one topic per thread" approach.

Any forum has a central group of "key" users, and this idea that messages go missing is frustrating some of them, in spite of my step-wise explanations.

The occasional message has, I believe, genuinely gone "missing" because its parent was deleted by that parent's author, but that's not the point here.

And finally they've persuaded me. Links to messages (e.g. in notifications) tend to show the message in question immediately below its parent. In order to achieve this, it's clear that siblings, and their children, are sometimes not just moved into a different order, but actually omitted. I had to work through an example to convince myself of this.

I can see how it makes sense, in that a link to a message gives a view centred on that message, whereas a link to a thread gives a whole view. However, sometimes, several sibling messages are closely inter-related, so a message-centric listing gives an incomplete view. I'm not sure that there's a perfect answer, although it might be good to have a link to "Show whole thread" and/or some other indication that the current view of the conversation was incomplete.

Is there documentation of the rules used to decide which messages to include? That would be helpful, as an alternative to my having to reverse engineer them!

How this works currently:

If you load a reply directly (link in a reply notification), and the thread has less than the pagesize of replies, the entire thread is loaded.  If the thread has more than the pagesize of replies, the reply is loaded and all its parents, but not its sibilings.

A couple current options to alleviate this:

• Increase the pagesize setting for your thread widgets.  That will increase the likelihood of the full thread being loaded, but could have a performance impact.
• Turn off threading by setting the depth setting on the widget to 1.  You will no longer have the "threading" behavior, but the contextual loading will no longer be an issue.  Not a solution if you want to have threaded comments, but it is an option.

In the next release their are some improvements to forum thread loading.

• Always loading the first full page of replies.  This will show the more often show the reply in its full context, but if that reply is not part of the first page it would only be shown with its parents.
• Load Next/Load Previous indicators are now accurate, they will only be shown if there are actually more items to load.

Thanks, that's helpful. I've tried doubling pagesize, to mask the issue, at risk of performance, as you say. I've also increased indentation (depth). I'm reluctant to abandon threading; doing that seems to lead to an excessive amount of quoting, as users try to show to which message they are replying, and that's not good either. I realise that these are not straightforward choices.

There are two things that, I believe, increase the confusion, however:

It seems that message sequence is related to depth. So, when the configured depth is exceeded, it's not just a matter of messages no longer being indented; rather, level 4 messages are treated entirely as level 3. So, let's say that there are two replies at level 3, call them 3A and 3B, followed by a reply to 3A, call it 4A. If depth is 4, the sequence is 3A, 4A, 3B, with 4A indented, which makes sense. If depth is 3, then 4A is treated as a level-3 message, so the sequence changes to date order, i.e. 3A, 3B, 4A, which doesn't make as much sense, especially if the author of 4A didn't quote 3A. It also doesn't make sense because that's a different sort rule from the one applying in higher levels of the same display of the same thread. Is that right? Certainly, when I changed our depth setting, what was the last message in the thread shot to the top, because it was a reply to a very early message.

Although siblings are hidden in the way that you describe, we're seeing children of those siblings still appearing. This makes it really obvious that a message is "missing" (hidden), but leaves the reader hunting the thread for the missing message. Of course you can click on In Reply To to reload the thread, but it's actually quite hard to see why a thread listing designed to show context for one message would then show another one specifically out of its own context.

I hope that my observations are accurate. It's actually quite hard to work out what's going on, short of constructing a test thread of messages "This is a reply to 3A" and so on.

I don't suppose that the next version will have any enhancements around finding the latest messages in a thread? That's pretty tricky too. And again, I realise that building rules for marking those is non-trivial, but it's really fundamental, for building good conversations, that contributors can see quickly what's changed since their last visits.

I've had another think about this. I can't really see why you're going always to load the first page (the one containing the oldest messages, and those of least interest to those checking for new developments in the conversation). Of course, loading the last page would be difficult because there is quite possibly no such thing, and recent messages may be scattered around the thread.

A relatively simple solution would be a "Show all" button that displayed the full thread - could be slow if it's large and complex,  but the user takes that risk when asking to see all messages. Although you might still need to mark the message that was the entry point into the thread.

These missing messages remain a topic of significant feedback from our users and, as above, I am now convinced that the effect is real; if you follow a link to a message in a thread, and then expand the thread with Load Next/Previous, some messages from the thread may never be displayed. Since we're talking about a scenario in which a discussion is getting significant numbers of participants, many people following the link are probably already familiar with the thread. They get hopelessly confused when messages that they have already read are simply missing from the thread.

This functionality has been improved in the latest release of 10.2

Yes, this has been significantly improved in 10.2. For more information, please see the release notes or this more in-depth discussion of the improvements.

Thanks. I've commented there - we'll need to try it to find out whether it addresses this issue of messages that are missing from the thread irrespective of how many times Load Next/Previous are used. I've seen it happen but I can't define how to reproduce it, so it's very hard to say whether a change will fix it without seeing it used "in anger". I don't think we'll know for sure until it's been in live use for a few weeks or months.

Best regards