Open Bug 141167 Opened 22 years ago Updated 2 years ago

New thread on subject change (threading by subject)

Categories

(MailNews Core :: Database, enhancement)

Product:
MailNews Core
Component:
Database
Type:
enhancement

Tracking

(Not tracked)

Milestone:
Future

People

(Reporter: 3.14, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: helpwanted) 

I cannot believe there is no bug for this yet, but I am not able to find it.

It is best practice to change the subject if the topic moved away from the
original one. To take advantage of this we need the ability to see those parts
of a thread as new ones.

There are two typical situations:

1) I don't care about some topic, so I choose to ignore the thread. Somehow the
discussions moves to something which would be interesting. Luckily the subject
changed. In Mozilla I will not see this and hence cannot enter the discussion.

2) The other way round. There is a long discussion I follow. Some aspect I don't
care about splits of. So far I cannot get rid of this part.

pi
QA Contact: gayatri → laurel
It's really up to the sender to specify which thread a message belongs to using
the reference header, and changing the subject should *not* always change the
thread, so there's no way to win, really, and we just have to use the reference
headers.
Status: NEW → RESOLVED
Closed: 22 years ago
Resolution: --- → WONTFIX
Quoting Son-of-1036:
---cut---
NOTE: There is a semi-standard  convention,  often
used, in which a subject change is flagged by mak-
ing the new Subject-content of the form:

     new topic (was: old topic)

possibly  with  "old  topic"  somewhat  truncated.
Posters  wishing  to  do  something  like this are
urged to use this exact form,  to  simplify  auto-
mated analysis.
---cut---

Note that it talks explicitly about automated analysis, i.e., newsreader action.

So this constitutes the proper way to handle the situation I described. You take
up an existing article, hence there are References. Anyways, there is no reason
to argue if this is reasonable, it is simply used. Mozilla cannot handle this.
Saying WONTFIX means it should be a very limited reader.

I cannot see *any* reason not to allow this at least as an option.

pi
the sheer bloody amount of effort it would take is why I don't think we'll fix this.
OK, I'll re-open and reassign to nobody@mozilla.org.
Status: RESOLVED → REOPENED
Keywords: helpwanted
Resolution: WONTFIX → ---
Target Milestone: --- → Future
the hard part about this is that the "de-threading" would need to be persistent
outside of the db (the .msf file) because we need to be able to throw away the
db and recreate it from just the mailbox or news/imap server. There's no obvious
solution to that problem.
Assignee: bienvenu → nobody
Status: REOPENED → NEW
Of course, I have no idea about the code, but my idea for the display would be
to do it "on the fly", i.e., when entering the group.

Further, it would be crucial not to extend watch/ignore. My guess is that we
look at the references and if this is labeled. This would need anthoer check if
the subject is identical.

Technical remark: Due to the habit of adding additional whitespace to the
subject (e.g., Mozilla before 0.9.9, all Versions of Netscape so far), it might
be a good idea to ignore whitespace when comparing.

A typical discussion might look like this:
Subject: old topic
Subject: Re: old topic
Subject: Re: old topic
Subject: new things (was: old topic)
Subject: Re: new things
Subject: Re: new things (was: old topic)

Some people/readers delete the (was: part, others don't. So this part should
also been ignored when checking if subjects are identical.

pi
If bug 164115 works (you have a patch) it should also be possible to do it
here:-)) Would you take this one, David?

pi
@bienvenu:
> the hard part about this is that the "de-threading" would need to be persistent
> outside of the db (the .msf file) because we need to be able to throw away the
> db and recreate it from just the mailbox or news/imap server.

Why? If all the threading information is part of the db (isn't it?), anyway,
then what prevents us from do the de-threading again when recreating the db? Or
do you mean that there are situations where we need (de)threading information
_without_ having a db?
If this is to be solely automatic, i.e., ignore references when threading if the
subject has changed, we could do that. It's the opposite of bug 164115, however.
From your description, I got the impression you wanted to do this on a case by
case basis as a user, and not automatically.  Automatically, we can do. The
questions are:

1. Should we only do this whenever the subject changes, or when we have the
(was: ...) form of the subject change?
2. Should we have a ui for this pref (this would be off by default, I believe)
Just jumping in:

> 1. Should we only do this whenever the subject changes,
>    or when we have the (was: ...) form of the subject change?

I think we should only destroy the references if the user
(a) has changed the subject 
*and*
(b) has *not* used the (was: ...) form of the subject change.

The "(was:" indicates that the thread context should be kept intact.
If he changes the subject *without* the "(was:", the references may be skipped.

> 2. Should we have a ui for this pref
>    (this would be off by default, I believe)

Definitely *yes*, and off by default.
David:

>If this is to be solely automatic, i.e., ignore references when
>threading if the subject has changed, we could do that.

Yes, it is start a new thread for different subject. The notion of different
need to be defined, e.g., two subjects are different iff after decoding and
removing leading "Re: " and " (was: .*" as well as whitespace (or only multiple
whitespace) they are equal.

>It's the opposite of bug 164115, however.

In a sense. So these both should be options (excluding one another?).

>From your description, I got the
>impression you wanted to do this on a case by case basis as a user,
>and not automatically.  Automatically, we can do. The questions are:
>
>1. Should we only do this whenever the subject changes, or when we
>have the (was: ...) form of the subject change?

In any case as described above. Often users change the subject incorrectly, but
want the same effect.

>2. Should we have a ui for this pref (this would be off by default, I
>believe)

I don't have a strong opinion about the default, a UI is always nice, but to me
not all that important. If so it might sit in the account setting (or do we want
it for all accounts at the same time?).


Karsten:

>> 1. Should we only do this whenever the subject changes,
>>    or when we have the (was: ...) form of the subject change?
>
>I think we should only destroy the references

Maybe you mean it, but just to be sure: References MUST NOT be destroyed for any
answer. They MIGHT be ignored for displaying thread, which is what this bug
suggests.

>The "(was:" indicates that the thread context should be kept intact.

No. What is the reason to change the subject: You want to indicate a change in
the topic. What is it good for: People should be able to ignore this part of the
thread if they read the original thread *and* they shall be able to join in
here. Both is not possible so far: For the first you can only apply ignore to
the complete thread. For the second if you ignored the original thread you will
never see the new topic.

pi
pi:
>>I think we should only destroy the references
>
>Maybe you mean it, but just to be sure: References MUST NOT be
>destroyed for any answer. They MIGHT be ignored for displaying 
>thread, which is what this bug suggests.

If the references are kept intact and only ignored for display, my objections
are meaningless: beginning a new thread whenever the subject really changes can
be quite useful.

(
>Yes, it is start a new thread for different subject. The notion of different
>need to be defined, e.g., two subjects are different iff after decoding and
>removing leading "Re: " and " (was: .*" as well as whitespace (or only
>multiple whitespace) they are equal.

I guess this should have been "...they are different."?
)
1a. As long the subject change is not in the form "new topic (was: old topic)"
Mozilla should do nothing. I.e. any other differences in the subject don't
matter (Because ppl often do this wrong and Mozilla should only mind "correct"
changes.).

1b. Follow-ups should only have the subject "new topic" (without " (was: old
topic)"), because after some subject changes, the subject would end up like this:

"new topic (was: old topic (was: even older (was: foo (was: (bar)))))"

Not very nice to look at. :/

2. Personally I don't need an UI as long I can set up Mozilla in any way to do this.

3. The references should *never* be destroyed!

4. _suggestion:_

4.1 If "[something] (was: [s/t here] ) [nothing]" shoes up in the subject, we
have a new thread.

4.2 If MID of 4.1 shoes up in the references, the article belongs to the new
thread a should be sorted appropriately.

4.3 further sorting after references with MID of the new subject as root.

4.4 answers/follow-ups like described under 1b

4.4 Decoding and stuff is not needed, I think. (I am not a programmer)

(disclaimer: Just a rough idea, but with potential I think.)

5. Pi is right with the "on-the-fly" thing. Nice idea.
sorry for some misspellings. #-/

To avoid misunderstandings:

4.2 a = and
4.3 further sorting after references, where article with new subject is the root
article

Tobias Schacht wrote:

> 1a. As long the subject change is not in the form "new topic (was: old topic)"
> Mozilla should do nothing. I.e. any other differences in the subject don't
> matter (Because ppl often do this wrong and Mozilla should only mind "correct"
> changes.).

No, many people don't do it right, so this would render most of this useles.

> 1b. Follow-ups should only have the subject "new topic" (without " (was: old
> topic)"), because after some subject changes, the subject would end up like this:

Mozilla already does it, with one bug: Bug 191275.

> 3. The references should *never* be destroyed!

No, they MUST NOT be destroyed.

pi, small p as in pi
If this option is ever going to be implemented, it needs to be clearly defined,
what a "changed subject" is. 

The following are some of the issues I see:
- In Germany, some people who are using clients which are not Euro compatible,
are manually "decoding" a "€" sign to something like "[Euro]". 
- Outlook Express removes everything before a : when forming a replies subject
(ie. "FAQ: " -> "Re: "). 
- The Bat! (I think) adds a "Re[2]: " when doing a second reply.
- Some people are also changing "ä" -> "ae". 
- Some clients are not as good in decoding encoded subjects (eg.
"=?windows-1252?Q?=80uro?=" which is "€uro") as others and thus sometimes have
left over =?windows-1252?Q? "artifacts". 
- Ignoring white space when doing the check if this is a "changed subject" is a
good idea, because especially with long subjects, a folding has to be done and
some clients take the resulting space as belonging to the encoded string
(wrong), some don't (correct).

All of these issues have to be dealt with when implementing a
"Thread-By-Reference:-and-detect-changed-subjects" mechanism is to be implemented.

Personally, I don't think this is worthwile. A better solution would be to have
the user manually "de-thread" a thread. This of course should be done in a
persistent way, so that the .dbf file can be deleted anytime.
> The following are some of the issues I see:
> - In Germany, some people who are using clients which are not Euro compatible,
> are manually "decoding" a "€" sign to something like "[Euro]". 

I almost never see that. Anyhow, this is a subject change.

> - Outlook Express removes everything before a : when forming a replies subject
> (ie. "FAQ: " -> "Re: "). 

Another subject change. Software cannot know if this is intended.

> - The Bat! (I think) adds a "Re[2]: " when doing a second reply.

Also broken, but we could have a list of Re: like patterns (Forte Agent has it
and it works nicely), e.g.: Re:, Re[\d]:, AW: ...

> - Some people are also changing "ä" -> "ae". 

Broken, but also not happening often.

> - Some clients are not as good in decoding encoded subjects (eg.
> "=?windows-1252?Q?=80uro?=" which is "€uro") as others and thus sometimes have
> left over =?windows-1252?Q? "artifacts". 

Totally broken, but a subject change.

> - Ignoring white space when doing the check if this is a "changed subject" is
> a good idea, because especially with long subjects, a folding has to be done
> and some clients take the resulting space as belonging to the encoded string
> (wrong), some don't (correct).

ACK

> Personally, I don't think this is worthwile. A better solution would be to
> have the user manually "de-thread" a thread.

Bad idea. A followup MUST have References. Software should help users doing
things correctly. It is a very bad idea to teach users to make mistakes because
software is limited or broken.

Let me again say why this is an important feature. There are basically two
situations. Both start with one thread which changes topic with subject change
(as is best practice):

1) You read the original thread and don't care about the modified one. With
splitting off the latter you can easily ignore it.

2) You don't read (ignore) the original thrad. With Mozilla you will never see
the new topic, so in effect you lose it completely. This happens all the time to
me. So Mozilla is not a usable reader for me.

pi
*** Bug 231764 has been marked as a duplicate of this bug. ***
Blocks: 236849
I'm surprised that this hasn't become more of an issue. Many of us subscribe to
mailing lists and keep the folders threaded (and collapsed). Just this evening,
I lost track of an earlier part of a thread, because the poster erroneously
started it by replying to a previous thread (which was some months old). The
initial posting of this new subject ended up well up and out of reach, forcing
me to search the folder for it.

At the very least, we should have some way to split/combine threads at the
recipient side, so that we can organize our folders manually. I agree that the
AI required to determine which threads have changed subject could be quite
involved, but a simple key binding or context menu would be a big step beyond
what we've got now.
Product: MailNews → Core
I just noticed this bug while thinking of reporting something similar. My take
on the thread-breaking by subject change is that it *should* happen as far as
the UI is concerned but that the references should never be destroyed. Make it a
UI option if you like, but I would tend to default to having it enabled.

Incidentally, I'm coming at this from an angle of just getting lots of email
from people who reply to a thread, delete all the message body, change the
subject and assume that that means they're starting a new thread. I reported bug
266594 as a way of encouraging users to *really* start a new thread with correct
references but the receiving UI should do its best to handle the *intended*
meaning seamlessly. I can think of a lot more circumstances where I'd *want* to
see a new subject as a new thread than have it continued (given a certain
technical flexibility on the definition of when a thread has been re-named, of
course...)

Well, that's my 0.02 of your preferred currency... thought the issue could do
with being re-awakened :-)
QA Contact: laurel → database
Product: Core → MailNews Core
Is this still an issue? I can't recall having a problem recently with changed subjects *not* re-threading. I don't spend a lot of time on newsgroups anymore, so I'd have to check to be sure. Can we get soem confirmation, here, perhaps?
Nothing has changed. You can also test this with e-mail.

pi
See Also: → 369068
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.