1644135 - filters with forward are failing

Status: RESOLVED INCOMPLETE

(Thunderbird :: Filters, defect)

Description

[TREMBLAY,Bernard]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36

Steps to reproduce:

Simply let run old filters.
Or create (for test) a new filter

Actual results:

A message with an old look (brown) "failed" appear wh"en filter is sending a transfer message.
Occurs with all transfers.
The message is repeated as many messages as to be tranfered, the user has to send OK for each. (after creating a new filter, for which the transfer update was needing many transfers, I have clicked 200 time until I decide to cancel TB with task manager (impossible to get control).
After the restart I had to quickly inactivate all filters and begin to do it manually.

Expected results:

Simply run the filter and send the messages to transfer.

Some explanations:

• Checked the smtp (I don't know which smtp is used to transfer, I suppose that it is the default smtp of the installation).
  Note that the "to" has no corresponding valid smtp because it is an alias (for example I uses currently destinations bty-xxxx9@<domain> for a smtp bty-xxxx@<domain>. All alis are gtrouped to a main mailbox.
  There are near 30 main mail boxes managed on the current TB installation.

The "failed" is intantaneous.
Anyway the fact of the error should stop temporarily the filter action to avoid a dead lock of TB.
The errors are produced at a rythm must more faster than a time out from receiver.
Generally the transfer are performed at each reception which allows to manage something if there is an error, but when a new filter is created and a manual execution is launched even normally or on an error done into the filter syntax there can be several thousands or more that can be sent or failed, so :

• a stop button must be able to inturrupt the process
• a number of mail limit to transfer in one "session" should be defined.

It is very difficult to debug because there is no mean to access the parameters of the filters independently of the UI, no possibility to produce a list of filters

Comment 1

[Wayne Mery (:wsmwk)]

The message is repeated as many messages as to be tranfered, the user has to send OK for each.

Not sure what to make of this. Do you also see the problem when using version 78?

Comment 2

[TREMBLAY,Bernard]

The failed forwarding was coming from a logfile full (filterlog.html 119Mo ?).
But other problem and suggestions remain.
see bug 1664693.

Comment 3

[Wayne Mery (:wsmwk)]

I don't understand what problem comment 0 is attempting to describe, do you?

Comment 4

[Dan Pernokis]

First, I don't understand what "transfer" means. I'm guessing it is the "Move to Message Folder..." or "Forward Message..." options.
What is "clean"? Is this "Clear" (empty the log)? If so, I take it to mean the reporter can't empty a full log.
And I don't understand "old look (brown) failed" -- and I have no experience with clicking "OK" for whatever is happening here.

It isn't clear what happens with 30 mailboxes. Is each processed independently and once only? Or are messages re-routed and distributed among them? I myself also use aliases for mailboxes (in fact, a whole funnel) where a few aliases form a group, and several groups go to a specific mailbox -- handy for, say, spam control, or disabling certain types of emails -- and there are filters on many of them.

I get the impression that perhaps a couple of filters overlap and loop recursively -- for example, Filter on A moves to B, then Filter on B moves to C, and Filter on C moves to A. Obvious in this 3-filter illustration, but if the nesting is deep enough or convoluted & complex enough, the programmer of the filters might not see it. And I have found by observation that if the "Stop Filter Execution" isn't included, a filter usually continues to run and may do additional unexpected results (but always exactly what the filter actually says to do) -- so it becomes very easy to overlook any recursion. Run enough filters like this and there could easily be a lot of logging for a single message processed umpteen times. Not all messages trigger everything, and not all mailboxes may be affected -- but one single message that meets too many criteria might loop for awhile (the "200" clicks as mentioned) or forever (until terminated by end-task). The fact that a "mail limit to transfer" is needed tells me there is a lot of recursion rather than a large volume of messages -- unless a new filter is applied retroactively on existing large mailboxes.

Comment 5

[TREMBLAY,Bernard]

Hello,

I read today again for the first time my message.
I have three remarks about it.
1- Because I use the French version, I don't know the terms used in English version and I need to imagine a right-valid meaning translation to English. More because there is a precise term used a synonym can not be accurate. Nevertheless generally I have been understood.
2- I had not well explained several details and not hierarchized the problem.
I am going to clear all.
3- often small screen copy (messages in small windows) should be clear inserted into the text to explain what the user see, but to join several files images files can be difficult to be used preserving the meaning. This is inherent to the bugzilla interface.

---

1- Clarify : Dan Pernokis says :
<<First, I don't understand what "transfer" means. I'm guessing it is the "Move to Message Folder..." or "Forward Message..." options.>>
Yes, "transfer" means" : "move (or copy) from one place to another" (my English to French dictionary) and is here "Copy" or "move" to a new folder.

2- Just after I tell that a "message" appears telling "Failed" . This happens if "transfer" fails obviously, and this message is displayed into "a box which has an 'old' look" this mean "design" (a square box with a brown background, like messages displayed in the DOS environment at the end of 1980th, this can help to identify it).

3- When you acknowledge (typing OK) the soft walks a step forward to the next instance (if you do not, the interface remains locked - a "modal" message)

4- Then, the loop goes on until they are mails to "transfer" applying the same filter. This can lock TB because you can have thousand of mails into the stack filtered, and you have lost the control. Note : I had to cancel TB using task manager (I had to stop connection to the mail server to restart and check the behavior.

I suggests that when the error occurs (many different causes are possible, can be a no existing or not identified target for example) to identify it and send a message like
"Failed to execute : <the action> on <target folder> because of : <the target error cause> Actions : skip all of this error - skip this type of error - stop filter - deactivate filter"

This is the lonely problem, this is the whole problem.

---

Another but secondary question is : When and why the failure occurs ?.

Dan Pernokis says :
<<What is "clean"? Is this "Clear" (empty the log)? If so, I take it to mean the reporter can't empty a full log.>>

The word "clean" is not used in this thread. So I don't understand.

Dan Pernokis says :
<<And I don't understand "old look (brown) failed" >>
I have not written this but <<A message with an old look (brown) "failed">>
means that a message is displayed with text "failed", a message which box have an "old look (design) using brown color as background "an old brown look message".

<<-- and I have no experience with clicking "OK" for whatever is happening here>> The message box is using the default acknowledge "OK" (for example in VB : <msgbox "Failed">)

---

Sub subjects about situations where "failures" of "transfers" can occurs
(the subject is "which action(s) when a failure occurs and secondarily how to help to analyse the situation - filters trace - start stop, hot key to stop filter process etc.)

Dan Pernokis says :
<<
is n't clear what happens with 30 mailboxes. Is each processed independently and once only? Or are messages re-routed and distributed among them? I myself also use aliases for mailboxes (in fact, a whole funnel) where a few aliases form a group, and several groups go to a specific mailbox -- handy for, say, spam control, or disabling certain types of emails -- and there are filters on many of them.

I get the impression that perhaps a couple of filters overlap and loop recursively -- for example, Filter on A moves to B, then Filter on B moves to C, and Filter on C moves to A. Obvious in this 3-filter illustration, but if the nesting is deep enough or convoluted & complex enough, the programmer of the filters might not see it. And I have found by observation that if the "Stop Filter Execution" isn't included, a filter usually continues to run and may do additional unexpected results (but always exactly what the filter actually says to do) -- so it becomes very easy to overlook any recursion. Run enough filters like this and there could easily be a lot of logging for a single message processed umpteen times. Not all messages trigger everything, and not all mailboxes may be affected -- but one single message that meets too many criteria might loop for awhile (the "200" clicks as mentioned) or forever (until terminated by end-task). The fact that a "mail limit to transfer" is needed tells me there is a lot of recursion rather than a large volume of messages -- unless a new filter is applied retroactively on existing large mailboxes.

It is not needed to have multiple loops. Simply receive several thousand of "[SPAM]" (marked) and filter to move mail which title contains "[SPAM]" to folder "SPAMS". If for any reason (included disk access problem to the corresponding system folder) currently the issue is what I have described.

Best regards

Trebly.

Comment 6

[TREMBLAY,Bernard]

Hello,

Sorry, I read again my last message today.

Important note about my previous message

An incident occurred at the end into a copy-past process (I have used the option "use citation of the message" and then I began to split it and gone on redact.
At the end it seems that I could not delete the end of the message (an html block not well filtered have been pasted). So practically the remaining content of the message of Dan Pernokis must be ignored from (I cannot and could not update it):
"Sub subjects about situations" (great chars bolded) to "retroactively on existing large mailboxes".

The valid text is my important and short answer :

It is not needed to have multiple loops to generate the problem. Simply receive several thousand of "[SPAM]" (marked with this string or any other) and define a filter which moves the mails which title contains "[SPAM]" to folder "SPAMS".
If, for any reason (included disk access problem to the corresponding system folder), the migration fails, currently the issue is what I have described.

Best regards

Trebly.

Comment 7

[Wayne Mery (:wsmwk)]

Please provide a screen shot of the filter(s) and filter rule criteria.

Comment 8

[Phoenix]

Resolved per whiteboard