Closed Bug 1853060 Opened 2 years ago Closed 2 years ago

Support Forum emails are delayed two or more days

Categories

(support.mozilla.org :: Questions, defect)

Product:
support.mozilla.org
Component:
Questions
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: wsmwk, Unassigned)

References

Details

Attachments

(1 file)

DSN-mozilla.jpg
344.34 KB, image/jpeg
Details

Originally reported at https://support.mozilla.org/en-US/forums/contributors/716514 and also mentioned in matrix #sumo:mozilla.org

We have emails coming in at two plus days later.

Today Roland posted a test for me at https://support.mozilla.org/en-US/questions/1423762#answer-1603319 and 40 minutes later I haven't gotten the email. Nothing in my spam folder.

This is rather serious, because we have a very large rate of users still adopting version 115.

Another example, I just received an email for https://support.mozilla.org/en-US/questions/1423233#answer-1603165 which was posted a day ago

I'm getting notifications that are 2-3 days late. It's annoying but manageable for contributors, but probably maddening for users who aren't receiving notifications for their topics, especially first-time posters.

Whenever this happens, once a year or so, there should be a banner at the top of the forum e.g. "Due to issues with the notification system, users are advised to periodically check their topics on this forum for new comments".

(In reply to Wayne Mery (:wsmwk) from comment #0)

...
Today Roland posted a test for me at https://support.mozilla.org/en-US/questions/1423762#answer-1603319 and 40 minutes later I haven't gotten the email.

I just got the "test reply" posting in email, so roughly 2.5 hours later.

Maybe the problem is clearing up?

This morning, I received a response within just a few minutes. HUGE improvement. Maybe it's fixed?

Does seem better.
But no one knows what happened?

It seems to be better:
Headers says:
Received: from a27-214.smtp-out.us-west-2.amazonses.com (54.240.27.214) by re-prd-rgin-006.btmx-prd.synchronoss.net (5.8.818)
id 64752EA211BDD469 for my email address; Sat, 16 Sep 2023 02:12:21 +0100
Date: Sat, 16 Sep 2023 01:12:17 +0000

Date of comment in forum says: Sep 15, 2023, 7:12:15 PM
But you have to bare in mind that the Support Forum never shows the date time of UK, so all those date times are always crazy for anyone ouside of the USA which is real pain. So you always have to do a mental adjustment for the 7 hour difference.

So it seems emails are now actually sending ok.

It seems to have barfed again, but this time the lag is several hours instead of days. Does anyone at Mozilla care about it or have any explanation for the breakdown? Rhetorical question.

Even though this bug was filed under the Questions component, it also affects discussion forums, localizers and KB reviewers/editors, as I posted here: https://support.mozilla.org/en-US/forums/contributors/716514#post-86102

Agree with sfhowes the last few days have gone bad again
Just got an email Date : - Wed, 27 Sep 2023 15:56:13 +0100
a comment from Support Forum dated Sep 26, 2023, 4:03:55 AM
The comment data in forum is always a pain for people not in the USA. It never shows local time. so even if I add on a generous 8 hours so in my time it would be close to midday on Tuesday - that email was still more than 27 hours late.
Looks like the delays are getting worse again.

It's a pain to have to keep checking in the forum whether a comment has been made on dozens of questions. Then seeing response in forum answering it only a day later to get email of comments I saw days ago.

Could anyone affected by this issue please share the raw headers of a delayed email with me? How to view the raw headers varies by email client. In GMail, the relevant function is called "Show original". If you don't want to share the raw headers publicly, you can email them to me (address in my profile).

(In reply to Sven Marnach from comment #10)

Could anyone affected by this issue please share the raw headers of a delayed email with me? How to view the raw headers varies by email client. In GMail, the relevant function is called "Show original". If you don't want to share the raw headers publicly, you can email them to me (address in my profile).

Sent some examples.
Note: today 28/09/23, I have received emails that I would consider received in good time. This is the first time this as occurred in a couple of weeks.
Date of comment : Sep 28, 2023, 4:23:42 AM
'Date' in source: Date: Thu, 28 Sep 2023 10:23:46 +0000
REceived server date and column header: Thu, 28 Sep 2023 11:23:48 +0100
Baring in mind date in comment is US time and Date for me is UK time.

Thanks, Anje! Unfortunately, I haven't received anything yet. Maybe your emails are delayed?

Attached image DSN-mozilla.jpg —

(In reply to Sven Marnach from comment #12)

Thanks, Anje! Unfortunately, I haven't received anything yet. Maybe your emails are delayed?

I sent on 28th at 12:26PM UK time
Nothing is delayed when sending to just about anywhere. I'm sending using same email address as in profile.

I've sent again asking for Delivery Status Notification. However, it seems the receiving server does not support this feature. I've added an image of reply from postmaster of btinternet.com server.

Maybe mozilla.com is blocking my emails at server level
Or maybe it's got something to do with your email address using @mozilla.com ?

I am forwarding samples from Anje to Sven

Thanks! I received all examples. All emails were delivered within seconds of being received by Amazon SES, so the problem must be in the SUMO software. I will work with the team to figure out what's going on.

Notifications dead again on Oct. 21.

Most of the people who noticed this problem are contributors who used to monitor SUMO notification regularly. I don't think a normal user will immediately notice this (esp if they have only posted once or twice). That being said, I think we should understand first how big of our user base is impacted by this issue to determine the priority of this bug.

Is there a way to measure that? Looking at Tasos/Sven here.

Flags: needinfo?(tasos)
Flags: needinfo?(sven)
Flags: needinfo?(sven)

The SUMO dev team is looking into this issue, which we suspect might be related to load in parts of our infrastructure. We are working to formalize some changes that we hope will resolve this as well as improve our overall logging.

(In reply to Smith from comment #18)

The SUMO dev team is looking into this issue, which we suspect might be related to load in parts of our infrastructure. We are working to formalize some changes that we hope will resolve this as well as improve our overall logging.

Thanks for the update, Smith!

Flags: needinfo?(tasos)

Hey everyone!

We had some infra changes this week and we are wondering if email notifications are still arriving with delays.

I would love if you could monitor this for a couple of days just to be sure if things have improved or not.

Thank you so much!

Flags: needinfo?(vseerror)
Flags: needinfo?(sfhowes)
Flags: needinfo?(anjeyelf)

(In reply to Emil Ghitta, QA [:emilghitta] from comment #20)

Hey everyone!

We had some infra changes this week and we are wondering if email notifications are still arriving with delays.

I would love if you could monitor this for a couple of days just to be sure if things have improved or not.

Thank you so much!

No notifications from SUMO in month of Nov. Notifications from Bugzilla and other forums arrive without delay.

Flags: needinfo?(sfhowes)

Response on SUMO has been prompt today. Thanks for what you're doing to fix it.

Sorry - been AFK due to a personal Bug !
Emails currently seem to be arriving without delay.

Flags: needinfo?(anjeyelf)

I haven't been super active on SUMO the last four weeks, so hard to say.

Flags: needinfo?(vseerror)

Performance on SUMO has been excellent. It seems to be working well. Thanks for doing this.

Thank you all for taking from your precious time to confirm that things are optimal again!

I'm going to go ahead and close this ticket as fixed.

Please feel free to re-open this ticket if you notice delays again!

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: