1438022 - Figure out how to handle email to firebot and bugbot

Status: RESOLVED WORKSFORME

(bugzilla.mozilla.org :: General, enhancement)

Description

[Dylan Hardison [:dylan] (he/him)]

Apparently the size of emails sent to firebot and bugbot might be more than SES can handle. We'll need to figure out how to handle this.

Comment 1

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

Can we use a mechanism other than email to signal to firebot and bugbot?

Comment 2

[:glob ✱]

i don't have anything to do with bugbot, so i'll just talk about firebot.

dylan - when you say "size of emails", do you mean volume/quantity?  firebot isn't receive emails that are larger in size than any other user.

currently the only way for firebot to ingest this firehose is via email, but that isn't a large isse; the question is more what bmo can support.  the obvious choice is i add a https endpoint that bmo posts json to, but that ability doesn't exist within bmo today.

here's the requirements from a firebot perspective:
- near instantaneous delivery of notifications .. email is surprisingly fast and i get pings when it's laggy
- bug metadata (id, product, component, keywords)
- what changed (who, field, old value, new value, extra-data)

an example of changed extra-data is a flag's requestee.

all this is currently parsed from bugmail's plaintext format (with a degree of "fun" when it comes to parsing the diff table).
https://github.com/globau/firebot/blob/master/BotModules/BugzillaMailHandler.pl
(note the age of this file and its author)

Comment 3

[:glob ✱]

dylan let me know that this _is_ about email size, and that ckolos reported seeing emails as large as 41 meg being sent to firebot.

this is not reflected in the emails that firebot receives - here's the stats since the start of feb:

  'count' => 36374,
  'average_size_bytes' => 8271,
  'max_size_bytes' => 111161,

ckolos - can you provide more info about the large emails you saw?  i can't imagine a scenario where bugzilla generates an email anywhere near that size.

Comment 4

[:glob ✱]

that said, firebot's mail server will reject any email over 10mb.

Comment 5

[Chris Kolosiwsky [:ckolos] (ckolos has left the building)]

Yes, this is correct. The reporting tool being used aggregated *all* messages sent to that account as a single item. Further investigation proves that it's impossible for jq1/2 to send any email over 10MB in size.