Stability problems on AMO since 2014-08 push

RESOLVED FIXED in 2014-09

Status

P1
critical
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: jorgev, Unassigned)

Tracking

unspecified
2014-09
Dependency tree / graph

Details

(Reporter)

Description

4 years ago
Since the August push, we have been receiving constant complaints from developers that AMO is failing for them. The problems we have seen reported so far:

* Uploading a new add-on or new version fails after validation.
* After a new version is reviewed, updates aren't pushed to users.
* Sporadic breakage in review tools: missing files in code viewer, errors when submitting reviews, and multiple review form submissions.

This is a critical problem that may be costing us developers and needs to be addressed ASAP.
CCing people who may have specific ideas.  Developers - did we adjust any of this (looks like several bugs with the validator?).  Jason - any changes on the ops side or obvious broken bits/alerts?
We've also been getting a lot of 504s, both in the editor tools and developer pages, especially when trying to submit reviews. The latter case is probably a factor in comments for reviews appearing multiple times in review logs.

Comment 3

4 years ago
I've been experiencing the same issue, especially when trying to upload a new add-on or an updated version, either of which would be queued for full review.  It's not clear what was and wasn't successfully uploaded. 

> This is a critical problem that may be costing us developers and needs to be
> addressed ASAP.
I can definitely vouch for that.  After several attempts at uploading my first public add-on and getting 504s and other errors and having to go back to the beginning, I was one attempt away from completely giving up and determining that I was not able and not going to be able to contribute public Firefox add-ons because I just couldn't figure out how to get the system to work.  Fortunately, it actually worked on that last attempt, and the add-on later passed full review.

Comment 4

4 years ago
 504 Gateway Time-out
when submitting addon

Comment 5

4 years ago
This sounds like the same problem I was seeing in Bug 897317. You can use the XPI attached there to reproduce on dev and stage.

Updated

4 years ago
Depends on: 1060019

Comment 6

4 years ago
I've reverted https://github.com/washort/django-cache-machine/commit/fecd5a281500018d42c6c7c2ff8d8bfc92545c1a#diff-d41d8cd98f00b204e9800998ecf8427e, which makes django-cache-machine compatible with redis sharding. Addon submission is still slow, but no longer 504ing, as far as I can tell.

Something, in the submit_license and version_delete views, is causing mass invalidation to occur. This resulted in 2,000+ SMEMBER calls and now ~15 giant SUNION calls. We should either keep this bug open to debug that issue or close this and open another bug to address it.
Jorge, Kris, are the issues listed here still there since Jeremy's fix?

Updated

4 years ago
Flags: needinfo?(kmaglione+bmo)
Flags: needinfo?(jorge)
(Reporter)

Comment 8

4 years ago
I haven't seen new reports recently. We should give it a couple of days before calling this fixed, however.
Flags: needinfo?(jorge)
Agree, let's give it until the end of the week before we call it fixed.
Flags: needinfo?(kmaglione+bmo)

Comment 10

4 years ago
Hi all,

I uploaded new version of a addon for android during AMO had problem.
After validation finished, AMO said new version is existing.
But It seems upload new version was finished in background. Because "Manage Status & Versions" shows new version and queued for review.
Then new version was approved for full review.

Yesterday, I noticed download button is active when I access addon page from desktop firefox.
But Version Informations said "Works with Firefox for Android 26.0 - 35.0" only, not for desktop.

Did I made any mistake or something related to AMO problem?


"Cache Tool for Android"
https://addons.mozilla.org/en-US/android/addon/cache-tool-for-android/versions/

Thank you for your help,
backy0175
(Reporter)

Comment 11

4 years ago
Both versions show up as compatible only with Android. You need to select "All Platforms" when uploading a new version so that Firefox compatibility is displayed. I think you can add it here: https://addons.mozilla.org/developers/addon/cache-tool-for-android/versions/1595024

In any case, this appears unrelated to this bug.

Comment 12

4 years ago
jorge, thank you for a reply.

I checked version information. It had been already "All Platforms".
I uploaded new version as usual. But this time, something weird.

I tried to add and remove desktop compatibility. But it didn't fix this problem.
What should I do next?

backy0175
(Reporter)

Comment 13

4 years ago
(In reply to backy0175 from comment #12)
> I tried to add and remove desktop compatibility. But it didn't fix this
> problem.
> What should I do next?

Please email the amo-admins list. This discussion is offtopic for this bug.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.