726811 - Extension file is corrupt, after toggling the extension visibility on AMO

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Public Pages, defect, P3)

Description

[3e2224c2]

User Agent: Opera/9.80 (X11; Linux x86_64; U; en) Presto/2.10.229 Version/11.61

Steps to reproduce:

I have disabled some extensions, then enabled them again.


Actual results:

Users cannot install or download the extensions anymore. They get a connection error, file corruption or file doesn't exist.


Expected results:

Users should be able to install the version already reviewed and listed in the extension frontpage.

Comment 1

[3e2224c2]

I just uploaded two new files to one of my addons and they are broken. I cannot delete them either.

Comment 2

[Jorge Villalobos [:jorgev] (he/him)]

Which add-ons are broken? Broken how?

Comment 3

[3e2224c2]

(In reply to Jorge Villalobos [:jorgev] from comment #2)
> Which add-ons are broken? Broken how?

Please ignore my last comment. It was due to the fact that the extension was disabled, so I wasn't able to download the uploaded file from AMO, even through Developer HUb.

Comment 4

[Jorge Villalobos [:jorgev] (he/him)]

A similar situation happens for the Firefox Hotfix add-on (https://addons.mozilla.org/en-US/firefox/addon/firefox-hotfix/?src=search):

1) I enabled it and uploaded a new version
2) an editor approved it
3) I dev-disabled it because alex requested that
4) alex enabled it again later

The files are similarly missing and the download doesn't work.

Comment 5

[u279076]

Full disclosure, I had no such problems with the hotfix during internal testing.

Comment 6

[Dave Townsend [:mossop]]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #5)
> Full disclosure, I had no such problems with the hotfix during internal
> testing.

We saw about 30k users get the hotfix correctly, I'm guessing that happened between the add-on being approved and it getting disabled at alex's request. Did we do the QA testing before it was disabled too?

Comment 7

[u279076]

I believe so, though I've only been a stand-in for any PST testing which was needed late in the game. Vlad Ghetiu has been the one doing most of the testing. I'll let him answer to more specific details.

Comment 8

[u279076]

PS The test plan Vlad used to track the feature testing can be found here:
https://wiki.mozilla.org/Features/Desktop/Add-on_hotfix/Test_Plan

Comment 9

[Vlad [QA]]

Like Anthony said, we performed all kind of tests and everything behaved as expected. 
When Hotfix v1 was disabled, the "automatic" installation of it didn't occur, I performed the installation manual. In all the tests that I've performed I didn't encounter any errors what so ever.
The only bug that I found so far is bug 727398.

Comment 10

[3e2224c2]

I still can't install two of my extensions:

https://addons.mozilla.org/en-US/firefox/addon/checkit/
https://addons.mozilla.org/en-US/firefox/addon/foxtester/

The other two still active have new versions, so they work as expected.

Comment 11

[Kumar McMillan [:kumar]]

Jorge, were you able to reproduce this on addons-dev? I cannot reproduce it there. Unfortunately, the checkit addon was disabled a couple months ago so I have to request a restoration of logs to find out what happened (bug 737248). Are there any addons in production within the last 60 days that this happened to?

Comment 12

[Jorge Villalobos [:jorgev] (he/him)]

Developers don't disable their add-ons often, so I don't expect this to happen to other people. I haven't tried to reproduce this bug.

Comment 13

[Kumar McMillan [:kumar]]

Unless someone has a more recent instance I could look at I'll have to wait until I can get the old logs restored for more clues in bug 737248.

Comment 14

[Wil Clouser [:clouserw]]

QA: can you reproduce this so we can have recent logs?

Comment 15

[Kumar McMillan [:kumar]]

I was able to check the log after bug 737248 and I think I see what's happening. It looks like we have a bug in File.unhide_disabled_file() where it is not re-copying the file to our mirror. I'm a little surprised we can't reproduce it though. This should be fixed regardless.


Here are the gory details...


Downloading the file is a 404 because it does not exist on the mirror:

$ wget 'https://addons.mozilla.org/firefox/downloads/file/88572/checkit-1.0.2-fx.xpi?src=version-history'
HTTP request sent, awaiting response... 302 FOUND
Location: http://releases.mozilla.org/pub/mozilla.org/addons/127758/checkit-1.0.2-fx.xpi [following]
HTTP request sent, awaiting response... 404 Not Found

So why does this happen?

This is where the file originally got uploaded and copied to the mirrors: https://github.com/mozilla/zamboni/blob/fe3deb1ae580f5015491d036f415abb25438dc8e/apps/files/models.py#L179

This is where it got disabled and removed from the mirror: https://github.com/mozilla/zamboni/blob/fe3deb1ae580f5015491d036f415abb25438dc8e/apps/files/models.py#L317

This is where it got re-enabled. Notice that it was not copied back to the mirror: https://github.com/mozilla/zamboni/blob/fe3deb1ae580f5015491d036f415abb25438dc8e/apps/files/models.py#L324

The file was moved to all the correct master locations so we could also fix historic files by combing through each one and making sure it exists on the mirror.

Comment 16

[Kumar McMillan [:kumar]]

This should be fixed going forward: https://github.com/mozilla/zamboni/commit/ce741d7

Comment 17

[Wil Clouser [:clouserw]]

Thanks Kumar!