686337 - Android nightly builds should download the previous.apk *before* uploading a new gecko-unsigned-unaligned.apk to the same URL

Status: RESOLVED WONTFIX

(Infrastructure & Operations Graveyard :: CIDuty, task, P3)

Description

[Phil Ringnalda (:philor)]

Well, I don't really know if it's intermittent, I just assume everything is.

https://tbpl.mozilla.org/php/getParsedLog.php?id=6374752
Android mozilla-central nightly on 2011-09-12 03:08:43 PDT for push 569a45bfb71c

command: START
command: wget -O previous.apk http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android/gecko-unsigned-unaligned.apk
command: cwd: /builds/slave/m-cen-lnx-andrd-ntly/build
command: output:
--04:19:08--  http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android/gecko-unsigned-unaligned.apk
Resolving ftp.mozilla.org... 63.245.209.137
Connecting to ftp.mozilla.org|63.245.209.137|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16307815 (16M) [application/vnd.android.package-archive]
...
04:19:10 (11.1 MB/s) - `previous.apk' saved [16307815/16307815]

command: END (1.48s elapsed)

Traceback (most recent call last):
  File "/builds/slave/m-cen-lnx-andrd-ntly/tools/scripts/android/android_snippet.py", line 246, in <module>
    main()
  File "/builds/slave/m-cen-lnx-andrd-ntly/tools/scripts/android/android_snippet.py", line 233, in main
    options.download_subdir)
  File "/builds/slave/m-cen-lnx-andrd-ntly/tools/scripts/android/android_snippet.py", line 172, in getPreviousBuildID
    return parseApk('previous.apk')[0]
  File "/builds/slave/m-cen-lnx-andrd-ntly/tools/scripts/android/android_snippet.py", line 93, in parseApk
    appini = StringIO(zipfile.ZipFile(apk_filename).read('application.ini'))
  File "/tools/python-2.5.1/lib/python2.5/zipfile.py", line 346, in __init__
    self._GetContents()
  File "/tools/python-2.5.1/lib/python2.5/zipfile.py", line 366, in _GetContents
    self._RealGetContents()
  File "/tools/python-2.5.1/lib/python2.5/zipfile.py", line 378, in _RealGetContents
    raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file
program finished with exit code 1

Comment 1

[Phil Ringnalda (:philor)]

Retriggered it, so perhaps we'll know if it's intermittent in an hour or so.

Comment 2

[[e-mail] (:bhearsum)]

John, can you look into this?

Comment 3

[Phil Ringnalda (:philor)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=7108828&tree=Firefox

Comment 4

[Aki Sasaki (not active)]

a) we upload fennec
b) we download the "previous" gecko-unsigned-unaligned.apk

This is the wrong order.  What's probably happening is post_upload.py is in the middle of copying the file into the latest directory when we try to download it and pretend it's the previous apk.

If we keep this order, we can just use the existing files on disk since downloading the same bits we just uploaded is silly.

If we reverse the order, we'll actually get the previous apk's buildid.

Comment 5

[Aki Sasaki (not active)]

Also, we may be getting rid of the gecko-unsigned-unaligned.apk file entirely.
Since we already know the version and filename of existing fennec, we can try to grab the same thing from ftp instead.

Comment 6

[John Ford [:jhford] CET/CEST Berlin Time]

Phil, have you seen this happen again?  I suggest marking this as WORKSFORME if we haven't seen this happen again in 7 months.

Comment 7

[Aki Sasaki (not active)]

John: the logic here is broken, see comment 4 and comment 5.
We need to download the previous apk before we overwrite it, otherwise it's not the previous apk.

Comment 8

[John Ford [:jhford] CET/CEST Berlin Time]

Putting this bug back in the pool.  I haven't had a chance to look into this bug yet, so I have no insight into the solution.

Comment 9

[Justin Wood (:Callek)]

(In reply to Aki Sasaki [:aki] from comment #5)
> Also, we may be getting rid of the gecko-unsigned-unaligned.apk file
> entirely.
> Since we already know the version and filename of existing fennec, we can
> try to grab the same thing from ftp instead.

aki, joel, philor:

I can't say I've ever noticed this error in my work so far, so I want to make sure it is still an issue before I devote any of my cycles to looking into this, rather than WFM/WONTFIX. Do any of you have insight?

Comment 10

[Phil Ringnalda (:philor)]

See comment 7.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

well, I haven't seen it, but comment 7 does make some sense.

Comment 12

[Pete Moore [:pmoore][:pete]]

I can pick this up in a month or so (end of July 2014) when I clear the higher priority items on my plate.

I appreciate comment 7 explains there is a problem (thanks Aki for looking into this) but am just curious why we don't see it more often. Do we have an explanation for this?

Or maybe it is happening frequently and this bug is not getting updated?

Philor, what are your thoughts?

Pete

Comment 13

[Phil Ringnalda (:philor)]

Bug 894360 comment 1, bug 907640.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

I think we should just WONTFIX this, and rip out all the snippet upload steps for Nightly/Aurora builds (both desktop and mobile). We've transitioned to Balrog now, and the only reason to keep uploading to AUS2 is some sanity checks post bug 748698.

Comment 15

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Thomas [:nthomas] from comment #14)
> I think we should just WONTFIX this, and rip out all the snippet upload
> steps for Nightly/Aurora builds (both desktop and mobile). We've
> transitioned to Balrog now, and the only reason to keep uploading to AUS2 is
> some sanity checks post bug 748698.

Can we do this?  My yak looks like:

I don't want to generate and upload gecko-unsigned-unaligned.apk;
it's used in android_snippet.py;
android_snippet.py is not relevant anymore.

nthomas: can you comment on the technical details of removing this snippet generation stuff?

Comment 16

[Nick Alexander :nalexander [he/him]]

nthomas: I've moved this discussion to Bug 1063966, which is my actual issue.

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

I'm going to WONTFIX this, since this code is going to go away in bug 933426.

Comment 18

[Pete Moore [:pmoore][:pete]]

Thanks Nick.