705065 - talos "di" failure - unpack failed for dirtyMaxDBs.zip: End-of-central-directory signature not found

Status: RESOLVED WORKSFORME

(Release Engineering :: General, defect)

Description

[Jonathan Kew [:jfkthame]]

https://tbpl.mozilla.org/php/getParsedLog.php?id=7564683&tree=Firefox

========= Started unpack failed (results: 2, elapsed: 0 secs) ==========
unzip -o dirtyMaxDBs.zip
 in dir /home/cltbld/talos-slave/test/../talos-data/talos (timeout 1200 secs)
 watching logfiles {}
 argv: ['unzip', '-o', 'dirtyMaxDBs.zip']
 environment:
  CVS_RSH=ssh
  DISPLAY=:0.0
  G_BROKEN_FILENAMES=1
  HISTCONTROL=ignoreboth
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=talos-r3-fed-005.build.mozilla.org
  LANG=en_US.UTF-8
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  MAIL=/var/spool/mail/cltbld
  PATH=/home/cltbld/bin:/tools/buildbot-0.8.0/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
  PWD=/home/cltbld/talos-slave/talos-data/talos
  SHELL=/bin/bash
  SHLVL=1
  SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
  TERM=xterm
  USER=cltbld
  _=/home/cltbld/bin/python
 using PTY: False
Archive:  dirtyMaxDBs.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of dirtyMaxDBs.zip or
        dirtyMaxDBs.zip.zip, and cannot find dirtyMaxDBs.zip.ZIP, period.
program finished with exit code 9
elapsedTime=0.003846
======== Finished unpack failed (results: 2, elapsed: 0 secs) ========

Comment 1

[Jonathan Kew [:jfkthame]]

This seems to have become a permanent failure:
https://tbpl.mozilla.org/php/getParsedLog.php?id=7564793&tree=Firefox
https://tbpl.mozilla.org/php/getParsedLog.php?id=7564948&tree=Firefox
https://tbpl.mozilla.org/php/getParsedLog.php?id=7564910&tree=Firefox
https://tbpl.mozilla.org/php/getParsedLog.php?id=7564942&tree=Firefox
https://tbpl.mozilla.org/php/getParsedLog.php?id=7564914&tree=Firefox

It means that the Talos ts_places_generated_max results we're getting are now bogus.

Comment 2

[Ed Morley [:emorley]]

Permared on mozilla-central and mozilla-inbound, including on previously known green pushes.

Is there any way we can try to announce rollouts such as these more clearly, since this is now the third time in ~2-3 days that I've had to play "guess the surprise broken rollout bug" or else just pick a random bunch of people to CC. Thanks! :-)

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I am not sure how this is possible other than the .zip file is corrupted.  I don't recall any bugs about changing dirtymaxdbs.zip.

I just did a wget as we do in the above mentioned log file that failed and I get:
joel@joel-laptop:~/Downloads/mossop$ wget --progress=dot:mega -N http://build.mozilla.org/talos/profiles/dirtyMaxDBs.zip
--2011-11-24 08:02:26--  http://build.mozilla.org/talos/profiles/dirtyMaxDBs.zip
Resolving build.mozilla.org... 10.2.74.128
Connecting to build.mozilla.org|10.2.74.128|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 56502533 (54M) [application/zip]
Saving to: `dirtyMaxDBs.zip'

     0K ........ ........ ........ ........ ........ ........  5%  544K 96s
  3072K ........ ........ ........ ........ ........ ........ 11%  810K 75s
  6144K ........ ........ ........ ........ ........ ........ 16%  264K 1m45s
  9216K ........ ........ ........ ........ ........ ........ 22%  754K 88s
 12288K ........ ........ ........ ........ ........ ........ 27%  440K 83s
 15360K ........ ........ ........ ........ ........ ........ 33%  411K 79s
 18432K ........ ........ ........ ........ ........ ........ 38%  311K 77s
 21504K ........ ........ ........ ........ ........ ........ 44%  849K 66s
 24576K ........ ........ ........ ........ ........ ........ 50%  276K 64s
 27648K ........ ........ ........ ........ ........ ........ 55%  232K 62s
 30720K ........ ........ ........ ........ ........ ........ 61% 1.98M 50s
 33792K ........ ........ ........ ........ ........ ........ 66% 1.80M 40s
 36864K ........ ........ ........ ........ ........ ........ 72% 1.72M 31s
 39936K ........ ........ ........ ........ ........ ........ 77%  493K 25s
 43008K ........ ........ ........ ........ ........ ........ 83%  813K 18s
 46080K ........ ........ ........ ........ ........ ........ 89%  849K 12s
 49152K ........ ........ ........ ........ ........ ........ 94%  184K 6s
 52224K ........ ........ ........ ........ ........ ......  100%  587K=1m58s

2011-11-24 08:04:24 (469 KB/s) - `dirtyMaxDBs.zip' saved [56502533/56502533]

joel@joel-laptop:~/Downloads/mossop$ 


Then I unzip and it works great.  What is concerning is that I get:
2011-11-24 08:04:24 (469 KB/s) - `dirtyMaxDBs.zip' saved [56502533/56502533]


and the log file gets:
2011-11-24 00:14:40 (10.1 MB/s) - `dirtyMaxDBs.zip' saved [47833088/47833088]


The file size is much different when I downloaded it by hand (and it unzips just fine).

So something is wrong with the machines or the network?

Comment 4

[Marco Bonardo [:mak]]

maybe the network, the failure solved by itself! but there are a bunch of other purples due to network issues, that may be related.

Comment 5

[Marco Bonardo [:mak]]

Surely there are network issues, so it's possible a partial dirtyMaxDBs.zip was downloaded and it just tried to use it?

Comment 6

[[e-mail] (:bhearsum)]

Possibly related to bug 705126

Comment 7

[[e-mail] (:bhearsum)]

resetting importance to reflect reality.

Comment 8

[Chris Cooper [:coop] (he/him)]

I imagine we ended up with a bad copy in the cache due to the network outage. Re-open if necessary.