thunderbird mac da 15.0.1 -> 17.0 partial broken

RESOLVED FIXED

Status

Release Engineering
Releases
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: kmoir, Assigned: kmoir)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

6 years ago
END 20:15:09 (12 seconds elapsed): /pub/mozilla.org/thunderbird/nightly/17.0-candidates/build2/win32/zh-TW/Thunderbird Setup 17.0.exe

Command failed for the following files:
  /pub/mozilla.org/thunderbird/nightly/17.0-candidates/build2/update/mac/da/thunderbird-15.0.1-17.0.partial.mar
command: ERROR
Traceback (most recent call last):
  File "/builds/slave/tb-rel-c-rel-av/scripts/scripts/release/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python-2.5.1/lib/python2.5/subprocess.py", line 461, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['ssh', '-l', 'tbirdbld', '-i', '/home/cltbld/.ssh/tbirdbld_dsa', 'stage.mozilla.org', 'nice', 'ionice', '-c2', '-n7', 'extract_and_run_command.py', '-j2', 'clamdscan', '-m', '--no-summary', '--', '/pub/mozilla.org/thunderbird/nightly/17.0-candidates/build2/']' returned non-zero exit status 1
command: END (4762.32s elapsed)

Traceback (most recent call last):
  File "/builds/slave/tb-rel-c-rel-av/scripts/scripts/release/stage-tasks.py", line 232, in <module>
    buildNumber=buildNumber)
  File "/builds/slave/tb-rel-c-rel-av/scripts/scripts/release/stage-tasks.py", line 98, in runAntivirusCheck
    sshKey=stageSshKey)
  File "/builds/slave/tb-rel-c-rel-av/scripts/scripts/release/../../lib/python/util/commands.py", line 59, in run_remote_cmd
    return run_cmd(cmd_prefix + cmd, **kwargs)
  File "/builds/slave/tb-rel-c-rel-av/scripts/scripts/release/../../lib/python/util/commands.py", line 40, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python-2.5.1/lib/python2.5/subprocess.py", line 461, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ssh', '-l', 'tbirdbld', '-i', '/home/cltbld/.ssh/tbirdbld_dsa', 'stage.mozilla.org', 'nice', 'ionice', '-c2', '-n7', 'extract_and_run_command.py', '-j2', 'clamdscan', '-m', '--no-summary', '--', '/pub/mozilla.org/thunderbird/nightly/17.0-candidates/build2/']' returned non-zero exit status 1
program finished with exit code 1
elapsedTime=4775.115991
(Assignee)

Updated

6 years ago
Assignee: nobody → kmoir
Blocks: 796966
Strangely, I don't see the script actually trying to do anything with that file. These are the only references to it:
START 19:52:58: /pub/mozilla.org/thunderbird/nightly/17.0-candidates/build2/update/mac/da/thunderbird-15.0.1-17.0.partial.mar.asc
/pub/mozilla.org/thunderbird/nightly/17.0-candidates/build2/update/mac/da/thunderbird-15.0.1-17.0.partial.mar.asc: OK
END 19:52:58 (0 seconds elapsed): /pub/mozilla.org/thunderbird/nightly/17.0-candidates/build2/update/mac/da/thunderbird-15.0.1-17.0.partial.mar.asc


However, update verify for that MAR failed with:
SOURCE DIRECTORY ../../update
DESTINATION DIRECTORY .
UPDATE TYPE partial
PREPARE ADD precomplete
PREPARE PATCH Contents/_CodeSignature/CodeResources
PREPARE PATCH Contents/MacOS/updater.ini
PREPARE PATCH Contents/MacOS/updater.app/Contents/MacOS/updater
PREPARE PATCH Contents/MacOS/update-settings.ini
PREPARE PATCH Contents/MacOS/thunderbird-bin
PREPARE PATCH Contents/MacOS/thunderbird
PREPARE PATCH Contents/MacOS/searchplugins/wikipedia-da.xml
PREPARE PATCH Contents/MacOS/searchplugins/amazon-co-uk.xml
PREPARE PATCH Contents/MacOS/plugin-container.app/Contents/MacOS/plugin-container
PREPARE PATCH Contents/MacOS/platform.ini
PREPARE PATCH Contents/MacOS/omni.ja
failed: 8
calling QuitProgressUI
FAIL: update status was not succeeded: failed: 8

Which confirms we have a broken MAR for whatever reason. At this point I'm going to suggest that we simply remove the partial snippet + MAR rather than try to fix this, as it's not worth our time for a single broken from 15.0.1. Kim, can you relay this to the tb-drivers list to make sure this is an OK course of action before proceeding?
Summary: Anti-virus failed for Thunderbird 17.0 → thunderbird mac da 15.0.1 -> 17.0 partial broken
Created attachment 683087 [details]
logs from repack

I don't see anything out of the ordinary in these logs.
I tried to find an unsigned version of this file to see if it was broken out of box, or only after it was signed. It had already been deleted from the signing server.

cc'ing nick, because he often has insight into issues like this.
Component: Release Engineering: Automation (Release Automation) → Release Engineering: Releases
(Assignee)

Comment 4

6 years ago
Standard8 

Since this issue only relates to one update from 15.0.1 to 17.0, we're thinking it would make sense just to delete the files related to this MAR, not rebuild for this issue.

This is just a corrupt MAR file, not related to a real virus scan issue. 

Thoughts?
Yes, lets drop that file, we don't have many users on 15.0.1 now anyway and I wasn't really expecting partials from 15.0.1 either.
(Assignee)

Comment 6

6 years ago
The files have been deleted.  I'll start the push to mirrors for Thunderbird 17.0 once the one for FF has completed.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
I did some checking around but didn't get to the bottom of it. Update verify checks the mar file matches the hash and size in the update.xml, and it fails later than that so we know the mar was bad somewhere on the slave. After it's signed everything is handled consistently so probably before that. The mar utility doesn't have any trouble unpacking the file, so this smells like some problem with the omni.ja patch itself. From a quick read of the updater source I can't see how we'd get a return value of 8 in an update verify log, because that's now deprecated in favor of more finegrained errors (in toolkit/mozapps/update/ see: common/errors.h, updater/updater.cpp, updater/archivereader.cpp).

bld-lion-r5-053 doesn't have a particularly orange or red build history, and the slave tracking is fairly clean - just a reimage early November.

So lets spin the BOFH excuse wheel of fortune and come up with .... static from plastic slide rulers.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.