Closed Bug 1293651 Opened 8 years ago Closed 8 years ago

Releng OSX Update Tests failing from 49.0b1 to newer

Tracking

(Not tracked)

Status:

RESOLVED FIXED

People

(Reporter: Callek, Assigned: Callek)

Details

Attachments

(2 files)

debugging 8 years ago Justin Wood (:Callek) 828 bytes, patch		Details \| Diff \| Splinter Review
Bug 1293651 - use the found updater binary to actually update with. 8 years ago Justin Wood (:Callek) 58 bytes, text/x-review-board-request	jlund : review+	Details

Justin Wood (:Callek)

Assignee

Description

•

8 years ago

https://mozilla-release-logs-dev.s3.amazonaws.com/mozilla-beta/firefox-49.0b2/build1/macosx64_beta_update_verification_2_12-macosx64-lVx1GHP3QwmNhME2G_cFpQ-0

Justin Wood (:Callek)

Assignee

Comment 1

•

8 years ago

So some details, after using this on a "loaner" for a few min, and marking up the script slightly

installing downloads/Firefox 49.0b1.dmg
"disk3" unmounted.
"disk3" ejected.
installing downloads/Firefox 49.0b2.dmg
"disk3" unmounted.
"disk3" ejected.
+ '[' -d source/Firefox.app ']'
+ cd source/Firefox.app
+ for updater in '$updaters'
+ '[' -e Contents/MacOS/updater.app/Contents/MacOS/updater ']'
+ for updater in '$updaters'
+ '[' -e Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater ']'
+ echo 'Found updater at Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater'
Found updater at Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater
++ pwd
+ echo 'Current Dir: /builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app'
Current Dir: /builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app
+ cp Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater ../../update
+ break
+ set -e
+ '[' 0 = 1 ']'
+ ../../update/updater ../../update . . 0
../common/check_updates.sh: line 64: ../../update/updater: No such file or directory
command: ERROR
Traceback (most recent call last):
  File "/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/scripts/release/updates/../../../lib/python/util/commands.py", line 52, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python27/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['bash', 'verify.sh', '-c', '/tmp/tmpIIrs8f']' returned non-zero exit status 1
command: END (56.95s elapsed)

Traceback (most recent call last):
  File "/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/scripts/release/updates/chunked-verify.py", line 83, in <module>
    run_cmd(UPDATE_VERIFY_COMMAND + [configFile], cwd=UPDATE_VERIFY_DIR)
  File "/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/scripts/release/updates/../../../lib/python/util/commands.py", line 52, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python27/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bash', 'verify.sh', '-c', '/tmp/tmpIIrs8f']' returned non-zero exit status 1
[cltbld@bld-lion-r5-070.build.releng.scl3.mozilla.com rel-m-beta_fx_m64_u_v-00000000]$ file /builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app 
/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app: directory
[cltbld@bld-lion-r5-070.build.releng.scl3.mozilla.com rel-m-beta_fx_m64_u_v-00000000]$ file /builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app/Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater
/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app/Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater: Mach-O universal binary with 2 architectures
/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app/Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater (for architecture x86_64):	Mach-O 64-bit executable x86_64
/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app/Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater (for architecture i386):	Mach-O executable i386
[cltbld@bld-lion-r5-070.build.releng.scl3.mozilla.com rel-m-beta_fx_m64_u_v-00000000]$ file /builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app/../../update                                                 
/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app/../../update: directory

Justin Wood (:Callek)

Assignee

Comment 2

•

8 years ago

Attached patch debugging — Details — Splinter Review

With the attached debugging patch... I got output like:

installing downloads/Firefox 49.0b1.dmg
"disk3" unmounted.
"disk3" ejected.
installing downloads/Firefox 49.0b2.dmg
"disk3" unmounted.
"disk3" ejected.
+ '[' -d source/Firefox.app ']'
+ cd source/Firefox.app
+ for updater in '$updaters'
+ '[' -e Contents/MacOS/updater.app/Contents/MacOS/updater ']'
+ for updater in '$updaters'
+ '[' -e Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater ']'
+ echo 'Found updater at Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater'
Found updater at Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater
++ pwd
+ echo 'Current Dir: /builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app'
Current Dir: /builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/release/updates/source/Firefox.app
+ cp -v Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater ../../update
Contents/MacOS/updater.app/Contents/MacOS/org.mozilla.updater -> ../../update/org.mozilla.updater
+ break
+ ls -la ../../update
total 322768
drwxr-xr-x   6 cltbld  staff       204 Aug  9 10:16 .
drwxr-xr-x  51 cltbld  staff      1734 Aug  9 10:16 ..
-rw-r--r--   1 cltbld  staff  82498838 Aug  8 12:11 complete.mar
-rw-r--r--   1 cltbld  staff         9 Aug  9 10:15 complete.size
-rwxr-xr-x   1 cltbld  staff    247664 Aug  9 10:16 org.mozilla.updater
-rw-r--r--   1 cltbld  staff  82498838 Aug  9 10:15 update.mar
+ set -e
+ '[' 0 = 1 ']'
+ ../../update/updater ../../update . . 0
../common/check_updates.sh: line 65: ../../update/updater: No such file or directory
command: ERROR
Traceback (most recent call last):
  File "/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/scripts/release/updates/../../../lib/python/util/commands.py", line 52, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python27/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['bash', 'verify.sh', '-c', '/tmp/tmpWAxuZm']' returned non-zero exit status 1
command: END (57.19s elapsed)

Traceback (most recent call last):
  File "/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/scripts/release/updates/chunked-verify.py", line 83, in <module>
    run_cmd(UPDATE_VERIFY_COMMAND + [configFile], cwd=UPDATE_VERIFY_DIR)
  File "/builds/slave/rel-m-beta_fx_m64_u_v-00000000/scripts/scripts/release/updates/../../../lib/python/util/commands.py", line 52, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python27/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bash', 'verify.sh', '-c', '/tmp/tmpWAxuZm']' returned non-zero exit status 1


Which certainly looked odd to me, so looking deeper... I found Bug 1275632 which appears incomplete.

Assignee: nobody → bugspam.Callek

Status: NEW → ASSIGNED

Comment hidden (mozreview-request)

Review commit: https://reviewboard.mozilla.org/r/70482/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/70482/

Jordan Lund (:jlund)

Comment 4

•

8 years ago

mozreview-review

Comment on attachment 8779494 [details]
Bug 1293651 - use the found updater binary to actually update with.

https://reviewboard.mozilla.org/r/70482/#review67938

::: release/common/check_updates.sh:58
(Diff revision 1)
>              cp $updater ../../update
>              break
>          fi
>      done
>      if [ "$use_old_updater" = "1" ]; then
> -        ../../update/updater ../../update . 0
> +        ../../update/$(basename $updater) ../../update . 0

should this go in the $updater scope block? Seems fragile to rely on last $updater value in previous loop before the break. Just in case we later update this logic.

r+ if this blocks a release and you need to land it as a temp fix.

note: this will have the side effect of using "updater.exe" instead of "updater" for windows but assuming there is no voodoo windows magic, I expect this to be a no op and, in fact, more accurate anyway.

Comment hidden (mozreview-request)

Comment on attachment 8779494 [details]
Bug 1293651 - use the found updater binary to actually update with.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/70482/diff/1-2/

Jordan Lund (:jlund)

Comment 6

•

8 years ago

mozreview-review

Comment on attachment 8779494 [details]
Bug 1293651 - use the found updater binary to actually update with.

https://reviewboard.mozilla.org/r/70482/#review68124

lgtm :)

Attachment #8779494 - Flags: review?(jlund) → review+

Justin Wood (:Callek)

Assignee

Comment 7

•

8 years ago

https://hg.mozilla.org/build/tools/rev/2a9d6ef50f0e

Status: ASSIGNED → RESOLVED

Closed: 8 years ago

Resolution: --- → FIXED

You need to log in before you can comment on or make changes to this bug.

Bugzilla

Quick Search

Releng OSX Update Tests failing from 49.0b1 to newer

Categories

(Release Engineering :: Release Automation: Other, defect)

Tracking

(Not tracked)

People

(Reporter: Callek, Assigned: Callek)

References

Details

Crash Data

Security

(public)

User Story

Attachments

(2 files)

Description

Comment 1

Comment 2

Comment 3

Comment 4

Comment 5

Comment 6

Comment 7

Attachment

General

Description

File Name

Content Type