634809 - 'make upload' fails for xulrunner release builds on macosx

Status: RESOLVED WORKSFORME

(Firefox Build System :: General, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

'make -j4 -C obj-firefox/i386 upload' successfully sends up 
  "../../dist/runtimes/xulrunner-2.0b12.en-US.mac-pkg.dmg"
  "../../dist/sdk/xulrunner-2.0b12.en-US.mac-i386.sdk.tar.bz2"
  "../../dist/runtimes//xulrunner-2.0b12.en-US.mac-pkg.txt"
  "../../dist/runtimes//xulrunner-2.0b12.en-US.mac-pkg.checksums"

then we fail to upload the 64bit sdk:

make -j4 -C obj-firefox/x86_64 upload
Makefile:112: FULL_NSPR_CFLAGS=-I\${includedir}
make[3]: dpkg-architecture: Command not found
make[3]: dpkg-architecture: Command not found
Traceback (most recent call last):
  File "/builds/slave/rel-cen-xr-osx64-bld/build/build/checksums.py", line 187, in <module>
    main()
  File "/builds/slave/rel-cen-xr-osx64-bld/build/build/checksums.py", line 184, in main
    process_files(files, options.outfile, options.digest, options.strip)
  File "/builds/slave/rel-cen-xr-osx64-bld/build/build/checksums.py", line 100, in process_files
    output = open(output_filename, 'w+')
IOError: [Errno 2] No such file or directory: '../../dist/runtimes//xulrunner-2.0b12.en-US.mac-pkg.checksums'

It's failing to generate the checksum file. As it's a release we have these set in the environment
  MOZ_PKG_PRETTYNAMES=1
  MOZ_PKG_VERSION=2.0b12


By comparison, a nightly doesn't have those two envvars and does
CHECKSUM FILE START
31345620e5a6dceca30a9acda9eb60d378aaac120433e83b3b7d5643d3540612836b06e5ed30f2a1b6fb52fd5e7ea2ce69e0d764c454b4aec0e889911b5cee25 sha512 50182515 sdk/xulrunner-2.0b12pre.en-US.mac-x86_64.sdk.tar.bz2
CHECKSUM FILE END

and uploads
 ../../dist/sdk/xulrunner-2.0b12pre.en-US.mac-x86_64.sdk.tar.bz2
 ../../dist//xulrunner-2.0b12pre.en-US.mac-pkg.checksums

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

i386 call:
/tools/buildbot/bin/python2.6 /builds/slave/rel-cen-xr-osx64-bld/build/build/checksums.py \
 -o "../../dist/runtimes//xulrunner-2.0b12.en-US.mac-pkg.checksums" \
 -d 'sha512' \
 -s "../../dist" \
 "../../dist/runtimes/xulrunner-2.0b12.en-US.mac-pkg.dmg"       "../../dist/sdk/xulrunner-2.0b12.en-US.mac-i386.sdk.tar.bz2" "../../dist/runtimes//xulrunner-2.0b12.en-US.mac-pkg.txt"   

x86_64 call:
/tools/buildbot/bin/python2.6 /builds/slave/rel-cen-xr-osx64-bld/build/build/checksums.py \
 -o "../../dist/runtimes//xulrunner-2.0b12.en-US.mac-pkg.checksums" \
 -d 'sha512' \
 -s "../../dist" \
 "../../dist/sdk/xulrunner-2.0b12.en-US.mac-x86_64.sdk.tar.bz2"    

So it's barfing because there is no obj-firefox/x86_64/dist/runtimes directory. If you create one then it uploads the sdk, but also stomps on the checksums for i386 when the upload happens. Here's the equivalent for nightlies:
http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/2011-02-16-03-mozilla-central/xulrunner-2.0b12pre.en-US.mac-pkg.checksums

We don't hit this for Firefox because we call 'make upload' in obj-firefox/i386, instead of calling 'make -f client.mk upload' in the src dir.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Work around: upload the x86_64 sdk manually. eg,

cd /builds/slave/rel-cen-xr-osx64-bld/build/obj-firefox/x86_64/dist/sdk
scp -i ~/.ssh/xrbld_dsa \
 xulrunner-2.0b12.en-US.mac-x86_64.sdk.tar.bz2 \
 [e-mail]:/pub/mozilla.org/xulrunner/nightly/2.0b12-candidates/build1/sdk/

Comment 3

[[e-mail] (:bhearsum)]

Any chance someone can pick this up soon or provide some guidance on where the issue lies? It bites us every release.

Comment 4

[Michael Wu [:mwu]]

Attachment: Fix

Can someone try this patch and see if this fixes it?

Comment 5

[[e-mail] (:bhearsum)]

Rerunning 4.0rc2build3 with this patch applied.

Comment 6

[[e-mail] (:bhearsum)]

That fix didn't work, unfortunately :(. I ran the upload target with "make -d" however, and found a curious thing: after the upload had completed, this was in the output:
Commands of `upload' are being run.
Live child 0x100329320 (upload) PID 2004 
Uploading /builds/slave/rel-2.0-xr-osx64-bld/build/obj-x86_64-apple-darwin10.2.0/i386/dist/runtimes/xulrunner-2.0.en-US.mac-pkg.dmg
Uploading /builds/slave/rel-2.0-xr-osx64-bld/build/obj-x86_64-apple-darwin10.2.0/i386/dist/sdk/xulrunner-2.0.en-US.mac-i386.sdk.tar.bz2
Uploading /builds/slave/rel-2.0-xr-osx64-bld/build/obj-x86_64-apple-darwin10.2.0/i386/dist/runtimes/xulrunner-2.0.en-US.mac-pkg.checksums
Upload complete
Reaping winning child 0x100329320 PID 2004 
Removing child 0x100329320 PID 2004 from chain.
Considering target file `upload'.
File `upload' was considered already.
Reaping winning child 0x100322e80 PID 1975 
Removing child 0x100322e80 PID 1975 from chain.
Successfully remade target file `upload'.
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin10.0
Reading makefiles...
<snip>
Updating goal targets....
Considering target file `upload'.
 File `upload' does not exist.
  Considering target file `obj-x86_64-apple-darwin10.2.0/x86_64/Makefile'.
    Considering target file `/builds/slave/rel-2.0-xr-osx64-bld/build/configure'.
      Considering target file `/builds/slave/rel-2.0-xr-osx64-bld/build/configure.in'.
<snip>

Which makes me think that it's running the upload target a second time, against the x86-64 version. That makes sense, because it has to upload the 64-bit SDK, but I guess it's getting choked up on the fact that the runtime only exists for the i386 version?

Comment 7

[[e-mail] (:bhearsum)]

I think the patch in bug 667012 addresses this.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Working these days.