Closed Bug 841935 Opened 11 years ago Closed 11 years ago

Publish B2G update smoketest flash zips for each build

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(blocking-b2g:tef+, b2g18 fixed, b2g18-v1.0.0 wontfix, b2g18-v1.0.1 fixed)

RESOLVED FIXED
B2G C4 (2jan on)
blocking-b2g tef+
Tracking Status
b2g18 --- fixed
b2g18-v1.0.0 --- wontfix
b2g18-v1.0.1 --- fixed

People

(Reporter: marshall, Assigned: mozilla)

References

Details

(Whiteboard: [NPOTB])

Attachments

(5 files, 2 obsolete files)

These flash zips are how the b2g update tests reset between each set of tests on updates.
Aki, can you test this in a real releng environment? I've only been able to test this by symlinking "build" to my development B2G dir, and only executing my new action, i.e:

./scripts/b2g_build.py \
  --target unagi \
  --repo https://git.mozilla.org/releases/gecko.git \
  --config-file users/marshall/b2g-updates.py \
  --only-build-update-testdata

I'll attach my config next.
Attachment #714644 - Flags: review?(aki)
Attached file b2g-updates.py (example) (obsolete) —
Example config I used to test this. Note you'll need to change smoketest_config.public_key/private_key to point at your copy of the B2G source dir (though, we could do this in code too..)
Comment on attachment 714644 [details] [diff] [review]
stage b2g update testdata for upload - v1

scripts/b2g_build.py:726: local variable 'gecko_config' is assigned to but never used

>+        os.makedirs(upload_testdata_dir)

self.mkdir_p(upload_testdata_dir) , if only for the logging.

>+        for f in ('flash.zip', 'flash.sh'):
>+            self.copy_to_upload_dir(
>+                os.path.join(target_testdata_dir, f),
>+                os.path.join(upload_testdata_dir, f))

This will be problematic because the prep-upload action wipes abs_upload_dir (specifically because depend builds were uploading a lot of cruft).

We could either get this copy into prep_upload(), or we could move the abs_upload_dir clobber into the clobber action, or something similar.

I'll go ahead and run this though; the above issues shouldn't be too hard to fix.

(In reply to Marshall Culpepper [:marshall_law] from comment #2)
> Created attachment 714647 [details]
> b2g-updates.py (example)
> 
> Example config I used to test this. Note you'll need to change
> smoketest_config.public_key/private_key to point at your copy of the B2G
> source dir (though, we could do this in code too..)

I'm going to try os.path.abspath("build/target/...")
Currently looks like I'm going to have time for this later tomorrow; is that ok?
Dying here:
14:44:21     INFO - #####
14:44:21     INFO - ##### Running build-update-testdata step.
14:44:21     INFO - #####
14:44:21     INFO - Running command: ['./build.sh', 'fs_config'] in /builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build
14:44:21     INFO - Copy/paste: ./build.sh fs_config
14:44:21    ERROR -  build/envsetup.sh: line 44: make: command not found
14:44:21     INFO -  ** Don't have a product spec for: 'full_unagi'
14:44:21     INFO -  ** Do you have the right repo manifest?
14:44:21     INFO -  
14:44:21     INFO -  > Build failed! <
14:44:21     INFO -  Build with |./build.sh -j1| for better messages
14:44:21     INFO -  If all else fails, use |rm -rf objdir-gecko| to clobber gecko and |rm -rf out| to clobber everything else.
14:44:21    ERROR - Return code: 1
14:44:21    FATAL - failed to build fs_config
14:44:21    FATAL - Exiting 2

Landing this fix and retrying.
http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/da46c1a6465e
15:17:46     INFO -  Run |./flash.sh| to flash all partitions of your device
15:17:46     INFO - Return code: 0
15:17:46     INFO - Writing to file /tmp/tmpjKuzEg/smoketest-config.json
15:17:46     INFO - Contents:
15:17:46     INFO -  {"public_key": "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/target/product/security/testkey.x509.pem", "private_key": "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/target/product/security/testkey.pk8", "devices": {"unagi": {"serials": ["full_unagi"], "system_fs_type": "ext4", "sdcard_recovery": "/sdcard", "sdcard": "/mnt/sdcard", "data_fs_type": "ext4", "system_location": "/dev/block/mmcblk0p19", "data_location": "/dev/block/mmcblk0p22"}}}
15:17:46     INFO - Running command: ['/usr/bin/python', '/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py', 'unagi', '/tmp/tmpjKuzEg'] in /builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build
15:17:46     INFO - Copy/paste: /usr/bin/python /builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py unagi /tmp/tmpjKuzEg
15:17:46     INFO -  Traceback (most recent call last):
15:17:46     INFO -    File "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py", line 7, in <module>
15:17:46     INFO -      import argparse
15:17:46     INFO -  ImportError: No module named argparse
15:17:46    ERROR - Return code: 1
15:17:46    FATAL - failed to stage b2g update testdata
15:17:46    FATAL - Exiting 2
Comment on attachment 714644 [details] [diff] [review]
stage b2g update testdata for upload - v1

Minusing for both the upload issue (and nits in comment 3) plus the missing argparse error.

Plus http://hg.mozilla.org/users/asasaki_mozilla.com/mozharness/rev/da46c1a6465e .
Attachment #714644 - Flags: review?(aki) → review-
The problem above seems to be the "/usr/bin/python" in 15:17:46     INFO - Running command: ['/usr/bin/python', '/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py', 'unagi', '/tmp/tmpjKuzEg'] in /builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build

This appears to be sys.executable, which is /usr/bin/python, 2.6.6.
We probably need to call via /tools/buildbot/bin/python , which is 2.7.2.
It's possible our staging build slaves are at a lower rev than our production slaves, too.
Assignee: marshall → aki
Turn this on for all update-enabled builds, since we're not sure if we want this for unagi-eng or for more user-type builds.
Ok, just calling with /tools/buildbot/bin/python doesn't work because we call further python scripts down the line that assume we have a 2.7 python first in our PATH.

I think the fix here may be updating the env PATH we run this with.

14:43:58     INFO - Writing to file /tmp/tmp4hzB5T/smoketest-config.json
14:43:58     INFO - Contents:
14:43:58     INFO -  {"public_key": "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/target/product/security/testkey.x509.pem", "private_key": "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/target/product/security/testkey.pk8", "devices": {"unagi": {"serials": ["full_unagi"], "system_fs_type": "ext4", "sdcard_recovery": "/sdcard", "sdcard": "/mnt/sdcard", "data_fs_type": "ext4", "system_location": "/dev/block/mmcblk0p19", "data_location": "/dev/block/mmcblk0p22"}}}
14:43:58     INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py', 'unagi', '/tmp/tmp4hzB5T'] in /builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build
14:43:58     INFO - Copy/paste: /tools/buildbot/bin/python /builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py unagi /tmp/tmp4hzB5T
14:43:59     INFO -  Traceback (most recent call last):
14:43:59     INFO -    File "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/tools/update-tools/build-flash-fota.py", line 19, in <module>
14:43:59     INFO -      import argparse
14:43:59     INFO -  ImportError: No module named argparse
14:43:59     INFO -  Building flash zip for device unagi, version 18.0, build 20130221172727...
14:43:59     INFO -  Traceback (most recent call last):
14:43:59     INFO -    File "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py", line 95, in <module>
14:43:59     INFO -      main()
14:43:59     INFO -    File "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py", line 90, in main
14:43:59     INFO -      stage_update(args.device, args.stage_dir)
14:43:59     INFO -    File "/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/gecko/testing/marionette/update-smoketests/stage-update.py", line 59, in stage_update
14:43:59     INFO -      '--output', flash_zip])
14:43:59     INFO -    File "/tools/python27/lib/python2.7/subprocess.py", line 511, in check_call
14:43:59     INFO -      raise CalledProcessError(retcode, cmd)
14:43:59     INFO -  subprocess.CalledProcessError: Command '['/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/tools/update-tools/build-flash-fota.py', '--system-dir', '/builds/slave/b2g_m-b18_unagi_eng_ntly-00000/build/out/target/product/unagi/system', '--system-fs-type', u'ext4', '--system-location', u'/dev/block/mmcblk0p19', '--data-fs-type', u'ext4', '--data-location', u'/dev/block/mmcblk0p22', '--output', '/tmp/tmp4hzB5T/unagi/20130221172727/flash.zip']' returned non-zero exit status 1
14:43:59    ERROR - Return code: 1
UGH.

Setting python to /tools/python27/bin/python2.7 works for the first call, but the next one doesn't work, even with /tools/python27/bin/ in the PATH, because it's python2.7 not python.

Also, it's not clear to me if I can run this outside of mock, since the entire build directory structure is inside of mock.  So I'd have to change the

testdata_dir = tempfile.mkdtemp()

to write to somewhere accessible from mock.


Marshall -- is it possible to prepend sys.executable in front of the various python calls?  That way if I get the original python call right, we don't have to worry about mucking with PATH and getting python2.7 somehow aliased to python.
Flags: needinfo?(marshall)
Aki -- this platform patch should fix the calls to the b2g update tools.
Attachment #717290 - Flags: review?(jgriffin)
Flags: needinfo?(marshall)
Attachment #717290 - Flags: review?(jgriffin) → review+
Building flash zip for device unagi, version 18.0, build 20130222184650...
Error: java is required to be on your PATH for signing

We're going to have to add java to the list of packages installed in mock, I think.
http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/b2g/nightly/mozilla-b2g18-unagi-eng/latest/update-testdata/

[ ]	flash.sh	22-Feb-2013 17:28 	572
[ ]	flash.zip	22-Feb-2013 17:28 	38M

Patches incoming.
This and attachment 717290 [details] [diff] [review] will have to land in m-c, b2g18, and b2g18_v1_0_1.
Attachment #717411 - Flags: review?(catlee)
I suppose we don't need otoro config.json in the above patch, since it looks like we only have a smoketest_config for unagi atm.
This patch:

* adds build-update-testdata to default_actions for all unagi build configs
* adds the python exe to the same configs. Luckily, /tools/python27/bin/python2.7 holds true for both non-mock and mock, although it looks like we're only using this in mock currently.
* adds the smoketest_config for those configs
* adds the build-update-testdata action.  This had to come after the make-updates action, or the scripts would die on the missing mar.
* bases query_abs_dirs() off abs_dirs['abs_work_dir'] since that's already defined
* moves the upload_dir clobber to clobber() so build_update_testdata() can put files in the upload dir
* adds marshall's build_update_testdata(), with mock, python2.7, and checks to make sure we should be running that action

This needs to land after the above gecko patches land and are uplifted to b2g18{,_v1_0_1}.
Attachment #714644 - Attachment is obsolete: true
Attachment #714647 - Attachment is obsolete: true
Attachment #717435 - Flags: review?(catlee)
Requesting tef+ so we can get this patch uplifted. I've landed the gecko patch in inbound:
 https://hg.mozilla.org/integration/mozilla-inbound/rev/c863d79841e9
blocking-b2g: --- → tef?
Attachment #717411 - Flags: review?(catlee) → review+
Comment on attachment 717435 [details] [diff] [review]
(mozharness) unagi update smoketest flash zips

Review of attachment 717435 [details] [diff] [review]:
-----------------------------------------------------------------

Some more comments in build-update-testdata() would be great for future readers of this code.
Attachment #717435 - Flags: review?(catlee) → review+
https://hg.mozilla.org/mozilla-central/rev/c863d79841e9
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Whiteboard: [leave-open]
Comment on attachment 717411 [details] [diff] [review]
(inbound) add java-1.6.0-openjdk-devel to otoro+unagi mock_packages

https://hg.mozilla.org/integration/mozilla-inbound/rev/0e814444d4b3
Attachment #717411 - Flags: checked-in+
Attachment #717290 - Flags: checked-in+
Waiting to land the mozharness patch until we've merged the two gecko patches to b2g18 + b2g18_v1_0_1.  Once it's landed we can mark this fixed.
Comment on attachment 717411 [details] [diff] [review]
(inbound) add java-1.6.0-openjdk-devel to otoro+unagi mock_packages

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 

New feature: update testing smoketests.

User impact if declined: 

Lack of automated update testing.

Testing completed: 

Was able to build + upload the smoketest zip in staging.

Risk to taking this patch (and alternatives if risky): 

Low.

String or UUID changes made by this patch:

None.
Attachment #717411 - Flags: approval-mozilla-b2g18?
Comment on attachment 717290 [details] [diff] [review]
use sys.executable - v1

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 

New feature: update testing smoketests.

User impact if declined: 

Lack of automated update testing.

Testing completed: 

Was able to build + upload the smoketest zip in staging.

Risk to taking this patch (and alternatives if risky): 

Low.

String or UUID changes made by this patch:

None.
Attachment #717290 - Flags: approval-mozilla-b2g18?
Marking blocking so this can be uplifted to both v1-train and v1.0.1 without approval needed.
blocking-b2g: tef? → tef+
Attachment #717290 - Flags: approval-mozilla-b2g18?
Attachment #717411 - Flags: approval-mozilla-b2g18?
Whiteboard: [leave-open] → [leave-open][NPOTB]
unagi-eng update smoketest zip:
https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18-unagi-eng/latest/update-testdata/

We got this for the unagi nightly:
13:12:16     INFO - Running command: ['mock_mozilla', '-r', u'mozilla-centos6-i386', '-q', '--cwd', '/builds/slave/b2g_m-b18_unagi_ntly-000000000/build', '--unpriv', '--shell', u'/usr/bin/env VARIANT=user "LESSOPEN=|/usr/bin/lesspipe.sh %s" LOGNAME=cltbld USER=cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin:/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/compare-locales/scripts SYMBOL_SERVER_USER=b2gbld CCACHE_UMASK=002 LANG=en_US.UTF-8 CCACHE_HASHDIR= TERM=linux SHELL=/bin/bash MOZ_SIGNING_SERVERS=signing3.srv.releng.scl3.mozilla.com:9100 SHLVL=1 G_BROKEN_FILENAMES=1 HISTSIZE=1000 SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_b2g/ LOCALES_FILE=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gaia/locales/languages_dev.json SYMBOL_SERVER_HOST=symbols1.dmz.phx1.mozilla.com CCACHE_DIR=/builds/ccache PYTHONPATH=:/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/compare-locales/lib MOZ_BUILD_DATE=20130225112314 L10NBASEDIR=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko-l10n MOZILLA_OFFICIAL=1 "MOZ_CHROME_MULTILOCALE=es-ES pt-BR" "MOZ_SIGN_CMD=python /builds/slave/b2g_m-b18_unagi_ntly-000000000/scripts/release/signing/signtool.py --cachedir /builds/slave/b2g_m-b18_unagi_ntly-000000000/signing_cache -t /builds/slave/b2g_m-b18_unagi_ntly-000000000/token -n /builds/slave/b2g_m-b18_unagi_ntly-000000000/nonce -c /builds/slave/b2g_m-b18_unagi_ntly-000000000/scripts/release/signing/host.cert -H signing3.srv.releng.scl3.mozilla.com:9100" CCACHE_COMPRESS=1 _=/tools/buildbot/bin/python CCACHE_BASEDIR=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build B2GUPDATER=1 HOSTNAME=bld-linux64-ec2-662.build.aws-us-west-2.mozilla.com SYMBOL_SERVER_SSH_KEY=/home/mock_mozilla/.ssh/b2gbld_dsa LOCALE_BASEDIR=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gaia-l10n HISTCONTROL=ignoredups POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py PWD=/builds/slave/b2g_m-b18_unagi_ntly-000000000 PROPERTIES_FILE=/builds/slave/b2g_m-b18_unagi_ntly-000000000/buildprops.json MAIL=/var/spool/mail/cltbld /tools/python27/bin/python2.7 /builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko/testing/marionette/update-smoketests/stage-update.py unagi /builds/slave/b2g_m-b18_unagi_ntly-000000000/build/testdata'] in /builds/slave/b2g_m-b18_unagi_ntly-000000000/build
13:12:16     INFO - Copy/paste: mock_mozilla -r mozilla-centos6-i386 -q --cwd /builds/slave/b2g_m-b18_unagi_ntly-000000000/build --unpriv --shell "/usr/bin/env VARIANT=user \"LESSOPEN=|/usr/bin/lesspipe.sh %s\" LOGNAME=cltbld USER=cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin:/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/compare-locales/scripts SYMBOL_SERVER_USER=b2gbld CCACHE_UMASK=002 LANG=en_US.UTF-8 CCACHE_HASHDIR= TERM=linux SHELL=/bin/bash MOZ_SIGNING_SERVERS=signing3.srv.releng.scl3.mozilla.com:9100 SHLVL=1 G_BROKEN_FILENAMES=1 HISTSIZE=1000 SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_b2g/ LOCALES_FILE=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gaia/locales/languages_dev.json SYMBOL_SERVER_HOST=symbols1.dmz.phx1.mozilla.com CCACHE_DIR=/builds/ccache PYTHONPATH=:/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/compare-locales/lib MOZ_BUILD_DATE=20130225112314 L10NBASEDIR=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko-l10n MOZILLA_OFFICIAL=1 \"MOZ_CHROME_MULTILOCALE=es-ES pt-BR\" \"MOZ_SIGN_CMD=python /builds/slave/b2g_m-b18_unagi_ntly-000000000/scripts/release/signing/signtool.py --cachedir /builds/slave/b2g_m-b18_unagi_ntly-000000000/signing_cache -t /builds/slave/b2g_m-b18_unagi_ntly-000000000/token -n /builds/slave/b2g_m-b18_unagi_ntly-000000000/nonce -c /builds/slave/b2g_m-b18_unagi_ntly-000000000/scripts/release/signing/host.cert -H signing3.srv.releng.scl3.mozilla.com:9100\" CCACHE_COMPRESS=1 _=/tools/buildbot/bin/python CCACHE_BASEDIR=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build B2GUPDATER=1 HOSTNAME=bld-linux64-ec2-662.build.aws-us-west-2.mozilla.com SYMBOL_SERVER_SSH_KEY=/home/mock_mozilla/.ssh/b2gbld_dsa LOCALE_BASEDIR=/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gaia-l10n HISTCONTROL=ignoredups POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py PWD=/builds/slave/b2g_m-b18_unagi_ntly-000000000 PROPERTIES_FILE=/builds/slave/b2g_m-b18_unagi_ntly-000000000/buildprops.json MAIL=/var/spool/mail/cltbld /tools/python27/bin/python2.7 /builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko/testing/marionette/update-smoketests/stage-update.py unagi /builds/slave/b2g_m-b18_unagi_ntly-000000000/build/testdata"
13:12:17     INFO -  Traceback (most recent call last):
13:12:17     INFO -    File "/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko/testing/marionette/update-smoketests/stage-update.py", line 95, in <module>
13:12:17     INFO -      main()
13:12:17     INFO -    File "/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko/testing/marionette/update-smoketests/stage-update.py", line 90, in main
13:12:17     INFO -      stage_update(args.device, args.stage_dir)
13:12:17     INFO -    File "/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko/testing/marionette/update-smoketests/stage-update.py", line 27, in stage_update
13:12:17     INFO -      b2g_config = b2g.import_update_tools().B2GConfig()
13:12:17     INFO -    File "/builds/slave/b2g_m-b18_unagi_ntly-000000000/build/gecko/testing/marionette/client/marionette/b2ginstance.py", line 94, in import_update_tools
13:12:17     INFO -      import update_tools
13:12:17     INFO -  ImportError: No module named update_tools
13:12:17    ERROR - Return code: 1
13:12:17    FATAL - failed to stage b2g update testdata
13:12:17    FATAL - Exiting 2

RyanVM re-kicked it; hopefully it's a one time glitch =\
Nope, it's a real problem.

Backed out 8015ede70192: http://hg.mozilla.org/build/mozharness/rev/301d66af8e86

We need to figure out:

a) do we need this on unagi-beta as well as unagi-eng, and if so
b) what's update_tools and why can't we import it in unagi-beta as opposed to unagi-eng
Asking Marshall for info on comment 30 question (b).
Flags: needinfo?(marshall)
Asking Stephen for info on comment 30 question (a) (I think this is a yes) and comment 31.
Flags: needinfo?(stephen.donner)
Looks like we need new snapshots for unagi + unagi-stable to proceed.
Depends on: 845000
Flags: needinfo?(marshall)
(In reply to Aki Sasaki [:aki] from comment #33)
> Asking Stephen for info on comment 30 question (a) (I think this is a yes)
> and comment 31.

Sorry to pass the buck, but I'm not actually sure; I've always been going off https://bugzilla.mozilla.org/show_bug.cgi?id=826450#c2 and its associated links, so I think (Marshall, please correct me!) for now we'll target unagi engineering builds (because of the need to use Marionette), but once we get that working, and if we have the infra, I'd be happy to target beta -> beta updates, too.
Flags: needinfo?(stephen.donner)
(In reply to Stephen Donner [:stephend] from comment #35)
> (In reply to Aki Sasaki [:aki] from comment #33)
> > Asking Stephen for info on comment 30 question (a) (I think this is a yes)
> > and comment 31.
> 
> Sorry to pass the buck, but I'm not actually sure; I've always been going
> off https://bugzilla.mozilla.org/show_bug.cgi?id=826450#c2 and its
> associated links, so I think (Marshall, please correct me!) for now we'll
> target unagi engineering builds (because of the need to use Marionette), but
> once we get that working, and if we have the infra, I'd be happy to target
> beta -> beta updates, too.

unagi-eng is what's working atm, so we can turn this back on if we limit to just unagi-eng.  Or we can wait til we resolve bug 845000.

Does the testdata zip look good?
Attached file flash diff
(In reply to Aki Sasaki [:aki] from comment #31)
> c) if someone could verify
> https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18-
> unagi-eng/2013/02/2013-02-25-11-23-14/update-testdata/ , that would be
> awesome as well.

This is a quick comparison with a flash.zip I have from last month. It looks like the uploaded zip is missing at the least "webapps" directory, and some fonts that might be required for gaia to run. Any idea why the webapps dir might be missing from the out/target/product/unagi/system/b2g dir?

I tried flashing this and it's definitely not suitable for running tests :)
=\

I'll try again when all the snapshots are updated.  Hopefully that's tomorrow.
We still don't have update_tools for unagi after the snapshot update.
Should I enable for just unagi-eng, or continue to debug, or both?
Since we need these for tests that require Marionette, I would say unagi-eng is enough, unless marshall says otherwise.
Comment on attachment 717435 [details] [diff] [review]
(mozharness) unagi update smoketest flash zips

Relanded, without the releng or releng-beta configs: http://hg.mozilla.org/build/mozharness/rev/af724a45b16b
Merged to production.

We should get unagi-eng smoketest flash zips again without the unagi bustage.  
Kicking off some nightlies to verify.

If we still have issues with the zips themselves, we can address those issues while still leaving this patch landed.
Attachment #717435 - Flags: checked-in+
Ok, we've got the flash zips for unagi-eng again:

https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18-unagi-eng/latest/update-testdata/
https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18_v1_0_1-unagi-eng/latest/update-testdata/

However, they both appear to be missing the webapps directory, per comment 37.

(In reply to Marshall Culpepper [:marshall_law] from comment #37)
> This is a quick comparison with a flash.zip I have from last month. It looks
> like the uploaded zip is missing at the least "webapps" directory, and some
> fonts that might be required for gaia to run. Any idea why the webapps dir
> might be missing from the out/target/product/unagi/system/b2g dir?

Looking at https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18_v1_0_1-unagi-eng/2013/02/2013-02-27-07-02-00/logs/log_info.log , and searching for 'webapps', I don't really see it in the system/b2g/ directory ever.

I do see

12:38:15     INFO -  rm -rf out/target/product/unagi/data/local/webapps

and some references to webapps in the multilocale portion of the build.

As to why, not really; this seems to be in the murky b2g build system that I have very little familiarity with.
Where does the system/b2g/webapps directory come from?
Is that on master + v1-train + v1.0.1 of whatever repo?
(In reply to Aki Sasaki [:aki] from comment #43)
> Where does the system/b2g/webapps directory come from?
> Is that on master + v1-train + v1.0.1 of whatever repo?

Ahh -- the fact that we're using the "eng" build variant might be the root of the problem. Is it possible for us to change to userdebug? Marionette and all the rest should still be enabled there (I test with it here a lot). Without changing to userdebug, it looks like we *might* be able to squeeze by with B2G_SYSTEM_APPS=1.. see this block of code:

https://github.com/mozilla-b2g/gaia/blob/master/Android.mk#L22-32

Barring those options, we might need another build patch :(
I don't know the history of calling this eng vs userdebug, so I don't know what will happen if we change.  I should be able to add B2G_SYSTEM_APPS=1 though.
Attached patch b2g_system_appsSplinter Review
Attachment #719803 - Flags: review?(catlee)
Attachment #719803 - Flags: review?(catlee) → review+
Comment on attachment 719803 [details] [diff] [review]
b2g_system_apps

http://hg.mozilla.org/build/mozharness/rev/287605cb149a
Merged to production.

I need to kick off nightlies for bug 843303 anyway; doing so.
Attachment #719803 - Flags: checked-in+
I see system/b2g/webapps/...

https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-b2g18-unagi-eng/2013/03/2013-03-01-12-00-39/update-testdata/flash.zip

Are we done here or is there something else still broken?
Flags: needinfo?(marshall)
Ack, apologies, I missed this :) Downloading / flashing this now..
Flags: needinfo?(marshall)
Yup, this is working now on my unagi. Thanks again for all your help Aki :)
Awesome.
Status: REOPENED → RESOLVED
Closed: 11 years ago11 years ago
Resolution: --- → FIXED
Whiteboard: [leave-open][NPOTB] → [NPOTB]
Target Milestone: --- → B2G C4 (2jan on)
Product: mozilla.org → Release Engineering
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.