865066 - b2g_build.py steps aren't independent

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Two issues I noticed working on bug 864474:

1, update-source-manifest doesn't work if not run in same execution as checkouts:

scripts/scripts/b2g_build.py --target inari --config b2g/releng-otoro.py --gaia-languages-file locales/languages_basecamp.json --gecko-languages-file gecko/b2g/locales/all-locales --update-source-manifest --build --build-symbols --prep-upload 2>&1 | tee build.log
...
17:28:27     INFO - #####
17:28:27     INFO - ##### Running update-source-manifest step.
17:28:27     INFO - #####
17:28:27     INFO - retry: Calling <bound method B2GBuild.load_json_from_url of <__main__.B2GBuild object at 0xf2e350>> with args: ('http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/raw-file/2c37df73e3e941dfdaa11c4f14007f66d0bd3f06/b2g/config/inari/config.json',), kwargs: {}, attempt #1
17:28:27     INFO - Reading from file /builds/slave/bug864474/build/sources.xml.original
17:28:27     INFO - Found git_base_url of https://git.mozilla.org/releases/ in manifest.
Traceback (most recent call last):
  File "scripts/scripts/b2g_build.py", line 1387, in <module>
    myScript.run()
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 800, in run
    self._possibly_run_method(method_name, error_if_missing=True)
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 759, in _possibly_run_method
    return getattr(self, method_name)()
  File "scripts/scripts/b2g_build.py", line 814, in update_source_manifest
    (self.buildbot_config['properties']['repo_path'], self.buildbot_properties['gecko_revision']))
KeyError: 'gecko_revision'

Previously I'd done |--clobber --checkout-sources --get-blobs| then futzed around with the gaia checkout (wouldn't have broken it). I think the problem here is that properties/* isn't read on start to populate self.buildbot_properties.
---------------------

2, build fails if not run in same execution as checkouts:

scripts/scripts/b2g_build.py --target inari --config b2g/releng-otoro.py --gaia-languages-file locales/languages_basecamp.json --gecko-languages-file gecko/b2g/locales/all-locales --build --build-symbols --prep-upload 2>&1 | tee build.log
...
17:44:19     INFO - #####
17:44:19     INFO - ##### Running build step.
17:44:19     INFO - #####
17:44:19     INFO - retry: Calling <bound method B2GBuild.load_json_from_url of <__main__.B2GBuild object at 0x2886310>> with args: ('http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/raw-file/2c37df73e3e941dfdaa11c4f14007f66d0bd3f06/b2g/config/inari/config.json',), kwargs: {}, attempt #1
Traceback (most recent call last):
  File "scripts/scripts/b2g_build.py", line 1387, in <module>
    myScript.run()
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 800, in run
    self._possibly_run_method(method_name, error_if_missing=True)
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 759, in _possibly_run_method
    return getattr(self, method_name)()
  File "scripts/scripts/b2g_build.py", line 849, in build
    env['MOZ_CHROME_MULTILOCALE'] = " ".join(self.locales)
TypeError

Perhaps self.query_locales() instead ?

aki suggests these might be fallout from the removal of snapshots.

Comment 1

[Chris AtLee [:catlee]]

I think 2) is bug 820140

Comment 2

[Aki Sasaki (not active)]

I think we can make the build action independent of checkout-sources if we had a query_SOMETHING_revision() method that checked for the local buildbot property, then the buildprops.json revision property, then returned the branch name if nothing else were available.

Pete hit this during the work week while working on darwin emulator builds.

Comment 3

[Steve Fink [:sfink] [:s:]]

Attachment: Allow using an existing Gecko checkout

I hit this when working on some configure.in changes that broke on b2g. I wish I would've found this bug sooner.

(In reply to Aki Sasaki [:aki] (out Feb14-Mar12) from comment #2)
> I think we can make the build action independent of checkout-sources if we
> had a query_SOMETHING_revision() method that checked for the local buildbot
> property, then the buildprops.json revision property, then returned the
> branch name if nothing else were available.

I don't fully understand this. I don't know the difference between "local buildbot property" and "buildprops.json revision property". Nor am I sure which branch name to fall back on. So I'll put this patch up since it worked for what I needed, and perhaps somebody can turn it into whatever it needs to be.

Comment 4

[Aki Sasaki (not active)]

Comment on attachment 8381755 [details] [diff] [review]
Allow using an existing Gecko checkout

lgtm :)

Comment 5

[Rail Aliiev [:rail]]

no more b2g