b2g_build.py steps aren't independent

RESOLVED WONTFIX

Status

Release Engineering
General Automation
RESOLVED WONTFIX
5 years ago
a year ago

People

(Reporter: nthomas, Assigned: sfink)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mozharness][b2g])

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
Two issues I noticed working on bug 864474:

1, update-source-manifest doesn't work if not run in same execution as checkouts:

scripts/scripts/b2g_build.py --target inari --config b2g/releng-otoro.py --gaia-languages-file locales/languages_basecamp.json --gecko-languages-file gecko/b2g/locales/all-locales --update-source-manifest --build --build-symbols --prep-upload 2>&1 | tee build.log
...
17:28:27     INFO - #####
17:28:27     INFO - ##### Running update-source-manifest step.
17:28:27     INFO - #####
17:28:27     INFO - retry: Calling <bound method B2GBuild.load_json_from_url of <__main__.B2GBuild object at 0xf2e350>> with args: ('http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/raw-file/2c37df73e3e941dfdaa11c4f14007f66d0bd3f06/b2g/config/inari/config.json',), kwargs: {}, attempt #1
17:28:27     INFO - Reading from file /builds/slave/bug864474/build/sources.xml.original
17:28:27     INFO - Found git_base_url of https://git.mozilla.org/releases/ in manifest.
Traceback (most recent call last):
  File "scripts/scripts/b2g_build.py", line 1387, in <module>
    myScript.run()
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 800, in run
    self._possibly_run_method(method_name, error_if_missing=True)
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 759, in _possibly_run_method
    return getattr(self, method_name)()
  File "scripts/scripts/b2g_build.py", line 814, in update_source_manifest
    (self.buildbot_config['properties']['repo_path'], self.buildbot_properties['gecko_revision']))
KeyError: 'gecko_revision'

Previously I'd done |--clobber --checkout-sources --get-blobs| then futzed around with the gaia checkout (wouldn't have broken it). I think the problem here is that properties/* isn't read on start to populate self.buildbot_properties.
---------------------

2, build fails if not run in same execution as checkouts:

scripts/scripts/b2g_build.py --target inari --config b2g/releng-otoro.py --gaia-languages-file locales/languages_basecamp.json --gecko-languages-file gecko/b2g/locales/all-locales --build --build-symbols --prep-upload 2>&1 | tee build.log
...
17:44:19     INFO - #####
17:44:19     INFO - ##### Running build step.
17:44:19     INFO - #####
17:44:19     INFO - retry: Calling <bound method B2GBuild.load_json_from_url of <__main__.B2GBuild object at 0x2886310>> with args: ('http://hg.mozilla.org/releases/mozilla-b2g18_v1_0_1/raw-file/2c37df73e3e941dfdaa11c4f14007f66d0bd3f06/b2g/config/inari/config.json',), kwargs: {}, attempt #1
Traceback (most recent call last):
  File "scripts/scripts/b2g_build.py", line 1387, in <module>
    myScript.run()
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 800, in run
    self._possibly_run_method(method_name, error_if_missing=True)
  File "/builds/slave/bug864474/scripts/mozharness/base/script.py", line 759, in _possibly_run_method
    return getattr(self, method_name)()
  File "scripts/scripts/b2g_build.py", line 849, in build
    env['MOZ_CHROME_MULTILOCALE'] = " ".join(self.locales)
TypeError

Perhaps self.query_locales() instead ?

aki suggests these might be fallout from the removal of snapshots.
I think 2) is bug 820140
Product: mozilla.org → Release Engineering
I think we can make the build action independent of checkout-sources if we had a query_SOMETHING_revision() method that checked for the local buildbot property, then the buildprops.json revision property, then returned the branch name if nothing else were available.

Pete hit this during the work week while working on darwin emulator builds.
Component: Other → General Automation
Whiteboard: [mozharness][b2g]
(Assignee)

Comment 3

4 years ago
Created attachment 8381755 [details] [diff] [review]
Allow using an existing Gecko checkout

I hit this when working on some configure.in changes that broke on b2g. I wish I would've found this bug sooner.

(In reply to Aki Sasaki [:aki] (out Feb14-Mar12) from comment #2)
> I think we can make the build action independent of checkout-sources if we
> had a query_SOMETHING_revision() method that checked for the local buildbot
> property, then the buildprops.json revision property, then returned the
> branch name if nothing else were available.

I don't fully understand this. I don't know the difference between "local buildbot property" and "buildprops.json revision property". Nor am I sure which branch name to fall back on. So I'll put this patch up since it worked for what I needed, and perhaps somebody can turn it into whatever it needs to be.
Attachment #8381755 - Flags: feedback?(aki)
(Assignee)

Updated

4 years ago
Assignee: nobody → sphink
Status: NEW → ASSIGNED
Comment on attachment 8381755 [details] [diff] [review]
Allow using an existing Gecko checkout

lgtm :)
Attachment #8381755 - Flags: feedback?(aki) → feedback+
no more b2g
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.