1311199 - Windows builds fail on incubator/stylo repo with 'stage_server' KeyError in query_mach_build_env

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P1)

Description

[Chris Peterson [:cpeterson]]

The incubator/stylo repo [1] is unable to build Firefox on Windows due to a build script error about where `stage_server` is not set [2]. incubator/stylo's Linux, Mac, and Android builds work just fine. I see comments in the build scripts that "stage_server is dictated from build_pool_specifics.py".

[1] https://treeherder.mozilla.org/#/jobs?repo=stylo&selectedJob=16240
[2] https://treeherder.mozilla.org/logviewer.html#?job_id=16267&repo=stylo#L576

18:14:23    FATAL - Uncaught exception: Traceback (most recent call last):
18:14:23    FATAL -   File "Z:\task_1476293750\build\src\testing\mozharness\mozharness\base\script.py", line 2033, in run
18:14:23    FATAL -     self.run_action(action)
18:14:23    FATAL -   File "Z:\task_1476293750\build\src\testing\mozharness\mozharness\base\script.py", line 1972, in run_action
18:14:23    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
18:14:23    FATAL -   File "Z:\task_1476293750\build\src\testing\mozharness\mozharness\base\script.py", line 1912, in _possibly_run_method
18:14:23    FATAL -     return getattr(self, method_name)()
18:14:23    FATAL -   File "Z:\task_1476293750\build\src\testing\mozharness\mozharness\mozilla\building\buildbase.py", line 1618, in build
18:14:23    FATAL -     env.update(self.query_mach_build_env())
18:14:23    FATAL -   File "Z:\task_1476293750\build\src\testing\mozharness\mozharness\mozilla\building\buildbase.py", line 918, in query_mach_build_env
18:14:23    FATAL -     'stage_server': c['stage_server']
18:14:23    FATAL - KeyError: 'stage_server'
18:14:23    FATAL - Running post_fatal callback...
18:14:23 ERROR - setting return code to 2 because fatal was called

Comment 1

[Jonathan Griffin (:jgriffin)]

The taskcluster windows build config is missing 'stage_server'; I'm not sure why. catlee, garndt, any ideas?

Comment 2

[Chris AtLee [:catlee]]

I believe this is failing because the stylo branch isn't listed in branch_specifics.py (https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/builds/branch_specifics.py)

That code doesn't make sense in taskcluster though, so the proper fix is probably to modify buildbase.py so that it stops assuming these config values exist
https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/building/buildbase.py?q=buildbase.py&redirect_type=direct#908

Comment 3

[Greg Arndt [:garndt]]

removing ni? looks like catlee has pointed things in the right direction

Comment 4

[Chris Peterson [:cpeterson]]

Cameron, can you take a look at this bug? catlee described the problem and fix (to the mozharness script) in comment 2. IIUC, mozharness should be fixed in mozilla-central, not just patched in the incubator/stylo repo.

Comment 5

[Chris Peterson [:cpeterson]]

Manish, catlee described this Windows build problem and fix (to the mozharness script) in comment 2. IIUC, mozharness should be fixed in mozilla-central, not just patched in the incubator/stylo repo.

Comment 6

[Manish Goregaokar [:manishearth]]

Attachment: Bug 1311199 - Don't assume `stage_server` config properties exist when creating the mach build environment;

Review commit: https://reviewboard.mozilla.org/r/90750/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/90750/

Comment 7

[Manish Goregaokar [:manishearth]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ef1ce8b7619407049aea8b84a6cb69c7791c3039&selectedJob=30466826 with current patch. Build isn't finished yet, but it's gotten past the step where it was stuck and it's now running the actual build.

Comment 8

[Chris AtLee [:catlee]]

Comment on attachment 8807655 [details]
Bug 1311199 - Don't assume `stage_server` config properties exist when creating the mach build environment;

https://reviewboard.mozilla.org/r/90750/#review90434

Comment 9

[Manish Goregaokar [:manishearth]]

Comment on attachment 8807655 [details]
Bug 1311199 - Don't assume `stage_server` config properties exist when creating the mach build environment;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/90750/diff/1-2/

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5c253ee2e5bc
Don't assume `stage_server` config properties exist when creating the mach build environment; r=catlee

Comment 11

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/5c253ee2e5bc