1644553 - Perma mozbuild.configure.options.InvalidOptionError: --enable-profile-use is not available in this configuration on Android artifact builds.

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P3)

Description

[Greg Mierzwinski [:sparky]]

It's been several days now that we haven't been able to run raptor android tests on try because of the following error:

[task 2020-06-08T21:56:25.384Z] 21:56:25     INFO -  checking for vcs source checkout... hg
[task 2020-06-08T21:56:25.664Z] 21:56:25     INFO -  checking for a shell... /bin/sh
[task 2020-06-08T21:56:25.760Z] 21:56:25     INFO -  checking for host system type... x86_64-pc-linux-gnu
[task 2020-06-08T21:56:25.760Z] 21:56:25     INFO -  checking for target system type... arm-unknown-linux-androideabi
[task 2020-06-08T21:56:25.888Z] 21:56:25     INFO -  checking whether cross compiling... yes
[task 2020-06-08T21:56:26.004Z] 21:56:26    ERROR -  Traceback (most recent call last):
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -    File "/builds/worker/checkouts/gecko/configure.py", line 181, in <module>
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -      sys.exit(main(sys.argv))
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -    File "/builds/worker/checkouts/gecko/configure.py", line 52, in main
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -      sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 455, in run
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -      self._value_for(option)
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 545, in _value_for
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -      return self._value_for_option(obj)
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/util.py", line 1018, in method_call
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -      cache[args] = self.func(instance, *args)
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/configure/__init__.py", line 613, in _value_for_option
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -      % option_string.split('=', 1)[0])
[task 2020-06-08T21:56:26.004Z] 21:56:26     INFO -  mozbuild.configure.options.InvalidOptionError: --enable-profile-use is not available in this configuration
[task 2020-06-08T21:56:26.080Z] 21:56:26     INFO -  *** Fix above errors and then restart with\
[task 2020-06-08T21:56:26.080Z] 21:56:26     INFO -                 "./mach build"
[task 2020-06-08T21:56:26.080Z] 21:56:26     INFO -  client.mk:111: recipe for target 'configure' failed
[task 2020-06-08T21:56:26.080Z] 21:56:26     INFO -  make: *** [configure] Error 1
[task 2020-06-08T21:56:26.126Z] 21:56:26    ERROR - Return code: 2
[task 2020-06-08T21:56:26.126Z] 21:56:26  WARNING - setting return code to 2

Here's a push with the failure in it: https://treeherder.mozilla.org/#/jobs?repo=try&revision=1036570247d02babc35f58a75f1ccd45ca7d40b7&selectedTaskRun=L05MARpJSB6C1tE5PNpaXw-0

The android shippable tests work on mozilla-central but not on try for some reason. This failure is starting to block our work.

Comment 1

[Ricky Stewart]

You say this has been a thing for "several days". Do you have a regressing commit or at least an example of an as-recent-as-possible try push that succeeded? Finding the cause of this without any context is going to be substantially more difficult.

Comment 2

[Mike Hommey [:glandium]]

It could "just" be that artifact builds were enabled in the local build, and that makes mach try automatically switch to artifact builds. It's likely that artifact pgo builds on Android have always been broken, because the block that enables PGO is not testing for USE_ARTIFACT in mobile/android/config/mozconfigs/common.

Comment 3

[Ricky Stewart]

(I'm currently running non-artifact instances of these builds to test that hypothesis, by the way.)

Comment 4

[Greg Mierzwinski [:sparky]]

It happened sometime between this try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2165037d9e8fa3c583d7733718ee88afb9ea41d5
and this one: https://treeherder.mozilla.org/#/jobs?repo=try&revision=54c585b193ce733a4c06b1d36c873379f484c633

I have a feeling it was caused by the android-shippable change but I'm not sure: https://bugzilla.mozilla.org/show_bug.cgi?id=1623355

Comment 5

[Mike Hommey [:glandium]]

(In reply to Greg Mierzwinski [:sparky] from comment #4)

It happened sometime between this try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2165037d9e8fa3c583d7733718ee88afb9ea41d5

That try has an opt build, and the PGO one was cancelled.

Your last push with a non failed PGO is:
https://treeherder.mozilla.org/#/jobs?repo=try&searchStr=android%2Cpgo&revision=eb33304c3305ee06dda56d30f6853fc080ef0112

And it had artifacts enabled, but the PGO build wasn't actually an artifact build. So yeah, bug 1623355 probably changed that, but actual PGO android artifact builds have always been busted as per comment 2.

There's a secondary problem, which is that when doing artifact builds (which PGO builds on other platforms are), we're still running the instr and run builds, which makes no sense.

Comment 6

[Ricky Stewart]

Yeah, there's no issue if you don't enable artifact builds. The builds here haven't succeeded yet as of this writing but they've already progressed beyond configuration successfully.

(There's probably a usability improvement to be made as per comment 2, so I won't close the bug now.)

Comment 7

[Greg Mierzwinski [:sparky]]

Ok thanks!

Is there a way to force not using artifact builds when pushing to try? I have artifact builds enabled through my mozconfig file which is where its being picked up from.

Comment 8

[Mike Hommey [:glandium]]

task configuration arguments:
  --artifact            Force artifact builds where possible.
  --no-artifact         Disable artifact builds even if being used locally.

Comment 9

[Greg Mierzwinski [:sparky]]

Thanks :glandium!

Comment 10

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1644553: Don't enable profile use for artifact builds r?rstewart

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3387933a2ad2
Don't enable profile use for artifact builds r=glandium,geckoview-reviewers,agi

Comment 12

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/3387933a2ad2