Migrate Android SDK to toolchain dependencies

RESOLVED FIXED in mozilla58

Status

enhancement
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: nalexander, Assigned: nalexander)

Tracking

(Blocks 1 bug)

unspecified
mozilla58
Dependency tree / graph

Details

Attachments

(3 attachments)

This ticket tracks turning the Android SDK bits of the Deps task into a proper toolchain task.
Blocks: 1405413
It'll be helpful to have Bug 1406196 out of the way while doing this work.
Depends on: 1406196
Comment on attachment 8916133 [details]
Bug 1405412 - Pre: Allow toolchain task images to not cache tc-vcs.

https://reviewboard.mozilla.org/r/187378/#review192814

::: taskcluster/taskgraph/transforms/job/toolchain.py:51
(Diff revision 1)
>          'public',
>          'internal',
>      ),
>  
> +    # If true, tc-vcs will be enabled.  Not supported on Windows.
> +    Required('tc-vcs', default=True): bool,

Any chance we could default this to False?
Attachment #8916133 - Flags: review?(dustin) → review+
Comment on attachment 8916134 [details]
Bug 1405412 - Migrate Android SDK to android-sdk-linux toolchain task.

https://reviewboard.mozilla.org/r/187380/#review192816

this is so cool :)
Attachment #8916134 - Flags: review?(dustin) → review+
Comment on attachment 8916137 [details]
Bug 1405412 - Post: Remove JDK repackaging script.

https://reviewboard.mozilla.org/r/187382/#review192818
Attachment #8916137 - Flags: review?(dustin) → review+
Comment on attachment 8916133 [details]
Bug 1405412 - Pre: Allow toolchain task images to not cache tc-vcs.

https://reviewboard.mozilla.org/r/187378/#review192814

> Any chance we could default this to False?

I think we could, but I don't have a good way of discovering the (few? zero?) cases where it's actually required.  I think I'd rather leave it as it is now (to reduce risk), and then whoever finally excises "tc.?vcs" from the tree can kill it here too.
Assignee: nobody → nalexander
Status: NEW → ASSIGNED
Pushed by nalexander@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5bd941606a64
Pre: Allow toolchain task images to not cache tc-vcs. r=dustin
https://hg.mozilla.org/integration/autoland/rev/5f4ff0de7242
Migrate Android SDK to android-sdk-linux toolchain task. r=dustin
https://hg.mozilla.org/integration/autoland/rev/44f885bc7c22
Post: Remove JDK repackaging script. r=dustin
Backed out bug 1352599 and bug 1405412 for failing build at python/mozbuild/mozbuild/test/configure/lint.py::Lint::test_mobile_android:

bug 1352599
https://hg.mozilla.org/integration/autoland/rev/6c9706f9f534eb684abbb11c72cdb6f31d94c272
https://hg.mozilla.org/integration/autoland/rev/fbc942d188228d69b3f6d33eeaa45cce2a9e808a
https://hg.mozilla.org/integration/autoland/rev/64a408bde7d2f24606f0806ce36f27480251a442
https://hg.mozilla.org/integration/autoland/rev/ca38508b320378f568c295da00c90ae6edcd9ffd

bug 1405412
https://hg.mozilla.org/integration/autoland/rev/11376fd5366ebe76c926e2e919245e5039c0b9e4
https://hg.mozilla.org/integration/autoland/rev/a6fab9d472152a3582869bac6088eb31b2957403
https://hg.mozilla.org/integration/autoland/rev/0e81b421af3564b89e2a9a646af24d83c36ba9fc

Latest push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=cb5f4236d080bb250f7f485a584216f014a18fba&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable
Build log: https://treeherder.mozilla.org/logviewer.html#?job_id=136603646&repo=autoland

[task 2017-10-12T21:17:52.266Z] 21:17:52  WARNING - ../python/mozbuild/mozbuild/test/configure/lint.py::Lint::test_mobile_android TEST-UNEXPECTED-FAIL
[task 2017-10-12T21:17:52.266Z] 21:17:52     INFO - =================================== FAILURES ===================================
[task 2017-10-12T21:17:52.266Z] 21:17:52     INFO - ___________________________ Lint.test_mobile_android ___________________________
[task 2017-10-12T21:17:52.266Z] 21:17:52     INFO - self = <lint.Lint testMethod=test_mobile_android>
[task 2017-10-12T21:17:52.266Z] 21:17:52     INFO -     def test(self):
[task 2017-10-12T21:17:52.267Z] 21:17:52     INFO - >       return func(self, project)
[task 2017-10-12T21:17:52.267Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/test/configure/lint.py:26:
[task 2017-10-12T21:17:52.267Z] 21:17:52     INFO - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2017-10-12T21:17:52.267Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/test/configure/lint.py:58: in lint
[task 2017-10-12T21:17:52.267Z] 21:17:52     INFO -     sandbox.run(os.path.join(topsrcdir, 'moz.configure'))
[task 2017-10-12T21:17:52.268Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/lint.py:35: in run
[task 2017-10-12T21:17:52.268Z] 21:17:52     INFO -     self.include_file(path)
[task 2017-10-12T21:17:52.268Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:389: in include_file
[task 2017-10-12T21:17:52.268Z] 21:17:52     INFO -     exec_(code, self)
[task 2017-10-12T21:17:52.268Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/util.py:59: in exec_
[task 2017-10-12T21:17:52.269Z] 21:17:52     INFO -     exec(object, globals, locals)
[task 2017-10-12T21:17:52.269Z] 21:17:52     INFO - ../moz.configure:225: in <module>
[task 2017-10-12T21:17:52.269Z] 21:17:52     INFO -     include(include_project_configure)
[task 2017-10-12T21:17:52.269Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:693: in include_impl
[task 2017-10-12T21:17:52.269Z] 21:17:52     INFO -     self.include_file(what)
[task 2017-10-12T21:17:52.270Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:389: in include_file
[task 2017-10-12T21:17:52.270Z] 21:17:52     INFO -     exec_(code, self)
[task 2017-10-12T21:17:52.270Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/util.py:59: in exec_
[task 2017-10-12T21:17:52.270Z] 21:17:52     INFO -     exec(object, globals, locals)
[task 2017-10-12T21:17:52.270Z] 21:17:52     INFO - ../mobile/android/moz.configure:130: in <module>
[task 2017-10-12T21:17:52.271Z] 21:17:52     INFO -     include('../../build/moz.configure/java.configure')
[task 2017-10-12T21:17:52.271Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:693: in include_impl
[task 2017-10-12T21:17:52.271Z] 21:17:52     INFO -     self.include_file(what)
[task 2017-10-12T21:17:52.271Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:389: in include_file
[task 2017-10-12T21:17:52.271Z] 21:17:52     INFO -     exec_(code, self)
[task 2017-10-12T21:17:52.272Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/util.py:59: in exec_
[task 2017-10-12T21:17:52.272Z] 21:17:52     INFO -     exec(object, globals, locals)
[task 2017-10-12T21:17:52.272Z] 21:17:52     INFO - ../build/moz.configure/java.configure:86: in <module>
[task 2017-10-12T21:17:52.272Z] 21:17:52     INFO -     help='Path to proguard.jar')
[task 2017-10-12T21:17:52.272Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/lint.py:121: in option_impl
[task 2017-10-12T21:17:52.272Z] 21:17:52     INFO -     result = super(LintSandbox, self).option_impl(*args, **kwargs)
[task 2017-10-12T21:17:52.273Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:610: in option_impl
[task 2017-10-12T21:17:52.273Z] 21:17:52     INFO -     kwargs = {k: self._resolve(v) for k, v in kwargs.iteritems()
[task 2017-10-12T21:17:52.273Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:611: in <dictcomp>
[task 2017-10-12T21:17:52.273Z] 21:17:52     INFO -     if k != 'when'}
[task 2017-10-12T21:17:52.273Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/__init__.py:463: in _resolve
[task 2017-10-12T21:17:52.274Z] 21:17:52     INFO -     need_help_dependency)
[task 2017-10-12T21:17:52.274Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/util.py:944: in method_call
[task 2017-10-12T21:17:52.274Z] 21:17:52     INFO -     cache[args] = self.func(instance, *args)
[task 2017-10-12T21:17:52.274Z] 21:17:52     INFO - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2017-10-12T21:17:52.275Z] 21:17:52     INFO - self = {'msvs_version': <mozbuild.configure.SandboxDependsFunction object at 0x7f55f4...android': <mozbuild.configure.SandboxDependsFunction object at 0x7f55f40e29d0>}
[task 2017-10-12T21:17:52.275Z] 21:17:52     INFO - obj = <mozbuild.configure.DependsFunction proguard_jar_default()>
[task 2017-10-12T21:17:52.275Z] 21:17:52     INFO - need_help_dependency = True
[task 2017-10-12T21:17:52.275Z] 21:17:52     INFO -     @memoize
[task 2017-10-12T21:17:52.276Z] 21:17:52     INFO -     def _value_for_depends(self, obj, need_help_dependency=False):
[task 2017-10-12T21:17:52.276Z] 21:17:52     INFO -         with_help = self._help_option in obj.dependencies
[task 2017-10-12T21:17:52.276Z] 21:17:52     INFO -         if with_help:
[task 2017-10-12T21:17:52.276Z] 21:17:52     INFO -             for arg in obj.dependencies:
[task 2017-10-12T21:17:52.276Z] 21:17:52     INFO -                 if self._missing_help_dependency(arg):
[task 2017-10-12T21:17:52.277Z] 21:17:52     INFO -                     raise ConfigureError(
[task 2017-10-12T21:17:52.277Z] 21:17:52     INFO -                         "`%s` depends on '--help' and `%s`. "
[task 2017-10-12T21:17:52.277Z] 21:17:52     INFO -                         "`%s` must depend on '--help'"
[task 2017-10-12T21:17:52.277Z] 21:17:52     INFO -                         % (obj.name, arg.name, arg.name))
[task 2017-10-12T21:17:52.277Z] 21:17:52     INFO -         elif ((self._help or need_help_dependency) and
[task 2017-10-12T21:17:52.278Z] 21:17:52     INFO -               self._missing_help_dependency(obj)):
[task 2017-10-12T21:17:52.278Z] 21:17:52     INFO -             raise ConfigureError("Missing @depends for `%s`: '--help'" %
[task 2017-10-12T21:17:52.278Z] 21:17:52     INFO - >                                obj.name)
[task 2017-10-12T21:17:52.278Z] 21:17:52     INFO - E           ConfigureError: Missing @depends for `proguard_jar_default`: '--help'
[task 2017-10-12T21:17:52.279Z] 21:17:52     INFO - ../python/mozbuild/mozbuild/configure/lint.py:116: ConfigureError
Flags: needinfo?(nalexander)
Pushed by nalexander@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/bbc2354236d3
Pre: Allow toolchain task images to not cache tc-vcs. r=dustin
https://hg.mozilla.org/integration/autoland/rev/d4d6cad20606
Migrate Android SDK to android-sdk-linux toolchain task. r=dustin
https://hg.mozilla.org/integration/autoland/rev/27213145f391
Post: Remove JDK repackaging script. r=dustin
Comment on attachment 8916133 [details]
Bug 1405412 - Pre: Allow toolchain task images to not cache tc-vcs.

https://reviewboard.mozilla.org/r/187378/#review194606

::: taskcluster/taskgraph/transforms/job/toolchain.py:51
(Diff revision 3)
>          'public',
>          'internal',
>      ),
>  
> +    # If true, tc-vcs will be enabled.  Not supported on Windows.
> +    Required('tc-vcs', default=True): bool,

bwarf, just when I was testing a patch that entirely removes tc-vcs support (nothing uses it anymore afaict).
Yeah, that's why I wanted default=false.
Clearing NI since this has landed \o/  Sorry about the tc-vcs, folks -- somebody else can figure out if it's used anywhere.
Flags: needinfo?(nalexander)
You need to log in before you can comment on or make changes to this bug.