Closed Bug 1255686 Opened 4 years ago Closed 4 years ago

SpiderMonkey builds can't access internal tooltool files

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gps, Assigned: gps)

References

Details

Attachments

(3 files)

https://treeherder.mozilla.org/#/jobs?repo=try&revision=46e456744f3a is using an "internal" visibility tooltool file providing the Visual Studio 2015 toolchain. Windows builds are working. SpiderMonkey builds aren't.

nthomas says the tooltool.py invocation is missing the argument to specify the .tok file.
From the mozharness code:

class TooltoolMixin(object):
    def _get_auth_file(self):
        # set the default authentication file based on platform; this
        # corresponds to where puppet puts the token
        if 'tooltool_authentication_file' in self.config:
            fn = self.config['tooltool_authentication_file']
        elif self._is_windows():
            fn = r'c:\builds\relengapi.tok'
        else:
            fn = '/builds/relengapi.tok'

        # if the file doesn't exist, don't pass it to tooltool (it will just
        # fail).  In taskcluster, this will work OK as the relengapi-proxy will
        # take care of auth.  Everywhere else, we'll get auth failures if
        # necessary.
        if os.path.exists(fn):
            return fn

Since --authentication-file isn't getting passed in SpiderMonkey builds, my guess is c:\builds\relengapi.tok doesn't exist on the SpiderMonkey builder machines.
Unfortunately these aren't run using mozharness. The spidermonkey builds are done using this shell script:

http://hg.mozilla.org/build/tools/file/tip/scripts/spidermonkey_builds/spidermonkey.sh
Depends on: 1166420
Depends on: 1251328
I forget what we decided was the best course of action here:
- pass in the path to the authentication token from buildbot
- hardcode the token path in spidermonkey.sh
- hardcode the token path in the tooltool wrapper

Any preferences?
Flags: needinfo?(rail)
IIRC it's:
- hardcode the token path in spidermonkey.sh
Flags: needinfo?(rail)
Attachment #8730307 - Flags: review?(rail) → review+
Comment on attachment 8730307 [details]
MozReview Request: Bug 1255686 - Pass --ttauthfile to spidermonkey.sh; r?rail

https://reviewboard.mozilla.org/r/39795/#review36395
Comment on attachment 8730305 [details]
MozReview Request: Bug 1255686 - Support specifying tooltool authentication file in spidermonkey.sh; r?rail

https://reviewboard.mozilla.org/r/39791/#review36397
Attachment #8730305 - Flags: review?(rail) → review+
lgtm
https://hg.mozilla.org/build/tools/rev/d239061cf51513d8c6c82773d3e24917e8cea89b
Bug 1255686 - Support specifying tooltool authentication file in spidermonkey.sh; r=rail
Assignee: nobody → gps
Status: NEW → ASSIGNED
The previous attempt at this was completely wrong yet somehow didn't
bust automation. We were attempted to use hgtool_args but used
httool_args due to a typo.

In this patch, we establish a variable holding additional arguments to
the tooltool wrapper, populate it with --authentication-file and
pass it to the tooltool invocation.

Review commit: https://reviewboard.mozilla.org/r/39835/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/39835/
Attachment #8730397 - Flags: review?(rail)
Comment on attachment 8730397 [details]
MozReview Request: Bug 1255686 - Pass --authentication-file to tooltool; r?rail

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/39835/diff/1-2/
Attachment #8730397 - Attachment description: MozReview Request: Bug 1255686 - Fix passing --authentication-file to tooltool; r?rail → MozReview Request: Bug 1255686 - Pass --authentication-file to tooltool; r?rail
Comment on attachment 8730397 [details]
MozReview Request: Bug 1255686 - Pass --authentication-file to tooltool; r?rail

https://reviewboard.mozilla.org/r/39835/#review36439
Attachment #8730397 - Flags: review?(rail) → review+
From http://archive.mozilla.org/pub/spidermonkey/try-builds/gszorc@mozilla.com-2351b52c53524e606b417565ae127d861cf2b516/try-win32/try_win32_spidermonkey-compacting-bm79-try1-build628.txt.gz:

 Executing: ['c:\\mozilla-build\\python27\\python.exe', 'C:/mozilla-build/tooltool.py', '--authentication-file', 'c:/builds/relengapi.tok', '--url', 'https://api.pub.build.mozilla.org/tooltool/', '--overwrite', '-m', 'src/browser/config/tooltool-manifests/win32/releng.manifest', 'fetch']
 INFO - Attempting to fetch from 'https://api.pub.build.mozilla.org/tooltool/'...
 INFO - File vs2015u1.zip fetched from https://api.pub.build.mozilla.org/tooltool/ as c:\builds\moz2_slave\try_w32_sm-compacting-00000000\tmpfua6rr
 INFO - File integrity verified, renaming tmpfua6rr to vs2015u1.zip
 INFO - untarring "rustc-beta-i686-pc-windows-msvc.tar.bz2"
 INFO - rm tree: sccache
 INFO - untarring "sccache.tar.bz2"
 INFO - unzipping "vs2015u1.zip" 

This appears to be working now!

Thank you very much, catlee and rail! Hopefully releng is now off the hook for VS2015 work.
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.