1126534 - add local developer support for desktop mozharness builds

Status: RESOLVED INACTIVE

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Jordan Lund (:jlund)]

like tests, mh desktop builds should be able to avail of something akin to developer-config.py

As fx_desktop_build.py is recent, it has only been used in production. It would be nice to have the ability to run this script locally or via slave loan.

this bug will track that effort

Comment 1

[Jordan Lund (:jlund)]

Bryan, thanks for bearing this and trying to hack around.

I took a look at your pastebin changes[1] and it seems you have found a way to get past a few hurdles. I'm hoping that none of those changes will cripple you later down the line but let's come to that as we hit new issues.

First, we need to solve the issue of you not being able to hit tooltool. Armen has filed https://bugzil.la/1126503 to track a lessy hacky permanent fix but let's try to see if we can get it to work conceptually for your work in 1122859.

Essentially: we need to add this to ToolToolMixin[2]

More verbosely, we need to take parts of[3]

the files/changes we care about are:
   - configs/developer_config.py (most of this file helps test jobs but some of these could help)
   - mozharness/lib/python/authentication.py
   - mozharness/mozilla/tooltool.py


this means you'll want to try running with developer_config.py again.

applying the above will allow you to enter a LDAP username/password when tooltool tries to fetch files. Bear in mind you'll want to ensure the credential file it creates is removed when you're done running the script. :)

With maybe some extra tweaks, this should solve the ToolTool issue. There are two other areas I know off the bat that could be troublesome. 1) mock 2) 'mach build' (the compile step)

wrt - mock, have you gotten past that hurdle? It looks like you may have talked a bit to coop about this.

wrt 'mach build', this is the step that actually compiles ff but, in automation, we get it to run a number of production steps. You'll likely want to turn these off if you're only interested in compiling and not running the following targets: 
    make buildsymbols, make uploadsymbols, make package, make package-tests, make installer, create complete mar, create partial mar, make upload, l10n-check

To turn off those automation steps, you need to remove the following lines (depending on the variant of linux your running)[4]

Note: you only need to remove MOZ_AUTOMATION from releng_base_linux_32_builds.py or releng_base_linux_64_builds.py if you only care about generic opt builds.

Lot's of info here. I'm happy to help where needed.  

[1] http://pastebin.ubuntu.com/9904201/
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1126503#c0
[3] https://bug1086672.bugzilla.mozilla.org/attachment.cgi?id=8548226
[4] http://mxr.mozilla.org/build/search?string=MOZ_AUTOMATION&find=configs%2Fbuild&findi=&filter=%5E%5B%5E%5C0%5D*%24&hitlimit=&tree=build

Comment 2

[Bryan Quigley]

>applying the above will allow you to enter a LDAP username/password when tooltool tries to fetch files. 

AFAIK I don't have one of those,  I'm connecting via SSH directly, not the VPN.  

wrt - mock, have you gotten past that hurdle?
I used something like mock_mozilla --init -r config-templates/mozilla-centos6-x86_64

Since I've gotten to the point of having mozilla-central checked out, would there be a negative to try just building it from within the mock_mozilla shell using mock?  Then if successful, I'd just be working on adding the gstreamer1.0 to that build, which might not depend on any other parts of the build process.

Comment 3

[Jordan Lund (:jlund)]

> Since I've gotten to the point of having mozilla-central checked out, would
> there be a negative to try just building it from within the mock_mozilla
> shell using mock?  Then if successful, I'd just be working on adding the
> gstreamer1.0 to that build, which might not depend on any other parts of the
> build process.

no harm in skipping any steps and yielding a successful build. As long as you can get to the point of adding what you need to the build, go for it! The negative will be this won't replicate what we do in production but at this testing stage, it doesn't seem like a hard requirement.