Thunderbird is missing mozharness pinning

RESOLVED FIXED

Status

Release Engineering
Release Automation: Other
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: Fallen, Assigned: Fallen)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(5 attachments)

(Assignee)

Description

3 years ago
Thunderbird is not configured for mozharness pinning, therefore we always use production, which started failing recently.
(Assignee)

Comment 1

3 years ago
Created attachment 8602355 [details] [diff] [review]
Fix - v1

This might fix it. Unfortunately its seems there is no buildbot property with the mozilla revision, so I will use the default branch for getting the file.
Attachment #8602355 - Flags: review?(rail)
Comment on attachment 8602355 [details] [diff] [review]
Fix - v1

lgtm
Attachment #8602355 - Flags: review?(rail) → review+
(Assignee)

Comment 3

3 years ago
https://hg.mozilla.org/build/buildbot-configs/rev/285aa773a8aa
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Created attachment 8602828 [details] [diff] [review]
thunderbird-import

This should fix the test failures:

Traceback (most recent call last):
  File "/home/travis/build/mozilla/build-buildbot-configs/.tox/braindump/buildbot-related/dump_master_json.py", line 111, in dump_master
    c = loadMaster(path)
  File "/home/travis/build/mozilla/build-buildbot-configs/.tox/braindump/buildbot-related/dump_master_json.py", line 26, in loadMaster
    execfile(path, g, g)
  File "/home/travis/build/mozilla/build-buildbot-configs/test-output/bm81-build_scheduler/master.cfg", line 6, in <module>
    import thunderbird_config
  File "/home/travis/build/mozilla/build-buildbot-configs/test-output/bm81-build_scheduler/thunderbird_config.py", line 1066, in <module>
    for _, branch in items_at_least(BRANCHES, 'gecko_version', 30):
NameError: name 'items_at_least' is not defined
Attachment #8602828 - Flags: review?(rail)

Updated

3 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #8602828 - Flags: review?(rail) → review+

Comment 7

3 years ago
I don't see any evidence this is working in current comm-esr38 or comm-beta builds.
I've lost my initial analysis.

I think we don't use ScriptFactory for TB builds (bug NightlyScriptFactory), hence, pinning cannot be used.
For test jobs we should be able to fix it by adding that same block to mozilla-tests/thunderbid_config.py

rail: does this analysis make sense to you?

misc.py changes => http://hg.mozilla.org/build/buildbotcustom/diff/cb6ce28989c2/misc.py

Original bug: bug 1110286

armenzg@armenzg-thinkpad:~/repos/hg/buildbotcustom$ grep -r "script_repo_manifest" .
./process/factory.py:    def __init__(self, scriptRepo, scriptName, script_repo_manifest=None,
./process/factory.py:            if script_repo_manifest:
./process/factory.py:                        '--manifest-url %s' % script_repo_manifest)],
./process/factory.py:            if script_repo_manifest:
./process/factory.py:                        '--manifest-url %s' % script_repo_manifest)],
./misc.py:        script_repo_manifest=config.get('script_repo_manifest'),
./misc.py:                        script_repo_manifest=None, is_debug=None):
./misc.py:            script_repo_manifest=script_repo_manifest,
./misc.py:                                   script_repo_manifest=None):
./misc.py:        script_repo_manifest=script_repo_manifest,
./misc.py:                            'script_repo_manifest': branch_config.get(
./misc.py:                                 'script_repo_manifest'),
./misc.py:                                test_builder_kwargs['script_repo_manifest'] = branch_config.get('script_repo_manifest')
(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #8)
> I've lost my initial analysis.
> 
> I think we don't use ScriptFactory for TB builds (bug NightlyScriptFactory),
> hence, pinning cannot be used.

Sounds correct to me. We have not switched to mozharness based builds for TB.

Comment 10

3 years ago
So what is the next step forward, and who is going to do it?

Comment 11

3 years ago
(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #8)

> I think we don't use ScriptFactory for TB builds (bug NightlyScriptFactory),
> hence, pinning cannot be used.

I'm not an expert here, but I don't understand that conclusion from my reading of the code.

yes 'mozharness_desktop_build' is not set for Thunderbird builds, but 'mozharness_config' is (for example http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/thunderbird_config.py#66)

If 'mozharness_config' is set, then the factory we use is:

            factory = makeMHFactory(config, pf,
                                    signingServers=secrets.get(pf.get('dep_signing_servers')),
                                    use_credentials_file=True)

which uses ScriptFactory.

Or is there something I am missing here?
The change landed on this bug was for *mozilla*/thunderbid_config.py (that entails builds) and *mozilla-tests*/thunderbid_config.py does not contain for/loop, hence, not activating the pinning for *test* jobs.

Steps:
* Back out previous change
* Land the same change but for mozilla-tests

Outcome:
* Pinning for test jobs will be activated

Comment 13

3 years ago
The original change was done IIRC to solve some issues with the Lightning repacks. Do we really need to back out the original change, or can the code be in both mozilla/ and mozilla-tests/ ?
It can be in both.
We should, however, add a comment saying that the pinning block there does not affect TB builds.

Comment 15

3 years ago
Sorry to keep bugging you, but one more question.

The original patch only uses the default mozilla-* rev, while in practice we typically build from a branch. Is there any way from within the thunderbird_config.py code to get access to current mozilla-* rev used for the build?

Comment 16

3 years ago
Created attachment 8615514 [details] [diff] [review]
add mozharness pinning in mozilla-test thunderbird configurations

I don't believe that we need to deal with the issue of the mozilla branch at the moment, so let's try to get this in so that we can proceed with our release.

Thanks for all of your assistance!
Attachment #8615514 - Flags: review?(armenzg)
Attachment #8615514 - Flags: review?(armenzg) → review+
I have not touched these code often in the last year so my answers can be very limited.

In Firefox, we substitute both the repo_name and gecko revision:
https://hg.mozilla.org/%(repo_path)s/raw-file/%(revision)s/testing/mozharness/mozharness.json'

I don't know if it would be substituted for the TB jobs.

Let's start with default and see.

Comment 18

3 years ago
Comment on attachment 8615514 [details] [diff] [review]
add mozharness pinning in mozilla-test thunderbird configurations

https://hg.mozilla.org/build/buildbot-configs/rev/664bea6866b7
Attachment #8615514 - Flags: checked-in+

Comment 19

3 years ago
Created attachment 8615775 [details] [diff] [review]
define moz_repo_path in testing

xpcshell test runs failed because of missing moz_repo_path  I believe this is what is needed to define them.
Attachment #8615775 - Flags: review?(armenzg)
Attachment #8615775 - Flags: review?(armenzg) → review+
Thanks rail for taking the review! I was 1/2 day out.
Created attachment 8616218 [details] [diff] [review]
moz_repo_path.diff

Maybe?
Attachment #8616218 - Flags: feedback?(rail)
Comment on attachment 8616218 [details] [diff] [review]
moz_repo_path.diff

Yeah, something like that. Not sure if that's the right place though. There are some other places where the properties set.
Attachment #8616218 - Flags: feedback?(rail) → feedback+
Attachment #8616218 - Flags: review+
(Assignee)

Comment 26

3 years ago
Has been in production for a while, looks like this is good. Marking fixed.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.