The default bug view has changed. See this FAQ.

Allow to install additional packages in mock

RESOLVED WONTFIX

Status

Release Engineering
General Automation
RESOLVED WONTFIX
3 years ago
2 years ago

People

(Reporter: glandium, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

3 years ago
I had this idea to abuse try to build clang, gcc, and gtk3 packages for tooltool, which would be a more reliable way to do those builds than relying on people doing the right thing (which requires knowledge they might not have).

But building those require a few packages that aren't in our Firefox build environments. It would be nice to have a way to install additional packages in the mock environment.

What I was thinking was to add a build step that would take a manifest file in the tree (path of which would be defined in config.py), and run mock install with the list of packages that manifest contains.
(Reporter)

Comment 1

3 years ago
Created attachment 8445606 [details] [diff] [review]
Allow to install additional packages in mock

I don't feel like going through all the hurdles of testing this on staging, but as long as there is no syntax error, this is a no-op until there is a buildbot-config change, and we can test that by enabling it on a barely used try target (like, shell root analysis)
Attachment #8445606 - Flags: review?(catlee)
(Reporter)

Updated

3 years ago
Assignee: nobody → mh+mozilla
Status: NEW → ASSIGNED
(Reporter)

Comment 2

3 years ago
Created attachment 8445614 [details] [diff] [review]
Allow to install additional packages in mock on try
Attachment #8445614 - Flags: review?(catlee)
(Reporter)

Updated

3 years ago
Attachment #8445606 - Attachment is obsolete: true
Attachment #8445606 - Flags: review?(catlee)
(Reporter)

Updated

3 years ago
Attachment #8445606 - Attachment is obsolete: false
Attachment #8445606 - Flags: review?(catlee)
tempted to WONTFIX this given our efforts to migrate firefox builds to mozharness. it's much easier to do there.

is there an immediate need for this?
(Reporter)

Comment 4

3 years ago
I wanted to setup regular builds of gcc trunk.
(Reporter)

Comment 5

3 years ago
(In reply to Mike Hommey [:glandium] from comment #4)
> I wanted to setup regular builds of gcc trunk.

and it would have helped me when I had to update gtk for gtk3 builds.

Can we go forward here?
Flags: needinfo?(catlee)

Comment 6

2 years ago
(In reply to Mike Hommey [:glandium] from comment #5)
> (In reply to Mike Hommey [:glandium] from comment #4)
> > I wanted to setup regular builds of gcc trunk.
> 
> and it would have helped me when I had to update gtk for gtk3 builds.
> 
> Can we go forward here?

mentioned in irc:

1) if we only care about the try case, we can put in additional mozharness args via the try commit message:

bug ref https://bugzil.la/967630

essentially doing this[1] here[2]

2) have mozharness look for an in-tree config that has its own list of mock packages (I think this is better)

essentially doing something like this totally untested pseudo code:

diff --git a/mozharness/mozilla/mock.py b/mozharness/mozilla/mock.py
index f8587c0..27cfc63 100644
--- a/mozharness/mozilla/mock.py
+++ b/mozharness/mozilla/mock.py
@@ -191,7 +191,16 @@ class MockMixin(object):
             old_packages_hash = None

         if mock_packages is None:
-            mock_packages = list(c.get('mock_packages'))
+            in_tree_mock_pkgs_file = os.path.join(self.query_abs_dirs()['abs_src_dir'],
+                                                  c.get('in_tree_mock_packages_path'))
+            if os.path.exists(in_tree_mock_pkgs_file):
+                # right now setup_mock is called before we download src (at least in the desktop
+                # ff case) so we would have to reverse that or else have in_tree_mock_packages_path
+                # be a raw file url that we download ahead of src download e.g.
+                "https://hg.mozilla.org/{branch}/raw-file/{rev}/browser/config/mock_packages.py'
+                mock_packages = self.parse_config_file(in_tree_mock_pkgs_file)['mock_packages']
+            else:
+                mock_packages = list(c.get('mock_packages'))

         package_list_hash = hashlib.new('sha1')
         if mock_packages:

would that solve your issue?
    

[1] http://mxr.mozilla.org/build/source/mozharness/mozharness/mozilla/testing/talos.py?rev=6412b5ff437e#199
[2] http://mxr.mozilla.org/build/source/mozharness/mozharness/mozilla/mock.py#194
(Reporter)

Comment 7

2 years ago
The latter seems more interesting, because it could allow to more generally move the list in-tree, which would allow to unclutter those lists, too.
we should be using task cluster for gtk3.
Flags: needinfo?(catlee)
Comment on attachment 8445606 [details] [diff] [review]
Allow to install additional packages in mock

Review of attachment 8445606 [details] [diff] [review]:
-----------------------------------------------------------------

I don't want more stuff going into buildbot factories that we don't require.
Attachment #8445606 - Flags: review?(catlee) → review-

Updated

2 years ago
Attachment #8445614 - Flags: review?(catlee) → review-
(Reporter)

Comment 10

2 years ago
Note that when this bug was filed and the patch attached a year ago, buildbot factories were the only place where this could be hooked.
(Reporter)

Updated

2 years ago
Assignee: mh+mozilla → nobody
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.