If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Prevent sm-pkg from failing anytime it can't find some random unrelated file

NEW
Unassigned

Status

()

Core
Build Config
2 months ago
2 months ago

People

(Reporter: ahal, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 months ago
The sm-pkg task generates a package by picking and choosing various directories from the source (including moz.build) and then building with the included moz.build:
https://dxr.mozilla.org/mozilla-central/source/js/src/make-source-package.sh#69

Unfortunately, this means that if you add a file to any of those moz.builds (such as a test manifest) and neglect to remember to also copy that file in make-source-package.sh, you'll get backed out.

Is there some way we could do one of:
A) add a mode to the build system to not fail if stuff is missing
B) package this more sanely
C) automatically run sm-pkg on try anytime any of its included files are modified

I'd prefer A or B, but will gladly settle for C at this point.
(Reporter)

Comment 1

2 months ago
Comment 0 should read:
"by choosing various files and directories from the source"

Which leads to:

D) only copy entire directories into the package, never individual files

and while we're at it:

E) stop using a package and use a source checkout instead
F) Don't add gecko tests to moz.build in ways that affect non gecko builds.

But since you're not saying what you were doing, I can't tell you how you were doing it wrong. ;)
Status: NEW → UNCONFIRMED
Ever confirmed: false
(Reporter)

Comment 3

2 months ago
In this case (bug 1380338) I added a python manifest:
https://hg.mozilla.org/try/diff/a6751ca9f7aa/taskcluster/moz.build

That isn't a gecko test and needs to be read even with a clobbered objdir. In order to land that patch, I need to add the entire taskcluster/taskgraph/test directory to the package.

Also moz.build is just an example. I've been backed out for moving a file out of one of the directories it includes as well.
(Reporter)

Comment 4

2 months ago
I'll admit I might be more likely to hit breakage here than anyone else due to the nature of the things I tend to modify. So maybe this isn't very high priority, but it's a valid complaint.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Well, the whole problem is that the taskcluster directory shouldn't be added to DIRS unconditionally in the first place.
(Reporter)

Comment 6

2 months ago
Some python manifests (like the one in taskcluster/moz.build) need to be parsed unconditionally, they don't depend on any build state. E.g this needs to work:

$ ./mach clobber
$ ./mach python-test

But that's not the whole problem, there are several ways we can break sm-pkg:

1. Move a python module that's added to build/virtualenv_packages.txt
2. Add a test to an unconditional manifest like: [../path/to/test]
3. Reference a file from an included makefile

In my experience, sm-pkg is very fragile and it's hard to remember to run it on try.
You need to log in before you can comment on or make changes to this bug.