Closed Bug 1406536 Opened 4 years ago Closed 4 years ago

Create New Product for Build Config and move existing components

Categories

(bugzilla.mozilla.org :: Administration, task, P1)

Production

Tracking

()

RESOLVED FIXED
Due Date:

People

(Reporter: emceeaich, Assigned: dylan)

References

Details

User Story

Proposed Reorganization of Build / Tools / Automation Bug Components

New Product: Firefox Build System

Components:

General    <- Core :: Build Config

    File bugs here for general Firefox build system issues. This includes problems running `mach build`, `mach configure`, `mach package`, `mach artifact`, and other mach commands related to building Firefox. This component also tracks issues related to moz.build and make files.


General: Unsupported Platforms      (new component - to help us better isolate bugs)

    Like the "Build System" component but for issues related to platforms not officially supported by Mozilla's infrastructure (e.g. FreeBSD, NetBSD, non-MacOS Darwin, etc).

Task Configuration     <- TaskCluster :: Task Configuration

    Tracks changes to how Firefox continuous integration (CI) tasks run. This includes build and testing related tasks. If you want to request a change to how some part of Firefox automation runs, this is a good place to file a bug. (Automation could mean many things, whereas task evokes taskcluster)


Mach Core      <- Core :: Mach

    Tracking bugs and feature requests for mach: a generic CLI dispatching tool. *Issues with specific `mach` commands used to develop Firefox should be reported elsewhere. e.g. issues with `mach build` should be filed in the "Build System" component.


Bootstrap Configuration           (new component)

    Tracks issues related to the various tools that attempt to bootstrap and configure a system to optimally build and develop Firefox. This includes the standalone "mozboot" bootstrap.py script, `mach bootstrap`, `mach mercurial-setup`, and `mach doctor`.


Source Code Analysis  <- Core :: Rewriting and Analysis

    For issues related to static analysis tools (e.g. clang-analyze). Feature requests for source code analysis tools can also be filed here.

    Triage owner: sylvestre and mystor 


Lint and Formatting <- Testing :: Lint

    Issues related to linting tools (e.g flake8, eslint) and code formatting tools (clang-format). Feature requests for these tools can also be filed here

    Triage owner Sylvestre and ahal(?) and Standard8


Generated Documentation       (new component)

    For issues related to documentation generated from in-repository content. This covers the Sphinx documentation (`mach doc` and https://firefox-source-docs.mozilla.org/).


Developer Environment Integration          (new component - I think)

    For issues related to integrating Firefox development into other tools. e.g. Visual Studio and Eclipse integration. gdb and rr integration. Issues developing with the Emacs operating system.

Attachments

(1 file)

Build Config Product

Rec Reviewers: the alias account build-config-reviews@mozilla.bugs

Security Groups: 

Port over the groups from Core and FFx

Components

Core
Firefox (may be combined with Core)
Unsupported Platforms
Thunderbird
Sea Monkey

Rationale is to get build system bugs out of the browser components to make assessment easier
If this meets with all yall's okay, I'd like to get this cued up.
Flags: needinfo?(gps)
Flags: needinfo?(catlee)
I've said it many times before and I'll say it again "Core" and "Build Config" are not great names. If we're going to go through the trouble of renaming Core :: Build Config, I'd strongly prefer to ditch both "Core" and "Build Config." "Firefox" or "Firefox Desktop" is a better name for "Core" and "Build System" is better than "Build Config."

If we're going to establish a new Bugzilla product, I'd also encourage us to think long and hard about consolidating some other components with significant overlap with the build system. Mainly TaskCluster :: Task Configuration. I think there is general consensus that the Task Configuration component is a dumping grounds of random and it should exist outside of the TaskCluster product because it has little to do with the TaskCluster platform itself.

I'd like to propose a "[Firefox] Build and Automation" product with the following components:

* Bootstrap
* Build System
* Automation Configuration (or CI Configuration)
* Mach
* Build System (Unsupported Platforms)
* Thunderbird
* Sea Monkey

... or something along those lines.

I'd also love to have a central component where people can file "workflow" bugs without having to decipher which of the few dozen components is most appropriate. But I don't want to scope bloat this bug too much...
Flags: needinfo?(gps)
Big +1

I'd suggest renaming the `Mach` one to `Mach Core` (which I know you just said is a bad name, but otherwise every problem with any mach command will be filed there).

We could also move Testing::Lint (and maybe Testing::Mozbase) to this new product, but I also don't want to scope bloat. If it gets made, I'd be happy to file follow-up bugs.
Yes, "mach" should probably get renamed to "Mach Core" to denote that it is supposed to be generic.

I also think linting and mozbase would belong in the new component.

mozharness could probably go there too.

How about a "Firefox Development Workflow" product? Although that would invite a larger discussion around consolidating code review, version control, and automation components/products. I'd be in favor of this consolidation. But I'm not sure how wed people are to the components. Unfortunately, I'm not seeing an obvious way to carve out a product that includes build and related tools while excluding overlapping functionality. The 2 tier product-component and bug is-a single component of Bugzilla just doesn't translate well to the reality of build, automation, and tools - which frequently have issues that span multiple components and can't be shoehorned into a strict classification ontology.

Given that Bugzilla is a critical gateway to reporting productivity-draining workflow problems, I think the most user friendly thing to do is have a single Product holding components related to Firefox development workflow. If nothing else, maybe we use that Product as a triage queue and move bugs to other components in different products. But at least we'd be able to point people at a single product for reporting their "I can't hack on Firefox" issue.
imho "Firefox Development Workflow" is a bit of an oddball name; it's extremely broad and overlaps with other established products.  for example, bugzilla itself is part of the development workflow, however i wouldn't want it to be shifted to be a component under a workflow product.

a _component_ with that name which is used as a default bucket for triaging workflow related issues makes sense.
if we take that route we should expose it on the top level of the two bug entry forms.

a random/weird plan worth considering may be to make use of classifications, which allows for grouping of products.  this feature is not really used on bmo, resulting in it being minimally exposed in the ui.  it may be useful to split "firefox products" from "developer workflow products" and other things.

we could then work on the ui to bring some more focus on product classifications in an effort to simplify bug reporting (eg. allow users to report bugs by selecting just the high level classification, which automatically picks a triage product+component).

</massive_scope_creep>

+1 to a "Firefox Build and Automation" product with gps's components.
I'd like to get :gps' proposed organization out to dev-planning for comment, and then :dylan and I can implement.
Also, do we need to wait until after 57 to make this change?
(In reply to Emma Humphries β˜•οΈ (she/her) [:emceeaich] (UTC-8) +needinfo me from comment #7)
> I'd like to get :gps' proposed organization out to dev-planning for comment,
> and then :dylan and I can implement.

I tossed together https://public.etherpad-mozilla.org/p/build-bug-components as a more concrete proposal. Please help me wordsmith it!(In reply to Emma Humphries β˜•οΈ (she/her) [:emceeaich] (UTC-8) +needinfo me from comment #8)

> Also, do we need to wait until after 57 to make this change?

I'm not sure what the potential impact would be. But from where I sit I don't see a need to wait.
> Automation Configuration (or CI Configuration)

This would be the new home for `Taskcluster :: Task Configuration`, right?
Yes.


Automation Task Configuration     <- TaskCluster :: Task Configuration

    Tracks changes to how Firefox continuous integration (CI) tasks run. This includes build and testing related tasks. If you want to request a change to how some part of Firefox automation runs, this is a good place to file a bug. (Automation could mean many things, whereas task evokes taskcluster)
Discussed in dev-planning (and related) groups.

https://groups.google.com/forum/#!topic/mozilla.dev.planning/tTB-vZsllic

Okay to schedule this. 

Will file bugs for triage owners for components.
Assignee: nobody → dylan
Flags: needinfo?(dylan)
Duplicate of this bug: 1282143
Is this still moving along?  "Automation Task Configuration" is now officially a submodule of "Build Config", so I'd like to move it out of the Taskcluster product!  Anything I can do to help?
So sorry! I can do this tonight.
Status: NEW → ASSIGNED
I need the user story to reflect exactly which actions should be taken for this bug -- having to read over comments would make this more error prone.
Flags: needinfo?(ehumphries)
Not to sound like a broken record [1], but is this still moving along?

[1] "Records" were flat pieces of vinyl that encoded sounds in the texture on the surface, similar to a CD[2].  When "broken" they tended to repeat the same sound over and over.
[2] "CDs" were flat pieces of plastic that encoded sounds in a digital format in a metallized layer.
Priority: -- → P1
this is taking a bit longer because keeping a vpn connection open is apparently very hard for the latest version of OSX.
Since Build System is in the name, perhaps "Build System" component should be "General"?
"Firefox Build System and Automation"
Flags: needinfo?(ehumphries)
(In reply to Dylan Hardison [:dylan] (he/him) from comment #20)
> Since Build System is in the name, perhaps "Build System" component should
> be "General"?
> "Firefox Build System and Automation"

Yes.
Flags: needinfo?(ehumphries)
I'm asking for kmoir's opinion on the new product -- I know Testing :: , Task Cluster ::, Core :: are keen to get these components out of their products, but the suggested configuration should also be OK'd by the build system-y people.
Comment 21 + the user story seems good to me.
I'm going to block out a few hours friday morning to do this.
Due Date: 2018-03-02
Really the afternoon, but here we go.
User Story: (updated)
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Small regression: we hid tracking flags.
And if anyone changed the bugs, the tracking flags were removed.

I wipped up some sql and ran it. It seems fine now.

https://gist.github.com/dylanwh/504f31aac886d8fe6d8108ce5adf1361 is all tracking flags that changed today, just to check.
Attached file the sql I ran today β€”
Depends on: 1450995
Depends on: 1476102
You need to log in before you can comment on or make changes to this bug.