Closed Bug 1460487 Opened 3 years ago Closed 2 years ago

about:buildconfig shows wrong source


(Thunderbird :: Build Config, defect)

Not set


(Not tracked)

Thunderbird 64.0


(Reporter: t.matsuu, Assigned: rjl)




(3 files, 3 obsolete files)

Daily shows wrong source URL in about:buildconfig.
The URL is generated with base URL for comm-central and revision changeset ID of mozilla-central.

comm-central with comm-central changeset ID is expected (and additional mozilla-central changeset URL would be great).

about:buildconfig in Daily (Build ID: 20180508100117):
Built from
Output .txt files at also have the same issue.
comm-central URL is generated with base URL for comm-central and revision changeset ID of mozilla-central.

In addition, mozilla source URL is from mozilla-unified, not mozilla-central.
Is this right repository?
Jorg K,

If we need some work in comm-central side, I'll set this bug as blocker of bug 456360.
Otherwide I'll set this bug as dup. of bug 456360.

What do you think?
Flags: needinfo?(jorgk)
Sadly I'm not familiar with build issues at all. I'll NI Tom who can pass on the task to the build/release engineer we're in the process of hiring.
Flags: needinfo?(jorgk) → needinfo?(mozilla)
Flags: needinfo?(mozilla) → needinfo?(rob)
It looks like the .txt files come from toolkit/mozapps/installer/ (I think). There's some MOZ_SOURCE_CHANGESET in there that I think comes out of an environment variable of the same name. There's also MOZ_SOURCE_REPO that's pointing to comm-central, but the CHANGESET value is M-C's changeset. Those originate in comm/
I need to play with this some more and figure out what to change, but those are the pieces at least.
The 'SourceRepository' and 'SourceStamp' entries in "application.ini" and "platform.ini" are also the same.

This might be related.

"platform.ini" should point to Mozilla-Central and
"application.ini" should point to Comm-Central.

However, both SourceRepository entries point to Comm-Central and both SourceStamp refer to Mozilla-Central
I think once bug 1491907 is fixed, this will fall into place.
Flags: needinfo?(rob)
Depends on: 1491907
Assignee: nobody → rob
about:buildconfig on Thunderbird shows an incorrect mercurial source URL
that's a comm- repo, but a mozilla- changeset revision.

For Taskcluster builds at least, MOZ_SOURCE_CHANGESET is set as an environment
variable by mozharness. During the build, build/ writes out
source-repo.h with a few defines, including MOZ_SOURCE_URL that gets picked
up by buildconfig.html.
Attachment 9012763 [details] addresses most of the issues raised in this bug.

- about:buildconfig now shows the correct mercurial URL for the build. It's only showing the comm- information, not the mozilla- information. Getting the mozilla- changeset information would require making mozharness/ smart enough to know that there are two repositories in play when doing a comm- build. source-repo.h would need to be updated accordingly. Last, buildconfig.html would need updating to display the additional information. 

- The target.txt file generated by my try build contains the correct information.

- application.ini now has the correct information.

- platform.ini still has the wrong SourceRepository. I suspect this is because with the current state of things, there's really no support for multiple repositories. Solving this piece would likely get us a step closer to having the mozilla- changeset information in about:buildconfig.
FWIW, I'm currently looking into having a single repo for comm so we'd not have to deal with the multi repository junk. But that's not a solution for anything that needs a quick resolution.
Comment on attachment 9012763 [details]
Bug 1460487 - Update mozharness to set MOZ_SOURCE_CHANGESET for comm builds.

Dustin J. Mitchell [:dustin] pronoun: he has approved the revision.
Attachment #9012763 - Flags: review+
Attachment #9012763 - Attachment description: Bug 1460487 - Update mozharness to set MOZ_SOURCE_CHANGESET for comm builds → Bug 1460487 - Update mozharness to set MOZ_SOURCE_CHANGESET for comm builds.
Pushed by
Update mozharness to set MOZ_SOURCE_CHANGESET for comm builds. r=dustin
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 64.0
Rob, please manage backport requests here. m-esr60? m-b?
Flags: needinfo?(rob)
Jorg, this will need to be brought into esr60 and beta. 

Rico, I'll take another pass at this bug to see if I can clean the output up a bit more.
Flags: needinfo?(rob)
Resolution: FIXED → ---
(In reply to Rob Lemley [:rjl] from comment #15)
> Jorg, this will need to be brought into esr60 and beta. 
OK, I'll uplift when ready. Please stick the approval requests onto the patch in due course.
The latest nightly build still have an issue.
Base URL is from comm-central and  revision changeset ID is from mozilla-unified.

Build ID: 20181002100152
Generated URL in about:buildconfig:

Treeherder log:
Component: General → Build Config
No longer depends on: 1491907
Referencing bug 1267270 for some historical context.
As it turns out, this bug is a very similar problem with having incorrect source repo information in Thunderbird builds. The solution that went into place is along the same lines as what I did with the first pass at this patch. Set MOZ_SOURCE_CHANGESET and MOZ_SOURCE_REPO to comm- values, overriding the mozilla- ones.

They also ran into the same problem we are having with platform.ini have the incorrect information from comm- instead of mozilla- after applying their fix. There's a bit of code in comm/mail/installer/ that takes care of this in theory, but I suspect that in practice it does not seem to run for some reason.

Re comment 17: I just looked at the latest nightly builds at The win32 and win64 builds have the incorrect source data, linux, linux64, and macosx64 have the correct information in their target.txt files. I think I know where to fix that.
I've thought about this over the weekend, and this is how I plan to proceed.

- Revert my earlier patch that was applied to m-c in comment 12. I just don't think that relying on environment variables is the most stable way of doing this,
- Expand/enhance build/
  - The source_repo_header function will be rewritten to not worry about environment variables if being built from a VCS checkout
  - Get the URL and current revision hash for files found at comm/mail ($MOZ_BUILD_APP). It's not an hg root, but by either using slightly different commands or perhaps with hglib we can get the values we need. Store these values as MOZ_SOURCE_*.
  - Get the same info from the repo at $topsrcdir. Store these values as PLATFORM_SOURCE_*. Some magic may need to happen to identify this repository as something other than "moz-unified",
  - Write out source-info.h with the MOZ_SOURCE and PLATFORM_SOURCE values.

MOZ_SOURCE and PLATFORM_SOURCE values will wind up being the same on a Firefox build (where MOZ_BUILD_APP is "browser" and is in the same repository as $topsrcdir). In cases where it's a different repository the MOZ_SOURCE variables will have the application values while PLATFORM_SOURCE is guaranteed to have Firefox values, or platform.ini values.

There will be some cleanup that needs to happen in some buried Makefile's under comm/ that run sed against platform.ini in an attempt to fix the current state of affairs. The code that generates platform.ini will need updating.

The target.txt build artifact is generated by which will need some changes.

Sound reasonable?
MOZ_SOURCE_URL and MOZ_SOURCE_CHANGESET get used in the build process to create
application.ini and source-repo.h. It refers to the application's HG repo
("browser/" or "comm/mail")
The previous iteration did not address issues with the repo source and changeset
found in platform.ini, will now output additional #defines PLATFORM_SOURCE_REPO,
checkout at $topsrcdir.

I kept the same resolution order for determining the values.
  1) Check relevent environment variables. If set, use those values.
  2) Check for presence of sourcestamp.txt and use it's data if valid.
     (MOZ_SOURCE_* values only)
  3) Interrogate Mercurial for the values.

The MOZ_SOURCE_* will continue to refer to the repo for MOZ_BUILD_APP's dir.
In the case of Firefox ("browser/"), the repo is the same as
PLATFORM_SOURCE_REPO. In the case of Thunderbird ("comm/mail/"), the repository
at comm/ is found

Depends on D8151
Attachment #9012763 - Attachment is obsolete: true
Attachment #9015692 - Attachment is obsolete: true
about:buildconfig on Thunderbird 65.0a1
Patch on its way after more testing.
Attachment #9015691 - Attachment is obsolete: true
Sets MOZ_SOURCE_CHANGESET to be the comm-* changeset when building
Thunderbird on Windows. This mirrors a change at line 196 that sets the
same when building for Linux and OSX.
Ran successfully on try-comm-central producing a buildconfig page as shown.
Pushed by
Set MOZ_SOURCE_CHANGESET appropriately for TB. r=dustin
Closed: 3 years ago2 years ago
Resolution: --- → FIXED
Duplicate of this bug: 1356115
Duplicate of this bug: 1511732
You need to log in before you can comment on or make changes to this bug.