600931 - Rename firefox-*.mac64.dmg to firefox-*.mac.dmg

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[:Gavin Sharp [email: [e-mail]]]

Since the new i386/x86_64 universal is our only Mac deliverable now, I think it makes more sense to name it "mac" than "mac64" (mac64 kind of implies 64-only, which isn't the case).

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

It was left that way because of various expectations and contortions in the release automation & buildbot, which are too risky to unwind before b7.

Comment 2

[[e-mail] (:bhearsum)]

I am likely to be starting work on this soon. Closely related to bug 583671.

Comment 3

[Rail Aliiev [:rail]]

Attachment: buildbotcustom changes

Comment 4

[Rail Aliiev [:rail]]

Attachment: tools

Comment 5

[Rail Aliiev [:rail]]

Attachment: Bootsrap

Comment 6

[Rail Aliiev [:rail]]

Attachment: Bootstrap

Comment 7

[Rail Aliiev [:rail]]

Attachment: mozilla-central patch

Comment 8

[Rail Aliiev [:rail]]

Attachment: mozilla-central patch

Testing this patch. The previous one didn't work for l10n repacks. l10n mozconfig doesn't include $topsrcdir/build/macosx/universal/mozconfig and as a result package-name.mk uses this:

ifeq ($(TARGET_CPU),x86_64)
MOZ_PKG_PLATFORM := mac64
...

I tried to source $topsrcdir/build/macosx/universal/mozconfig (and change --with-l10n-base=$topsrcdir/../l10n-central), but in this case we have 2 objdirs for each platform and release automation, which assumes to find needed files in obj-l10n, fails during repacking.

As a simplest solution I used the following.

* we do opt (universal) builds on 10.6 only
* we do debug (not universal) builds on 10.5 and 10.6 and need to distinguish them by file name (actually we put them into the same directory, so they have to have different names)
* opt and debug directories are not the same

I suggest to use "mac" for opt (universal, i386+x86_64) and 10.5 debug (not universal, 32bit) builds, and "mac64" for 10.6 debug (not universal, 64bit) builds. The attached patch should handle this.

Comment 9

[Rail Aliiev [:rail]]

Attachment: tools

updated version (including 64bit platform exception handling)

Comment 10

[Rail Aliiev [:rail]]

Attachment: Bootstrap

updated version (including 64bit platform exception handling)

Comment 11

[[e-mail] (:bhearsum)]

Are we going to have to land the buildbotcustom/tools/m-c bits all in sync with each other?

Comment 12

[Rail Aliiev [:rail]]

(In reply to comment #11)
> Are we going to have to land the buildbotcustom/tools/m-c bits all in sync with
> each other?

Yeah, it would be safer. Additionally the CVS part (Bootstrap/Util.pm) should be re-tagged as UPDATE_PACKAGING_R13 after landing.

Comment 13

[Rail Aliiev [:rail]]

Attachment: Bootstrap

s/macos/macosx/

Comment 15

[[e-mail] (:bhearsum)]

(In reply to comment #12)
> (In reply to comment #11)
> > Are we going to have to land the buildbotcustom/tools/m-c bits all in sync with
> > each other?
> 
> Yeah, it would be safer. Additionally the CVS part (Bootstrap/Util.pm) should
> be re-tagged as UPDATE_PACKAGING_R13 after landing.

K. That'll have to land on the UPDATE_PACKAGING_R11_minibranch if we need it there, too.

A couple of other troublesome things:
- We'll need to land this on all of the m-c based project branches simultaneously with
- Try builds that happen after the Buildbot changes land, but which are based off of an earlier m-c revision will fail. We should blog/post to newsgroups about this.

Comment 16

[Rail Aliiev [:rail]]

Attachment: Merge mac and mac64 ABIs for snippets

This patch merges mac and mac64 ABIs introduced in bug 583671 attachment 500104 [details] [diff] [review]. Not sure if we need all of them, but since PPC is blocked by AUS2, it should be harmless.

Comment 17

[Rail Aliiev [:rail]]

So far everything went well, just one step should be done once manually.

Patcher uses a patcher config to generate partial MARs and snippets. After applying all of the patches, we still have to do 2 major changes in the config manually:

1) mac64 will be replaced with mac. From patcher's point of view it looks like dropping mac64 and introducing a new "mac" platform. As a result no update is being generated for the new platform (new platform, nothing to update!). 

Solution is easy, we just need to replace all mac64 entries in the patcher config for older versions, *before* running updates builder for the next version (4.0b9). This is a one time fix, no need to do it every release.

2) linux-x86_64 platform exists in the config, but it's not listed in the exception section for "ja" locale. As a result there is no update for linux64/ja. To fix this we need to add linux-x86_64 to the list of the locale/platform exceptions for all older versions, that have this locale (iirc, since beta 4 or 5). This is also one time fix.

I'll start requesting reviews for the patches after testing mac64->mac name transition for nightly builds (see bug 613301).

Comment 18

[[e-mail] (:bhearsum)]

Comment on attachment 500309 [details] [diff] [review]
Merge mac and mac64 ABIs for snippets

I'm not sure I understand why you want to merge these together. We only use the trunk of patcher (the branch where UPDATE_PACKAGING_R13 lives) for m-c/m-2.0. I'd recommend just getting rid of the old mac ABIs altogether there rather than having us generate a bunch of snippets that aren't even going to be used. 1.9.1 and 1.9.2 use UPDATE_PACKAGING_R11_1, which lives on UPDATE_PACKAGING_R11_minibranch, so they wouldn't be affected.

Comment 19

[Rail Aliiev [:rail]]

(In reply to comment #18)
> I'm not sure I understand why you want to merge these together. 

It's just because only I was lazy to check every ABI we need for 4.0  :) (and it was not the main point of the tests).I just wanted to test that the builder generates snippets for multiple ABIs and I can run update verify properly. The final list of ABIs will be reviewed before requesting review.

Comment 20

[[e-mail] (:bhearsum)]

Ah, ok. Cool!

Comment 21

[Rail Aliiev [:rail]]

Attachment: MozAUSlib.pm

Comment 22

[Rail Aliiev [:rail]]

Comment on attachment 500039 [details] [diff] [review]
Bootstrap

To be updated after bug 619980

Comment 23

[Rail Aliiev [:rail]]

Comment on attachment 500016 [details] [diff] [review]
tools

To be updated after bug 619980.

Comment 24

[[e-mail] (:bhearsum)]

Comment on attachment 499916 [details] [diff] [review]
mozilla-central patch

Looks ok to me, Needs Ted or Khuey review too, though.

Comment 25

[(not currently active) Ted Mielczarek]

Comment on attachment 499916 [details] [diff] [review]
mozilla-central patch

Why did you add the ifndef MOZ_DEBUG here?

Comment 26

[(not currently active) Ted Mielczarek]

Oh, er, should have re-read your earlier comment.

Comment 27

[(not currently active) Ted Mielczarek]

Comment on attachment 499916 [details] [diff] [review]
mozilla-central patch

I think you should just leave the UNIVERSAL_BINARY check, and s/mac64/mac on the line like you did. I don't think this should care whether you're doing a debug build or not. You could build a debug universal build, or an opt 64-bit only build, and we should name them appropriately.

Comment 28

[Rail Aliiev [:rail]]

Attachment: Bootstrap

Comment 29

[Rail Aliiev [:rail]]

Attachment: tools

Comment 30

[Rail Aliiev [:rail]]

Comment on attachment 503419 [details] [diff] [review]
tools

Needs some additional tweaks due to ted's r-.

Comment 31

[Rail Aliiev [:rail]]

Attachment: mozilla-central patch

The attached patch initially made by Ben. It works fine for en-US build.

l10n repacks don't source  $topsrcdir/build/macosx/universal/mozconfig (and don't work with that file sourced :/) and use guessed $(TARGET_CPU) (in our case "x86_64"). To make l10n repacks work as we want we need to export MOZ_PKG_PLATFORM=mac.

Comment 32

[Rail Aliiev [:rail]]

Attachment: tools

The same patch with additional hunk for lib/python/build/l10n.py.

Usual builds and nightlies still need some changes in buildbotcustom.

Comment 33

[Rail Aliiev [:rail]]

Attachment: buildbotcustom changes

Testing this one...

Comment 34

[Rail Aliiev [:rail]]

Attachment: patcher-configs

Once we switch from mac64 to mac, it would be better to rename mac64 to mac where it was shipped apart from mac. I'm leaving mac64 as is for releases with 2 platforms (mac+mac64).

Comment 35

[Rail Aliiev [:rail]]

Attachment: Fix mac64 updates

This script grabs all releases with mac+mac64 platforms and prints shell commands which:
* remove mac64 update directory which is not being updated anymore by patcher (mac64 is dropped)
* create a symlink to that directories pointing to "mac" directories

We have 7 releases with multiple platforms. If we want to provide updates for these users directly to the latest version, it's way to go.

At the moment the output of the script is:

rm -rv 3.7a5/Darwin_x86_64-gcc3/20100610053411 && ln -sv 20100610053455 3.7a5/Darwin_x86_64-gcc3/20100610053411
rm -rv 3.7a5/Darwin_x86-gcc3-u-i386-x86_64/20100610053411 && ln -sv 20100610053455 3.7a5/Darwin_x86-gcc3-u-i386-x86_64/20100610053411
rm -rv 3.7a5/Darwin_x86_64-gcc3-u-i386-x86_64/20100610053411 && ln -sv 20100610053455 3.7a5/Darwin_x86_64-gcc3-u-i386-x86_64/20100610053411
rm -rv 4.0b1/Darwin_x86_64-gcc3/20100630131936 && ln -sv 20100630131607 4.0b1/Darwin_x86_64-gcc3/20100630131936
rm -rv 4.0b1/Darwin_x86-gcc3-u-i386-x86_64/20100630131936 && ln -sv 20100630131607 4.0b1/Darwin_x86-gcc3-u-i386-x86_64/20100630131936
rm -rv 4.0b1/Darwin_x86_64-gcc3-u-i386-x86_64/20100630131936 && ln -sv 20100630131607 4.0b1/Darwin_x86_64-gcc3-u-i386-x86_64/20100630131936
rm -rv 4.0b2/Darwin_x86_64-gcc3/20100720180051 && ln -sv 20100720175703 4.0b2/Darwin_x86_64-gcc3/20100720180051
rm -rv 4.0b2/Darwin_x86-gcc3-u-i386-x86_64/20100720180051 && ln -sv 20100720175703 4.0b2/Darwin_x86-gcc3-u-i386-x86_64/20100720180051
rm -rv 4.0b2/Darwin_x86_64-gcc3-u-i386-x86_64/20100720180051 && ln -sv 20100720175703 4.0b2/Darwin_x86_64-gcc3-u-i386-x86_64/20100720180051
rm -rv 4.0b3/Darwin_x86_64-gcc3/20100805182338 && ln -sv 20100805181943 4.0b3/Darwin_x86_64-gcc3/20100805182338
rm -rv 4.0b3/Darwin_x86-gcc3-u-i386-x86_64/20100805182338 && ln -sv 20100805181943 4.0b3/Darwin_x86-gcc3-u-i386-x86_64/20100805182338
rm -rv 4.0b3/Darwin_x86_64-gcc3-u-i386-x86_64/20100805182338 && ln -sv 20100805181943 4.0b3/Darwin_x86_64-gcc3-u-i386-x86_64/20100805182338
rm -rv 4.0b4/Darwin_x86_64-gcc3/20100818121922 && ln -sv 20100818121614 4.0b4/Darwin_x86_64-gcc3/20100818121922
rm -rv 4.0b4/Darwin_x86-gcc3-u-i386-x86_64/20100818121922 && ln -sv 20100818121614 4.0b4/Darwin_x86-gcc3-u-i386-x86_64/20100818121922
rm -rv 4.0b4/Darwin_x86_64-gcc3-u-i386-x86_64/20100818121922 && ln -sv 20100818121614 4.0b4/Darwin_x86_64-gcc3-u-i386-x86_64/20100818121922
rm -rv 4.0b5/Darwin_x86_64-gcc3/20100831070403 && ln -sv 20100831070010 4.0b5/Darwin_x86_64-gcc3/20100831070403
rm -rv 4.0b5/Darwin_x86-gcc3-u-i386-x86_64/20100831070403 && ln -sv 20100831070010 4.0b5/Darwin_x86-gcc3-u-i386-x86_64/20100831070403
rm -rv 4.0b5/Darwin_x86_64-gcc3-u-i386-x86_64/20100831070403 && ln -sv 20100831070010 4.0b5/Darwin_x86_64-gcc3-u-i386-x86_64/20100831070403
rm -rv 4.0b6/Darwin_x86_64-gcc3/20100914073111 && ln -sv 20100914072643 4.0b6/Darwin_x86_64-gcc3/20100914073111
rm -rv 4.0b6/Darwin_x86-gcc3-u-i386-x86_64/20100914073111 && ln -sv 20100914072643 4.0b6/Darwin_x86-gcc3-u-i386-x86_64/20100914073111
rm -rv 4.0b6/Darwin_x86_64-gcc3-u-i386-x86_64/20100914073111 && ln -sv 20100914072643 4.0b6/Darwin_x86_64-gcc3-u-i386-x86_64/20100914073111

Comment 36

[[e-mail] (:bhearsum)]

Comment on attachment 503456 [details] [diff] [review]
tools

Is it possible to set MOZ_PKG_PLATFORM in the mozconfig instead? It doesn't need any runtime information, so that would be preferable IMHO. The other parts look fine.

Comment 37

[[e-mail] (:bhearsum)]

Comment on attachment 503469 [details] [diff] [review]
Fix mac64 updates

I don't think it's worthwhile worrying about this. There's only about ~5000 users on b6 and earlier mac builds, and Christian/Beltzner already signed off on not providing these users direct updates to the latest Beta. (From e-mail):

Sounds fine to me as well.

On Dec 21, 2010, at 12:32 PM, Ben Hearsum wrote:

> > It will only affect the beta audience, because we have no other audience on 4.0 at the moment :). Note that we've shipped exclusively new-style universals since beta 7, so *all* users on b7 and future builds will always get the latest version directly.
> > 
> > This has no impact on 3.6 -> 4.0 MUs.
> > 
> > So, unless Christian objects I'll take this as a "yes".
> > 
> > On 12/21/10 03:26 PM, Mike Beltzner wrote:
>> >> Fine by me if this only applies to the beta audience, and doesn't mean that Firefox 3.6 users wouldn't be able to go to Firefox 4.latest when they decide to upgrade.
>> >> 
>> >> cheers,
>> >> mike
>> >> 
>> >> On 2010-12-21, at 3:16 PM, Ben Hearsum wrote:
>> >> 
>>> >>> Hey Guys,
>>> >>> 
>>> >>> I'm working on fixing https://bugzilla.mozilla.org/show_bug.cgi?id=583671 at the moment, which is going to (finally) streamline Mac updates after the updater ABI changes back in August.
>>> >>> 
>>> >>> My question to you is: Is it an issue if we stop providing direct updates from old universals to the latest version.
>>> >>> 
>>> >>> It would simplify my work quite a bit if we could stop doing that, and only provide direct updates to the latest version from older versions of the new universals/64-bit mac builds.
>>> >>> 
>>> >>> This would mean that anyone on an old universal build, which we shipped up until 4.0b6, would have to update to 4.0b8, and then to whatever the latest is. By my count, this would affect a maximum of 16,000 users, and probably less once 4.0b8 is out the door.
>>> >>> 
>>> >>> - Ben
>> >>

Comment 38

[Rail Aliiev [:rail]]

(In reply to comment #36)
> Comment on attachment 503456 [details] [diff] [review]
> tools
> 
> Is it possible to set MOZ_PKG_PLATFORM in the mozconfig instead? It doesn't
> need any runtime information, so that would be preferable IMHO. The other parts
> look fine.

I tried to export this variable in .mozconfig, but looks like "make installers-$locale" doesn't use the environment set there. :/

Comment 39

[Rail Aliiev [:rail]]

Attachment: buildbotcustom changes

I have to add env=self.env for all "make unpack" steps. Otherwise those steps try to use mac64.

Comment 40

[[e-mail] (:bhearsum)]

Comment on attachment 503456 [details] [diff] [review]
tools

Turns out it's not possible to set this in the mozconfig, because the 'make installers-%' doesn't source it. r+ on this.

Comment 41

[Rail Aliiev [:rail]]

This bug requires a downtime to land all patches in sync.

Comment 42

[Robert Kaiser]

CCing Callek so we can make sure SeaMonkey picks this up and doesn't break (for too long).

Comment 43

[Dustin J. Mitchell [:dustin] (he/him)]

From some quick discussion in IRC, we're going to defer this until at least Monday when the builds go to QA.

Ben, you're up next week, if you want to start the ball rolling on announcing the tree closure tomorrow (Thursday).

Comment 44

[Rail Aliiev [:rail]]

Attachment: patcher-configs

Fix 4.0b10 mac64 entries too.

Comment 45

[[e-mail] (:bhearsum)]

Minor note, need to update the file that controls the nagios checks:
http://mxr.mozilla.org/mozilla/source/tools/tinderbox-configs/monitoring/Firefox_mozilla-central.txt

and maybe the Tracemonkey/Electrolysis ones too

Comment 46

[Chris AtLee [:catlee]]

Attachment: Fix for l10n dep builds

Comment 47

[Chris AtLee [:catlee]]

Comment on attachment 507131 [details] [diff] [review]
Fix for l10n dep builds

Landed on default and production-0.8

Comment 48

[Rail Aliiev [:rail]]

Comment on attachment 503418 [details] [diff] [review]
Bootstrap

$ cvs commit -m "Bug 600931 - Rename firefox-*.mac64.dmg to firefox-*.mac.dmg. p=rail, r=bhearsum"
cvs commit: Examining .
Checking in Util.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Util.pm,v  <--  Util.pm
new revision: 1.21; previous revision: 1.20
done
$ cvs tag UPDATE_PACKAGING_R13
cvs tag: Tagging .
W Util.pm : UPDATE_PACKAGING_R13 already exists on version 1.20 : NOT MOVING tag to version 1.21
$ cvs tag -F UPDATE_PACKAGING_R13
cvs tag: Tagging .
T Util.pm

Comment 49

[Rail Aliiev [:rail]]

Comment on attachment 501280 [details] [diff] [review]
MozAUSlib.pm

$ cvs commit -m "Bug 600931 - Rename firefox-*.mac64.dmg to firefox-*.mac.dmg. p=rail, r=bhearsum"
cvs commit: Examining .
Checking in MozAUSLib.pm;
/cvsroot/mozilla/tools/patcher/MozAUSLib.pm,v  <--  MozAUSLib.pm
new revision: 1.17; previous revision: 1.16
done
$ cvs tag UPDATE_PACKAGING_R13
cvs tag: Tagging .
W MozAUSLib.pm : UPDATE_PACKAGING_R13 already exists on version 1.16 : NOT MOVING tag to version 1.17
$ cvs tag -F UPDATE_PACKAGING_R13
cvs tag: Tagging .
T MozAUSLib.pm

Comment 50

[Rail Aliiev [:rail]]

Comment on attachment 506721 [details] [diff] [review]
patcher-configs

$ cvs commit -m "Bug 600931 - Rename firefox-*.mac64.dmg to firefox-*.mac.dmg. p=rail, r=bhearsum"
cvs commit: Examining .
Checking in moz20-branch-patcher2.cfg;
/cvsroot/mozilla/tools/patcher-configs/moz20-branch-patcher2.cfg,v  <--  moz20-branch-patcher2.cfg
new revision: 1.16; previous revision: 1.15
done

Comment 51

[Rail Aliiev [:rail]]

Comment on attachment 503455 [details] [diff] [review]
mozilla-central patch

http://hg.mozilla.org/mozilla-central/rev/af7e65f1ee6f

Comment 52

[Rail Aliiev [:rail]]

Comment on attachment 503456 [details] [diff] [review]
tools

http://hg.mozilla.org/build/tools/rev/68f0560d5d64

Comment 53

[Rail Aliiev [:rail]]

Comment on attachment 503814 [details] [diff] [review]
buildbotcustom changes

default: http://hg.mozilla.org/build/buildbotcustom/rev/c38d384732e0
merged to production-0.8: http://hg.mozilla.org/build/buildbotcustom/rev/87fb13375a48

Comment 54

[Rail Aliiev [:rail]]

We published en-US complete snippets on AUS2 yesterday, because current automation doesn't generate partial mars and doesn't push complete snippets if there is no previous complete mar with the same name pattern (mac vs mac64).

Seen green nightly builds of en-US and l10n repacks today. Dep builds and l10n repacks work fine as well. Closing.