Support updates for multiple mac universal builds

RESOLVED FIXED

Status

Release Engineering
General
P3
normal
RESOLVED FIXED
8 years ago
5 years ago

People

(Reporter: nthomas, Assigned: bhearsum)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [automation][updates][releases])

Attachments

(7 attachments, 1 obsolete attachment)

(Reporter)

Description

8 years ago
Mostly this is in updates, do support the changes described in bug 552924 comment 26. This affects Firefox 4 and onwards, so mozilla-central and derived repos plus mozilla-2.0 when we start using it.

We'll need to support nightly updates as soon as the change lands from the parent bug. If it is included in the 4.0b3 release then we'll need to support release updates at the time of 4.0b4.
(Reporter)

Comment 1

8 years ago
Initial support for nightlies provided by adding symlinks:
 cd /opt/aus2/incoming/2/Firefox
 for d in mozilla-central mozilla-2.0 tracemonkey electrolysis; do 
   ln -s Darwin_Universal-gcc3 $d/Darwin_ppc-gcc3-u-ppc-i386
   ln -s Darwin_Universal-gcc3 $d/Darwin_i386-gcc3-u-ppc-i386
 done

This will redirect users of the existing ppc+i386 universal to the right snippets once bug 552924 lands. We'll need to change these snippets once we turn the x64_64 build into i386+x86_64 universal and want to switch users over to it. We can change the name of the directories that snippets to better names once everything shakes out.

For releases we need to do more work for a long term fix. Assuming 552924 is in b5 then we at least need symlinks for b5 -> whatever's next.
(Reporter)

Comment 2

8 years ago
Darwin_i386-gcc3-u-ppc-i386 replaced with Darwin_x86-gcc3-u-ppc-i386, per bug 591817.
(Reporter)

Updated

8 years ago
Summary: Support multiple mac universal builds → Support updates for multiple mac universal builds
(Reporter)

Updated

8 years ago
No longer blocks: 571367
(Reporter)

Comment 3

8 years ago
For mozilla-central, Darwin_x86-gcc3-u-ppc-i386 now points to the Darwin_x86_64-gcc3 bucket where the snippets from the new universal are going. As do
  Darwin_x86-gcc3-u-i386-x86_64
  Darwin_x86_64-gcc3-u-i386-x86_64

We should do the same for TraceMonkey and E10s when they do a universal nightly in the new style.
(Reporter)

Comment 4

8 years ago
With all the fun from bug 600289 and friends we now have nightlies publishing into Darwin_x86_64-gcc3 and the following symlinks
  Darwin_x86-gcc3-u-ppc-i386       -> Darwin_x86_64-gcc3
  Darwin_x86-gcc3-u-i386-x86_64    -> Darwin_x86_64-gcc3
  Darwin_x86_64-gcc3-u-i386-x86_64 -> Darwin_x86_64-gcc3
PPC users on the old universal (Darwin_ppc-gcc3-u-ppc-i386) are no longer offered updates. 

I've set up the same mappings for tracemonkey, electrolysis and mozilla-2.0.
(Reporter)

Comment 5

8 years ago
I've set up these mappings for mozilla-1.9.1 and mozilla-1.9.2
  Darwin_ppc-gcc3-u-ppc-i386 -> Darwin_Universal-gcc3
  Darwin_x86-gcc3-u-ppc-i386 -> Darwin_Universal-gcc3
to support the change on those branches in bug 552924 (attachment 478623 [details] [diff] [review]).
(Reporter)

Updated

8 years ago
Blocks: 613301
(Assignee)

Comment 6

8 years ago
This is more critical now that we're shipping a release (3.6.13) with these updater changes. Currently, we have to manually modify the snippets to have Mac updates work for 3.6.13 users. It'll hurt even more in 3.6.14.
(Assignee)

Comment 7

8 years ago
I'll be working on this very soon.
Assignee: nobody → bhearsum
(Assignee)

Updated

8 years ago
Depends on: 619977
(Assignee)

Updated

8 years ago
Depends on: 619978
(Assignee)

Updated

8 years ago
Depends on: 600931
(Assignee)

Comment 8

8 years ago
Created attachment 498532 [details] [diff] [review]
first stab at support multiple ABIs for a platform

I haven't been able to verify this against existing snippets yet, but it _seems_ ok at a first glance. Needs a ton more testing, of course.
(Assignee)

Comment 9

7 years ago
Did some verification by regenerating 3.6.12 -> 3.6.13 snippets. Everything in the test channels was a match, except for the sums and filesizes of partial MARs, which were regenerated as part of my test run.

Done with a limited set of locales.

I'm going to do a full test run on 3.5, 3.6, and 4.0 next.
(Assignee)

Comment 10

7 years ago
Created attachment 499113 [details] [diff] [review]
support multiple ABIs in patcher, UPDATE_PACKAGING_R12 version

After a bunch more testing I believe that this is complete and ready for review. 

Summary of changes:
- Map "mac64" platform to "osx" in Bouncer
- Add platform -> filesystem mapping, to support what patcher thinks of as "mac64" living in the "mac" directory
- Adjust AUS2_PLATFORMS data structure to be a hash whose values are lists; add appropriate mac/mac64 Darwin ABIs.

The rest is just adjusting patcher to cope with the changes. It looks bigger than it is; all of the large hunks are just adding a loop and re-indenting.

With this patch applied, I re-run a full set of updates for 4.0b8. I diffed those against the ones we're shipping b8 with. The results were acceptable, with the following differences:
- My test run had real partial snippets for Windows. In the real Beta 8, we overwrote them with completes.
- My test run had no snippets for ja/ja-JP-mac. This will be fixed in bug 619522.
- My test run did not generate old universal -> new universal snippets. This is Really Hard to do correctly in patcher, and I got the OK from drivers to stop doing it. (The "Only in /home/cltbld/...." lines in the diff are about this.)
- My test run generated snippets for "Darwin_x86_64-gcc3-u-i386-x86_64" and "Darwin_x86-gcc3-u-i386-x86_64" for 3.7a5 through 4.0b7 and the original run did not, which had to be fixed by hand later. This is the thing we're fixing in this bug! These ABIs are only used starting on beta 5, but it doesn't hurt to have them for older releases.

This patch only applies to the UPDATE_PACKAGING_R12 tag. I have a similar patch for the _R11 branch of development.
Attachment #498532 - Attachment is obsolete: true
Attachment #499113 - Flags: review?(rail)
(Assignee)

Comment 11

7 years ago
Created attachment 499114 [details]
results of 4.0b8 original vs rerun diff
(Assignee)

Comment 12

7 years ago
Created attachment 499120 [details] [diff] [review]
support multiple ABIs in patcher, UPDATE_PACKAGING_R11 version

This is very very similar to the _R12 version. The only difference is that the 'mac64' bouncer platform isn't adjusted, because it's irrelevant on this branch, and the details of one of the large hunks is different, because of bug 514040.

Like the other patch, I tested this by running updates. In this case, I did: 3.5.x -> 3.5.16, 3.6.x -> 3.6.13, and 3.5.16 -> 3.6.13 MU. The only differences were:
- My test run generated snippets for the "Darwin_x86-gcc3-u-ppc-i386" ABI for all older 3.5 and 3.6 releases, the real one did not. Again, this what we're fixing here. Versions older than 3.5.16/3.6.13 don't need them, but it doesn't hurt to have them (and makes this patch a lot simpler...).
- My test run did not have any snippets for 3.6.13build1/2 -> build3 nor 3.5.16build1 -> build2. This is OK because these are generated after patcher runs, by a different script.

To land this, I'll need to create a branch whose parent is UPDATE_PACKAGING_R11, for all of the patcher/ directory. After landing I plan to retag with UPDATE_PACKAGING_R11_1. I'll be tagging _R12 similarly.
Attachment #499120 - Flags: review?(rail)
(Assignee)

Comment 13

7 years ago
Created attachment 499121 [details]
results of 3.5/3.6 reruns
Blocks: 620792
(Assignee)

Updated

7 years ago
Duplicate of this bug: 619977
(Assignee)

Updated

7 years ago
Duplicate of this bug: 619978
(Assignee)

Comment 16

7 years ago
Created attachment 499295 [details] [diff] [review]
fix 1:1 platform:ABI assumptions in tools repo

The only things affected by this are update verify, and generate-candidate-build-updates.py. I didn't do any testing on the former, but the patch to is is extremely simple and safe. For the latter, I ran it against the 3.5.16/3.6.13 snippets I generated yesterday, and re-ran the diffs. The results were the same as before, except the "missing 3.6.13/3.5.16" lines went away, which means I did it right!
Attachment #499295 - Flags: review?(rail)
(Assignee)

Comment 17

7 years ago
Besides the tools addressed in the posted patches, our nightly update logic, contained mostly in process/factory.py also assumes a 1:1 mapping of platform:update ABI. I don't think it's worth fixing it for a few reasons:
- Adding new ABIs for nightly updates is a one-off change on AUS, we don't need to fix it every time new updates are published
- The patch needed is fairly difficult, and very risky to all nightly updates
- This version of AUS is intended to be going away in the next 3 months.
Comment on attachment 499113 [details] [diff] [review]
support multiple ABIs in patcher, UPDATE_PACKAGING_R12 version

> @@ -354,8 +370,11 @@ sub SubstitutePath
>      my $platform = $args{'platform'} || 'UNDEFINED';
> ........... 
>      my %bouncer_platforms = GetBouncerPlatformStrings();
>      my $bouncer_platform = $bouncer_platforms{$platform};
> +    
> +    my %filepath_platforms = GetFilepathPlatformStrings();
> +    my $filepath_platform = $filepath_platforms{$platform};
>  
> -    $string =~ s/%platform%/$platform/g;
> +    $string =~ s/%platform%/$filepath_platform/g;

A nit. These changes should issue warnings (with -w) for SubstitutePath calls for $licenseUrl = SubstitutePath(...) because you don't pass platform. r+ if you pass platform. License is not supposed to be different per platform and contain %platform% placeholder, but just to be consistent with other calls.
Attachment #499113 - Flags: review?(rail) → review+
Comment on attachment 499120 [details] [diff] [review]
support multiple ABIs in patcher, UPDATE_PACKAGING_R11 version

See my comments for the previous patch.
Attachment #499120 - Flags: review?(rail) → review+
Comment on attachment 499295 [details] [diff] [review]
fix 1:1 platform:ABI assumptions in tools repo

> -        update_platform = buildbot2updatePlatform(platform)
> +        update_platforms = buildbot2updatePlatform(platform)

A nit. Wouldn't it be better to rename buildbot2updatePlatform to buildbot2updatePlatforms because it returns a list of platforms?

Great progress!!!
Attachment #499295 - Flags: review?(rail) → review+
(Assignee)

Comment 21

7 years ago
Comment on attachment 499113 [details] [diff] [review]
support multiple ABIs in patcher, UPDATE_PACKAGING_R12 version

Checking in MozAUSLib.pm;
/cvsroot/mozilla/tools/patcher/MozAUSLib.pm,v  <--  MozAUSLib.pm
new revision: 1.14; previous revision: 1.13
done
Checking in patcher2.pl;
/cvsroot/mozilla/tools/patcher/patcher2.pl,v  <--  patcher2.pl
new revision: 1.41; previous revision: 1.40
done

cvs co -r UPDATE_PACKAGING_R12 mozilla/client.mk
cd mozilla
make -f client.mk MOZ_CO_PROJECT=all checkout
cvs up -A tools/update-packaging
cvs up -AdP tools/release
cvs up -AdP tools/patcher
cvs -q tag UPDATE_PACKAGING_R13 2>&1 | tee ../tag.log
grep -v '^T' ../tag.log
cvs tag -b UPDATE_PACKAGING_R13_minibranch client.mk
cvs up -r UPDATE_PACKAGING_R13_minibranch client.mk

edited client.mk, bumped to UPDATE_PACKAGING_R13
commited client.mk

/cvsroot/mozilla/client.mk,v  <--  client.mk
new revision: 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1; previous revision: 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1

cvs -q tag UPDATE_PACKAGING_R13 client.mk
W client.mk : UPDATE_PACKAGING_R13 already exists on version 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1 : NOT MOVING tag to version 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1
foo-ix-blah:mozilla bhearsum$ cvs -q tag -F UPDATE_PACKAGING_R13 client.mk
T client.mk

Also created an UPDATE_PACKAGING_R13 tag on m-c as follows:
hg tag -r UPDATE_PACKAGING_R12 -m "Create UPDATE_PACKAGING_R13 tag for bug 583671. r=rail. a=npotb DONTBUILD" UPDATE_PACKAGING_R13
(Assignee)

Comment 22

7 years ago
Comment on attachment 499120 [details] [diff] [review]
support multiple ABIs in patcher, UPDATE_PACKAGING_R11 version

Before applying and commiting this patch, I had to create the UPDATE_PACKAGING_R11_minibranch for patcher, as follows:
cvs co -d patcher -r UPDATE_PACKAGING_R11 mozilla/tools/patcher
cd patcher
cvs tag -b UPDATE_PACKAGING_R11_minibranch .

I then applied and commited the patch as usual:
Checking in MozAUSLib.pm;
/cvsroot/mozilla/tools/patcher/MozAUSLib.pm,v  <--  MozAUSLib.pm
new revision: 1.13.2.1; previous revision: 1.13
done
Checking in patcher2.pl;
/cvsroot/mozilla/tools/patcher/patcher2.pl,v  <--  patcher2.pl
new revision: 1.38.2.1; previous revision: 1.38
done

And then did the following to create a new tag:
cvs co -r UPDATE_PACKAGING_R11 mozilla/client.mk
cd mozilla
make -f client.mk MOZ_CO_PROJECT=all checkout
cvs up -AdP -r UPDATE_PACKAGING_R11_minibranch tools/patcher
cvs -q tag UPDATE_PACKAGING_R11_1 2>&1 | tee ../tag.log
grep -v '^T' ../tag.log
cvs up -r UPDATE_PACKAGING_R11_minibranch client.mk

edited client.mk, bumped to UPDATE_PACKAGING_R11_1
commited client.mk

Checking in client.mk;
/cvsroot/mozilla/client.mk,v  <--  client.mk
new revision: 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.2; previous revision: 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1

cvs -q tag UPDATE_PACKAGING_R11_1 client.mk
W client.mk : UPDATE_PACKAGING_R11_1 already exists on version 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1 : NOT MOVING tag to version 1.314.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.1.2.2
cvs -q tag -F UPDATE_PACKAGING_R11_1 client.mk
T client.mk

As mentioned, I updated m-c with the new tag, like so:
hg tag -r UPDATE_PACKAGING_R11 UPDATE_PACKAGING_R12
(Assignee)

Comment 23

7 years ago
Oops, forgot to address the comment about SubstitutePath before I landed, fixed in follow-up landings to MozAUSLib.pm, and I moved the tags.
(Assignee)

Updated

7 years ago
Attachment #499113 - Flags: checked-in+
(Assignee)

Updated

7 years ago
Attachment #499120 - Flags: checked-in+
(Assignee)

Comment 24

7 years ago
(In reply to comment #20)
> Comment on attachment 499295 [details] [diff] [review]
> fix 1:1 platform:ABI assumptions in tools repo
> 
> > -        update_platform = buildbot2updatePlatform(platform)
> > +        update_platforms = buildbot2updatePlatform(platform)
> 
> A nit. Wouldn't it be better to rename buildbot2updatePlatform to
> buildbot2updatePlatforms because it returns a list of platforms?
> 
> Great progress!!!

Definitely. I'll fix that on check-in.
(Assignee)

Comment 25

7 years ago
Comment on attachment 499295 [details] [diff] [review]
fix 1:1 platform:ABI assumptions in tools repo

changeset:   1915:035c7877c38c
Attachment #499295 - Flags: checked-in+
(Assignee)

Comment 26

7 years ago
Had to fix up the _R11_1 tag a little bit because I forgot to include mozilla/tools/release in it. In my tagging work directory I did:
cd tools
cvs co -d release mozilla/tools/release
cvs up -AdP -r UPDATE_PACKAGING_R11 release
cd release
cvs tag -b UPDATE_PACKAGING_R11_minibranch .
cvs tag UPDATE_PACKAGING_R11_1 .
(Assignee)

Comment 27

7 years ago
Created attachment 500103 [details] [diff] [review]
add Darwin_ppc-gcc3-u-ppc-i386 ABI

We need this ABI so that Mac PPC users will still receive updates.
Attachment #500103 - Flags: review?(rail)
(Assignee)

Comment 28

7 years ago
Created attachment 500104 [details] [diff] [review]
add ppc ABI to the tools library
Attachment #500104 - Flags: review?(rail)
Attachment #500103 - Flags: review?(rail) → review+
Attachment #500104 - Flags: review?(rail) → review+
(Assignee)

Comment 29

7 years ago
So, other than the missing ppc ABI, for which I just posted the fix for, this was 100% good in my staging 3.6.14 run. Identical snippets were generated for the correct ABIs, update verify passed. That, combined with my previous verifications makes me very confident that this is nearly ready to flip in production. I just wanted to wait for Rail to finish his 4.0 staging run before calling it good and bumping the UPDATE_PACKAGING tags in the production configs.
(Assignee)

Comment 30

7 years ago
Comment on attachment 500103 [details] [diff] [review]
add Darwin_ppc-gcc3-u-ppc-i386 ABI

Landed this. On UPDATE_PACKAGING_R13:
Checking in MozAUSLib.pm;
/cvsroot/mozilla/tools/patcher/MozAUSLib.pm,v  <--  MozAUSLib.pm
new revision: 1.16; previous revision: 1.15
done
[] bhearsum@voot:~/tmp/1/patcher$ cvs tag UPDATE_PACKAGING_R13 MozAUSLib.pm 
W MozAUSLib.pm : UPDATE_PACKAGING_R13 already exists on version 1.15 : NOT MOVING tag to version 1.16
[] bhearsum@voot:~/tmp/1/patcher$ cvs tag -F UPDATE_PACKAGING_R13 MozAUSLib.pm 
T MozAUSLib.pm


And for UPDATE_PACKAGING_R11_1:
Checking in MozAUSLib.pm;
/cvsroot/mozilla/tools/patcher/MozAUSLib.pm,v  <--  MozAUSLib.pm
new revision: 1.13.2.3; previous revision: 1.13.2.2
done
[] bhearsum@voot:~/tmp/2/patcher$ cvs tag UPDATE_PACKAGING_R11_1 MozAUSLib.pm 
W MozAUSLib.pm : UPDATE_PACKAGING_R11_1 already exists on version 1.13.2.2 : NOT MOVING tag to version 1.13.2.3
[] bhearsum@voot:~/tmp/2/patcher$ cvs tag -F UPDATE_PACKAGING_R11_1 MozAUSLib.pm 
T MozAUSLib.pm
Attachment #500103 - Flags: checked-in+
(Assignee)

Comment 31

7 years ago
Comment on attachment 500104 [details] [diff] [review]
add ppc ABI to the tools library

changeset:   1948:3247678ba39e
Attachment #500104 - Flags: checked-in+
Worked fine while testing mac64->mac transition in bug 600931. 

No more pain with manual snippet manipulations!

           /(|
          (  :
         __\  \  _____
       (____)  `|
      (____)|   |
       (____).__|
        (___)__.|_____
(Assignee)

Comment 33

7 years ago
Everything is checked in, both my and Rail's tests passed, we're all done here, woohoo!
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED

Updated

7 years ago
Depends on: 629048
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.