The default bug view has changed. See this FAQ.

Update all OS X 10.6 test slaves to 10.6.6

RESOLVED FIXED

Status

Release Engineering
Other
P1
normal
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: Joe Drew (not getting mail), Assigned: armenzg)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(5 attachments, 4 obsolete attachments)

1.02 KB, patch
bhearsum
: review+
armenzg
: checked-in-
Details | Diff | Splinter Review
4.39 KB, patch
dustin
: review+
bhearsum
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
14.73 KB, patch
catlee
: review+
armenzg
: checked-in-
Details | Diff | Splinter Review
1.61 KB, text/plain
catlee
: review+
armenzg
: checked-in-
Details
8.85 KB, patch
lsblakk
: review+
armenzg
: checked-in-
Details | Diff | Splinter Review
(Reporter)

Description

6 years ago
Because of a bug in 10.6.2, which caused bug 629016, we need to update all our 10.6 OS X slaves to at least 10.6.3. My recommendation is to simply update to 10.6.6, to more accurately reflect our users' likely OS distribution.

This is quite important, because we need to block OpenGL on 10.6.2 and earlier, and if we do that without updating our 10.6 slaves, we won't be testing OpenGL at all.
Joe means testers, not builders, ie talos-r3-snow-*
Summary: Update all OS X 10.6 slaves to 10.6.6 → Update all OS X 10.6 test slaves to 10.6.6
(Assignee)

Updated

6 years ago
Assignee: nobody → armenzg
Status: NEW → ASSIGNED
Priority: -- → P1
Created attachment 507924 [details] [diff] [review]
Add 10.6.6 to the list of supported puppet platforms

shall I install also the iTunes update? :P

The 10.6.6 took 10mins to download and apply.
We can use "sudo softwareupdate --install MacOSXUpdCombo10.6.6-10.6.6" to install which means that we don't have to VNC.
A reboot was required for the change to take place.

We also need this puppet change to allow the machine to take updates.

I assume we will need a renaming of the builder name from 10.6.2 to 10.6.6 (patch coming).

I am running now unit tests and talos jobs on talos-r3-snow-001 to check what it does. The change I am testing is http://hg.mozilla.org/try/rev/e0dd77d96259 which re-enables again those changes.

You can see how they fail on production:
reftest 5626/5/230 - http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1296180493.1296181381.30447.gz
reftest 5622/9/250 -http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1296174673.1296179894.23329.gz
and how they succeed on staging after the 10.6.6 update (instead of 10.6.2).
reftest 5631/0/230 - http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296241070.1296241767.22623.gz&fulltext=1

This shows that the update works.

The builds are on my people account: http://people.mozilla.com/~armenzg/builds




##### FOR THE RECORD

I trigger it with:
buildbot sendchange --master talos-staging-master02.build.mozilla.org:9010 --username armenzg@mozilla.com --branch tryserver-macosx64-opt-unittest --revision e0dd77d96259 --property buildid:20110127143609  http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.dmg http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.tests.zip

Since the previous sendchange uses TryChooser (somewhere) it only triggers reftests. To fool the system I am pretending to be a mozilla-central change and therefore I can get the full run (yay for cheating!):
buildbot sendchange --master talos-staging-master02.build.mozilla.org:9010 --username sendchange-unittest --branch mozilla-central-macosx64-opt-unittest --revision e0dd77d96259 --property buildid: 20110127143609  http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.dmg http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.tests.zip


talos-r3-snow-001:talos-slave cltbld$ softwareupdate -l
Software Update Tool
Copyright 2002-2009 Apple

2011-01-28 08:16:34.054 softwareupdate[407:6517] PackageKit: Missing bundle path, skipping: <bundle id="com.apple.NetworkUtility"></bundle>
Software Update found the following new or updated software:
   * iPhoto_812-8.1.2
        iPhoto Update (8.1.2), 13259K [recommended]
   * MacOSXUpdCombo10.6.6-10.6.6
        Mac OS X Update Combined (10.6.6), 1034674K [recommended] [restart]
   * AirPortClientUpdate2009002-1.0
        AirPort Client Update 2009-002 (1.0), 13010K [recommended] [restart]
   * iMovie_805-8.0.6
        iMovie Update (8.0.6), 36271K [recommended]
   * iTunesX-10.1.2
        iTunes (10.1.2), 90946K [recommended]
   * RAWCamera27-2.7
        Digital Camera Raw Compatibility Update (2.7), 7110K [recommended]
   * AirPortUtility-5.5.2
        AirPort Utility (5.5.2), 11990K [recommended]
   * RemoteDesktopClient-3.4
        Remote Desktop Client Update (3.4), 5960K [recommended]
talos-r3-snow-001:talos-slave cltbld$ sudo time softwareupdate --install MacOSXUpdCombo10.6.6-10.6.6
Software Update Tool
Copyright 2002-2009 Apple

2011-01-28 08:17:53.295 softwareupdate[413:5803] PackageKit: Missing bundle path, skipping: <bundle id="com.apple.NetworkUtility"></bundle>

Downloading Mac OS X Update Combined
Waiting to install Mac OS X Update Combined
   Checking packages…
Installing 
   Waiting for other installations to complete…2011-01-28 08:19:37.327 softwareupdate[413:34a3] PackageKit: Missing bundle path, skipping: <bundle id="com.apple.NetworkUtility"></bundle>

   Validating packages…2011-01-28 08:19:37.634 softwareupdate[413:34a3] PackageKit: Missing bundle path, skipping: <bundle id="com.apple.NetworkUtility"></bundle>

   Writing files…
   Running package scripts…
   Removing old files…
   Optimizing system for installed software…
   Moving items into place…
   Registering updated components…
   Running package scripts…
   Registering updated applications…
   Writing package receipts…
Installed Mac OS X Update Combined
Done.

You have installed one or more updates that requires that you restart your
computer.  Please restart immediately.
      598.27 real       240.44 user        80.22 sys
Attachment #507924 - Flags: review?(dustin)
(In reply to comment #2)
> Since the previous sendchange uses TryChooser (somewhere) it only triggers
> reftests. To fool the system I am pretending to be a mozilla-central change and
> therefore I can get the full run (yay for cheating!):
> buildbot sendchange --master talos-staging-master02.build.mozilla.org:9010
> --username sendchange-unittest --branch mozilla-central-macosx64-opt-unittest
> --revision e0dd77d96259 --property buildid: 20110127143609 
> http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.dmg
> http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.tests.zip

Small typo:
buildbot sendchange --master talos-staging-master02.build.mozilla.org:9010 --username sendchange-unittest --branch mozilla-central-macosx64-opt-unittest --revision e0dd77d96259 http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.dmg http://people.mozilla.com/~armenzg/builds/firefox-4.0b11pre.en-US.mac.tests.zip
I tested the builds and packages provided by joe's commit on the 10.6.6 machine and I got every test suite to be green.

My testing and analysis is now completed.

I will keep the machine running for the following changeset just to keep on doing jobs over the weekend:
http://tbpl.mozilla.org/?tree=Firefox&rev=5e576c6e3d75

Results:
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296248331.1296248723.20883.gz&fulltext=1
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296248005.1296248227.18678.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296243216.1296244335.1320.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296246278.1296246951.13273.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296246950.1296247425.15351.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296245962.1296246279.10018.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296244973.1296245672.7658.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296245669.1296245963.8603.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296248721.1296248868.21223.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296247424.1296248006.17624.gz
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1296244334.1296244975.4425.gz
Created attachment 507990 [details] [diff] [review]
Rename from 10.6.2 to 10.6.6 for testing slaves

I recently noticed that we should update our builder name as well.
I have it running on staging so I don't know yet if it can has any problems.
I am afraid it might affect graph server posts.
Attachment #507990 - Flags: review?(bhearsum)
(Reporter)

Comment 6

6 years ago
I see no reason to avoid the upgrade to 10.6.6. I've verified that acceleration is enabled, and reftests are fine.
(In reply to comment #5)
> I am afraid it might affect graph server posts.

Should check this carefully. For starters we have 'Mac OSX 10.6.2' as a platform option, lots of 10.6.2 in data.sql, eg
 http://hg.mozilla.org/graphs/file/ac18c882f3f5/sql/data.sql#l17
Comment on attachment 507924 [details] [diff] [review]
Add 10.6.6 to the list of supported puppet platforms

In this patch, the added non-comment line should be 10.6.0, not 10.2.0.

If we had a newer version of puppet, we could use regexes :(

However, the string 10.2.0 appears lots of places in the puppet configs, and you'll need to test/fix them all.

./manifests/base/os.pp:        "10.2.0" => "darwin10"
./manifests/includes/functions.pp:        "10.2.0": {
./manifests/os/osx.pp:        "10.2.0": {
./manifests/os/osx.pp:        "10.2.0": {
./manifests/os/talos_osx.pp:        "10.2.0": {
./manifests/os/talos_osx.pp:       "10.2.0": {
./manifests/os/talos_osx.pp:        "10.2.0": {
./manifests/packages/devtools.pp:                "10.2.0": {
./manifests/packages/repackaging-tools.pp:                "10.2.0": {
Attachment #507924 - Flags: review?(dustin) → review-
Attachment #507990 - Flags: review?(bhearsum) → review+
Created attachment 508410 [details] [diff] [review]
Add 10.6.6 to the list of supported puppet platforms and update

Thanks for catching that!
We would stop managing puppet with only my change.

I have checked on talos-r3-snow-001 and moz2-darwin10-slave03 that there is no fallouts on talking with puppet.

There is no need to address os/osx.pp, packages/devtools.pp packages/repackaging-tools.pp as they are just for builder machines.
Attachment #507924 - Attachment is obsolete: true
Attachment #508410 - Flags: review?(dustin)
Attachment #508410 - Flags: review?(bhearsum)
Created attachment 508412 [details] [diff] [review]
Add 10.6.6 to the graph server

Nick how can I figure out why I don't get any problems on staging when I run talos jobs on staging?

AFAIK I have not landed anything on graphs-stage's DB.


TODO add scraping on tbox once this goes live
Attachment #508412 - Flags: review?(nrthomas)
I guess I can just check by looking at graph-stage's web UI.
Attachment #508410 - Flags: review?(dustin) → review+
Created attachment 508417 [details] [diff] [review]
Add 10.6.6 to the graph server [v1]

Interesting I can see the results:
http://graphs-stage.mozilla.org/graph.html#type=series&tests=[{%22test%22:%2236%22,%22branch%22:%221%22,%22machine%22:%22606%22,%22testrun%22:%221591819%22}]
but it says its a 10.6.2 machine even thought the builder name has 10.6.6 in it.
Tp4 on MacOSX 10.6.2
talos-r3-snow-001, Firefox branch
1/31/2011 8:14:19 AM
Build ID: 20110131035758

I think this is the right patch.
We move the snow slaves to point to the 21th os (10.6.6 - assuming that's the primary key we get) rather than the 17th (10.6.2).

I will try to get this into graphs-stage's DB before you wake up.
Attachment #508412 - Attachment is obsolete: true
Attachment #508417 - Flags: feedback?(nrthomas)
Attachment #508412 - Flags: review?(nrthomas)
Attachment #508410 - Flags: review?(bhearsum) → review+
Created attachment 508484 [details]
[sql statements] add 10.6.6 to talos

This is what catlee deployed for me on graphs-stage today.
Attachment #508484 - Flags: review?(nrthomas)
Armen, I don't know the graph server enough to review these patches. Please catlee or alice.
(Assignee)

Updated

6 years ago
Attachment #508417 - Attachment description: [not tested] Add 10.6.6 to the graph server [v1] → Add 10.6.6 to the graph server [v1]
Attachment #508417 - Flags: feedback?(nrthomas) → review?(catlee)
Comment on attachment 508484 [details]
[sql statements] add 10.6.6 to talos

You already did this for me.
Do you approve so I can ask IT to land it for me?
Attachment #508484 - Flags: review?(nrthomas) → review?(catlee)
joe we are scheduling this to happen on Thursday morning from 6AM to 8AM PST to tag along with another downtime going on.
We will bring it up on the Tuesday meeting for approval.
Sounds good?
Flags: needs-treeclosure?
(Reporter)

Comment 17

6 years ago
Sounds lovely.
Comment on attachment 508484 [details]
[sql statements] add 10.6.6 to talos

update machines SET os_id=21 WHERE MATCH(name) AGAINST ('talos-r3-snow-'); 

this needs to be rewritten using LIKE.

also, the os_id on production may be different.
Attachment #508484 - Flags: review?(catlee) → review-

Updated

6 years ago
Attachment #508417 - Flags: review?(catlee) → review+
Created attachment 508898 [details]
[sql statements] add 10.6.6 to talos [v1]

I can't make my local mysql to work right now.

Would this address the issues you mention?
Attachment #508484 - Attachment is obsolete: true
Attachment #508898 - Flags: review?(catlee)
Comment on attachment 508410 [details] [diff] [review]
Add 10.6.6 to the list of supported puppet platforms and update

http://hg.mozilla.org/build/puppet-manifests/rev/46a59bcbf7a7
Attachment #508410 - Flags: checked-in+
Created attachment 509197 [details]
[sql statements] add 10.6.6 to talos [v2]

Is this ready for IT to apply it? Shall I wait until tomorrow just before the downtime?
Attachment #508898 - Attachment is obsolete: true
Attachment #509197 - Flags: review?(catlee)
Attachment #508898 - Flags: review?(catlee)
Created attachment 509220 [details] [diff] [review]
[buildbotcustom] adjust tryserver to work with 10.6.6 renaming
Attachment #509220 - Flags: review?(lsblakk)
(Assignee)

Updated

6 years ago
Depends on: 631011
Comment on attachment 509220 [details] [diff] [review]
[buildbotcustom] adjust tryserver to work with 10.6.6 renaming

Thanks for updating the try_parser tests too.
Attachment #509220 - Flags: review?(lsblakk) → review+

Updated

6 years ago
Attachment #509197 - Flags: review?(catlee) → review+
(Assignee)

Updated

6 years ago
Attachment #507990 - Flags: checked-in-
(Assignee)

Updated

6 years ago
Attachment #508417 - Flags: checked-in-
(Assignee)

Updated

6 years ago
Attachment #509197 - Flags: checked-in-
(Assignee)

Updated

6 years ago
Attachment #509220 - Flags: checked-in-
We did not rename the slaves from 10.6.2 to 10.6.6 after all.
It would cause a discontinuity of the 10.6.2 posts on the graph server.

I have filed bug 631256 to keep track of the renaming work (I am proposing to do it after FF4) and talk with webdev on how to port forward the 10.6.2 posts.

All machines have been updated to 10.6.6 and I am using this changeset as the cut-line:
http://tbpl.mozilla.org/?tree=Firefox&rev=094a7967e171

Please ignore the set of talos jobs which were trying to run debug builds rather than opt builds (bug 631237). I triggered another 2 full sets to cover that up.

I have also updated and rebooted the ref image.

To determine if a machine has the change you can type:
> uname -r
> 10.6.0
instead of
> 10.2.0

Updated

6 years ago
Blocks: 631276
(Assignee)

Updated

6 years ago
No longer depends on: 631011
https://wiki.mozilla.org/ReferencePlatforms/Test/SnowLeopard#Update_from_10.6.2_to_10.6.6

Asked to update the ref image in https://bugzilla.mozilla.org/show_bug.cgi?id=617105#c16
Nothing left to be done.
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Incidentally, this update is no longer available.  I'll update the wiki accordingly.
I saw your changes. Thank you.

FTR it needs to be installed manually.
http://support.apple.com/kb/DL1349
I couldn't get that update to work - it failed for me without any real description of why.
(Assignee)

Updated

5 years ago
Flags: needs-treeclosure?
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.