Last Comment Bug 673131 - when minor talos changes land, the a-team should be able to deploy with minimal releng time required
: when minor talos changes land, the a-team should be able to deploy with minim...
Status: RESOLVED FIXED
[talos][qa-]
:
Product: Release Engineering
Classification: Other
Component: Other (show other bugs)
: other
: x86 Linux
: P2 normal (vote)
: ---
Assigned To: Armen Zambrano [:armenzg] (EDT/UTC-4)
:
:
Mentors:
Depends on: 701484
Blocks: 721822
  Show dependency treegraph
 
Reported: 2011-07-21 08:56 PDT by Joel Maher ( :jmaher)
Modified: 2013-08-12 21:54 PDT (History)
10 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
wontfix
affected
fixed
affected
?


Attachments
[poc] talos_from_code.py (1011 bytes, patch)
2011-12-01 14:39 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
no flags Details | Diff | Splinter Review
talos_from_code.py - v0.1 (2.46 KB, patch)
2011-12-09 14:51 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
jmaher: feedback+
Details | Diff | Splinter Review
[poc] factory/pu changes (2.38 KB, patch)
2011-12-09 14:52 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
no flags Details | Diff | Splinter Review
talos.json (69 bytes, patch)
2011-12-14 13:51 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
coop: review+
Details | Diff | Splinter Review
buildbot-config changes to run talos_from_code on mozilla-central (2.83 KB, patch)
2011-12-14 13:53 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
no flags Details | Diff | Splinter Review
talos_from_code.py - take 1 (2.83 KB, patch)
2011-12-14 13:53 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
coop: review+
jmaher: review+
armenzg: checked‑in+
Details | Diff | Splinter Review
buildbotcustom changes to run talos_from_code.py (5.63 KB, patch)
2011-12-14 13:53 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
coop: review+
armenzg: checked‑in+
Details | Diff | Splinter Review
talos.json patch (512 bytes, patch)
2011-12-15 07:06 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
armenzg: review+
akeybl: approval‑mozilla‑aurora+
akeybl: approval‑mozilla‑beta+
akeybl: approval‑mozilla‑release+
akeybl: approval1.9.2.25+
Details | Diff | Splinter Review
buildbot-configs - enable talos_from_source code only for mozilla-central (2.32 KB, patch)
2011-12-15 13:21 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
coop: review+
armenzg: checked‑in+
Details | Diff | Splinter Review
enable talos_from_source_code by default and exempt some release branches (4.45 KB, patch)
2011-12-20 12:31 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
coop: review+
armenzg: checked‑in-
Details | Diff | Splinter Review
use a python that has simplejson (2.19 KB, patch)
2011-12-21 13:57 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
coop: review+
armenzg: checked‑in+
Details | Diff | Splinter Review
enable for normal and project branches (2.46 KB, patch)
2011-12-22 07:00 PST, Armen Zambrano [:armenzg] (EDT/UTC-4)
coop: review+
armenzg: checked‑in+
Details | Diff | Splinter Review

Description Joel Maher ( :jmaher) 2011-07-21 08:56:11 PDT
many of the changes we make to talos do not change tests or performance numbers, they are bug fixes and stability changes to the tools.  

It would be nice if we could have some method where we can check into m-c a version number (hg revision, link to talos.zip, etc...) so it will be updated.

there are a lot of issues to discuss:
 - how to roll this out to all branches (bootstrapping)
 - updating the buildbot scripts
 - how to define what we are pulling
Comment 1 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-07-21 08:57:45 PDT
I will follow up with a blog post explaining all the pieces and changes on the work flow.
Comment 2 alice nodelman [:alice] [:anode] 2011-07-21 13:08:57 PDT
As a note, whatever system that we come up with we should try and do the same for the pageloader bundle - it is going through the same deployment woes that talos is.
Comment 3 alice nodelman [:alice] [:anode] 2011-07-21 13:10:43 PDT
To better define a 'minor' talos change:

- something that does not affect currently collected talos numbers
- something that does not add a new test

Basically, just a bug fix or feature addition to the talos code base that has passed ateam staging and is considered to be low risk of causing any alteration to the current testing infrastructure (other than making it better, of course).
Comment 4 Joel Maher ( :jmaher) 2011-08-22 07:15:23 PDT
can we start putting some details down about how to work around this?  It would be nice to have a game plan figured out even if we are not ready to execute it.
Comment 5 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-09-01 07:40:19 PDT
I was thinking about this last night and there are several parts that we would like to control from m-c.

As an example of a similar system we have jetpack which has a file checked-in on m-c:
http://mxr.mozilla.org/mozilla-central/source/testing/jetpack/jetpack-location.txt

I believe the right way to go is to have a json file that indicates the values of the following:
TALOS_TP_OPTS = {'plugins': 'zips/plugins.zip', 'pagesets': ['zips/tp5.zip']}
TALOS_TP4_OPTS = {'plugins': 'zips/plugins.zip', 'pagesets': ['zips/tp4.zip',]}
TALOS_ADDON_OPTS = {'addonTester' : True, 'releaseTester' : True, 'plugins': 'zips/plugins.zip'}
TALOS_BASELINE_ADDON_OPTS = {'releaseTester' : True, 'plugins': 'zips/plugins.zip'}

This file would be read on the fly at the beginning of a talos job and download the required files.

The added value to this are the following:
* a talos landing is visible to every one
* a performance variance can be reported by the regression emails
* it can be backed out without relengs
* you can use the try server to test new bundles and even use the compare-talos tool
* every push has a matching set of bundles that do not vary after the push (right now if we switch anything some test jobs for a changeset can grab the newer files)

For minor changes you would probably want to land the change with the comment DONTBUILD.

This system would improve both minor and major changes by not needing downtimes (hopefully) for the major changes (as the push would be blamed for regressions)

For now I would prefer to keep all the bundles in build.m.o by asking releng to pull them in. We should ensure that the json file (or whatever we use) would be matching integrity rules and not allow to grab files from places that are not build.m.o (perhaps in the future it would make sense to be more flexible).
Comment 6 Joel Maher ( :jmaher) 2011-09-01 08:27:20 PDT
So this makes a lot of sense, except things like tp5 and plugins.zip are highly likely to cause regressions or need rebaselining (maybe even a reconfig to absorb config.py changes).  likewise with pageloader.xpi

Most of the changes we make are minor changes to talos.zip (addons, xperf, remote) which do not affect talos proper or the numbers.  

Also when releng does a downtime would the process be the same?  I.e. we can look at a hg revision history and track what version we were using.

So I am envisioning a talos-locations.txt:
http://hg.mozilla.org/projects/talos/archive/31415926535/talos.zip
http://hg.mozilla.org/projects/talos/archive/31415926535/tp5.zip
http://hg.mozilla.org/projects/talos/archive/31415926535/plugins.zip
http://hg.mozilla.org/projects/talos/archive/31415926535/dirtyprofiles.zip
http://hg.mozilla.org/projects/talos/archive/31415926535/pageloader.xpi

So the work would need to be done in buildbot to accommodate these referenced .zip files instead of being hardcoded.  For talos, can we access http://mxr.mozilla.org/mozilla-central/source/testing/talos/talos-locations.txt?  We don't download a tests.zip file or anything, so I want to make sure we don't overlook that small detail.
Comment 7 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-09-01 08:36:13 PDT
(In reply to Joel Maher (:jmaher) from comment #6)
> 
> Also when releng does a downtime would the process be the same?  I.e. we can
> look at a hg revision history and track what version we were using.
I believe we could actually not have downtimes but we will see once we get there.
 
> So the work would need to be done in buildbot to accommodate these
> referenced .zip files instead of being hardcoded.  For talos, can we access
> http://mxr.mozilla.org/mozilla-central/source/testing/talos/talos-locations.
> txt?  We don't download a tests.zip file or anything, so I want to make sure
> we don't overlook that small detail.

We could access through hg and raw:
http://hg.mozilla.org/mozilla-central/raw-file/%(revision)s/testing/talos/talos-locations.
Comment 8 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-11-15 15:58:22 PST
I will be tackling this either this week or next one.
Comment 9 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-01 14:39:33 PST
Created attachment 578400 [details] [diff] [review]
[poc] talos_from_code.py

Feedback wanted:
* is testing/talos/talos.json a good location for the json file?
* is talos_from_code.py a good name for the script?
* in the future we will rewrite talos on mozharness but for now where should I put the script? should I land it on mozharness?

#####
This script downloads a talos.json file [1] which indicates to grab the talos.zip from a given location [2] (it can be from anywhere at this point).

This script will need to:
* retrieve the script to the talos machine (I don't care where)
* ask TalosFactory to give us the tree's revision
* run the script by passing [3]
** this will download the specified talos.zip on the json file

Needed:
* accept calling the script with the URL to the json file
* hack TalosFactory to download and run the script 

Optional:
* add progress for downloading it
* optimize to not download the same talos.zip twice
* extend the script to download all artifacts for a given talos job (pageloader.xpi, plugins, etc)

[1] http://people.mozilla.com/~armenzg/talos/talos.json
[2] http://hg.mozilla.org/build/talos/archive/5ae66b11330f.zip
[3] http://hg.mozilla.org/mozilla-central/raw-file/c120734d20ba/testing/talos/talos.json
Comment 10 Joel Maher ( :jmaher) 2011-12-02 11:31:28 PST
so if I put a talos.zip on my people account I see there being problems with every talos machine downloading talos.zip from there.

otherwise I like this, it will be much easier to change things around then.

as for mozharness, I know we have a script for remote and desktop versions of talos ready to go.  It needs some more love and deployment when there is bandwidth.  What I don't know is if the mozharness stuff will allow dynamic updating of a talos.zip.
Comment 11 Aki Sasaki [:aki] back dec19 2011-12-02 11:35:47 PST
(In reply to Joel Maher (:jmaher) from comment #10)
> as for mozharness, I know we have a script for remote and desktop versions
> of talos ready to go.  It needs some more love and deployment when there is
> bandwidth.  What I don't know is if the mozharness stuff will allow dynamic
> updating of a talos.zip.

Talos on mozharness should be able to both grab a talos zip or a specified revision from source when it's done.  We should be able to specify which talos zip, have that zip be different across branches if desired, and we can edit the config files on the fly to point at a different location if, say, people.m.o goes down.

However, re: the other bit, I do not like the idea of people.m.o being a critical piece of infrastructure.  It's not guaranteed to be up 24/7.  If we serve the talos zip off there for all talos jobs, all talos jobs will go red when people.m.o goes down.
Comment 12 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-02 11:37:37 PST
(In reply to Joel Maher (:jmaher) from comment #10)
> so if I put a talos.zip on my people account I see there being problems with
> every talos machine downloading talos.zip from there.
> 
On the try server I assume it would be good to allow it to happen.
I assume people would use reviews to land this on the development repos.

Would it makes sense to modify the script to stop the job if something different than http://build.m.o/talos/zips is used for non-try repos? We could add a cgi to upload official talos without releng's involvement.
Comment 13 Joel Maher ( :jmaher) 2011-12-02 11:45:04 PST
this is all sounding good.  Try server for anything; otherwise build.m.o via upload script for production.
Comment 14 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-09 14:51:38 PST
Created attachment 580561 [details] [diff] [review]
talos_from_code.py - v0.1

What do you think so far?

##############################
Armens-MacBook-Air:talos armenzg$ python talos_from_code.py --talos_json_url http://localhost/~armenzg/talos.json --branch try; echo $?
INFO: talos.json URL:  http://localhost/~armenzg/talos.json
INFO: talos.zip URL: 'http://localhost/~armenzg/talos.zip'
0
Armens-MacBook-Air:talos armenzg$ python talos_from_code.py --talos_json_url http://localhost/~armenzg/talos.json --branch wrong-branch; echo $?
INFO: talos.json URL:  http://localhost/~armenzg/talos.json
INFO: talos.zip URL: 'http://localhost/~armenzg/talos.zip'
ERROR: You have tried to download a talos.zip from a location different thanhttp://build.mozilla.org/talos/zips
1
Armens-MacBook-Air:talos armenzg$ cat ~/Sites/talos.json 
{
    "talos_zip": "http://localhost/~armenzg/talos.zip"
}
Comment 15 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-09 14:52:56 PST
Created attachment 580562 [details] [diff] [review]
[poc] factory/pu changes
Comment 16 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-09 14:54:34 PST
coop, what do you think of putting the file in:
http://hg.mozilla.org/build/tools/file/default/scripts/talos/talos_from_code.py ?

Does the script name seem good?
Comment 17 Joel Maher ( :jmaher) 2011-12-10 10:13:46 PST
Comment on attachment 580561 [details] [diff] [review]
talos_from_code.py - v0.1

>#! /usr/bin/env python
>import argparse
>import json

do we have json available on our boxes?  I know we don't for talos proper in 2.4, that is why we use pyyaml.  Also simplejson seems to be a common fallback:
try:
  import json
except:
  import simplejson as json


>def get_filename_from_url(url):
>    '''
>    This returns the filename of the file we're trying to download
>    '''
>    parsed = urlparse.urlsplit(url.rstrip('/'))
>    if parsed.path != '':
>        return parsed.path.rsplit('/', 1)[-1]
>    else:
>        file_name = parsed.netloc

we don't return anything in the else case.

>
>def download_file(url, saveAs=None):
>    '''
>    It downloads a file from the URL indicated and can be saved locally with
>    a different name if needed.
>    '''
>    req = urllib2.Request(url)
>    filename = get_filename_from_url(url)
>    f = urllib2.urlopen(req)
>    local_file = open(saveAs if saveAs else filename, 'wb')

this inline if/else blocks do not work in python 2.4.  It would be nice to put this in a try/catch block and return a value.

>if __name__ == '__main__':

normal style at Mozilla is to put this in a main() function and call it as so:
if __name__ == '__main__':
  main()

>    # retrieve the talos.zip as "talos.zip"
>    download_file(talos_zip_url, "talos.zip")

we need to determine if this was successful or not.  Maybe the network resource is not available.  Should we check the integrity of the talos.zip before exiting this fuction?

Overall, this is great and seems like it will work except for some corner cases.  I put comments in regarding python2.4 since that is what we run for talos, I believe python2.5+ will work just fine.
Comment 18 Mike Taylor [:bear] 2011-12-10 15:51:33 PST
(In reply to Joel Maher (:jmaher) from comment #17)
> Comment on attachment 580561 [details] [diff] [review]
> talos_from_code.py - v0.1
> 
> >#! /usr/bin/env python
> >import argparse
> >import json
> 
> do we have json available on our boxes?  I know we don't for talos proper in
> 2.4, that is why we use pyyaml.  Also simplejson seems to be a common
> fallback:

I believe aki has just explored this for mozharness but I can't recall what bug he documented the results in - you may want to ask him.  I believe the answer was "yes, but..."


> 
> Overall, this is great and seems like it will work except for some corner
> cases.  I put comments in regarding python2.4 since that is what we run for
> talos, I believe python2.5+ will work just fine.

really? 2.4 - gosh I hope not.  I thought the minimum was 2.5 now with 2.6 becoming more common.
Comment 19 Joel Maher ( :jmaher) 2011-12-10 15:53:36 PST
yeah, desktop talos runs on python2.4, we are working on a method to install the pywin32 modules automatically so we can upgrade to 2.5 (and then 2.6 afterwards will be a piece of cake since we are doing the work now)
Comment 20 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-14 13:51:11 PST
Created attachment 581770 [details] [diff] [review]
talos.json
Comment 21 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-14 13:53:10 PST
Created attachment 581772 [details] [diff] [review]
buildbot-config changes to run talos_from_code on mozilla-central
Comment 22 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-14 13:53:16 PST
Created attachment 581773 [details] [diff] [review]
talos_from_code.py - take 1
Comment 23 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-14 13:53:19 PST
Created attachment 581774 [details] [diff] [review]
buildbotcustom changes to run talos_from_code.py
Comment 24 Joel Maher ( :jmaher) 2011-12-14 14:01:47 PST
Comment on attachment 581773 [details] [diff] [review]
talos_from_code.py - take 1

Review of attachment 581773 [details] [diff] [review]:
-----------------------------------------------------------------

thanks, this looks good.
Comment 25 Chris Cooper [:coop] 2011-12-14 14:22:28 PST
Comment on attachment 581772 [details] [diff] [review]
buildbot-config changes to run talos_from_code on mozilla-central

Review of attachment 581772 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with nits addressed.

::: scripts/talos/talos_from_code.py
@@ +9,5 @@
> +import urlparse
> +
> +def main():
> +    parser = OptionParser()
> +    parser.add_option("--talos_json_url", dest="talos_json_url", type="string",

I think it might be standard practice to use, e.g., --talos-json-url for the command-line option vs. --talos_json_url, and then use talos_json_url internally.

@@ +20,5 @@
> +
> +    # json file with info on which talos.zip to use
> +    # the json file URL should look like this:
> +    #  %(repo_path)s/raw-file/%(revision)s/testing/talos/talos.json
> +    jsonFilename = download_file(options.talos_json_url)

Should probably have a try block around this instance of download_file too, just like you do below.
Comment 26 Chris Cooper [:coop] 2011-12-14 14:35:05 PST
Comment on attachment 581773 [details] [diff] [review]
talos_from_code.py - take 1

Review of attachment 581773 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with nits addressed.

::: scripts/talos/talos_from_code.py
@@ +9,5 @@
> +import urlparse
> +
> +def main():
> +    parser = OptionParser()
> +    parser.add_option("--talos_json_url", dest="talos_json_url", type="string",

I think it might be standard practice to use, e.g., --talos-json-url for the command-line option vs. --talos_json_url, and then use talos_json_url internally.

@@ +20,5 @@
> +
> +    # json file with info on which talos.zip to use
> +    # the json file URL should look like this:
> +    #  %(repo_path)s/raw-file/%(revision)s/testing/talos/talos.json
> +    jsonFilename = download_file(options.talos_json_url)

Should probably have a try block around this instance of download_file too, just like you do below.
Comment 27 Chris Cooper [:coop] 2011-12-14 14:37:50 PST
Comment on attachment 581772 [details] [diff] [review]
buildbot-config changes to run talos_from_code on mozilla-central

Looks like you double-posted the python script instead of the config changes.
Comment 28 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-15 07:06:46 PST
Created attachment 581954 [details] [diff] [review]
talos.json patch

Now in a patch format.
Comment 29 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-15 07:08:19 PST
Hi,
Could you please land for me attachment 581954 [details] [diff] [review] to mozilla-central or mozilla-inbound?

This is a NPOTB and DONTBUILD.

Thanks!
Comment 31 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-15 12:12:13 PST
Comment on attachment 581954 [details] [diff] [review]
talos.json patch

Hi release-drivers would you please be able to approve this patch to land on 1.9.2, aurora, beta and release?

This is a NPOTB and a DONTBUILD type of patch.
This patch has already landed on mozilla-inbound (.

This file is the new way to configure talos.zip for talos/performance jobs and it needs to land on every branch. Nevertheless, this does *not* block me as I have branch granularity.
Comment 32 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-15 12:22:57 PST
Comment on attachment 581773 [details] [diff] [review]
talos_from_code.py - take 1

http://hg.mozilla.org/build/tools/rev/667d675c733c

I have only landed the two patches that do not turn this new system live.
Comment 33 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-15 13:19:28 PST
Comment on attachment 581774 [details] [diff] [review]
buildbotcustom changes to run talos_from_code.py

This change can also land as it is a no-op since we default to the old version.
http://hg.mozilla.org/build/buildbotcustom/rev/4aeb797aea9a
Comment 34 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-15 13:21:45 PST
Created attachment 582083 [details] [diff] [review]
buildbot-configs - enable talos_from_source code only for mozilla-central

Once the patch goes from mozilla-inbound into mozilla-central we can land this.

There is no rush to land this so I'll wait until Wednesday morning.

On Thursday we can land a change that will enable talos_from_source_code if all approvals arrive on time and nothing goes wrong on Wednesday.
Comment 35 Ed Morley [:emorley] 2011-12-16 06:17:51 PST
https://hg.mozilla.org/mozilla-central/rev/53a13d60eaf5
Comment 36 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-16 06:45:43 PST
Thanks Ed for landing it!

There's one more patch to land on the releng side so reopening.
Comment 37 Alex Keybl [:akeybl] 2011-12-16 12:28:41 PST
Comment on attachment 581954 [details] [diff] [review]
talos.json patch

[Triage Comment]
Approving for Aurora and Beta.

As for release or 1.9.2 (both going out on the 20th), I understand that this won't cause build regressions. Is there any opportunity for this to cause test regressions or changes in behavior though? If so, let's wait until after we ship FF3.6.25/9.
Comment 38 Alex Keybl [:akeybl] 2011-12-16 12:31:38 PST
Comment on attachment 581954 [details] [diff] [review]
talos.json patch

[Triage Comment]
Just spoke with Armen who clarified that this would not affect automated testing. Approving for all branches requested.
Comment 39 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-16 13:42:04 PST
Hi checkin-needed hero,
Would you please be able to land attachment 581954 [details] [diff] [review] on aurora, beta, release and 1.9.2?
Thanks and sorry it is so many branches :(
Comment 40 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-20 12:30:40 PST
Comment on attachment 582083 [details] [diff] [review]
buildbot-configs - enable talos_from_source code only for mozilla-central

checked-in on default:
http://hg.mozilla.org/build/buildbot-configs/rev/27198f5bff61

This will be picked up next time we reconfig our masters (next scheduled time is Thursday).
Comment 41 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-20 12:31:32 PST
Created attachment 583252 [details] [diff] [review]
enable talos_from_source_code by default and exempt some release branches

This would exempt mozilla-beta, mozilla-release and mozilla-1.9.2 (since talos.json has not been landed there yet).
Comment 42 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-20 14:12:04 PST
Comment on attachment 583252 [details] [diff] [review]
enable talos_from_source_code by default and exempt some release branches

Landed on "default":
http://hg.mozilla.org/build/buildbot-configs/rev/073d4cd32699
Comment 43 Ben Hearsum (:bhearsum) 2011-12-20 18:43:45 PST
Attachments 582083 and 583252 made it to production tonight.
Comment 44 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-20 19:49:26 PST
I backed this out as it was causing reds.
I will be looking deeper into it tomorrow but we were having "simplejson" is missing for mac.
I thought I tested it properly on staging for all platforms.
Comment 45 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-21 13:57:03 PST
Created attachment 583615 [details] [diff] [review]
use a python that has simplejson

I'm still running it on staging but manual testing has proven to be good.
Comment 46 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-21 14:17:16 PST
Comment on attachment 583252 [details] [diff] [review]
enable talos_from_source_code by default and exempt some release branches

This was backed-out yesterday.
Comment 47 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-22 07:00:57 PST
Created attachment 583786 [details] [diff] [review]
enable for normal and project branches

I really have to unify those two sections.

talos_from_source_code has worked on all platforms on staging.

I had not tested it correctly previously.
Comment 48 Chris Cooper [:coop] 2011-12-22 07:16:55 PST
Comment on attachment 583615 [details] [diff] [review]
use a python that has simplejson

Review of attachment 583615 [details] [diff] [review]:
-----------------------------------------------------------------

::: process/factory.py
@@ -7067,5 @@
> -            return "/home/cltbld/bin/python"
> -        elif (platform == "leopard"):
> -            return "/usr/bin/python"
> -        elif (platform in ("snowleopard", "lion")):
> -            return "/Users/cltbld/bin/python"

I have never seen this code before, and now I cannot unsee it. My poor, virgin eyes.
Comment 49 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-22 08:09:07 PST
Comment on attachment 583615 [details] [diff] [review]
use a python that has simplejson

http://hg.mozilla.org/build/buildbotcustom/rev/eee5d445b9b6
Comment 50 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-22 08:09:20 PST
Comment on attachment 583786 [details] [diff] [review]
enable for normal and project branches

http://hg.mozilla.org/build/buildbot-configs/rev/f03744c11c9d
Comment 51 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-22 08:15:23 PST
The patches that landed on "default" are now on production and the masters have been reconfigured.
Comment 52 Armen Zambrano [:armenzg] (EDT/UTC-4) 2011-12-22 11:56:10 PST
This is completed.
http://armenzg.blogspot.com/2011/12/taloszip-talosjson-and-you.html

If anyone cares to land the approved patches on the releases branches please go ahead.
Otherwise in 11.5 weeks only 1.9.2 would be different that the other ones.
Comment 53 Ed Morley [:emorley] 2012-02-05 11:19:50 PST
Adjusting flags/whiteboard post uplift.

Armen, is this still wanted on beta? Presume 1.9.2 is wontfix at this point, given ESR - in which case presume wanted for ESR 10.1? 

I don't have a beta or 1.9.2 tree checked out (my poor HD is dying, so I've culled as many repos as possible), or I'd land these for you. I suspect most people who look at check-neededs have fixed bugs filtered out, so asking directly on #developers might be the best way forwards.
Comment 54 Armen Zambrano [:armenzg] (EDT/UTC-4) 2012-02-06 14:21:51 PST
Thanks edmorley. I found out I can land it myself plus I just kept the system disabled for those branches until we merge there.
Comment 55 Armen Zambrano [:armenzg] (EDT/UTC-4) 2012-07-18 14:08:28 PDT
We then moved the file to the source tree in bug 721822.

Note You need to log in before you can comment on or make changes to this bug.