turn on robocop talos tests

RESOLVED FIXED

Status

Release Engineering
General
P1
normal
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: jmaher, Assigned: armenzg)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mobile][talos])

Attachments

(8 attachments, 3 obsolete attachments)

5.27 KB, patch
armenzg
: review+
jmaher
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
1.31 KB, patch
jmaher
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
1.88 KB, patch
jmaher
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
2.26 KB, patch
jmaher
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
1.80 KB, patch
armenzg
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
1015 bytes, patch
armenzg
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
6.15 KB, patch
jmaher
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
1.16 KB, patch
armenzg
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
(Reporter)

Description

6 years ago
We need to replace tpan with a robocop test.  We also have tcheckerboard as well.

These will run just like other tests by doing the PerfConfigurator.py step with --activeTests=trobocop or --activeTests trobocheck.

We need to ensure that robocop.apk is installed properly first in bug 715215.  The other issue is figuring out where fennec_ids.txt will come from.  Right now this lives in tests.zip, but we don't download that for each talos run.  Would it be better to have this in the upload directory as we do fennec.apk and robocop.apk?

Comment 1

6 years ago
We need to stop proceeding any further with robocop until l10n packaging is fixed.

Comment 2

6 years ago
(In reply to Aki Sasaki [:aki] from comment #1)
> We need to stop proceeding any further with robocop until l10n packaging is
> fixed.

Aki: can you add the dependency here then, please?
Priority: -- → P3
Whiteboard: [mobile][talos]

Updated

6 years ago
Depends on: 719697
(Reporter)

Comment 3

6 years ago
the core of trobopan is landed, it is ready to start running.

note to releng: this requires us to have a fennec_ids.txt file somewhere in the food chain that we can point talos at.  We need to discuss where you might already have this file lying around or the best place to get it.

I could upload the fennec_ids.txt to the upload directory, or we could get it from the tests.zip.
(In reply to Joel Maher (:jmaher) from comment #3)
> note to releng: this requires us to have a fennec_ids.txt file somewhere in
> the food chain that we can point talos at.  We need to discuss where you
> might already have this file lying around or the best place to get it.
> 
> I could upload the fennec_ids.txt to the upload directory, or we could get
> it from the tests.zip.

We need more info to answer the question - how often does it change? Where is it built? is it big enough that copy is an issue? etc.

If it's something you generate, then likely tests.zip is the best place.
(Reporter)

Comment 5

6 years ago
the problem is tests.zip is massive (50mb) and we would have to unpack a bit of it to get a small <1k text file.  We do this already for all other test types, so if we want to do this for talos, then we can and call it good.

Alternatively I could upload fennec_ids.txt to the upload directory and it would live next to the .apk files.  This is sort of hacky, but it would make it easier in some sense.  

we generate fennec_ids.txt with each build as it is a map of the UI elements that are defined in fennec (these change quite frequently now, and probably on a regular basis going forward).

I am not sure of the exact target, but if it goes into m-c now, it would be fennec-13.  We might want to downport a bunch of tests to fennec-12 (currently m-a) so we could get good test coverage on that release.  I don't believe we would consider fennec-11.
(Reporter)

Updated

6 years ago
Depends on: 724097, 725348, 725301, 725352
(Reporter)

Comment 6

6 years ago
Here is an example commandline that I run locally:
python remotePerfConfigurator.py -v -e org.mozilla.fennec_jmaher -a tcheckerboard --output test.yml --develop --remoteDevice 192.168.1.103 --fennecIDs=fennec_ids.txt

Here is what it might look like adapted to a commandline that I see in the tbpl logs:
python remotePerfConfigurator.py -v -e org.mozilla.fennec -t tegra-074 --branchName Mozilla-Inbound --resultsServer graphs-old.mozilla.org --resultsLink /server/collect.cgi --symbolsPath ../symbols --remoteDevice 10.250.49.62 --sampleConfig remote.config --output local.yml --webServer bm-remote.build.mozilla.org --browserWait 60 --activeTests tcheckerboard --fennecIDs=fennec_ids.txt

The changes are just the values we pass to --activeTests and the --fennecIDs which is what we have been discussing in this bug to date.

the new values for --activeTests are:
 * --activeTests tcheckerboard
 * --activeTests trobopan

In my example above I am assuming we have a file called fennec_ids.txt in the same folder as run_tests.py.  You can have this point to anywhere on the local file system that the python script has access to.
I will be taking this. jmaher do you have some time tomorrow in the morning to help me understand this better? I will be looking at how talos for the tegras work this afternoon.
Assignee: nobody → armenzg
Priority: P3 → P2
Depends on: 725799
I have a patch on staging that I hope will give results soon after I managed to keep my tegra connected.
I should have a patch for review tomorrow (depending on buildduty it could be today).
I have a patch to download the fennec_ids.txt.

I hope on Tuesday the boards actually cooperate with me.

jmaher: is it enough to have the fennec_ids.txt in /builds/tegra-XXX/talos-data/talos?
(Reporter)

Comment 11

6 years ago
yes, that is the ideal location.
I am having trouble getting this to work:
http://tinderbox.mozilla.org/showlog.cgi?log=MobileTest/1330440249.1330442219.12671.gz&fulltext=1
http://tinderbox.mozilla.org/showlog.cgi?log=MobileTest/1330440249.1330441363.10021.gz&fulltext=1

jmaher is helping with this.

> getting files in '/mnt/sdcard/tests/profile/minidumps/'
> Failed trobopan: 
> 		Stopped Tue, 28 Feb 2012 06:57:27
> FAIL: Busted: trobopan
> FAIL: unrecognized output format
> Traceback (most recent call last):
>   File "run_tests.py", line 659, in <module>
>     main()
>   File "run_tests.py", line 656, in main
>     test_file(arg, options, parser.parsed)
>   File "run_tests.py", line 597, in test_file
>     raise e
> utils.talosError: 'unrecognized output format'
> program finished with exit code 1
> elapsedTime=161.549437
> TinderboxPrint:<a href = "http://hg.mozilla.org/mozilla-central/rev/6989376471f7">rev:6989376471f7</a>
> 
> TinderboxPrint:FAIL: Busted: trobopan
> 
> TinderboxPrint:FAIL: unrecognized output format
After I changed it to download to "build" I am actually sure that robocop.apk got installed:
http://dev-master01.build.scl1.mozilla.com:8043/builders/Android%20Tegra%20250%20mozilla-central%20talos%20remote-trobocheck/builds/34
http://tinderbox.mozilla.org/showlog.cgi?log=MobileTest/1330440249.1330441363.10021.gz&fulltext=1
but we are still having the "unrecognized output format".

> devroot /mnt/sdcard/tests
> Installing /mnt/sdcard/tests/robocop.apk
> in push file with: build/robocop.apk, and: /mnt/sdcard/tests/robocop.apk
> local hash returned: '2294cdf71a71eac8d7f4611d2aa0cff4'
> sending: push /mnt/sdcard/tests/robocop.apk
> push returned: 2294cdf71a71eac8d7f4611d2aa0cff4
Created attachment 601383 [details] [diff] [review]
[wip] patch for robocop and talos.zip

This is so you can have some visibility on what the patch is looking like.
I need to adjust it so it only adds the robocop step for the robocop jobs.
jmaher told me to pass --fennecIDs=<path>/fennec_ids.txt which you can see in the step [1] as --fennecIDs ../fennec_ids.txt since the pwd is /builds/tegra-026/test/../talos-data/talos

fenned_ids.txt is in 'talos-data'.

It now simply stops without much explanation [2].
> FIRE PROC: 'org.mozilla.fennec  -profile /mnt/sdcard/tests/profile http://bm-remote.build.mozilla.org/getInfo.html'
> process killed by signal 15

[1] http://dev-master01.build.scl1.mozilla.com:8043/builders/Android%20Tegra%20250%20mozilla-central%20talos%20remote-trobocheck/builds/37/steps/Update%20config/logs/stdio
[2] http://dev-master01.build.scl1.mozilla.com:8043/builders/Android%20Tegra%20250%20mozilla-central%20talos%20remote-trobocheck/builds/37/steps/Run%20performance%20tests/logs/stdio
(Reporter)

Updated

6 years ago
Depends on: 731637
(Reporter)

Updated

6 years ago
Depends on: 731755
(Reporter)

Comment 16

6 years ago
the talos changes are landed and here is a talos.zip bundle:
http://people.mozilla.org/~jmaher/taloszips/zips/talos.mobile.zip

We still need to resolve the OOM, but with this talos bundle you will get a lot further along than you were getting before.

Shall we land this bundle somewhere?  Could we just add this to talos.json?
I have the bundle on build.m.o. For now nothing else to be done.

I am getting this with robocop:
http://tinderbox.mozilla.org/showlog.cgi?log=MobileTest/1330618380.1330619117.30870.gz&fulltext=1#err0

error returned from pull: could not find metadata
error returned from pull: could not find metadata
reconnecting socket
DeviceManager: error pulling file '/mnt/sdcard/tests/browser_output.txt': No such file or directory
...
DeviceManager: error pulling file '/mnt/sdcard/tests/browser_output.txt': No such file or directory
getting files in '/mnt/sdcard/tests/profile/minidumps/'
Failed tcheckerboard: 
		Stopped Thu, 01 Mar 2012 08:25:17
FAIL: Busted: tcheckerboard
FAIL: no output from browser
Traceback (most recent call last):
  File "run_tests.py", line 666, in <module>
    main()
  File "run_tests.py", line 663, in main
    test_file(arg, options, parser.parsed)
  File "run_tests.py", line 604, in test_file
    raise e
utils.talosError: 'no output from browser'
program finished with exit code 1
elapsedTime=419.815983
I also got this on another suite:
removing file: /mnt/sdcard/tests/browser_output.txt
NOISE: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFileInputStream.init]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: chrome://pageloader/content/pageloader.js :: plLoadURLsFromURI :: line 634"  data: no]
reconnecting socket
FIRE PROC: 'org.mozilla.fennec  -profile /mnt/sdcard/tests/profile -tp page_load_test/tp4m.manifest -tpnoisy -tpformat tinderbox -tpcycles 2 -rss -tpdelay 1000'
NOISE: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFileInputStream.init]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: chrome://pageloader/content/pageloader.js :: plLoadURLsFromURI :: line 634"  data: no]
getting files in '/mnt/sdcard/tests/profile/minidumps/'
Failed tp4m: 
		Stopped Wed, 29 Feb 2012 14:17:16
FAIL: Busted: tp4m
FAIL: unrecognized output format
Traceback (most recent call last):
  File "run_tests.py", line 659, in <module>
    main()
  File "run_tests.py", line 656, in main
    test_file(arg, options, parser.parsed)
  File "run_tests.py", line 597, in test_file
    raise e
utils.talosError: 'unrecognized output format'
program finished with exit code 1
elapsedTime=1324.439399
and this:
NOISE: _x_x_mozilla_cycle_collect,1027
NOISE: __end_cc_report
NOISE: __startTimestamp1330591597557__endTimestamp
NOISE: 
getting files in '/mnt/sdcard/tests/profile/minidumps/'
Failed tsvg: 
		Stopped Thu, 01 Mar 2012 09:02:55
FAIL: Busted: tsvg
FAIL: unrecognized output format
Traceback (most recent call last):
  File "run_tests.py", line 666, in <module>
    main()
  File "run_tests.py", line 663, in main
    test_file(arg, options, parser.parsed)
  File "run_tests.py", line 604, in test_file
    raise e
utils.talosError: 'unrecognized output format'
program finished with exit code 1
elapsedTime=1310.658404

and this one:
DeviceManager: error pulling file '/mnt/sdcard/tests/browser_output.txt': No such file or directory
getting files in '/mnt/sdcard/tests/profile/minidumps/'
Failed trobopan: 
		Stopped Thu, 01 Mar 2012 07:01:27
FAIL: Busted: trobopan
FAIL: no output from browser
Traceback (most recent call last):
  File "run_tests.py", line 666, in <module>
    main()
  File "run_tests.py", line 663, in main
    test_file(arg, options, parser.parsed)
  File "run_tests.py", line 604, in test_file
    raise e
utils.talosError: 'no output from browser'
program finished with exit code 1
elapsedTime=425.284433
Updated summary:
* bug 731637 - we hit a permanent OOM issue with robocop
* I will use this bug to switch over to talos.zip model while bug 731637 gets figured it out
Created attachment 603431 [details] [diff] [review]
move to talos.zip and prepare for robocop

I am doing a last run with tegra-025 (since it is the one behaving).

You can wait until tomorrow for me to post back on the results but it is looking good so far.

I am very inclined to switch all DownloadFile steps to be retry capable. What do you think?
Attachment #601383 - Attachment is obsolete: true
Attachment #603431 - Flags: review?(jmaher)
Attachment #603431 - Flags: review?(bear)
(Reporter)

Comment 23

6 years ago
Comment on attachment 603431 [details] [diff] [review]
move to talos.zip and prepare for robocop

Review of attachment 603431 [details] [diff] [review]:
-----------------------------------------------------------------

overall, I like it.  I am r-'ing this due to the self.suies.find('robocop').

::: process/factory.py
@@ +7441,5 @@
>           haltOnFailure=True,
>           ignore_certs=self.ignoreCerts,
>           name="Download build",
>          ))
> +        if self.OS == "android" and self.suites.find('robocop') != -1:

this won't work because tcheckerboard uses robocop.  Can we key off --fennecIDs being present instead?

@@ +7444,5 @@
>          ))
> +        if self.OS == "android" and self.suites.find('robocop') != -1:
> +            def get_fennec_ids_url(build):
> +                url = build.source.changes[-1].files[0]
> +                return url[:-len('fennec-XX.0XX.en-US.android-arm.apk')] + "fennec_ids.txt" 

this seems fragile, but will work for now, maybe a comment?
Attachment #603431 - Flags: review?(jmaher) → review-

Comment 24

6 years ago
Comment on attachment 603431 [details] [diff] [review]
move to talos.zip and prepare for robocop

cannot wait for mozharness so this kind of code can be forgotten!

is robocop an unique enough string?  like Joel mentioned in his review I would be worried that it may cause more test suites than you expect to add these steps
Attachment #603431 - Flags: review?(bear) → review+
Created attachment 603819 [details] [diff] [review]
move to talos.zip and prepare for robocop (take 2)
Attachment #603431 - Attachment is obsolete: true
Attachment #603819 - Flags: review?(jmaher)
Attachment #603819 - Flags: review?(bear)
Created attachment 603820 [details] [diff] [review]
add robocop tests disabled

Let's get it in until we figure out the last issues for robocop (I got unrecognized output on my last run).
Attachment #603820 - Flags: review?(jmaher)
ftr robocop jobs failed without much in the logs:
reconnecting socket
RETURN:<a href = "http://hg.mozilla.org/mozilla-central/rev/3dcb40ebd487">rev:3dcb40ebd487</a>
tegra-025: 
		Started Wed, 07 Mar 2012 13:15:20
Running test trobopan: 
		Started Wed, 07 Mar 2012 13:15:20
reconnecting socket
FIRE PROC: 'org.mozilla.fennec  -profile /mnt/sdcard/tests/profile http://bm-remote.build.mozilla.org/getInfo.html'
process killed by signal 15
program finished with exit code -1
elapsedTime=426.407035

[1]
http://dev-master01.build.scl1.mozilla.com:8043/builders/Android%20Tegra%20250%20mozilla-central%20talos%20remote-trobocop/builds/41/steps/Run%20performance%20tests/logs/stdio
You might have noticed by the 938 emails that I've sent you (except you're too cunning to watch all of releng, aren't you?) about it through bug comments that bug 660480 is not uncommon, though ;)

If you're using production slaves, I'd start getting interested in killed by signal 15 after it happens 3 times in a row on 2 slaves; if you're using staging slaves, which are the most vile and uncooperative of a population that's nothing but vile and uncooperative, I'd rerun something more like 10 times on at least 3 slaves before getting interested in a lack of successes.
(Reporter)

Comment 29

6 years ago
Comment on attachment 603820 [details] [diff] [review]
add robocop tests disabled

Review of attachment 603820 [details] [diff] [review]:
-----------------------------------------------------------------

one small change.

::: mozilla-tests/config.py
@@ +308,5 @@
> +        'options': (TALOS_REMOTE_FENNEC_OPTS, ANDROID),
> +    },
> +    'remote-trobocheck': {
> +        'enable_by_default': False,
> +        'suites': GRAPH_CONFIG + ['--activeTests', 'trobopan', '--noChrome', '--fennecIDs', '../fennec_ids.txt'],

s/trobopan/tcheckerboard/
Attachment #603820 - Flags: review?(jmaher) → review+
(Reporter)

Comment 30

6 years ago
Comment on attachment 603819 [details] [diff] [review]
move to talos.zip and prepare for robocop (take 2)

Review of attachment 603819 [details] [diff] [review]:
-----------------------------------------------------------------

this looks right.  I am not a factory.py expert, but nothing stands out as questionable or incorrect.
Attachment #603819 - Flags: review?(jmaher) → review+
Comment on attachment 603819 [details] [diff] [review]
move to talos.zip and prepare for robocop (take 2)

r+ from IRC.
Attachment #603819 - Flags: review?(bear) → review+
These changes went live in a reconfiguration around 9AM.
While I am still working on the sut agent upgrade I have noticed that I get this cannot remove browser_output.txt.[1]

In fact, when I list the directory I don't see it there.
$>ls /mnt/sdcard/tests
profile
robocop.apk
fennec-13.0a1.en-US.android-arm.apk

[1]
reconnecting socket
RETURN:<a href = "http://hg.mozilla.org/mozilla-central/rev/3dcb40ebd487">rev:3dcb40ebd487</a>
tegra-022: 
		Started Mon, 12 Mar 2012 14:35:37
Running test trobopan: 
		Started Mon, 12 Mar 2012 14:35:37
reconnecting socket
FIRE PROC: 'org.mozilla.fennec  -profile /mnt/sdcard/tests/profile http://bm-remote.build.mozilla.org/getInfo.html'
reconnecting socket
FIRE PROC: 'am instrument -w -e class org.mozilla.fennec.tests.testPan org.mozilla.roboexample.test/android.test.InstrumentationTestRunner'
DeviceManager: error pulling file '/mnt/sdcard/tests/browser_output.txt': No such file or directory
reconnecting socket
pushing directory: /tmp/tmp79VX21/profile to /mnt/sdcard/tests/profile
	Screen width/height:1024/768
	colorDepth:24
	Browser inner width/height: 980/671

removing file: /sdcard/fennec_ids.txt
removing file: /sdcard/robotium.config
removing file: /mnt/sdcard/tests/browser_output.txt
Right now, jmaher is dealing with one of the boards to determine why comment 35 happens.

On my side, I have the patches for the SUT Agent almost to be ready to be deployed.

I have also been running unit tests and talos jobs to determine what happens after the SUT upgrade.

So far everything is running well except for two suites:
* for jsreftest-2 I keep on hitting bug 719377 (I have submitted a patch).
* for robocop unit tests I have hit twice:
> 11 INFO TEST-UNEXPECTED-FAIL | testWebContentContextMenu | Bookmark removal verified - got false, expected true

I am running three more jobs of each in hopes is transient.
Depends on: 719377
I noticed that I should also give Android XUL a run.
Depends on: 735856
Created attachment 606585 [details] [diff] [review]
[buildbot-configs] only run robocop talos for native and enable it for pine
Attachment #606585 - Flags: review?(jmaher)
(Reporter)

Comment 39

6 years ago
Comment on attachment 606585 [details] [diff] [review]
[buildbot-configs] only run robocop talos for native and enable it for pine

does PROJECT_BRANCHES == 'pine' or all project branches?
Attachment #606585 - Flags: review?(jmaher) → review+
Only "pine" [1]

BTW do you want this enabled today? or can it wait until Monday?
I just did a reconfig but I could do another one if wanted.


[1]
diff --git a/mozilla/project_branches.py b/mozilla/project_branches.py
--- a/mozilla/project_branches.py
+++ b/mozilla/project_branches.py
@@ -255,37 +255,39 @@ PROJECT_BRANCHES = {
     'pine': {
         'enable_unittests': False,
         'enabled_products': ['firefox'],
         'talos_suites': {
             'tp': [1, {'suites': ['--sampleConfig', 'cycles.config']}],
             'chrome': [1, {'suites': ['--sampleConfig', 'cycles.config']}],
             'chrome_mac': [1, {'suites': ['--sampleConfig', 'cycles.config']}],
             'nochrome': [1, {'suites': ['--sampleConfig', 'cycles.config']}],
             'dirty': 0,
             'svg': [1, {'suites': ['--sampleConfig', 'cycles.config']}],
             'dromaeo': 0,
             # hate that remote talos still show up when mobile product is not being requested
             # but that's not part of the bug i'm doing this for, so leaving for now
             'remote-ts': 0,
             'remote-tdhtml': 0,
             'remote-tsvg': 0,
             'remote-tsspider': 0,
             'remote-tpan': 0,
             'remote-tp4m': 0,
             'remote-tp4m_nochrome': 0,
             'remote-twinopen': 0,
             'remote-tzoom': 0,
+            'remote-trobocop': 1,
+            'remote-trobocheck': 1,
Comment on attachment 606585 [details] [diff] [review]
[buildbot-configs] only run robocop talos for native and enable it for pine

http://hg.mozilla.org/build/buildbot-configs/rev/59b0ab57d21b

Landed on "default".
Attachment #606585 - Flags: checked-in+
Depends on: 736587
No longer depends on: 719377
(Reporter)

Updated

6 years ago
Depends on: 737411
Passing it to Joel Maher to figure out some fennec issues.
We will enable this as soon as it starts working.

I will meanwhile work on bug 736587 to have robocop enabled there and then we can enable it everywhere.
Assignee: armenzg → jmaher
No longer depends on: 736587
jmaher, does bug 736587 really block us?
Is it so we have more memory for robocop? (since we don't allocate it for the higher resolution?)
(Reporter)

Comment 44

6 years ago
it doesn't block us.  more memory for robocop doesn't help as we run robocop right now with 1024x768.  If we left the resolution at 1600x1200 (the current reftest resolution), then it would matter a lot.
deployed as part of this morning's reconfig
(Reporter)

Comment 46

6 years ago
with my local testing of a patch to fix bug 737411, I am able to get robocop tests working again.
I have triggered a job on staging based on this change:
https://tbpl.mozilla.org/?tree=Try&rev=b12a718cb1f1
jmaher, you should be able to see the results in here:
* http://dev-master01.build.scl1.mozilla.com:8043/builders/Android%20Tegra%20250%20try%20talos%20remote-trobocheck
* http://dev-master01.build.scl1.mozilla.com:8043/builders/Android%20Tegra%20250%20try%20talos%20remote-trobocop

So far the first one fails with:
> TypeError: 'NoneType' object is not iterable
and the second is still running.
(Reporter)

Comment 49

6 years ago
it looks like trobopan worked great (although the reporting failed), but tcheckerboard is failing...getting close!
(Reporter)

Comment 50

6 years ago
s/trobocheck/tcheckerboard/
Created attachment 609327 [details] [diff] [review]
add trobopan and trobocheck to mozilla-central and mozilla-inbound
Assignee: jmaher → armenzg
Status: NEW → ASSIGNED
Attachment #609327 - Flags: review?(jmaher)
(Reporter)

Updated

6 years ago
Attachment #609327 - Flags: review?(jmaher) → review+
Comment on attachment 609327 [details] [diff] [review]
add trobopan and trobocheck to mozilla-central and mozilla-inbound

checked-in 35addb0fc98d

This should go in with today's scheduled reconfig or I can do one tomorrow morning.
Attachment #609327 - Flags: checked-in+
(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #52)
> Comment on attachment 609327 [details] [diff] [review]
> add trobopan and trobocheck to mozilla-central and mozilla-inbound
> 
> checked-in 35addb0fc98d

That is a production landing which should have been gone on default, given the test masters weren't reconfiged (they're on 36ce86337cb2 right now). I've merged it to default and backed it out of production, because I need to reconfig for bug 739505.
nthomas, somehow, hg merge default would not add my changes on default back to production.

I had to merge from production to default. Re-add my changes. Push back to production.

I am waiting for jhford's reconfig to happen so I can do mine.
Created attachment 609842 [details] [diff] [review]
really add robocop

r+ from jmaher in IRC.

Deploying.
Attachment #609842 - Flags: review+
Attachment #609842 - Flags: checked-in+
Created attachment 609856 [details] [diff] [review]
add trobopan and trobocheck (for real) to m-i

Because nothing gets fixed in two places the same way *sigh*

I wish that patches this simple would not need this disparate ways to make them work.
Attachment #609856 - Flags: review?(jmaher)
(Reporter)

Comment 57

6 years ago
Comment on attachment 609856 [details] [diff] [review]
add trobopan and trobocheck (for real) to m-i

Review of attachment 609856 [details] [diff] [review]:
-----------------------------------------------------------------

I could be missing the obvious...

::: mozilla/project_branches.py
@@ +109,5 @@
>          },
>          'talos_suites': {
>              'v8': 1,
> +            'remote-trobopan': 1,
> +            'remote-trobocheck': 1,

I don't understand why these are here?  Shouldn't they be with the other tests like tp4m or remote_ts?
Attachment #609856 - Flags: review?(jmaher) → review+
Depends on: 739974
Depends on: 739977
(In reply to Joel Maher (:jmaher) from comment #57)
> Comment on attachment 609856 [details] [diff] [review]
> add trobopan and trobocheck (for real) to m-i
> 
> Review of attachment 609856 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I could be missing the obvious...
> 
> ::: mozilla/project_branches.py
> @@ +109,5 @@
> >          },
> >          'talos_suites': {
> >              'v8': 1,
> > +            'remote-trobopan': 1,
> > +            'remote-trobocheck': 1,
> 
> I don't understand why these are here?  Shouldn't they be with the other
> tests like tp4m or remote_ts?

These are here because they are not enabled by default like the other ones you mention.

jmaher, trobopan is working. Should we just enable it everywhere?

I filed bug 739977 for trobocheck failures.
(Reporter)

Comment 59

6 years ago
we can't run trobopan on aurora/beta/release/esr10/1.9.2.  I am fine enabling it everwhere.
(Reporter)

Comment 60

6 years ago
Created attachment 610142 [details] [diff] [review]
run ts_paint and tcheckerboard
Attachment #610142 - Flags: review?(armenzg)
Comment on attachment 610142 [details] [diff] [review]
run ts_paint and tcheckerboard

Review of attachment 610142 [details] [diff] [review]:
-----------------------------------------------------------------

Sweet :)
Attachment #610142 - Flags: review?(armenzg) → review+
Created attachment 610189 [details] [diff] [review]
enable trobopan and tcheckerboard by default (not for m-a/m-b/m-r)

There is a little bit of re-ordering that I believe makes things clearer.

This patch does not enable these tests for m-a/m-b/m-r.
Attachment #609856 - Attachment is obsolete: true
Attachment #610189 - Flags: review?(jmaher)
(Reporter)

Comment 64

6 years ago
Comment on attachment 610189 [details] [diff] [review]
enable trobopan and tcheckerboard by default (not for m-a/m-b/m-r)

Review of attachment 610189 [details] [diff] [review]:
-----------------------------------------------------------------

looks good
Attachment #610189 - Flags: review?(jmaher) → review+
Comment on attachment 610189 [details] [diff] [review]
enable trobopan and tcheckerboard by default (not for m-a/m-b/m-r)

I landed it on "default" and disable it for 1.9.2 as well:
http://hg.mozilla.org/build/buildbot-configs/rev/b16777a3388d
Attachment #610189 - Flags: checked-in+
Comment on attachment 610142 [details] [diff] [review]
run ts_paint and tcheckerboard

Landed on "default":
http://hg.mozilla.org/build/buildbot-configs/rev/40b64f4b4bb3
Attachment #610142 - Flags: checked-in+
These changes went live around 11:55AM PDT.
Priority: P2 → P1

Comment 68

6 years ago
Backed out ts_paint in bug 740196.
This bug is done! yay!
https://tbpl.mozilla.org/?jobname=Android%20Tegra%20250%20mozilla-central%20talos%20remote-trobo

bug 739974 will fix the robocop jobs show up as a "T" symbol on tbpl but it is not a blocker for having trobopan and trobocheck enabled.
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED

Comment 70

6 years ago
[16:47]	<philor>	hmm, https://tbpl.mozilla.org/php/getParsedLog.php?id=12365739&tree=Mozilla-Beta - "wget --progress=dot:mega -N http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-beta-android/133884418fennec_ids.txt" isn't going to go well
[16:48]	<philor>	aki|brb: ^?
[16:49]	<philor>	seems to be post-clobber, post-reconfig, and there's rather a lot of them lacking that slash
[16:52]	<philor>	oh, I should probably yell at the numbskull who blindly turned robopan on on beta without checking whether it works, shouldn't I?
[16:53]	<catlee-away>	the trains must go on!
[16:53]	<philor>	or asking one of multiple |afks who know about android talos what would be missing the trailing slash
[16:54]	<philor>	cunning armen, not even being afk
[16:54]	<catlee-away>	WTH
[16:55]	<catlee-away>	splitting based on / seems safer that fixed length string manipulation
[16:55]	<catlee-away>	hehehe
[16:56]	<catlee-away>	because we lost the a2 from 14.0a2
[16:56]	<catlee-away>	http://hg.mozilla.org/build/buildbotcustom/file/default/process/factory.py#l7541
[16:56]	<philor>	yeah, I just got to there :)
[16:57]	<philor>	and said... WTH

Should we back out or fix?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
They're hidden on beta (with rather an excess of glee), so there's no real need to back out, they fail fast.
Created attachment 630428 [details] [diff] [review]
fix beta bustage

Really, who *doesn't* think a bit of my untested unidiomatic Python is exactly what we need for major, critical, right up against the releasing edge bustage?
Attachment #630428 - Flags: review?(armenzg)

Comment 75

6 years ago
This made it to production today.
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago6 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.