1243877 - fix config_seta.py to parse api15 data

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Kim Moir [:kmoir] ET]

I added some quick fixes one night when our tests were failing to ignore this platform but now it has been enabled in our configs.

Comment 1

[Kim Moir [:kmoir] ET]

Attachment: bug1243877.patch

Comment 2

[Kim Moir [:kmoir] ET]

:jmaher Yesterday Jordan deprecated api11 in our configs in bug 1219094 and moved it to api15.  When do you anticipate the seta data for 15 to be included?  (Also, you can probably deprecate it for api11 since these jobs don't run on fx-team or m-i now)

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I have api15 in the seta api:
http://alertmanager.allizom.org/data/setadetails/?date=2015-01-28&buildbot=1&active=1

api11 is still there, leaving that alone, I assume you can safely ignore it.  Right now there will be no jobs run for api15 :)  it is perfect!  I assume in short time api15 will have a set of jobs to run.  I can preseed it with jobs if we wish.

Comment 4

[Kim Moir [:kmoir] ET]

There is some unexpected data in the seta source data, i.e. [funsize] jobs which don't parse well, and jobs like this "Ubuntu VM 12.04 x64 fx-team opt test  Opt"
I'm going to add some code to fix this or exclude them

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

I bet that is coming from taskcluster jobs...so many variables.

Comment 6

[Kim Moir [:kmoir] ET]

I have a patch 90% done to fix this and exclude the TC data, will post tomorrow.

Comment 7

[Kim Moir [:kmoir] ET]

Attachment: bug1243877-2.patch

Patch to add support for api15 data which Joel added on Friday.  Also to change the way that the data is parsed so we split the job string on branch instead of assuming it will be in certain element of the split string. Also excluded TC, funsize and talos jobs from the resulting dictionary as SETA doesn't change their scheduling.

Comment 8

[Jordan Lund (:jlund)]

Comment on attachment 8714338 [details] [diff] [review]
bug1243877-2.patch

Review of attachment 8714338 [details] [diff] [review]:
-----------------------------------------------------------------

I think this looks good. hard to say now that the logic requires more complexity. some comments but no blockers

::: mozilla-tests/config_seta.py
@@ +92,2 @@
>          elif t.split()[-1].startswith('jsreftest'):
> +            tests_by_worker_platform[worker_platform[1]].append(t)

side note: since the three above conditions do the same thing, maybe simplify to something like:

# untested :)
if any([t.split()[-1].startswith(target) for target in ['plain-reftest', 'crashtest', 'jsreftest']]):
    tests_by_worker_platform[worker_platform[1]].append(t)

@@ +102,5 @@
>          test_config = {}
>          for t in test_dict[sp]:
>              test = t.split()[-1]
> +            test_type = (t.split(branch)[1]).split()[0]
> +            if test_type in test_type_exclusions:

can this part go next to the other exclusion conditions in define_configs? or am I reading that wrong?

@@ +109,5 @@
>              BRANCHES[branch]['platforms'][plat][str(sp)]['skipconfig'] = test_config
>  
>  
>  def define_configs(branch, platforms, BRANCHES):
> +  

nit: w/s

Comment 9

[Kim Moir [:kmoir] ET]

Attachment: bug1243877-3.patch

patch incorporating review comments

Comment 10

[Kim Moir [:kmoir] ET]

So the scheduling bug in bug 1223042 bit us again. I fixed it like this.

I noticed that the api15 data doesn't include all the SETA data for all tests that have been chunked into 50 pieces etc. This will cause load issues. So I think we need to hack it to make api15 look like api11 again, I have a patch to attach.

mysql> select buildrequests.id from buildsets, sourcestamp_changes, changes where changes.changeid = sourcestamp_changes.changeid and sourcestamp_changes.sourcestampid = buildsets.sourcestampid and buildrequests.buildsetid = buildsets.id and buildrequests.complete = 0 and buildrequests.claimed_at =0 and buildername like 'Android 4.3 armv7 API 15+%' and when_timestamp < 1454357249;
ERROR 1054 (42S22): Unknown column 'buildrequests.id' in 'field list'
mysql> create temporary table ids select buildrequests.id from buildrequests, buildsets, sourcestamp_changes, changes w
    -> here changes.changeid = sourcestamp_changes.changeid and sourcestamp_changes.sourcestampid = buildsets.sourcestampid an
    -> d buildrequests.buildsetid = buildsets.id and buildrequests.complete = 0 and buildrequests.claimed_at =0 and buildernam
    -> e like 'Android 4.3 armv7 API 15+%' and when_timestamp < 1454357249;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'here changes.changeid = sourcestamp_changes.changeid and sourcestamp_changes.sou' at line 2
mysql> create temporary table ids select buildrequests.id from buildrequests, buildsets, sourcestamp_changes, changes where changes.changeid = sourcestamp_changes.changeid and sourcestamp_changes.sourcestampid = buildsets.sourcestampid and buildrequests.buildsetid = buildsets.id and buildrequests.complete = 0 and buildrequests.claimed_at =0 and buildername like 'Android 4.3 armv7 API 15+%' and when_timestamp < 1454357249;
Query OK, 11116 rows affected (0.53 sec)
Records: 11116  Duplicates: 0  Warnings: 0

mysql> update buildrequests, ids set complete=1, results=2, complete_at=1454360349 where buildrequests.id=ids.id and complete=0 and claimed_at=0;
Query OK, 11074 rows affected (3.26 sec)
Rows matched: 11074  Changed: 11074  Warnings: 0

Comment 11

[Kim Moir [:kmoir] ET]

Attachment: bug1243877-4.patch

Patch to make api15 jobs use api11 seta data until we have api15 seta data.  There are too many chunks to run these jobs w/o seta.

Comment 12

[Kim Moir [:kmoir] ET]

Looks like this worked and no huge pending counts were created, perhaps because of the changes in bug 1174870