jsreftests run too frequently

RESOLVED FIXED in mozilla59

Status

Taskcluster
General
RESOLVED FIXED
12 days ago
a day ago

People

(Reporter: gbrown, Assigned: dustin)

Tracking

unspecified
mozilla59

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

(Reporter)

Description

12 days ago
Since bug 1403322 landed, jsreftests seem to run on nearly every push (at least on some platform). I believe they should only run when there are files modified under js/src or js/public. 

https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=jsreftest&fromchange=e188cb9fe8f5f3e73e06a13cdebde190b142d432&tochange=54015b4eb2ed3b79d5513c0a7547bae586dc6715

(test-verify and jittest seem okay this time!)
(Reporter)

Updated

12 days ago
Flags: needinfo?(dustin)
(Assignee)

Comment 1

11 days ago
In task-graph.json for a recent push on autoland:

https://public-artifacts.taskcluster.net/ARQoxkKHRwSSa16BAa68Jw/0/public/task-graph.json
  "APPUn87gTY-7gI8fQIGFpA": {
    "attributes": {
      ...
      "unittest_flavor": "jsreftest",
      "unittest_suite": "reftest",
      "unittest_try_name": "jsreftest"
    },
    "label": "test-android-4.3-arm7-api-16/debug-jsreftest-15",
    "optimization": {
      "skip-unless-schedules-or-seta": [
        "reftest",
        "android"
      ]
    },

yet python/mozbuild/mozbuild/schedules.py has

INCLUSIVE_COMPONENTS = [ 
    ...
    'jsreftest',
]

so I think the fix here is to make this suite `jsreftest`.  I don't really understand the ramifications of that kind of change, though.  Do you?  Would something break?  I think it just ends up in the task.extra.  Maybe Joel knows if that would cause issues?
Flags: needinfo?(dustin) → needinfo?(jmaher)
I don't know for sure, but we could validate on try.  Reftest is a harness and there are 3 main test types: reftest, crashtest, jsreftest.  jsreftest is the most unique of the 3.  What I don't know is how taskcluster and mozharness play into decisions with suites and specific names.  The only concern I might have here is that we could have odd try scheduling quirks with the old syntax vs fuzzy syntax.  

In short, we should be able to fix any fallout :)
Flags: needinfo?(jmaher)
Comment hidden (mozreview-request)
(Assignee)

Comment 5

9 days ago
[task 2018-01-12T14:40:15.614Z] desktop_unittest.py: error: no such option: --jsreftest-suite

I think what I will do is add a parameter to test descriptions that can override the scheduling component -- so usually it's the suite, but can be set to something else.  In this case, to 'jsreftest' :)
(Assignee)

Updated

9 days ago
Assignee: nobody → dustin

Comment 6

9 days ago
mozreview-review
Comment on attachment 8942189 [details]
Bug 1429236: put jsreftests in their own scheduling component;

https://reviewboard.mozilla.org/r/212448/#review218206

while I like the name chose, the try push failed:
desktop_unittest.py: error: no such option: --jsreftest-suite

please ensure that there is tests for Android as well, that is treated differently.
Attachment #8942189 - Flags: review?(jmaher) → review-
(Assignee)

Comment 7

9 days ago
How is Android different?
different harness scripts- cli args might be handled differently
(Assignee)

Comment 9

9 days ago
Thanks, I'll include those in the next try push
(Reporter)

Updated

6 days ago
See Also: → bug 1430668
Comment hidden (mozreview-request)

Comment 12

2 days ago
mozreview-review
Comment on attachment 8942189 [details]
Bug 1429236: put jsreftests in their own scheduling component;

https://reviewboard.mozilla.org/r/212448/#review219978

slight change in direction, looks good,
Attachment #8942189 - Flags: review?(jmaher) → review+

Comment 13

2 days ago
We're sorry, Autoland could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg rebase -s 699c807f3f6b50e70a6c3ee1a9ae1a679f2882a7 -d 7438fea5bc44: rebasing 443284:699c807f3f6b "Bug 1429236: put jsreftests in their own scheduling component; r=jmaher" (tip)
merging taskcluster/ci/test/reftest.yml
merging taskcluster/taskgraph/transforms/tests.py
warning: conflicts while merging taskcluster/ci/test/reftest.yml! (edit, then use 'hg resolve --mark')
unresolved conflicts (see hg resolve, then hg rebase --continue)
Comment hidden (mozreview-request)

Comment 15

2 days ago
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9c1e9d7bec5f
put jsreftests in their own scheduling component; r=jmaher

Comment 16

a day ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/9c1e9d7bec5f
Status: NEW → RESOLVED
Last Resolved: a day ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
You need to log in before you can comment on or make changes to this bug.