Closed Bug 1429236 Opened 6 years ago Closed 6 years ago

jsreftests run too frequently

Categories

(Taskcluster :: General, defect)

Product:
Taskcluster
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla59

People

(Reporter: gbrown, Assigned: dustin)

References

Details

Attachments

(1 file)

Bug 1429236: put jsreftests in their own scheduling component;
59 bytes, text/x-review-board-request
jmaher
: review+
Details 
Since bug 1403322 landed, jsreftests seem to run on nearly every push (at least on some platform). I believe they should only run when there are files modified under js/src or js/public. 

https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=jsreftest&fromchange=e188cb9fe8f5f3e73e06a13cdebde190b142d432&tochange=54015b4eb2ed3b79d5513c0a7547bae586dc6715

(test-verify and jittest seem okay this time!)
Flags: needinfo?(dustin)
In task-graph.json for a recent push on autoland:

https://public-artifacts.taskcluster.net/ARQoxkKHRwSSa16BAa68Jw/0/public/task-graph.json
  "APPUn87gTY-7gI8fQIGFpA": {
    "attributes": {
      ...
      "unittest_flavor": "jsreftest",
      "unittest_suite": "reftest",
      "unittest_try_name": "jsreftest"
    },
    "label": "test-android-4.3-arm7-api-16/debug-jsreftest-15",
    "optimization": {
      "skip-unless-schedules-or-seta": [
        "reftest",
        "android"
      ]
    },

yet python/mozbuild/mozbuild/schedules.py has

INCLUSIVE_COMPONENTS = [ 
    ...
    'jsreftest',
]

so I think the fix here is to make this suite `jsreftest`.  I don't really understand the ramifications of that kind of change, though.  Do you?  Would something break?  I think it just ends up in the task.extra.  Maybe Joel knows if that would cause issues?
Flags: needinfo?(dustin) → needinfo?(jmaher)
I don't know for sure, but we could validate on try.  Reftest is a harness and there are 3 main test types: reftest, crashtest, jsreftest.  jsreftest is the most unique of the 3.  What I don't know is how taskcluster and mozharness play into decisions with suites and specific names.  The only concern I might have here is that we could have odd try scheduling quirks with the old syntax vs fuzzy syntax.  

In short, we should be able to fix any fallout :)
Flags: needinfo?(jmaher)
[task 2018-01-12T14:40:15.614Z] desktop_unittest.py: error: no such option: --jsreftest-suite

I think what I will do is add a parameter to test descriptions that can override the scheduling component -- so usually it's the suite, but can be set to something else.  In this case, to 'jsreftest' :)
Assignee: nobody → dustin
Comment on attachment 8942189 [details]
Bug 1429236: put jsreftests in their own scheduling component;

https://reviewboard.mozilla.org/r/212448/#review218206

while I like the name chose, the try push failed:
desktop_unittest.py: error: no such option: --jsreftest-suite

please ensure that there is tests for Android as well, that is treated differently.
Attachment #8942189 - Flags: review?(jmaher) → review-
How is Android different?
different harness scripts- cli args might be handled differently
Thanks, I'll include those in the next try push
See Also: → 1430668
Comment on attachment 8942189 [details]
Bug 1429236: put jsreftests in their own scheduling component;

https://reviewboard.mozilla.org/r/212448/#review219978

slight change in direction, looks good,
Attachment #8942189 - Flags: review?(jmaher) → review+
We're sorry, Autoland could not rebase your commits for you automatically. Please manually rebase your commits and try again.

hg error in cmd: hg rebase -s 699c807f3f6b50e70a6c3ee1a9ae1a679f2882a7 -d 7438fea5bc44: rebasing 443284:699c807f3f6b "Bug 1429236: put jsreftests in their own scheduling component; r=jmaher" (tip)
merging taskcluster/ci/test/reftest.yml
merging taskcluster/taskgraph/transforms/tests.py
warning: conflicts while merging taskcluster/ci/test/reftest.yml! (edit, then use 'hg resolve --mark')
unresolved conflicts (see hg resolve, then hg rebase --continue)
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9c1e9d7bec5f
put jsreftests in their own scheduling component; r=jmaher
https://hg.mozilla.org/mozilla-central/rev/9c1e9d7bec5f
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla59
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: