Closed Bug 1283791 Opened 4 years ago Closed 3 years ago

Firefox-ui-tests taskcluster tasks should also upload reports (html, xml) as artifacts

Categories

(Testing :: Firefox UI Tests, defect)

49 Branch
defect
Not set

Tracking

(firefox49 wontfix, firefox50 fixed, firefox51 fixed)

RESOLVED FIXED
mozilla51
Tracking Status
firefox49 --- wontfix
firefox50 --- fixed
firefox51 --- fixed

People

(Reporter: whimboo, Assigned: whimboo)

References

Details

Attachments

(1 file)

When we submit job results from Mozmill CI for jobs which are run in Jenkins we also create HTML and XUnit reports. The same reports are getting created for the execution in Taskcluster but we actually do not add the reports subfolder to the list of artifacts. As result nothing gets uploaded to the public section of artifacts on Taskcluster.

I will get this added once the refactoring of tasks on bug 1283596 has been landed.
We best should use the `abs_blob_upload_dir` for that so that taskcluster is doing that automatically. The upcoming change on bug 1275674 will help.
Depends on: 1275674
By default artifacts will be picked up by the following directories:

"artifacts": {
    "public/logs/": {
      "path": "/home/worker/workspace/build/upload/logs/",
      "expires": "2017-08-11T20:37:13.025401Z",
      "type": "directory"
    },
    "public/test": {
      "path": "/home/worker/artifacts/",
      "expires": "2017-08-11T20:37:13.025401Z",
      "type": "directory"
    },
    "public/test_info/": {
      "path": "/home/worker/workspace/build/blobber_upload_dir/",
      "expires": "2017-08-11T20:37:13.025401Z",
      "type": "directory"
    }
  },

As of now reports end-up at "/home/worker/workspace/build/reports". So we should change it that reports are getting stored in "blobber_upload_dir" too like all the other non-log related files.

A change like this will have consequences for results from mozmill-ci and older branches given that we do not differentiate between branches:

https://github.com/mozilla/mozmill-ci/blob/staging/jenkins-master/jobs/scripts/workspace/config.py#L21

But I think we should do it any way, report.html was never that important yet and we most likely won't need it for release, and esr45. Once this patch lands we can backport it to aurora after mozmill-ci got updated too.
Assignee: nobody → hskupin
Status: NEW → ASSIGNED
The HTTP log is only for update tests, which we currently do not run via in-tree taskcluster. Instead this is done by mozmill-ci.
Summary: Firefox-ui-tests taskcluster tasks should also upload reports (html, http) as artifacts → Firefox-ui-tests taskcluster tasks should also upload reports (html, xml) as artifacts
Comment on attachment 8780492 [details]
Bug 1283791 - Firefox-ui-tests taskcluster tasks have to upload reports (html, xml) as artifacts.

https://reviewboard.mozilla.org/r/71194/#review69098

In-tree taskcluster jobs are working great, but mozmill-ci needs updates before we can push this to autoland.
Attachment #8780492 - Flags: review?(mjzffr)
Whiteboard: [needs mozmill-ci config fix first]
Comment on attachment 8780492 [details]
Bug 1283791 - Firefox-ui-tests taskcluster tasks have to upload reports (html, xml) as artifacts.

https://reviewboard.mozilla.org/r/71196/#review69438
Attachment #8780492 - Flags: review?(mjzffr) → review+
I pushed the mozmill-ci changes which let me run try builds with the above changes. Here the results:

https://treeherder.allizom.org/#/jobs?repo=try&revision=fa48dadee6e9&filter-tier=1&filter-tier=2&filter-tier=3&selectedJob=26365761

As you can see we do not include the reports anymore for mozmill-ci triggered builds. But I don't think that is that important as long as older branches still upload those. Once this feature rode the trains down to release we can change mozmill-ci - if it still exits then.

I will go ahead and land this patch now.
Whiteboard: [needs mozmill-ci config fix first]
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a218d8647a97
Firefox-ui-tests taskcluster tasks have to upload reports (html, xml) as artifacts. r=maja_zf
https://hg.mozilla.org/mozilla-central/rev/a218d8647a97
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla51
I will wait for the mozmill-ci results today. If all works well, I would propose that we get this backported to aurora and beta.
It would be good to have this extra HTML report available for TC test jobs on aurora and beta. So please get it backported to aurora and beta. Thanks.
Whiteboard: [checkin-needed-aurora][checkin-needed-beta]
https://hg.mozilla.org/releases/mozilla-aurora/rev/d8e648eee1db
Whiteboard: [checkin-needed-aurora][checkin-needed-beta] → [checkin-needed-beta]
Needs rebasing for Beta.
Flags: needinfo?(hskupin)
Whiteboard: [checkin-needed-beta]
Not worse the hassle. Lets ride the trains.
Flags: needinfo?(hskupin)
You need to log in before you can comment on or make changes to this bug.