Closed Bug 1339179 Opened 4 years ago Closed 4 years ago

Set up pine for activity-stream builds & tests

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

task
Not set
normal

Tracking

(firefox54 fixed)

RESOLVED FIXED
Tracking Status
firefox54 --- fixed

People

(Reporter: catlee, Assigned: aselagea)

References

Details

Attachments

(8 files, 4 obsolete files)

We need to get the pine project branch set up to run the regular Firefox desktop builds & tests. No mobile automation needed.
Depends on: 1339849
Assignee: nobody → aselagea
Attached patch bug_1339179_bb-configs.patch (obsolete) — Splinter Review
Similar jobs to what we have on graphics.
Attachment #8839478 - Flags: review?(kmoir)
Attached file tests_diff.txt
Attached file builds_diff.txt
Comment on attachment 8839478 [details] [diff] [review]
bug_1339179_bb-configs.patch

It looks good except for linux64 and linux64-debug.  These need to be enabled in taskcluster if this is not already done.  We don't run linux64 and linux64-debug on buildbot anymore.
Attached file disable_mobile_jobs.patch (obsolete) —
Only asking for feedback since I followed the steps here: 
http://gecko.readthedocs.io/en/latest/taskcluster/taskcluster/how-tos.html

but I get no diff on my local computer (not sure why).
Attachment #8839492 - Flags: feedback?(kmoir)
Attachment #8839478 - Flags: review?(kmoir) → review-
Comment on attachment 8839478 [details] [diff] [review]
bug_1339179_bb-configs.patch

actually just remembered that if you need talos tests for Linux64 on this branch, you need to enable the buildbot jobs. Is that a requirement?
Attachment #8839478 - Flags: review-
Comment on attachment 8839492 [details]
disable_mobile_jobs.patch

hmm, so what steps do you use to generate the new graph locally?  Is a new full task graph generated locally? I would think that this would include the new pine tasks, the patch looks good to me
Attachment #8839492 - Flags: feedback?(kmoir) → feedback+
(In reply to Kim Moir [:kmoir] from comment #6)
> Comment on attachment 8839478 [details] [diff] [review]
> bug_1339179_bb-configs.patch
> 
> actually just remembered that if you need talos tests for Linux64 on this
> branch, you need to enable the buildbot jobs. Is that a requirement?

I'll shift that to :catlee, maybe he has an idea if talos tests are needed on pine.
Flags: needinfo?(catlee)
(In reply to Kim Moir [:kmoir] from comment #7)
> Comment on attachment 8839492 [details]
> disable_mobile_jobs.patch
> 
> hmm, so what steps do you use to generate the new graph locally?  Is a new
> full task graph generated locally? I would think that this would include the
> new pine tasks, the patch looks good to me

From what I can see, TC jobs are already enabled on pine: https://treeherder.mozilla.org/#/jobs?repo=pine
Unfortunately, the desktop builds are all failing, so we don't have any tests to be run.

In order to generate the old graph, I copied the 'parameters.yml' file for the most recent pine run on my local m-i clone and ran './mach taskgraph'. 

As for the new graph, I applied the patch above and ran the same command again (redirecting the output to a different file of course). Yet no task difference between the two can be spotted.
Attached patch bug1339179.patchSplinter Review
I tried it with patch and the following parameters.yml 

base_repository: https://hg.mozilla.org/mozilla-central
build_date: 1487259202
filters:
- target_tasks_method
head_ref: 782b85278179d243acb385ddbb032355620e360f
head_repository: https://hg.mozilla.org/try/
head_rev: 782b85278179d243acb385ddbb032355620e360f
level: '1'
message: 'try: -b o -p linux64-stylo -u refest-stylo-1 -t none'
moz_build_date: '20170216153322'
optimize_target_tasks: true
owner: kmoir@mozilla.com
project: try
pushdate: 1487259202
pushlog_id: '170732'
target_tasks_method: pine_tasks
triggered_by: push

And it seemed to filter out the android tasks
From what I recall from our buildduty meeting, we should move on and don't enable talos on pine. Updated the patch to account for that.
Attachment #8839478 - Attachment is obsolete: true
Attachment #8840885 - Flags: review?(kmoir)
Attachment #8840885 - Flags: review?(kmoir) → review+
(In reply to Kim Moir [:kmoir] from comment #10)
> Created attachment 8840106 [details] [diff] [review]
> bug1339179.patch
> 
> I tried it with patch and the following parameters.yml 
...
> pushlog_id: '170732'
> target_tasks_method: pine_tasks
> triggered_by: push
> 
> And it seemed to filter out the android tasks

Thanks for looking into this Kim!
It seems the reason I wasn't getting any difference was that I used 'target_tasks_method: default' instead of 'target_tasks_method: pine_tasks'.
Attached file task_diff.txt
The android tasks are indeed filtered properly now.
Pushed by aselagea@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/dd8d6b4f3054
Set up pine for activity-stream builds & tests, r=kmoir
Attachment #8840106 - Flags: checked-in+
Attachment #8839492 - Attachment is obsolete: true
Attachment #8839492 - Attachment is patch: false
Flags: needinfo?(catlee)
https://hg.mozilla.org/mozilla-central/rev/dd8d6b4f3054
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
(In reply to Alin Selagea [:aselagea][:buildduty] from comment #8)
> (In reply to Kim Moir [:kmoir] from comment #6)
> > Comment on attachment 8839478 [details] [diff] [review]
> > bug_1339179_bb-configs.patch
> > 
> > actually just remembered that if you need talos tests for Linux64 on this
> > branch, you need to enable the buildbot jobs. Is that a requirement?
> 
> I'll shift that to :catlee, maybe he has an idea if talos tests are needed
> on pine.

yes, we need talos tests on pine.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attached patch enable_talos.patch (obsolete) — Splinter Review
I checked the list of tasks for pine and noticed that 'talos' was enabled. Since these tests are scheduled in TC, but run in BB, we would need to alter bb-configs as well.
Attachment #8841612 - Flags: review?(kmoir)
Comment on attachment 8841612 [details] [diff] [review]
enable_talos.patch

Could you provide a builder diff?  Do these also enable the associated builders or are they removed somewhere else since this builds now run on tc? (we just want to enable talos)
I missed the fact that 'enable_talos' was set to False, so no talos tests showed up. Updated the patch.
Attachment #8841612 - Attachment is obsolete: true
Attachment #8841612 - Flags: review?(kmoir)
Attachment #8842841 - Flags: review?(kmoir)
Attached file diff_talos.txt
Here's the tests diff following the change above. I also tested the patch on a build master, but no extra builds were triggered.
Attachment #8842842 - Flags: review+
Attachment #8842841 - Flags: review?(kmoir) → review+
Attachment #8842842 - Flags: review+
Attachment #8842841 - Flags: checked-in+
Can this be closed?
Flags: needinfo?(aselagea)
(In reply to Kim Moir [:kmoir] from comment #23)
> Can this be closed?

Yeah, I think so - automation has been enabled for the pine branch. 
I was just waiting to confirm that talos tests were enabled too. There are some job failures on this branch, but I don't think they make the scope of this bug. 

Thanks!
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Flags: needinfo?(aselagea)
Resolution: --- → FIXED
Attached patch fix_pine.diff (obsolete) — Splinter Review
Attachment #8855118 - Flags: review?(jlorenzo)
Attached patch fix_pine2.diffSplinter Review
Also disable asan, per dmose.

20:02 <dmose> aki: wonderful; thanks!   if you're so inclined, i think it'd be fine to remove ASAN as well.
Attachment #8855118 - Attachment is obsolete: true
Attachment #8855118 - Flags: review?(jlorenzo)
Attachment #8855163 - Flags: review?(jlorenzo)
Comment on attachment 8855163 [details] [diff] [review]
fix_pine2.diff

LGTM!
Attachment #8855163 - Flags: review?(jlorenzo) → review+
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.