Closed Bug 1262123 Opened 4 years ago Closed 3 years ago

Remove datazilla credentials & configs

Categories

(Infrastructure & Operations :: CIDuty, task)

task
Not set

Tracking

(firefox48 fixed)

RESOLVED FIXED
Tracking Status
firefox48 --- fixed

People

(Reporter: catlee, Assigned: vciobancai)

Details

Attachments

(6 files, 1 obsolete file)

We no longer submit data to datazilla. We should remove all references to it from buildbot / mozharness / puppet.
Component: General Automation → Buildduty
QA Contact: catlee → bugspam.Callek
Assignee: nobody → vlad.ciobancai
Attached you can find the patch were I removed datazilla from puppet
Attachment #8739421 - Flags: review?(kmoir)
Attached you can find the patch were I removed datazilla from puppet
Attachment #8739422 - Flags: review?(kmoir)
(In reply to Vlad Ciobancai [:vladC] from comment #2)
> Created attachment 8739422 [details] [diff] [review]
> bug1262123_buildbot-configs.patch
> 
> Attached you can find the patch were I removed datazilla from puppet

The patch remove datazilla from buildbot-config not from puppet
Attached you can find the patch were I removed datazilla from buildbotcustom
Attachment #8739423 - Flags: review?(kmoir)
Attached you can find the patch were I removed datazilla from mozharness
Attachment #8739424 - Flags: review?(kmoir)
Attached file bug1262123_output.txt
Attached you can find the tests made on my master
Attachment #8739421 - Flags: review?(kmoir) → review+
Attachment #8739422 - Flags: review?(kmoir) → review+
Attachment #8739423 - Flags: review?(kmoir) → review+
Comment on attachment 8739424 [details] [diff] [review]
bug1262123_mozharness.patch

So you can land this patch.  But I think this is for the hg.mozilla.org/build/mozharness.  We don't really use this project anymore.  Instead, we use mozharness scripts in tree.  So you should look for references to datazilla in tree starting with ssh://hg.mozilla.org/integration/mozilla-inbound and create patches for this.  Once they land, they will be uplifted to other branches by the sheriffs.
Attachment #8739424 - Flags: review?(kmoir)
Attachment #8739421 - Flags: checked-in+
Attachment #8739422 - Flags: checked-in+
Attachment #8739423 - Flags: checked-in+
Attached patch bug1262123_mozilla-inbound.patch (obsolete) — Splinter Review
Created the patch in order to disable datazilla from mozilla-inbound
Attachment #8743313 - Flags: feedback?(kmoir)
Comment on attachment 8743313 [details] [diff] [review]
bug1262123_mozilla-inbound.patch

So I'm not really familiar with this code and I'm a bit wary of the line  
# another hack; datazilla stands for Perfherder
Perfherder is a active project 
so I would recommend asking for review for someone on ateam that works with talos/perherfer, perhaps wlach or jmaher.  Another approach would be to run this code on try with talos enabled and see if anything breaks.
Attachment #8743313 - Flags: feedback?(kmoir)
jmaher I would like your help for feedback or review for the following attachment 8743313 [details] [diff] [review]
Flags: needinfo?(jmaher)
Comment on attachment 8743313 [details] [diff] [review]
bug1262123_mozilla-inbound.patch

Review of attachment 8743313 [details] [diff] [review]:
-----------------------------------------------------------------

I think remove compare.py, and consider changing datazilla_urls to be output_urls in all the code.

::: testing/talos/talos/compare.py
@@ -302,5 @@
> -
> -    for item in jdata:
> -        alldata[jdata[item]['revisions'][0]] = \
> -            getDatazillaCSET(jdata[item]['revisions'][0], branchid)
> -    return alldata

removing this code will require some additional work, I think we might want to consider removing compare.py completely as it doesn't use perfherder and we don't submit data to graph server anymore.  Please remove this entire file!

::: testing/talos/talos/output.py
@@ -278,5 @@
>                                 'subtests': counter_subtests})
>          return test_results
>  
> -# available output formats
> -formats = {'datazilla_urls': PerfherderOutput}

this is needed, it just has the old name, please don't remove this.

::: testing/talos/talos/run_tests.py
@@ -169,5 @@
> -        )
> -    else:
> -        # local mode, output to files
> -        results_urls = dict(datazilla_urls=[os.path.abspath('local.json')])
> -    talos_results.check_output_formats(results_urls)

all of this code needs to remain, again reusing an old variable.

@@ -214,5 @@
>      if results_urls:
>          talos_results.output(results_urls)
> -        if browser_config['develop']:
> -            print ("Thanks for running Talos locally. Results are in %s"
> -                   % (results_urls['datazilla_urls']))

this needs to stay as well.
Attachment #8743313 - Flags: review-
feel free to ask me for a review on the next patch.
Flags: needinfo?(jmaher)
Vlad: I just had to back out the puppet patch again so I could land the follow-up for bug 1265758. If this puppet patch is really ready to go, let's get it merged tomorrow. Otherwise, don't land the partial patch again until the rest is ready so we don't block any other deployments. Thanks.
Attachment #8739421 - Flags: checked-in+ → checked-in-
Thank you for the feedback.
I recreated the patch by using your suggestion
Attachment #8743313 - Attachment is obsolete: true
Attachment #8743730 - Flags: review?(jmaher)
Comment on attachment 8743730 [details] [diff] [review]
bug1262123_mozilla-inbound.patch

Review of attachment 8743730 [details] [diff] [review]:
-----------------------------------------------------------------

please test this in try |try: -b o -p linux64 -u none -t all|, I would expect all green results, but please paste the try url in here so I can validate numbers are posted.
Attachment #8743730 - Flags: review?(jmaher) → review+
(In reply to Joel Maher (:jmaher) from comment #15)
> Comment on attachment 8743730 [details] [diff] [review]
> bug1262123_mozilla-inbound.patch
> 
> Review of attachment 8743730 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> please test this in try |try: -b o -p linux64 -u none -t all|, I would
> expect all green results, but please paste the try url in here so I can
> validate numbers are posted.

I tried to test the change on try but I was not able, can you please test it for me ?
Flags: needinfo?(jmaher)
try run looks great.
Flags: needinfo?(jmaher)
I do not think I have access to m-i to land the change, can somebody push it ?
https://hg.mozilla.org/mozilla-central/rev/869c852dff02
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
I will reopen the bug because I haven't landed all the patches.
kmoir can I land the following patches bug1262123_mozharness.patch and bug1262123_puppet-BuildSlaves-tests.py.erb.patch
Status: RESOLVED → REOPENED
Flags: needinfo?(kmoir)
Resolution: FIXED → ---
Yes I think so,
Flags: needinfo?(kmoir)
Attachment #8739421 - Flags: checked-in- → checked-in+
Attachment #8739424 - Flags: checked-in+
The changes have been landed
Status: REOPENED → RESOLVED
Closed: 3 years ago3 years ago
Resolution: --- → FIXED
I merged your puppet change to production too.
Product: Release Engineering → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.