create_talos_zip should pull dependencies from pypi, not github

RESOLVED FIXED

Status

RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: wlach, Assigned: wlach)

Tracking

Trunk
x86_64
Linux
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

The github repo for mozbase is abandoned. For the deps we want to include in talos.zip, let's fetch them from pypi.

Comment 1

4 years ago
I could be misunderstanding. Please correct me if needed!
I think we should spend time deprecating the need to create a talos zip for mobile.
ideally yes, we just need the android 4.0 mozharness script to be re-written
Created attachment 8493985 [details] [diff] [review]
Update talos.zip creation process

This patch actually does a few things:

1. What this bug says it does.
2. Use latest versions of all mozbase packages (instead of ancient github versions)
3. Remove simplejson copy (don't need it anymore)
4. Create the .zip file from a temporary directory. This saves us the trouble of needing to clean up after ourselves (just blow away the temporary directory at the end and we're done).
Attachment #8493985 - Flags: review?(jmaher)
(In reply to Armen Zambrano - Automation & Tools Engineer (:armenzg) from comment #1)
> I could be misunderstanding. Please correct me if needed!
> I think we should spend time deprecating the need to create a talos zip for
> mobile.

Yes, absolutely we should. This is just a stopgap.
before I review too much, these versions are pegged at certain versions of modules for a reason.  Maybe those reasons are not needed anymore.  Have you tested this creates a .zip file that we can use on try server to run tests on android?
(In reply to Joel Maher (:jmaher) from comment #6)
> before I review too much, these versions are pegged at certain versions of
> modules for a reason.  Maybe those reasons are not needed anymore.  Have you
> tested this creates a .zip file that we can use on try server to run tests
> on android?

Of course. :) That's what's being run through try right now.
Created attachment 8494010 [details] [diff] [review]
Same patch with versions pegged to current

Actually, it's probably better not to mess with the version numbers here. Let's try to just keep them as they are for now, we can always upgrade later quite easily.

This patch keeps the version in sync with setup.py and what we were using before. The only exception is mozcrash, because 0.9 was a .zip (and thus doesn't work with the fetching function), but I don't think that should cause any issues.
Attachment #8493985 - Attachment is obsolete: true
Attachment #8493985 - Flags: review?(jmaher)
Attachment #8494010 - Flags: review?(jmaher)
(In reply to William Lachance (:wlach) from comment #5)
> Try run for .zip generated with new script:
> https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=9df20b75ad4a

Try run looks good; I think we can commit this if it looks ok to :jmaher.
Comment on attachment 8494010 [details] [diff] [review]
Same patch with versions pegged to current

Review of attachment 8494010 [details] [diff] [review]:
-----------------------------------------------------------------

this looks like a great addition!

::: create_talos_zip.py
@@ +80,4 @@
>                     for f in httplib2_files]
>  
>  # all dependencies
> +manifest = datazilla_client + yaml + oauth2 + httplib2_oauth2

didn't you find we don't need yaml anymore?
Attachment #8494010 - Flags: review?(jmaher) → review+
(In reply to Joel Maher (:jmaher) from comment #10)
> Comment on attachment 8494010 [details] [diff] [review]
> ::: create_talos_zip.py
> @@ +80,4 @@
> >                     for f in httplib2_files]
> >  
> >  # all dependencies
> > +manifest = datazilla_client + yaml + oauth2 + httplib2_oauth2
> 
> didn't you find we don't need yaml anymore?

There's still stuff referencing yaml in the mozharness scripts, so it's probably not safe to remove yet. Thinking about it a bit more, I think changing that is pretty low priority. Others things would have vastly more bang/buck.
Pushed: https://hg.mozilla.org/build/talos/rev/c104a3296ee3

I also needed to update the version of mozlog used, because mozcrash now depends on structured logging. A retriggered try run of rck2 looked ok and jmaher gave approval on irc. :)
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.