Closed Bug 1180949 Opened 7 years ago Closed 7 years ago

`mach artifact install` fails without wget installed

Categories

(Firefox for Android Graveyard :: General, defect)

All
Android
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: mcomella, Unassigned, Mentored)

References

Details

(Whiteboard: [lang=python][good next bug])

Attachments

(1 file)

Attached file Failure output
My OS X install (10.10) did not have wget pre-installed.

I installed via brew and `mach artifact install` was able to complete.

I believe `curl` comes pre-installed - perhaps that should be used if wget is not available?
We're using wget specifically so it will handle --timestamp for us.  (Curl doesn't make this as easy.)

I'd prefer that we actually use Python to do the fetch and file caching.  Using something like

https://github.com/mozilla/mozregression/blob/master/mozregression/download_manager.py
https://github.com/mozilla/mozregression/blob/master/mozregression/limitedfilecache.py

would be good, I think.  This is a great contributor ticket, although maybe not a great first ticket.
Mentor: nalexander, michael.l.comella
Whiteboard: [lang=python][good next bug]
Assignee: nobody → rshenthar
You can find the relevant code by using:

nalexander@chocho ~/M/gecko> ./mach mach-debug-commands artifact
artifact
========

File: /Users/nalexander/Mozilla/gecko/mobile/android/mach_commands.py
Class: PackageFrontend
Method: artifact

In this case, that's a wrapper around the real code, which is at https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/artifacts.py.

The relevant code is at https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/artifacts.py#261.  The caching is a little tricky, so you might want to disable it entirely at first (by removing the @cachedmethod wrapper.)
Thanks Nick! I searched mozilla-central but could not find any use of mozregression modules in it. Could you point me to a file where download manager and LimitedFileCache are used?

Are the highlighted code areas the ones that we need to change in the below link?
https://pastebin.mozilla.org/8839224

Also I am not familiar with caching - for the wget or for any download, could you please give a brief explanation of how the process works.(with respect to https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/artifacts.py#261 would be even simpler.)
Flags: needinfo?(nalexander)
(In reply to rshenthar from comment #3)
> Thanks Nick! I searched mozilla-central but could not find any use of
> mozregression modules in it. Could you point me to a file where download
> manager and LimitedFileCache are used?

There are no uses.  mozregression is in pypi (https://pypi.python.org/pypi/mozregression/); see the |mach artifact| command to see examples of using pip to install pypi dependencies.  You should be able to add the dependency and then use the code in the command easily.  This is how we use pylru, for example.
 
> Are the highlighted code areas the ones that we need to change in the below
> link?
> https://pastebin.mozilla.org/8839224

Yes, mostly.  You'll need to rewrite this method entirely to not use wget and to use the mozregression modules.  (Assuming that it's possible; I haven't investigated the mozregression API thoroughly.  You might need to work on mozregression to make it work for your situation as well.)

> Also I am not familiar with caching - for the wget or for any download,
> could you please give a brief explanation of how the process works.(with
> respect to
> https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/
> artifacts.py#261 would be even simpler.)

The caching is built into the mozregression download_manager.  You shouldn't need to know too much here.  A demo that didn't cache (but used mozregression) would be a big help.
Flags: needinfo?(nalexander)
Unassigning as I am occupied with high priority bugs for now. Free to take and work on.
Assignee: rshenthar → nobody
Can I work on this one?  Just a fore warning, I haven't worked with Python a whole lot.  But, I defintely want to learn.
(In reply to Alex Johnson (:alex_johnson) from comment #6)
> Can I work on this one?  Just a fore warning, I haven't worked with Python a
> whole lot.  But, I defintely want to learn.

I think this has been fixed organically.  Now we're using mozdownload, so this isn't an issue:

https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/artifacts.py#265

Alex, you might be interested in https://bugzilla.mozilla.org/show_bug.cgi?id=1221200.  That ticket has significant work in progress but has stalled.  I'd want to check in with Harshit before jumping on that, but take a look at it.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WORKSFORME
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.