Closed Bug 1036064 Opened 10 years ago Closed 10 years ago

--installer-url and --test-url should accept local files

Categories

(Release Engineering :: Applications: MozharnessCore, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gps, Assigned: gps)

Details

Attachments

(1 file)

I'm running mozharness locally and want to make it easier to point it at installers/tests generated from local builds. Patch will be uploaded momentarily.
--installer-path ?
We don't have an equivalent for --test-zip-path but I don't think it would be difficult to add.
If the argument given to --installer-url and --test-url looks like a
path (we only support UNIX paths), we prepend file:// to the value. The
existing URL handling code transparently "downloads" these URLs by
copying the file.

As part of testing this, I realized my local build wasn't producing a
jsshell archive and failed "downloads" would retry. I added some logic
to the download error handling logic to detect local filesystem errors
so this error is treated as fatal.

The .errno comparison should probably catch a few more failures, such as
permissions failures. But I'm not sure which ones are exclusive to
file:// vs other protocols. I suppose we could have certain protocol
handlers not retry. But this was the simplest patch.
Attachment #8452626 - Flags: review?(aki)
Assignee: nobody → gps
Status: NEW → ASSIGNED
(In reply to Aki Sasaki [:aki] from comment #1)
> --installer-path ?
> We don't have an equivalent for --test-zip-path but I don't think it would
> be difficult to add.

Well, I tried --installer-path with the code path I was using and it didn't like it! I think there are differences between --installer-path and --installer-url. I didn't want to track those down.

I think computers should take the cognitive load of "url" vs "path" away from the user. IMO we should trend towards a single argument (--installer/--tests-zip) that accepts either both URLs and paths. This patch gets us closer to that.
Comment on attachment 8452626 [details] [diff] [review]
Accept local paths to --installer-url and --test-url

Hm, I thought file:// didn't work, but now it does.  Not sure if this was faulty testing back then or a change in urllib2.
Attachment #8452626 - Flags: review?(aki) → review+
pyflakes sez:
mozharness/base/script.py:208: undefined name 'errno'
Merged to production branch of mozharness.
Is comment 6 a problem? Do we need to pull it out?
Flags: needinfo?(gps)
It's not a problem for automation use - only local use. I pushed a trivial followup (missing import of a stdlib module).

I keep forgetting I don't have my Mercurial extension for running pyflakes on commit globally installed. I need to fix that...
Flags: needinfo?(gps)
Merged to production, and deployed.
If this is deployed to production, why is the bug still open?
We leave it to the owner to resolve, since it's not always clear whether a bug is closable after the initial set of patches is landed.
Ahh, didn't know that.

In that case...
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
mozharness/base/script.py:215: [E, ScriptMixin._download_file] Module 'errno' has no 'NOENT' member

https://docs.python.org/2/library/errno.html

Did you mean errno.ENOENT ?
In production.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: