Closed
Bug 1244189
Opened 8 years ago
Closed 8 years ago
Add a mach command to download docker images from artifacts
Categories
(Taskcluster Graveyard :: Docker Images, defect)
Taskcluster Graveyard
Docker Images
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla48
People
(Reporter: dustin, Assigned: dustin)
Details
Attachments
(1 file)
./mach docker-load desktop-test would do the usual arithmetic to figure out what task artifact corresponds to the desktop-test directory in-tree, docker-load it, and tag it as `desktop-test:latest`.
Assignee | ||
Comment 1•8 years ago
|
||
The command helpfully downloads a docker image created by a taskcluster task, which can be helpful when trying to replicate subtle bugs that only occur in-tree. This also fixes a bug in hashing Dockerfile directories, where the full pathname was taken into account. While this pathname is consistent from decision task to decision task, it is not consistent with developers' home directories. This change omits the directory prefix, which will cause a one-time shift in all directory hashes. Review commit: https://reviewboard.mozilla.org/r/42505/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/42505/
Attachment #8734951 -
Flags: review?(garndt)
Comment 2•8 years ago
|
||
Comment on attachment 8734951 [details] MozReview Request: Bug 1244189: add 'mach taskcluster-load-image'; r?garndt https://reviewboard.mozilla.org/r/42505/#review39015 ::: testing/taskcluster/mach_commands.py:238 (Diff revision 1) > + @CommandArgument('--task-id', > + help="Load the image at public/image.tar in this task, rather than " > + "searching the index") > + @CommandArgument('image_name', nargs='?', > + help="Load the image of this name based on the current contents of the tree " > + "(as built for mozilla-central or mozilla-inbound)") Hrm, why is this "or mozilla-inbound" but we only seem to be loading from mozilla-central below. ::: testing/taskcluster/taskcluster_graph/image_builder.py:250 (Diff revision 1) > + filename = 'temp-docker-image.tar' > + > + print("Downloading {}".format(url)) > + subprocess.check_call(['curl', '-#', '-L', '-o', filename, url]) > + > + print("Loading image into docker") So the tarball is downloaded, which could be many gigabytes, before trying to load the image. Would it be helpful to catch CalledProcessError in the case that docker load fails or that docker daemon is unreachable (happens a lot when running docker-machine and the current environment isn't configured for whatever machine you want to use). This way we could include an error message indicating what might have happened, where the image was stored, and what to do once docker is configured/working correctly. Maybe the user could then run docker load themselves or something since the file was already downloaded. ::: testing/taskcluster/taskcluster_graph/image_builder.py:251 (Diff revision 1) > + > + print("Downloading {}".format(url)) > + subprocess.check_call(['curl', '-#', '-L', '-o', filename, url]) > + > + print("Loading image into docker") > + subprocess.check_call(['docker', 'load', '-i', filename]) I wish 'load' returned back the image name, or at least the image ID that it loaded it with, but after talking with the docker team I guess that would not be nice since an image tarball could actually contain multiple images.
Attachment #8734951 -
Flags: review?(garndt)
Assignee | ||
Comment 3•8 years ago
|
||
https://reviewboard.mozilla.org/r/42505/#review39015 > Hrm, why is this "or mozilla-inbound" but we only seem to be loading from mozilla-central below. Whoops, I changed the message but not the implementation. The idea is to check both branches so you can use the latest and greatest even if it hasn't merged to central yet. I'll fix. > So the tarball is downloaded, which could be many gigabytes, before trying to load the image. Would it be helpful to catch CalledProcessError in the case that docker load fails or that docker daemon is unreachable (happens a lot when running docker-machine and the current environment isn't configured for whatever machine you want to use). This way we could include an error message indicating what might have happened, where the image was stored, and what to do once docker is configured/working correctly. Maybe the user could then run docker load themselves or something since the file was already downloaded. Good idea - thanks! > I wish 'load' returned back the image name, or at least the image ID that it loaded it with, but after talking with the docker team I guess that would not be nice since an image tarball could actually contain multiple images. You would think it could just output one `name:tag` combination per line or something. It ain't rocket science. And having to depend on the structure of the tarball feels pretty gross.
Assignee | ||
Comment 4•8 years ago
|
||
Comment on attachment 8734951 [details] MozReview Request: Bug 1244189: add 'mach taskcluster-load-image'; r?garndt Review request updated; see interdiff: https://reviewboard.mozilla.org/r/42505/diff/1-2/
Attachment #8734951 -
Flags: review?(garndt)
Comment 5•8 years ago
|
||
Comment on attachment 8734951 [details] MozReview Request: Bug 1244189: add 'mach taskcluster-load-image'; r?garndt https://reviewboard.mozilla.org/r/42505/#review39023
Attachment #8734951 -
Flags: review?(garndt) → review+
Assignee | ||
Comment 6•8 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/4840ae780ac1b30464fd93be2c1170939a512887 Bug 1244189: add 'mach taskcluster-load-image'; r=garndt
Comment 7•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/4840ae780ac1
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
Updated•6 years ago
|
Product: Taskcluster → Taskcluster Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•