Closed Bug 1237681 Opened 5 years ago Closed 5 years ago

set up the desktop-test image to build on push

Categories

(Taskcluster Graveyard :: Docker Images, defect)

Product:
Taskcluster Graveyard
Component:
Docker Images
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dustin, Assigned: dustin)

Details

Attachments

(1 file)

MozReview Request: Bug 1237681: remove REGISTRY and VERSION for in-tree-generated images; r?garndt
58 bytes, text/x-review-board-request
wcosta
: review+
garndt
: review+
Details 
It can keep its FROM line pulling from docker hub -- if someone wants to edit ubuntu1204-test, they're on their own.  But for experimenting in try, it's easy enough to add something to the desktop-test Dockerfile and push.
taskcluster/centos6-build-upd:20151201141000 didn't get pushed somehow.  Anyway, it has the wrong name (should have the base version as a prefix).  So I rebuilt and re-pushed it.
Note that this does not build their predecessors (ubuntu1204-test-upd and
centos6-build-upd) -- those still come from docker hub.

Review commit: https://reviewboard.mozilla.org/r/31829/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/31829/
Attachment #8710653 - Flags: review?(wcosta)
I have no idea why the build failed to mkdir:

21:10:46     INFO -  mkdir -p '/home/worker/workspace/build/src/obj-firefox/addon-sdk/source/test/addons/'
21:10:46     INFO -  mkdir: cannot create directory `/home/worker/workspace/build/src/obj-firefox/addon-sdk/source/test/addons/': No such file or directory
21:10:46     INFO -  gmake[5]: *** [/home/worker/workspace/build/src/obj-firefox/addon-sdk/source/test/addons/.mkdir.done] Error 1

maybe aufs failed?  Anyway, I can't see how it's related to this patch.
..and that build succeeded on a retrigger, so hmph.
Comment on attachment 8710653 [details]
MozReview Request: Bug 1237681: remove REGISTRY and VERSION for in-tree-generated images; r?garndt

https://reviewboard.mozilla.org/r/31829/#review28687
Attachment #8710653 - Flags: review?(wcosta) → review+
https://hg.mozilla.org/mozilla-central/rev/fa4ec02a4411
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Turns out we should delete VERSION and REGISTRY too.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
From push to push, is it going to be the same being pulled? (unless we change something under docker/desktop-test/)

Related steps:
> curl -L -o image.tar https://queue.taskcluster.net/v1/task/Q-kdufr9QC64qHr5aNc45Q/artifacts/public/image.tar
> image=$(tar xf image.tar -O repositories | jq -r 'keys[0]')
> image_tag=$(tar xf image.tar -O repositories | jq -r '.[keys[0]] | keys[0]')
> image_name=$image:$image_tag
>
> # Load docker image from tarball
> docker load < image.tar
Yes[1].  The decision task will compute the hash of the directory contents at testing/docker/desktop-test and then based on the branch and hash, it will look in the index for the task that has been indexed under that namespace.  If nothing has changed in that directory, the hash should be the same and should return the same task that was indexed.

[1] Once the image lands on m-c, that image will be used for all tasks regardless of branch as long as the hash of the directory contents is the same.
That's pretty cool!

If we were to want to run Linux64 tests from a developer's local host by using mach to instantiate docker, what would be the right way to determine from where should the image be fetched?
And what would be the way to determine if we already have the image on disk?
Right now the images are indexed under the following namespaces:
https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/routes.json#18


If there is a more user friendly way of indexing these images that helps out, I'm happy to help out.

the context hash could be used, but I would first want to make sure that the hash is consistent regardless of where the clone of your repo is located locally.  Currently it might take the full path into account rather than the path relative to the root of the repo.
Thanks! That should help as long as I do it consistently.
Attachment #8710653 - Attachment description: MozReview Request: Bug 1237681: build desktop-build and desktop-test on demand; r?wcosta → MozReview Request: Bug 1237681: remove REGISTRY and VERSION for in-tree-generated images; r?garndt
Attachment #8710653 - Flags: review?(garndt)
Comment on attachment 8710653 [details]
MozReview Request: Bug 1237681: remove REGISTRY and VERSION for in-tree-generated images; r?garndt

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/31829/diff/1-2/
Comment on attachment 8710653 [details]
MozReview Request: Bug 1237681: remove REGISTRY and VERSION for in-tree-generated images; r?garndt

How will this affect users who want to use the docker image on their desktops?  Should we just worry about that later?
(In reply to Dustin J. Mitchell [:dustin] from comment #19)
> Comment on attachment 8710653 [details]
> MozReview Request: Bug 1237681: remove REGISTRY and VERSION for
> in-tree-generated images; r?garndt
> 
> How will this affect users who want to use the docker image on their
> desktops?  Should we just worry about that later?

I assume that is why bug 1237681 was created which is definitely a lot more friendly for grabbing the image than using the run locally script found for a task.  Thanks for entering that!
Attachment #8710653 - Flags: review?(garndt) → review+
Comment on attachment 8710653 [details]
MozReview Request: Bug 1237681: remove REGISTRY and VERSION for in-tree-generated images; r?garndt

https://reviewboard.mozilla.org/r/31829/#review29667
https://hg.mozilla.org/mozilla-central/rev/b451c824ecc1
Status: REOPENED → RESOLVED
Closed: 5 years ago → 5 years ago
Resolution: --- → FIXED
Product: Taskcluster → Taskcluster Graveyard
You need to log in before you can comment on or make changes to this bug.