1682972 - Push pioneer-debug container to pioneer-debug project for use as custom ai platform notebook container

Status: RESOLVED FIXED

(Data Platform and Tools Graveyard :: Operations, enhancement)

Description

[Anthony Miyaguchi [:amiyaguchi]]

A gcr.io/moz-fx-data-airflow-prod-88e0/pioneer-debug_docker_etl container is being published by docker-etl. This can be used within AI platform notebooks. I would like to test this container in a rally project to close the loop on the custom containers proposal: https://docs.google.com/document/d/1OsHeG2YaEu1rx4La1MVH8NV9xYBqZaOZxNRn0WopkMI/edit?usp=sharing

Due to the security perimeter in the rally infrastructure, the container would need to be copied to the specific project so it could be used. There is some discussion around this in bug 1656112.

Comment 1

[Wesley Dawson [:whd]]

I've pushed this manually to gcr.io/moz-fx-data-pioneer-debug/pioneer-debug_docker_etl.

jenkins@jenkins-node-default:~$ docker push gcr.io/moz-fx-data-pioneer-debug/pioneer-debug_docker_etl:latest
The push refers to repository [gcr.io/moz-fx-data-pioneer-debug/pioneer-debug_docker_etl]
59d338f9e0fb: Pushed 
2f287174c30a: Pushed 
58c37b024800: Pushed 
093955a9f693: Pushed 
292c93aa8921: Pushed 
25e90c4f31bb: Pushed 
5ed5b5583a70: Pushed 
fed2ce1b9bf5: Pushed 
a2a7397c9263: Pushed 
135d5d53f509: Pushed 
28952c0fc305: Pushed 
1fff2aeddb5e: Pushed 
193419df8fce: Pushed 
9d1088ee89e7: Pushed 
98868f5e88f9: Pushed 
efa6a40d1ffb: Pushed 
7a694df0ad6c: Pushed 
3fd9df553184: Pushed 
805802706667: Pushed 
latest: digest: sha256:f53670e6e4aa59267fdb397c34cf318cf5baf637e90e0d266a8ba292226feb5f size: 4287

https://console.cloud.google.com/gcr/images/moz-fx-data-pioneer-debug/GLOBAL/pioneer-debug_docker_etl?project=moz-fx-data-pioneer-debug&organizationId=442341870013&gcrImageListsize=30

It took quite a long time (minutes) which is probably either due to the security perimeter or the image being fairly large.

Comment 2

[Anthony Miyaguchi [:amiyaguchi]]

Thank you, it looks like this works fine in the environment. I had to specify the full tag of the image (gcr.io/moz-fx-data-pioneer-debug/pioneer-debug_docker_etl:latest) for this to work correctly, otherwise running into the following issue:

Dec 16 22:48:46 r-3-20201216-144629 c2d-startup[417]: Error: No such object: gcr.io/moz-fx-data-pioneer-debug/pioneer-debug_docker_etl
Dec 16 22:48:46 r-3-20201216-144629 c2d-startup[417]: ERROR: unable to pull  gcr.io/moz-fx-data-pioneer-debug/pioneer-debug_docker_etl.
Dec 16 22:49:08 r-3-20201216-144629 c2d-startup[417]: ERROR: (gcloud.compute.instances.add-metadata) Could not fetch resource:
Dec 16 22:49:08 r-3-20201216-144629 c2d-startup[417]:  - Required 'compute.instances.setMetadata' permission for 'projects/moz-fx-data-pioneer-debug/zones/us-west1-b/instances/r-3-20201216-144629'

Once I got into the environment, I confirmed that this was indeed the correct container

root@5943e47472bd:/app# ls /app/tutorials/
README-storage.ipynb  README.ipynb

I copied the tutorial notebooks into the jupyter home directory, and ran a few cells to confirm that the environment was working as expected.

I'll update the "using the analysis environment" document with a footnote that it is possible to utilize custom containers if the default image doesn't have the necessary software via docker-etl.

Comment 3

[Wesley Dawson [:whd]]

Infrastructure for automating pushing of containers to all projects is expected to be part of bug #1660350 but for now these pushes will occur manually as needed.