1442013 - Migrate CI off CircleCI 1.0 before August 31st 2018

Status: RESOLVED FIXED

(Conduit :: General, enhancement)

Description

[Steven MacLeod [:smacleod]]

Circle CI is sunsetting their 1.0 product: https://circleci.com/blog/sunsetting-1-0/

Many of our services rely on circle CI 1.0 for our builds and must be migrated to 2.0 or to another CI product. 1.0 builds will stop working after August 31st 2018.

Repos to migrate:
 - mozilla-conduit/lando-ui
 - mozilla-conduit/lando-api
 - mozilla-services/mozphab
 - mozilla-services/phabricator-extensions

(Possibly others, but I think I got most of them).

Comment 1

[Dylan Hardison [:dylan] (he/him)]

Attachment: lando-api PR

Comment 2

[Dylan Hardison [:dylan] (he/him)]

I've started on lando-api. The basics are there, but the stuff in docker-compose.yml won't port easily.

Comment 3

[Dylan Hardison [:dylan] (he/him)]

fun pip error:

Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 7, in <module>
    from compose.cli.main import main
  File "/usr/local/lib/python3.5/site-packages/compose/cli/main.py", line 23, in <module>
    pip_packages = pip.get_installed_distributions()
AttributeError: module 'pip' has no attribute 'get_installed_distributions'

Solution: downgrade pip
Seriously, who is docker-compose still this fragile?

Comment 4

[Dylan Hardison [:dylan] (he/him)]

  #!/bin/sh -eo pipefail
Next up: undeclared dependency on bash... somewhere. Not in our code.

invoke build
Traceback (most recent call last):
  File "/usr/local/bin/invoke", line 11, in <module>
    sys.exit(program.run())
  File "/usr/local/lib/python3.5/site-packages/invoke/program.py", line 270, in run
    self.execute()
  File "/usr/local/lib/python3.5/site-packages/invoke/program.py", line 381, in execute
    executor.execute(*self.tasks)
  File "/usr/local/lib/python3.5/site-packages/invoke/executor.py", line 113, in execute
    result = call.task(*args, **call.kwargs)
  File "/usr/local/lib/python3.5/site-packages/invoke/tasks.py", line 111, in __call__
    result = self.body(*args, **kwargs)
  File "/root/project/tasks.py", line 81, in build
    '-f ./docker/Dockerfile-prod .'.format(image_name=DOCKER_IMAGE_NAME)
  File "/usr/local/lib/python3.5/site-packages/invoke/context.py", line 53, in run
    return runner_class(context=self).run(command, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/invoke/runners.py", line 259, in run
    self.start(command, shell, env)
  File "/usr/local/lib/python3.5/site-packages/invoke/runners.py", line 966, in start
    stdin=PIPE,
  File "/usr/local/lib/python3.5/subprocess.py", line 676, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python3.5/subprocess.py", line 1289, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: '/bin/bash'

Comment 5

[Dylan Hardison [:dylan] (he/him)]

So... the dev build feeds into the prod build?

That is, does the dev docker file modify any files that are bind-mounted in?

Comment 6

[Dylan Hardison [:dylan] (he/him)]

Next error: 

ERROR: Cannot start service lando-api: b'OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \\"docker/entrypoint.sh\\": stat docker/entrypoint.sh: no such file or directory": unknown'
Traceback (most recent call last):
  File "/usr/local/bin/invoke", line 11, in <module>
    sys.exit(program.run())

I suspect some refactoring will be needed.

Comment 7

[Steven MacLeod [:smacleod]]

(In reply to Dylan Hardison [:dylan] (he/him) from comment #5)
> So... the dev build feeds into the prod build?
> 
> That is, does the dev docker file modify any files that are bind-mounted in?

It doesn't actually feed into it, we just use the dev container for running tests. Eventually I'd like to migrate to a single image for both, but that's out of scope here.

We discussed over vidyo and the bind-mounted directories are going to be removed for CI, as we don't actually need that.

Comment 8

[Dylan Hardison [:dylan] (he/him)]

Attachment: circle-ci: migrate lando-api to circleci 2.0 (Bug 1442013).

Comment 9

[Steven MacLeod [:smacleod]]

Comment on attachment 9004382 [details]
circle-ci: migrate lando-api to circleci 2.0 (Bug 1442013).

Steven MacLeod [:smacleod] has approved the revision.

Comment 10

[Dylan Hardison [:dylan] (he/him)]

Attachment: circleci: remove unsupported '-e' argument to docker login

docker login used to support an -e ("email") option.
Since we're using a newer docker now, this is an error.

Comment 11

[Dylan Hardison [:dylan] (he/him)]

Attachment: circleci: migrate from 1.0 to 2.0 config

Migrate from CircleCI 1.0 to 2.0.
Migration also required upgrading flask-pyoidc because installing
the version from GitHub broke the 'build' job.

Comment 12

[Steven MacLeod [:smacleod]]

Comment on attachment 9005657 [details]
circleci: remove unsupported '-e' argument to docker login

Steven MacLeod [:smacleod] has approved the revision.

Comment 13

[Steven MacLeod [:smacleod]]

Comment on attachment 9005658 [details]
circleci: migrate from 1.0 to 2.0 config

Steven MacLeod [:smacleod] has approved the revision.