1514205 - ./mach bootstrap fails with error IOError: [Errno 2] No such file or directory: u'taskcluster/docker/update-verify/Dockerfile'

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Mike de Boer [:mikedeboer]]

I get this on MacOSX Mojave (I just upgraded, if that matters), doing:

1. ./mach bootstrap,
2. Option 2 (Firefox for Desktop non-artifact),
3. (Y) To update my Mercurial config
4. (Y) Will be submitting commits to Mozilla

Then it fails at the command

Error running mach:

    ['artifact', 'toolchain', '--from-build', 'macosx64-cbindgen']

The details of the failure are as follows:

IOError: [Errno 2] No such file or directory: u'taskcluster/docker/update-verify/Dockerfile'

  File "/Users/mikedeboer/Projects/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1434, in artifact_toolchain
    toolchains = load_tasks_for_kind(params, 'toolchain', root_dir=root_dir)
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/generator.py", line 394, in load_tasks_for_kind
    for task in tgg.full_task_set
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/generator.py", line 141, in full_task_set
    return self._run_until('full_task_set')
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/generator.py", line 342, in _run_until
    k, v = self._run.next()
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/generator.py", line 263, in _run
    new_tasks = kind.load_tasks(parameters, list(all_tasks.values()))
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/generator.py", line 76, in load_tasks
    for task_dict in transforms(trans_config, inputs)]
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1909, in check_run_task_caches
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1855, in check_task_dependencies
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1843, in check_task_identifiers
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1826, in chain_of_trust
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1672, in build_task
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1642, in add_index_routes
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1460, in validate
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1437, in task_name_from_label
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/task.py", line 1396, in set_defaults
    for task in tasks:
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/cached_tasks.py", line 64, in cache_task
    for task in order_tasks(config, tasks):
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/cached_tasks.py", line 22, in order_tasks
    pending = deque(tasks)
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/transforms/docker_image.py", line 117, in fill_template
    GECKO, context_path, image_name, args)
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/util/docker.py", line 161, in generate_context_hash
    return stream_context_tar(topsrcdir, image_path, VoidWriter(), image_name, args)
  File "/Users/mikedeboer/Projects/mozilla-central/taskcluster/taskgraph/util/docker.py", line 221, in stream_context_tar
    with open(os.path.join(context_dir, 'Dockerfile'), 'rb') as fh:


Please let me know if you need more information from me. Thanks!

Comment 1

[Mike de Boer [:mikedeboer]]

Running `./mach artifact toolchain --from-build macosx64-cbindgen` directly from the command line works fine.

This looks like a relative vs. absolute path issue to me...

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Seeing it on Windows too.

Comment 3

[Nathan Froyd [:froydnj]]

It looks like https://hg.mozilla.org/mozilla-central/rev/c5904125b48c might have broken this?  It's the only relevant change that appears recently that touches the files in that error stack.  Tom, does that seem plausible to you?

Comment 4

[Mike de Boer [:mikedeboer]]

(Ad. this makes it so that I don't reach the rustup part, which leaves me with a tree I can't build :/ )

Comment 5

[Tom Prince [:tomprince]]

Attachment: Bug 1514205: [taskgraph] Look for docker images in the provided source directory, in stream_context_tar; r?dustin

This allows `mach artifact toolchain` (via `load_tasks_for_kind` to work
outside the root of the source directory.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/8641f3fc2eec
[taskgraph] Look for docker images in the provided source directory, in stream_context_tar; r=dustin a=tomprince

Comment 7

[Nathan Froyd [:froydnj]]

Thanks Tom!

Comment 8

[Tom Prince [:tomprince]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/2e419660137b