1402409 - ./mach try doesn't work anymore, only ./mach try fuzzy

Status: RESOLVED DUPLICATE of bug 1402233

(Firefox Build System :: General, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

in trying to fix a bug, I have been doing a lot of try pushes- I get failures, at first it was:
Traceback (most recent call last):
  File "c:\Users\elvis\mozilla-inbound\taskcluster/mach_commands.py", line 409, in show_taskgraph
    parameters.check()
  File "c:\Users\elvis\mozilla-inbound\taskcluster\taskgraph\parameters.py", line 53, in check
    raise Exception("; ".join(msg))
Exception: missing parameters: try_mode, try_task_config, try_options; extra parameters: morph_templates, target_task_labels


:ahal, mentioned I needed to update to pick up :dustin's changes in bug 1383880, that works.  Sadly ./mach try fuzzy doesn't solve my needs all the time, I tried pushing with |./mach try -b do -p win32,win64 -u all -t none| and I get a failure:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a8ed78224a69323e643ebe35990cad93299a3cec

which shows this error:
[task 2017-09-22T16:30:47.642334Z] Traceback (most recent call last):
[task 2017-09-22T16:30:47.642444Z]   File "/builds/worker/checkouts/gecko/taskcluster/mach_commands.py", line 154, in taskgraph_decision
[task 2017-09-22T16:30:47.642496Z]     return taskgraph.decision.taskgraph_decision(options)
[task 2017-09-22T16:30:47.642556Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/decision.py", line 135, in taskgraph_decision
[task 2017-09-22T16:30:47.642741Z]     write_artifact('task-graph.json', tgg.morphed_task_graph.to_json())
[task 2017-09-22T16:30:47.642817Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/generator.py", line 178, in morphed_task_graph
[task 2017-09-22T16:30:47.642868Z]     return self._run_until('morphed_task_graph')
[task 2017-09-22T16:30:47.642923Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/generator.py", line 283, in _run_until
[task 2017-09-22T16:30:47.643033Z]     k, v = self._run.next()
[task 2017-09-22T16:30:47.643397Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/generator.py", line 270, in _run
[task 2017-09-22T16:30:47.643469Z]     do_not_optimize)
[task 2017-09-22T16:30:47.643533Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/optimize.py", line 55, in optimize_task_graph
[task 2017-09-22T16:30:47.643574Z]     do_not_optimize=do_not_optimize)
[task 2017-09-22T16:30:47.643839Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/optimize.py", line 126, in remove_tasks
[task 2017-09-22T16:30:47.643891Z]     if opt.should_remove_task(task, params, arg):
[task 2017-09-22T16:30:47.644158Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/optimize.py", line 376, in should_remove_task
[task 2017-09-22T16:30:47.644219Z]     scheduled = self.scheduled_by_push(params['head_repository'], params['head_rev'])
[task 2017-09-22T16:30:47.644306Z]   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/util.py", line 944, in method_call
[task 2017-09-22T16:30:47.644537Z]     cache[args] = self.func(instance, *args)
[task 2017-09-22T16:30:47.644639Z]   File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/optimize.py", line 367, in scheduled_by_push
[task 2017-09-22T16:30:47.644683Z]     for p, m in rdr.files_info(changed_files).items():
[task 2017-09-22T16:30:47.644920Z]   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/reader.py", line 1342, in files_info
[task 2017-09-22T16:30:47.645007Z]     paths, _ = self.read_relevant_mozbuilds(paths)
[task 2017-09-22T16:30:47.645071Z]   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/reader.py", line 1306, in read_relevant_mozbuilds
[task 2017-09-22T16:30:47.645328Z]     self.config, metadata=metadata):
[task 2017-09-22T16:30:47.645398Z]   File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/reader.py", line 1061, in read_mozbuild
[task 2017-09-22T16:30:47.645583Z]     raise bre
[task 2017-09-22T16:30:47.645662Z] BuildReaderError: 
[task 2017-09-22T16:30:47.645704Z] ==============================
[task 2017-09-22T16:30:47.645746Z] ERROR PROCESSING MOZBUILD FILE
[task 2017-09-22T16:30:47.645821Z] ==============================
[task 2017-09-22T16:30:47.645852Z] 
[task 2017-09-22T16:30:47.645895Z] The error occurred while processing the following file:
[task 2017-09-22T16:30:47.645916Z] 
[task 2017-09-22T16:30:47.645980Z]     /builds/worker/checkouts/gecko/testing/web-platform/moz.build
[task 2017-09-22T16:30:47.646006Z] 
[task 2017-09-22T16:30:47.646167Z] The error was triggered on line 9 of this file:
[task 2017-09-22T16:30:47.646201Z] 
[task 2017-09-22T16:30:47.646240Z]     ('mozilla/meta/MANIFEST.json', 'mozilla/tests/')
[task 2017-09-22T16:30:47.646266Z] 
[task 2017-09-22T16:30:47.646345Z] An error was encountered as part of executing the file itself. The error appears to be the fault of the script.
[task 2017-09-22T16:30:47.646377Z] 
[task 2017-09-22T16:30:47.646416Z] The error as reported by Python is:
[task 2017-09-22T16:30:47.646438Z] 
[task 2017-09-22T16:30:47.646509Z]     ["UnboundLocalError: local variable 'wptmanifest' referenced before assignment\n"]
[task 2017-09-22T16:30:47.646534Z] 
[task


after spending about an hour trying to figure out what is broken, I realize that a previous push of ./mach try fuzzy on the same base revision worked- I experimented and learned that ./mach try fails, but ./mach try fuzzy works.

I am not sure if |./mach try fuzzy| is invalid and I should go ahead and land my code- I have less confidence now.  I assume this is related to bug 1383880, I did an |hg update <rev_before_bug_1383880>| and ./mach try worked and ./mach try fuzzy failed- which is why I believe it is related to bug 1383880.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

It looks related to bug 1203266 -- I see an `import` as the last statement in a block.. is that a typo, or intentional?

def read_wpt_manifest(context, paths):
    manifest_path, tests_root = paths
    full_path = mozpath.normpath(mozpath.join(context.srcdir, manifest_path))
    old_path = sys.path[:]
    try:
        # Setup sys.path to include all the dependencies required to import
        # the web-platform-tests manifest parser. web-platform-tests provides
        # a the localpaths.py to do the path manipulation, which we load,
        # providing the __file__ variable so it can resolve the relative
        # paths correctly.
        paths_file = os.path.join(context.config.topsrcdir, "testing",
                                  "web-platform", "tests", "tools", "localpaths.py")
        _globals = {"__file__": paths_file}
        execfile(paths_file, _globals)
        import manifest as wptmanifest
    finally:
        sys.path = old_path
        f = context._finder.get(full_path)
        return wptmanifest.manifest.load(tests_root, f)

Comment 2

[Justin Wood (:Callek)]

Possible dupe of: Bug 1402233

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

oh, good find on the potential dup.  Probably that would make the error go away, still curious why we don't see this with |./mach try fuzzy| though- maybe a different parsing route for tasks or builds

Comment 4

[Gregory Szorc [:gps]]

Yes, it's a dupe. Long term solution is bug 1402010.

And, yes, the import being the last line is intentionally. That localpaths.py file adjusts sys.path. So hacky.