In the interactive task, it looks like `mach` attempts to load the `mach_initialize` module in `/builds/worker/workspace/build/tests/tools/mach_initialize.py`, which seems to be [this file](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/testing/tools/mach_test_package_initialize.py) and it only contains a [`bootstrap` function](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/testing/tools/mach_test_package_initialize.py#165). Fixing this issue does not make things better, though. The next error is: ``` root@7bb12ab96ace:~/workspace/build/tests# mach help Traceback (most recent call last): File "/builds/worker/bin/mach", line 101, in <module> main(sys.argv[1:]) File "/builds/worker/bin/mach", line 93, in main mach = check_and_get_mach(os.path.dirname(os.path.realpath(__file__))) File "/builds/worker/bin/mach", line 40, in check_and_get_mach return load_mach(dir_path, mach_path) File "/builds/worker/bin/mach", line 26, in load_mach return mach_initialize.bootstrap(dir_path) File "/builds/worker/workspace/build/tests/tools/mach_initialize.py", line 169, in bootstrap import mach.main File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 33, in <module> from .dispatcher import CommandAction File "/builds/worker/workspace/build/tests/tools/mach/mach/dispatcher.py", line 20, in <module> from .decorators import SettingsProvider File "/builds/worker/workspace/build/tests/tools/mach/mach/decorators.py", line 14, in <module> from mozbuild.base import MachCommandBase ModuleNotFoundError: No module named 'mozbuild' ``` That seems to be Bug 1515743 or very similar, right? I ran `run-wizard` again to checkout gecko and then I ran the following command to install `mozbuild`: ``` root@7bb12ab96ace:~/workspace/build/tests# pip install ../../../gecko/python/mozbuild/ ``` That didn't work better so I first made sure to activate the provided `venv` (I didn't find much docs about interactive tasks to run mochitests so I am flying blind here): ``` root@7bb12ab96ace:~/workspace/build/tests# source ../venv/bin/activate (venv) root@7bb12ab96ace:~/workspace/build/tests# ``` Then I installed all the python packages with the usual try (`mach help`)/fail/install/repeat. Some packages (starting with `moz`) were installed from the `../../../gecko/python/`. Others from the PyPI registry. Eventually, I got the output below: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# mach mochitest browser/components/downloads/test/browser/browser_pdfjs_preview.js /builds/worker/workspace/build/venv/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) Error running mach: ['mochitest', 'browser/components/downloads/test/browser/browser_pdfjs_preview.js'] The error occurred in mach itself. This is likely a bug in mach itself or a fundamental problem with a loaded module. You can invoke |./mach busted| to check if this issue is already on file. If it isn't, please use |./mach busted file general| to report it. If |./mach busted| is misbehaving, you can also inspect the dependencies of bug 1543241. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: TypeError: expected str, bytes or os.PathLike object, not NoneType File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 399, in run return self._run(argv) File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 433, in _run topsrcdir = Path(self.populate_context_handler("topdir")) File "/usr/lib/python3.6/pathlib.py", line 1001, in __new__ self = cls._from_parts(args, init=False) File "/usr/lib/python3.6/pathlib.py", line 656, in _from_parts drv, root, parts = self._parse_args(args) File "/usr/lib/python3.6/pathlib.py", line 640, in _parse_args a = os.fspath(a) ``` At least the dependencies were resolved... Fortunately, this error was related to some error reporting (sentry) and we could use a [`NoopErrorReporter`](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/python/mach/mach/main.py#436) instead. ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# mach help /builds/worker/workspace/build/venv/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) usage: mach [global arguments] command [command arguments] Global Arguments: -v, --verbose Print verbose output. -l FILENAME, --log-file FILENAME Filename to write log data to. --log-interval Prefix log line with interval from last message rather than relative time. Note that this is NOT execution time if there are parallel operations. --no-interactive Automatically selects the default option on any interactive prompts. If the output is not a terminal, then --no-interactive is assumed. --log-no-times Do not prefix log lines with times. By default, mach will prefix each output line with the time since command start. -h, --help Show this help message. --debug-command Start a Python debugger when command is dispatched. --profile-command Capture a Python profile of the mach process as command is dispatched. --settings FILENAME Path to settings file. Testing: Run tests. geckoview-junit Run the geckoview-junit harness. marionette-test Run a Marionette test (Check UI or the internal JavaScript using marionette). mochitest Run the mochitest harness. ``` I then tried to run `mach mochitest` and it failed, ugh. A wrong path somewhere that I fixed with a symlink: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# ln -s /builds/worker/gecko/ /builds/worker/checkouts/gecko ``` Then, a missing `.mozconfig` file: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# touch ~/gecko/.mozconfig ``` Then, a missing shell script when I installed `mozbuild` manually: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# ln -s /builds/worker/gecko/python/mozbuild/mozbuild/mozconfig_loader ../venv/lib/python3.6/site-packages/mozbuild/ ``` Things still didn't work, likely because Bug 1636251 changed a few things around `mach`'s context, especially the `populate_context()` signature: ```diff - def populate_context(context, key=None): + def populate_context(key=None): ``` I butchered some more python files to successfully run the mochitest framework but I couldn't run the test file I was interested in... (it ran nothing because it didn't find any tests to run or something like that). :ahal, any ideas? Maybe mochitests aren't (or are no longer) supported in interactive tasks?
Bug 1730731 Comment 15 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
In the interactive task, it looks like `mach` attempts to load the `mach_initialize` module in `/builds/worker/workspace/build/tests/tools/mach_initialize.py`, which seems to be [this file](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/testing/tools/mach_test_package_initialize.py) and it only contains a [`bootstrap` function](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/testing/tools/mach_test_package_initialize.py#165). Fixing this issue does not make things better, though. The next error is: ``` root@7bb12ab96ace:~/workspace/build/tests# mach help Traceback (most recent call last): File "/builds/worker/bin/mach", line 101, in <module> main(sys.argv[1:]) File "/builds/worker/bin/mach", line 93, in main mach = check_and_get_mach(os.path.dirname(os.path.realpath(__file__))) File "/builds/worker/bin/mach", line 40, in check_and_get_mach return load_mach(dir_path, mach_path) File "/builds/worker/bin/mach", line 26, in load_mach return mach_initialize.bootstrap(dir_path) File "/builds/worker/workspace/build/tests/tools/mach_initialize.py", line 169, in bootstrap import mach.main File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 33, in <module> from .dispatcher import CommandAction File "/builds/worker/workspace/build/tests/tools/mach/mach/dispatcher.py", line 20, in <module> from .decorators import SettingsProvider File "/builds/worker/workspace/build/tests/tools/mach/mach/decorators.py", line 14, in <module> from mozbuild.base import MachCommandBase ModuleNotFoundError: No module named 'mozbuild' ``` That seems to be Bug 1515743 or very similar, right? I ran `run-wizard` again to checkout gecko and then I ran the following command to install `mozbuild`: ``` root@7bb12ab96ace:~/workspace/build/tests# pip install ../../../gecko/python/mozbuild/ ``` That didn't work better so I first made sure to activate the provided `venv` (I didn't find much docs about interactive tasks to run mochitests so I am flying blind here): ``` root@7bb12ab96ace:~/workspace/build/tests# source ../venv/bin/activate (venv) root@7bb12ab96ace:~/workspace/build/tests# ``` Then I installed all the python packages with the usual try (`mach help`)/fail/install/repeat. Some packages (starting with `moz`) were installed from the `../../../gecko/python/`. Others from the PyPI registry. Eventually, I got the output below: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# mach mochitest browser/components/downloads/test/browser/browser_pdfjs_preview.js /builds/worker/workspace/build/venv/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) Error running mach: ['mochitest', 'browser/components/downloads/test/browser/browser_pdfjs_preview.js'] The error occurred in mach itself. This is likely a bug in mach itself or a fundamental problem with a loaded module. You can invoke |./mach busted| to check if this issue is already on file. If it isn't, please use |./mach busted file general| to report it. If |./mach busted| is misbehaving, you can also inspect the dependencies of bug 1543241. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: TypeError: expected str, bytes or os.PathLike object, not NoneType File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 399, in run return self._run(argv) File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 433, in _run topsrcdir = Path(self.populate_context_handler("topdir")) File "/usr/lib/python3.6/pathlib.py", line 1001, in __new__ self = cls._from_parts(args, init=False) File "/usr/lib/python3.6/pathlib.py", line 656, in _from_parts drv, root, parts = self._parse_args(args) File "/usr/lib/python3.6/pathlib.py", line 640, in _parse_args a = os.fspath(a) ``` At least the dependencies were resolved... Fortunately, this error was related to some error reporting (sentry) and we could use a [`NoopErrorReporter`](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/python/mach/mach/main.py#436) instead. ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# mach help /builds/worker/workspace/build/venv/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) usage: mach [global arguments] command [command arguments] Global Arguments: -v, --verbose Print verbose output. -l FILENAME, --log-file FILENAME Filename to write log data to. --log-interval Prefix log line with interval from last message rather than relative time. Note that this is NOT execution time if there are parallel operations. --no-interactive Automatically selects the default option on any interactive prompts. If the output is not a terminal, then --no-interactive is assumed. --log-no-times Do not prefix log lines with times. By default, mach will prefix each output line with the time since command start. -h, --help Show this help message. --debug-command Start a Python debugger when command is dispatched. --profile-command Capture a Python profile of the mach process as command is dispatched. --settings FILENAME Path to settings file. Testing: Run tests. geckoview-junit Run the geckoview-junit harness. marionette-test Run a Marionette test (Check UI or the internal JavaScript using marionette). mochitest Run the mochitest harness. ``` I then tried to run `mach mochitest` and it failed, ugh. A wrong path somewhere that I fixed with a symlink: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# ln -s /builds/worker/gecko/ /builds/worker/checkouts/gecko ``` Then, a missing `.mozconfig` file: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# touch ~/gecko/.mozconfig ``` Then, a missing shell script when I installed `mozbuild` manually: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# ln -s /builds/worker/gecko/python/mozbuild/mozbuild/mozconfig_loader ../venv/lib/python3.6/site-packages/mozbuild/ ``` Things still didn't work, likely because Bug 1636251 changed a few things around `mach`'s context, especially the `populate_context()` signature: ```diff - def populate_context(context, key=None): + def populate_context(key=None): ``` I butchered some more python files to successfully run the mochitest framework but I couldn't run the test file I was interested in... (it ran nothing because it didn't find any tests to run or something like that). ``` [...] 0:00.39 ERROR No tests were found for flavor 'plain' and the following manifest filters: skip_if, run_if, fail_if, subsuite(name=None), pathprefix(['./browser/components/downloads/test/browser/browser_pdfjs_preview.js']) Make sure the test paths (if any) are spelt correctly and the corresponding --flavor and --subsuite are being used. See `mach mochitest --help` for a list of valid flavors. 0:00.39 SUITE_START: mochitest-plain - running 0 tests 0 INFO TEST-START | Shutdown 1 INFO Passed: 0 2 INFO Failed: 0 3 INFO Todo: 0 4 INFO Mode: e10s 5 INFO SimpleTest FINISHED 0:00.39 INFO Buffered messages finished 0:00.39 SUITE_END ``` :ahal, any ideas? Maybe mochitests aren't (or are no longer) supported in interactive tasks?
In the interactive task, it looks like `mach` attempts to load the `mach_initialize` module in `/builds/worker/workspace/build/tests/tools/mach_initialize.py`, which seems to be [this file](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/testing/tools/mach_test_package_initialize.py) and it only contains a [`bootstrap` function](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/testing/tools/mach_test_package_initialize.py#165). Fixing this issue does not make things better, though. The next error is: ``` root@7bb12ab96ace:~/workspace/build/tests# mach help Traceback (most recent call last): File "/builds/worker/bin/mach", line 101, in <module> main(sys.argv[1:]) File "/builds/worker/bin/mach", line 93, in main mach = check_and_get_mach(os.path.dirname(os.path.realpath(__file__))) File "/builds/worker/bin/mach", line 40, in check_and_get_mach return load_mach(dir_path, mach_path) File "/builds/worker/bin/mach", line 26, in load_mach return mach_initialize.bootstrap(dir_path) File "/builds/worker/workspace/build/tests/tools/mach_initialize.py", line 169, in bootstrap import mach.main File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 33, in <module> from .dispatcher import CommandAction File "/builds/worker/workspace/build/tests/tools/mach/mach/dispatcher.py", line 20, in <module> from .decorators import SettingsProvider File "/builds/worker/workspace/build/tests/tools/mach/mach/decorators.py", line 14, in <module> from mozbuild.base import MachCommandBase ModuleNotFoundError: No module named 'mozbuild' ``` That seems to be Bug 1515743 or very similar, right? I ran `run-wizard` again to checkout gecko and then I ran the following command to install `mozbuild`: ``` root@7bb12ab96ace:~/workspace/build/tests# pip install ../../../gecko/python/mozbuild/ ``` That didn't work better so I first made sure to activate the provided `venv` (I didn't find much docs about interactive tasks to run mochitests so I am flying blind here): ``` root@7bb12ab96ace:~/workspace/build/tests# source ../venv/bin/activate (venv) root@7bb12ab96ace:~/workspace/build/tests# ``` Then I installed all the python packages with the usual try (`mach help`)/fail/install/repeat. Some packages (starting with `moz`) were installed from the `../../../gecko/python/`. Others from the PyPI registry. Eventually, I got the output below: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# mach mochitest browser/components/downloads/test/browser/browser_pdfjs_preview.js /builds/worker/workspace/build/venv/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) Error running mach: ['mochitest', 'browser/components/downloads/test/browser/browser_pdfjs_preview.js'] The error occurred in mach itself. This is likely a bug in mach itself or a fundamental problem with a loaded module. You can invoke |./mach busted| to check if this issue is already on file. If it isn't, please use |./mach busted file general| to report it. If |./mach busted| is misbehaving, you can also inspect the dependencies of bug 1543241. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: TypeError: expected str, bytes or os.PathLike object, not NoneType File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 399, in run return self._run(argv) File "/builds/worker/workspace/build/tests/tools/mach/mach/main.py", line 433, in _run topsrcdir = Path(self.populate_context_handler("topdir")) File "/usr/lib/python3.6/pathlib.py", line 1001, in __new__ self = cls._from_parts(args, init=False) File "/usr/lib/python3.6/pathlib.py", line 656, in _from_parts drv, root, parts = self._parse_args(args) File "/usr/lib/python3.6/pathlib.py", line 640, in _parse_args a = os.fspath(a) ``` At least the dependencies were resolved... Fortunately, this error was related to some error reporting (sentry) and we could use a [`NoopErrorReporter`](https://searchfox.org/mozilla-central/rev/9b8ebf06145feeccf34dc126cf45b07e86556392/python/mach/mach/main.py#436) instead. ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# mach help /builds/worker/workspace/build/venv/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) usage: mach [global arguments] command [command arguments] Global Arguments: -v, --verbose Print verbose output. -l FILENAME, --log-file FILENAME Filename to write log data to. --log-interval Prefix log line with interval from last message rather than relative time. Note that this is NOT execution time if there are parallel operations. --no-interactive Automatically selects the default option on any interactive prompts. If the output is not a terminal, then --no-interactive is assumed. --log-no-times Do not prefix log lines with times. By default, mach will prefix each output line with the time since command start. -h, --help Show this help message. --debug-command Start a Python debugger when command is dispatched. --profile-command Capture a Python profile of the mach process as command is dispatched. --settings FILENAME Path to settings file. Testing: Run tests. geckoview-junit Run the geckoview-junit harness. marionette-test Run a Marionette test (Check UI or the internal JavaScript using marionette). mochitest Run the mochitest harness. ``` I then tried to run `mach mochitest` and it failed, ugh. A wrong path somewhere that I fixed with a symlink: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# ln -s /builds/worker/gecko/ /builds/worker/checkouts/gecko ``` Then, a missing `.mozconfig` file: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# touch ~/gecko/.mozconfig ``` Then, a missing shell script when I installed `mozbuild` manually: ``` (venv) root@7bb12ab96ace:~/workspace/build/tests# ln -s /builds/worker/gecko/python/mozbuild/mozbuild/mozconfig_loader ../venv/lib/python3.6/site-packages/mozbuild/ ``` Things still didn't work, likely because Bug 1636251 changed a few things around `mach`'s context, especially the `populate_context()` signature: ```diff - def populate_context(context, key=None): + def populate_context(key=None): ``` I butchered some more python files to successfully run the mochitest framework but I couldn't run the test file I was interested in... (it ran nothing because it didn't find any tests to run or something like that). ``` [...] 0:00.09 INFO Checking for xpcshell processes... 0:00.39 ERROR No tests were found for flavor 'plain' and the following manifest filters: skip_if, run_if, fail_if, subsuite(name=None), pathprefix(['browser/browser/components/downloads/test/browser/browser_pdfjs_preview.js']) Make sure the test paths (if any) are spelt correctly and the corresponding --flavor and --subsuite are being used. See `mach mochitest --help` for a list of valid flavors. 0:00.39 SUITE_START: mochitest-plain - running 0 tests 0 INFO TEST-START | Shutdown 1 INFO Passed: 0 2 INFO Failed: 0 3 INFO Todo: 0 4 INFO Mode: e10s 5 INFO SimpleTest FINISHED 0:00.39 INFO Buffered messages finished 0:00.40 SUITE_END mochitest-plain ~~~~~~~~~~~~~~~ Ran 0 checks () Expected results: 0 Unexpected results: 0 OK ``` :ahal, any ideas? Maybe mochitests aren't (or are no longer) supported in interactive tasks?