Closed Bug 1644437 Opened 5 years ago Closed 5 years ago

add --test-iterations

Categories

(Testing :: mozperftest, enhancement, P1)

Product:
Testing
Component:
mozperftest
Type:
enhancement

Tracking

(firefox79 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla79
Tracking Flags:
Tracking Status
firefox79 --- fixed

People

(Reporter: tarek, Assigned: tarek)

Details

Attachments

(1 file)

Bug 1644437 - add --test-iterations
47 bytes, text/x-phabricator-request
Details

add a new option to allow running the whole test several times. a whole environment is re-created each time

this also adds the before_iterations hook that runs before all iterations,
unlike before_runs, that runs at the beginning of each iteration

Adds a new option to allow running the whole test several times,
and a before_iterations hook

Backed out 3 changesets (bug 1644706, bug 1644437, bug 1644368) for python related bustage

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=aopnPC0nQAC-wMdY45rtyQ-0&searchStr=build&fromchange=da95a35fb07571d59e9346cab07c4ca8a56c3fde&tochange=1a5b8a1f2abc54eddaf4ef9e9caf3c77397b7af6

Backout link: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedTaskRun=aopnPC0nQAC-wMdY45rtyQ-0&searchStr=build&fromchange=da95a35fb07571d59e9346cab07c4ca8a56c3fde&tochange=1a5b8a1f2abc54eddaf4ef9e9caf3c77397b7af6

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=305859506&repo=autoland&lineNumber=1635

...
[task 2020-06-10T22:00:19.143Z] 22:00:19     INFO -  js/src> updating cache /builds/worker/workspace/obj-build/./config.cache
[task 2020-06-10T22:00:19.144Z] 22:00:19     INFO -  js/src> creating ./config.data
[task 2020-06-10T22:00:19.154Z] 22:00:19     INFO -  js/src> Creating config.status
[task 2020-06-10T22:00:19.185Z] 22:00:19     INFO -  Creating config.status
[task 2020-06-10T22:00:19.343Z] 22:00:19     INFO -  Reticulating splines...
[task 2020-06-10T22:00:19.379Z] 22:00:19    ERROR -  Traceback (most recent call last):
[task 2020-06-10T22:00:19.379Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/emitter.py", line 1567, in _process_test_manifest
[task 2020-06-10T22:00:19.379Z] 22:00:19     INFO -      path, ', '.join(missing)), context)
[task 2020-06-10T22:00:19.379Z] 22:00:19     INFO -  mozbuild.frontend.reader.SandboxValidationError:
[task 2020-06-10T22:00:19.380Z] 22:00:19     INFO -  ==============================
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -  FATAL ERROR PROCESSING MOZBUILD FILE
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -  ==============================
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -  The error occurred while processing the following file or one of the files it includes:
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      /builds/worker/checkouts/gecko/python/moz.build
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -  The error occurred when validating the result of the execution. The reported error is:
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      Test manifest (/builds/worker/checkouts/gecko/python/mozperftest/mozperftest/tests/python.ini) lists test that does not exist: test_scriptinfo.py
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -  During handling of the above exception, another exception occurred:
[task 2020-06-10T22:00:19.382Z] 22:00:19    ERROR -  Traceback (most recent call last):
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/configure.py", line 182, in <module>
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      sys.exit(main(sys.argv))
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/configure.py", line 58, in main
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      return config_status(config)
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/configure.py", line 177, in config_status
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      return config_status(args=[], **normalize(sanitized_config))
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/config_status.py", line 145, in config_status
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      definitions = list(definitions)
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/emitter.py", line 181, in emit
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      objs = list(emitfn(out))
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/emitter.py", line 1383, in emit_from_context
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      for obj in self._process_test_manifests(context):
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/emitter.py", line 1530, in _process_test_manifests
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      for obj in self._process_test_manifest(context, info, path, manifest):
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -    File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/emitter.py", line 1635, in _process_test_manifest
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -      context)
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -  mozbuild.frontend.reader.SandboxValidationError:
[task 2020-06-10T22:00:19.382Z] 22:00:19     INFO -  ==============================
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -  FATAL ERROR PROCESSING MOZBUILD FILE
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -  ==============================
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -  The error occurred while processing the following file or one of the files it includes:
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -      /builds/worker/checkouts/gecko/python/moz.build
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -  The error occurred when validating the result of the execution. The reported error is:
[task 2020-06-10T22:00:19.383Z] 22:00:19    ERROR -      Error processing test manifest file /builds/worker/checkouts/gecko/python/mozperftest/mozperftest/tests/python.ini: Traceback (most recent call last):
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -        File "/builds/worker/checkouts/gecko/python/mozbuild/mozbuild/frontend/emitter.py", line 1567, in _process_test_manifest
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -          path, ', '.join(missing)), context)
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -      mozbuild.frontend.reader.SandboxValidationError:
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -      ==============================
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -      FATAL ERROR PROCESSING MOZBUILD FILE
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -      ==============================
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -      The error occurred while processing the following file or one of the files it includes:
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -          /builds/worker/checkouts/gecko/python/moz.build
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -      The error occurred when validating the result of the execution. The reported error is:
[task 2020-06-10T22:00:19.383Z] 22:00:19     INFO -          Test manifest (/builds/worker/checkouts/gecko/python/mozperftest/mozperftest/tests/python.ini) lists test that does not exist: test_scriptinfo.py
[task 2020-06-10T22:00:19.489Z] 22:00:19     INFO -  *** Fix above errors and then restart with\
[task 2020-06-10T22:00:19.489Z] 22:00:19     INFO -                 "./mach build"
[task 2020-06-10T22:00:19.489Z] 22:00:19     INFO -  make: *** [client.mk:115: configure] Error 1
[task 2020-06-10T22:00:19.550Z] 22:00:19    ERROR - Return code: 2
[task 2020-06-10T22:00:19.550Z] 22:00:19  WARNING - setting return code to 2
[task 2020-06-10T22:00:19.550Z] 22:00:19    FATAL - 'mach build -v' did not run successfully. Please check log for errors.
[task 2020-06-10T22:00:19.550Z] 22:00:19    FATAL - Running post_fatal callback...
[task 2020-06-10T22:00:19.550Z] 22:00:19    FATAL - Exiting -1
[task 2020-06-10T22:00:19.550Z] 22:00:19     INFO - [mozharness: 2020-06-10 22:00:19.550225Z] Finished build step (failed)
Flags: needinfo?(tarek)

regression from bug 1644368 (stack push)

Flags: needinfo?(tarek)

https://hg.mozilla.org/mozilla-central/rev/39003988bd7b

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox79: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla79
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: