Closed Bug 1158276 Opened 5 years ago Closed 5 years ago

Autophone - adapt to the split tests implemented in bug 917999

Categories

(Testing :: Autophone, defect)

defect
Not set

Tracking

(firefox40 affected)

RESOLVED FIXED
Tracking Status
firefox40 --- affected

People

(Reporter: bc, Assigned: bc)

References

Details

Attachments

(2 files)

No description provided.
Bug 917999 is getting close. We already have what we should need to build in compatibility with the split test zip -- a json manifest (test_packages.json) is already being uploaded with each build. The file contains a mapping from test harness names to a list of file names in the build directory that the harness needs to run. I'm not sure which tests Autophone needs to run, but I'm planning to initially split out cppunittest, reftest, xpcshell, and mochitest.

One approach would be to download an unpack all the files that appear in the manifest (this will have the same end result as downloading an unpacking the monolithic testszip today).
Chris, do I understand correctly that for reftests I need to download common mozbase reftest and for mochitests I need to download common mozbase mochitest ?
Attachment #8621049 - Flags: review?(gbrown)
Attachment #8621049 - Flags: feedback?(cmanchester)
Blocks: 1173853
Comment on attachment 8621049 [details] [diff] [review]
bug-1158276-split-tests-configs.patch

Review of attachment 8621049 [details] [diff] [review]:
-----------------------------------------------------------------

Not exactly, sorry if I was unclear. The idea is to download test_packages.json, and look for your suite category in the keys (for instance, "reftest", or "mochitest"), and the corresponding value will be a list of packages to download.

For the initial implementation, a suite category will either have everything it needs in the "common" zip, or need the "common" zip and one other. http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/cmanchester@mozilla.com-dfeaa184de1c/try-linux/ is an example of the uploads that result when this is turned on. (It's possible, but I don't think likely, some of this will change before landing.)
Attachment #8621049 - Flags: feedback?(cmanchester)
This uses the test package names introduced in the test configs to determine which test zip files to download.

We store the test_packages.json in the build cache directory as well as the individual test zip files so we can determine later if they have been downloaded.

While testing the override-build-dir option and the use of the already downloaded test_packages.json to determine if the requested test packages had already been downloaded, I found an existing bug in buildserver.py:BuildCacheHandler that would not properly deal with an exception from the build_cache.get which would leave the server.cache_lock unreleased. 

I'm also taking advantage of this bug to fix a pyflakes warning I introduced earlier in PerfTest.publish_results.
Attachment #8621113 - Flags: review?(gbrown)
(In reply to Chris Manchester [:chmanchester] from comment #3)

Ok. A single category is sufficient. Thanks.

gbrown: consider the common and mozbase entries removed from the configs. I'll leave the ability to provide a list of category names though.
Comment on attachment 8621049 [details] [diff] [review]
bug-1158276-split-tests-configs.patch

Review of attachment 8621049 [details] [diff] [review]:
-----------------------------------------------------------------

s/common mozbase//
Attachment #8621049 - Flags: review?(gbrown) → review+
Comment on attachment 8621113 [details] [diff] [review]
bug-1158276-split-tests-download.patch

Review of attachment 8621113 [details] [diff] [review]:
-----------------------------------------------------------------

::: builds.py
@@ +581,5 @@
>              if enable_unittests and not os.path.exists(tests_path):
> +                raise BuildCacheException(
> +                    'Override Build Directory %s does not contain a tests directory.' %
> +                    self.override_build_dir)
> +            test_packages_json_path = os.path.join(self.override_build_dir,

it looks like test_packages_json_path could be set inside the following "if test_packages_name"...

@@ +689,4 @@
>                  # XXX: assumes fixed buildurl-> fennec_ids.txt mapping
>                  fennec_ids_url = urlparse.urljoin(buildurl, 'fennec_ids.txt')
>                  fennec_ids_path = os.path.join(cache_build_dir, 'fennec_ids.txt')
> +                if not os.path.exists(fennec_ids_path):

This reminds me, weren't you going to try removing use of fennec_ids.txt?
Attachment #8621113 - Flags: review?(gbrown) → review+
I remember something about it not being necessary anymore but are you sure it isn't used by robocop any more?
Oh right, fennec_ids.txt is still used by robocop.
(In reply to Geoff Brown [:gbrown] from comment #8)
> Comment on attachment 8621113 [details] [diff] [review]
> bug-1158276-split-tests-download.patch
> 

> it looks like test_packages_json_path could be set inside the following "if
> test_packages_name"...
> 

ok
Depends on: 1204288
You need to log in before you can comment on or make changes to this bug.