796068 - Unzip reader mode pageset in objdir/_test/testing/mochitests/tests/rococop/reader_mode_sites

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Lucas Rocha (:lucasr)]

The rococop tests for reader mode assume the pageset is unzipped under objdir/_test/testing/mochitests/tests/rococop/reader_mode_sites (or whatever equivalent directory in the test infra).

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

we should be able to treat this as a pageset or other tools zip file.  

:lucasr, can you post a link to a place where we can get the .zip file?
releng, can we upload that file to the build network and then work on patches to use that for robocop tests?

Comment 2

[Lucas Rocha (:lucasr)]

I posted the zip on bug 786638 here:
  https://bugzilla.mozilla.org/attachment.cgi?id=665918

Comment 3

[Lucas Rocha (:lucasr)]

Ping. Any news?

Comment 4

[Aki Sasaki (not active)]

This is only Android?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

yes, android only.

Comment 6

[Justin Wood (:Callek)]

(In reply to Lucas Rocha (:lucasr) from comment #3)
> Ping. Any news?

Sorry this fell off my radar entirely (somehow missed it in my bugmail)

So to recap, what is needed [and by who]:

* [releng] put the reader-mode zip of pagetests somewhere.
* [ateam] patch automation code to use said reader-mode zip of pagetests, as downloaded from our infra and unpacked.

Does that ToDo list [and who is responsible] look accurate?

Open Questions:

* What [build network] machine do we host this on?
* Are the pagetests DO-NOT-DISTRIBUTE just like our tp4/5 sets?
* Do we expect to host them as a zip file that we actually *do* unpack, or should it be hosted in its unpacked form
* How often do we expect said pagesets to be updated
* When we update the pagesets is it an update across-all-branches, or would we be having any pageset updates ride a train
* When we update the pagesets would this entail an entirely new graphserver entry, and run with both sets in parallel on the same branch for a while

I can give ETA's once we have at least some of these open question answers solved.

Comment 7

[Justin Wood (:Callek)]

--> taking the bug so we don't lose it, but not actually doing work until we know some of these answers.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

* we need to host this on a build network.
* TODO: I don't know about the pagesets, I believe they are private.
* It will need to be unpacked (like the tp5 pageset)
* this pageset won't be updated very often
* pagesets would follow the train if they are updated, they will be test specific
* if we update the pageset it would mean new pages and new graph server entries.  Since this is rare, we will probably be on datazilla before we need to update.

Comment 9

[Justin Wood (:Callek)]

(In reply to Joel Maher (:jmaher) from comment #8)
> * we need to host this on a build network.

Now At:
http://build.mozilla.org/talos/zips/tp_reader.zip [renaming this or using a new pageset before this goes live should be trivial]
(For docs, I used basically the same steps as https://wiki.mozilla.org/ReleaseEngineering:Buildduty#How_to_update_the_talos_zips except I used tp_reader.zip based on the attachment mentioned in this bug)

> * TODO: I don't know about the pagesets, I believe they are private.

I'm pretending they are best hosted private for now, let me know if you have ideas on a more-public place to host them.

> * It will need to be unpacked (like the tp5 pageset)

I'm assuming you do mean like [what uses] (http://mxr.mozilla.org/build/source/buildbot-configs/mozilla-tests/config.py#31) and not like tp4m (http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#5720) The latter would need me to know a test name.

> * this pageset won't be updated very often
> * pagesets would follow the train if they are updated, they will be test
> specific
> * if we update the pageset it would mean new pages and new graph server
> entries.  Since this is rare, we will probably be on datazilla before we
> need to update.

SGTM and like we can treat this as a normal test.

Joel, what is our next step here, who is on point - etc?

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

next steps are:
* host the pageset on the build network
* I would prefer the (http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#5720) method as this will only be for a few select tests and it won't be talos specific.
* tests would be placed in tests.zip:/mochitests/tests/rococop/reader_mode_sites/

Once this is hosted and the buildbot changes are in effect for mochitest robocop, then we can turn on the tests in mochitest robocop.

Once that is successful, we can resolve this for talos.

Comment 11

[Lucas Rocha (:lucasr)]

Any update here?

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

I guess I was assuming releng would take the 3 steps outlined in comment #10.  Callek, how do you want to split this up, I would need you guys to do some of it as I can't upload a file.

Comment 13

[Justin Wood (:Callek)]

(In reply to Joel Maher (:jmaher) from comment #10)
> next steps are:
> * host the pageset on the build network

Been done for a while:
http://build.mozilla.org/talos/zips/tp_reader.zip

> * tests would be placed in
> tests.zip:/mochitests/tests/rococop/reader_mode_sites/

If they are not public we can't place them in the zip itself, and must download on the test machine. If they are ok to be public we should check them into tree and have the build process add them to tests.zip (*remember* the tests.zip is publicly uploaded)

...

What is still not clear to me (and I take teh fault for not yet seeking clarity) is where *during a test run* should this be downloaded, where *in the test run/directory structure* should it extract to, and what class of tests is it to be used in (inclusive) e.g. mochi/talos. and a confirmation of *how* it is invoked.

Comment 14

[Justin Wood (:Callek)]

This hasn't had traction for over a year, and has had my outstanding confusion left unresolved. I'm unassigning from myself for now, let me/releng know if this has newfound priority

Comment 15

[:Margaret Leibovic]

We decided to go with a different approach in bug 786638 and just land our testcases in the tree, so I don't think we need to pursue this bug anymore.

Comment 16

[Justin Dolske [:Dolske]]

I will boldly WONTFIX this based on last comment, and that we're generally talking about testing Reader View in a different way now.