Closed Bug 1331129 Opened 7 years ago Closed 7 years ago

get mainthreadio working in talos again, and clean up old entries in the mtio_whitelist.json

Categories

(Testing :: Talos, defect)

defect
Not set
normal

Tracking

(firefox57 fixed)

RESOLVED FIXED
mozilla57
Tracking Status
firefox57 --- fixed

People

(Reporter: jmaher, Assigned: igoldan)

References

Details

(Whiteboard: [PI:September])

Attachments

(2 files, 1 obsolete file)

it appears that mainthreadIO detection is not working in talos anymore.  I am not sure why, but I had to hack around a bit to get it to report:
https://hg.mozilla.org/try/rev/e0a5ba4a43c0dd64ff0d6684446b2675c1f8ddfb

figuring out why this isn't working (possibly a bug and it is turned off?, possibly a bug in talos on accident?) and paving a path to turning this on would be excellent.

In addition there is a mtio_whitelist.json file that we should attempt to reduce.  this should be tested on try with opt and pgo builds as pgo appears to do things differently with fileIO.
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #8828421 - Flags: review?(rwood)
Comment on attachment 8828421 [details] [diff] [review]
enable mainthreadio and cleanup whitelist

LGTM
Attachment #8828421 - Flags: review?(rwood) → review+
apologies for requesting another review- I have done a lot of testing and there are some intermittents here, in this case I wanted to include as many of them as possible- we should be good on opt and pgo- although I do suspect we will have some intermittents and I can keep an eye on them.
Attachment #8828421 - Attachment is obsolete: true
Attachment #8829206 - Flags: review?(rwood)
Comment on attachment 8829206 [details] [diff] [review]
enable mainthreadio and cleanup whitelist (2.0)

LGTM
Attachment #8829206 - Flags: review?(rwood) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/a82768d2e585
get mainthreadio working in talos again, and clean up old entries in the mtio_whitelist.json. r=rwood
Backed out for unexpected access to xperf in pageloader@mozilla.org.xpi:

https://hg.mozilla.org/integration/mozilla-inbound/rev/6b60555ed1be7100f435023e9adea50a7934fd68

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=18c503e50ae50020440c1d83feb32efa69ea99d8
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=71233333&repo=mozilla-inbound

TEST-UNEXPECTED-FAIL | mainthreadio | File '{profile}\extensions\pageloader@mozilla.org.xpi' was accessed and we were not expecting it: {'Count': 54, 'Duration': 1.8178559999999997, 'RunCount': 54}

TEST-UNEXPECTED-FAIL | mainthreadio | File '{profile}\extensions\talos-powers@mozilla.org.xpi' was accessed and we were not expecting it: {'Count': 54, 'Duration': 1.964928, 'RunCount': 54}
Flags: needinfo?(jmaher)
Flags: needinfo?(jmaher)
Whiteboard: [PI:June]
Flags: needinfo?(ionut.goldan)
Whiteboard: [PI:June] → [PI:July]
Assignee: jmaher → ionut.goldan
if you look in the original patch:
https://bugzilla.mozilla.org/page.cgi?id=splinter.html&bug=1331129&attachment=8829206

you can see the changes to config.py, that is needed to force reporting here- I recommend pushing to try to see what happens to xperf with that one line removed.
:igoldan, this is an area of code and data which can be confusing, don't hesitate to ask questions!
Whiteboard: [PI:July] → [PI:August]
Whiteboard: [PI:August] → [PI:September]
Comment on attachment 8905412 [details]
Bug 1331129 - address unexpected file accesses

https://reviewboard.mozilla.org/r/177224/#review182246

one comment to address- also do you have a try push- for this I want to see 10 retriggers on win7/xperf for both opt and pgo (hint try using |mach try fuzzy|)

::: testing/talos/talos/mainthreadio.py:45
(Diff revision 1)
>                        'thumbnails': '{thumbnails}',
> -                      'windows media player': '{media_player}'}
> +                      # {appdata}\locallow\mozilla\temp-{*}
> +                      'temp-{': '{temp}',
> +                      'cltbld.': '{cltbld}',
> +                      'windows media player': '{media_player}',
> +                      re.compile(r'{\w{8}-\w{4}-\w{4}-\w{4}-\w{12}}'): '{uuid}'}

I do not see you using the new {temp}, {cltbld}, etc. 

did you mean to do that, I think the list of files in the whitelist.txt is easy enough to follow
Attachment #8905412 - Flags: review?(jmaher) → review-
Comment on attachment 8905412 [details]
Bug 1331129 - address unexpected file accesses

will wait on a new patch with mainthreadIO enabled fully.
Attachment #8905412 - Flags: review?(jmaher)
Comment on attachment 8905412 [details]
Bug 1331129 - address unexpected file accesses

https://reviewboard.mozilla.org/r/177224/#review182792

lets get the file access in, but not turn this on until investigating the intermittents
Attachment #8905412 - Flags: review-
Comment on attachment 8905412 [details]
Bug 1331129 - address unexpected file accesses

https://reviewboard.mozilla.org/r/177224/#review182808
Attachment #8905412 - Flags: review?(jmaher) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/41f350766d7d
address unexpected file accesses r=jmaher
https://hg.mozilla.org/mozilla-central/rev/41f350766d7d
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
:igoldan, can you file a new bug to investigate the intermittent and turn on mainthreadIO- we have the main file access sorted out here now.
Flags: needinfo?(ionut.goldan)
See Also: → 1399024
You need to log in before you can comment on or make changes to this bug.