910837 - display graphics information from about:support in talos

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

many times we wonder what type of graphics card we are using, or what preferences are enabled/disabled.  This can be determined by looking at the graphics adapter section in about:support.

I believe this should be in the getInfo.html section of talos.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

here is what I can do pretty easily:
	Screen width/height:1366/768
	colorDepth:24
	Browser inner width/height: 1024/626


Graphics Adapter: {"numTotalWindows":1,"numAcceleratedWindows":0,"windowLayerManagerType":"Basic","windowLayerManagerRemote":false,"numAcceleratedWindowsMessage":[""],"adapterDescription":"Intel Open Source Technology Center -- Mesa DRI Intel(R) Ivybridge Mobile ","adapterVendorID":"Intel Open Source Technology Center","adapterDeviceID":"Mesa DRI Intel(R) Ivybridge Mobile ","adapterRAM":"","adapterDrivers":"","driverVersion":"3.0 Mesa 9.1.3","driverDate":"","webglRenderer":"Intel Open Source Technology Center -- Mesa DRI Intel(R) Ivybridge Mobile ","info":{"AzureCanvasBackend":"cairo","AzureSkiaAccelerated":0,"AzureFallbackCanvasBackend":"none","AzureContentBackend":"none"}}

This would show up in the logs, is this useful, should I graph more?  format it differently?

Comment 2

[Mike Conley (:mconley) (:⚙️)]

If you use JSON.parse(obj, null, " "), it formats it like this:

{
 "numTotalWindows": 1,
 "numAcceleratedWindows": 0,
 "windowLayerManagerType": "Basic",
 "windowLayerManagerRemote": false,
 "numAcceleratedWindowsMessage": [
  ""
 ],
 "adapterDescription": "IntelOpen Source Technology Center -- Mesa DRI Intel(R) Ivybridge Mobile",
 "adapterVendorID": "Intel Open Source TechnologyCenter",
 "adapterDeviceID": "Mesa DRI Intel(R) Ivybridge Mobile",
 "adapterRAM": "",
 "adapterDrivers": "",
 "driverVersion": "3.0 Mesa9.1.3",
 "driverDate": "",
 "webglRenderer": "Intel Open Source Technology Center --Mesa DRI Intel(R) Ivybridge Mobile",
 "info": {
  "AzureCanvasBackend": "cairo",
  "AzureSkiaAccelerated": 0,
  "AzureFallbackCanvasBackend": "none",
  "AzureContentBackend": "none"
 }
}

Which is a little more readable.

Comment 3

[Mike Conley (:mconley) (:⚙️)]

Sorry, my bad - that should be:

JSON.stringify(obj, null, " ")

Where obj is the item to be serialized. This returns a string.

Comment 4

[Matthew N. [:MattN]]

Thanks for doing this. I had this idea myself and was going to implement but this is even better :)

I think we should wait for HWA to be enabled (after some delay) before reading the values or the number of accelerated windows won't match value during the test runs.

Comment 5

[Avi Halachmi (:avih)]

Another option if we must wait (I don't know if we do), is to read this value just before talos closes the browser.

bas, can we open about:support right after the browser opens and expect d2d/d3d values there to be the same as after 10 seconds?

I'm asking wrt to me recalling that LayerManager replaces the software LM with HWA one after about 5s from startup.

Comment 6

[Bas Schouten (:bas.schouten)]

(In reply to Avi Halachmi (:avih) from comment #5)
> Another option if we must wait (I don't know if we do), is to read this
> value just before talos closes the browser.
> 
> bas, can we open about:support right after the browser opens and expect
> d2d/d3d values there to be the same as after 10 seconds?
> 
> I'm asking wrt to me recalling that LayerManager replaces the software LM
> with HWA one after about 5s from startup.

In the case of D3D9, no. I believe about:support will report Basic in the first 5s and D3D9 after those 5s in the case of a system where D2D/D3D10 is blacklisted bug D3D9 is not.

Comment 7

[Matthew N. [:MattN]]

Is there a notification or some way to detect that this changed? Or what is the magic amount of time to wait for this? Is it safe to rely on waiting 5s? 6s? 7s?

Comment 8

[Bas Schouten (:bas.schouten)]

I'd argue 10 seconds is a safe bet. We currently have the timer set at 5s but I wouldn't go too close to that.

Comment 9

[Avi Halachmi (:avih)]

I believe the experimental patch which joel is working on waits 10s when the talos run starts before it gathers this info, and then proceeds normally.

Since it seems that talos runs already have some wait before the tests themselves start running (or at least the pageloader tests do), then with the patch it now waits 10s, gets the gfx info, then the browser opens/pageloader-start-running, waits a bit more, then starts to actually test.

I think we should unify those delays into a single 10s one, after which we gather the gfx info and right after that the actual test should start without further wait.

This also makes me think about the other talos tests which are possibly affected by this switch from SW to HWA layer manager. The effect could be either:

1. Different results on the runs before/after the switch (mostly where gfx stuff is tested/used).

2. If this switch takes non-negligible duration, then the specific test during which the switch happens will probably show worse result than the rest.

All that being said, I think that the wait which happens also before this gfx-info patch is long enough to not fall into this switch trap. I can't tell if it's 10s, but it's most probably more than 5. So I don't think we actually suffer from this switch, but we should still remember that it's there.

Overall however, a single wait of 10s before the gfx info and the test runs would cover us properly on this.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

should we look at making the default for pageloader tests launch the browser, wait 10 seconds, then start the tests?  Maybe we are running tests prior to the gfx layer being fully initialized?

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: initial patch to dump this out

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

I would like to make a decision on this and get it landed.  There is a lot of value here.

I see these options:
1) leave this in getInfo.html with the 10second delay.  getInfo.html is called to initialize the fresh profile before the test is run, this would incur a 10 second delay on all talos tests.  For reference, after getInfo.html is loaded, it will shutdown the browser and then we relaunch the browser with the test.

2) add this to the start of the test.  For pageloader tests, this would delay the start of the test (we wouldn't loadURI) for 10 seconds until we get the graphics information.  This again would incur a 10 second cost per entire test.  The advantage here is that the tests we are running will be guaranteed to have the graphics initialized.

2b) for startup tests we are measuring startup time, we would have to not print out graphics information or go with option 1.  the startup tests are ts_paint, tspaint_places_generated_[med|max], tpaint, tresize.

Please weigh in on what makes the most sense.  I am open to other options if they exist.

Comment 13

[Benoit Girard (:BenWa)]

I hate the idea of slowing down a test by 10 seconds. We need to make our test faster and this is going to be annoying to fix so no one will ever do it.

How about printing this information before shutdowning down the process? We'd have to make sure we have been running for 10 seconds but that's already much better. We could also add a way to async query this information and make it either return after we've activated accelerated graphics or give us a better way to query the information before we've activated them.

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

printing this at the end is very valid.  There is a slight concern that if we are testing before the accelerated graphics are activated then our data could be invalid/skewed without us knowing it.  If this isn't a concern, we could just dump this at the beginning and not care about it.

Comment 15

[Benoit Girard (:BenWa)]

Right, perhaps we need to turn off the layers switch for things like tp5 but continue to measure them for startup tests.

Comment 16

[Joel Maher ( :jmaher ) (UTC -8)]

how do you turn off the layers switch?  I am unfamiliar with this concept and what it means.

Comment 17

[Benoit Girard (:BenWa)]

I'm suggesting we introduce a preference that makes it so we startup directly with the graphics settings we want to use instead of using the basic code paths on startup that are used to speed up the browser load. For tests like tp5 where we don't measure the time it takes to startup we don't have to wait 10 seconds for the graphics engine to stabilize. Of course we can't use this for tests where we measure startup time.

Actually never mind since we have to wait for other things to stabilize like GC and delayed loading so I don't think it's avoidable.

Comment 18

[Joel Maher ( :jmaher ) (UTC -8)]

final summary?

for startup tests we continue to do what we have been doing
for pageload tests, we delay 10 seconds, dump graphics information, then do what we have always done?

Comment 19

[Benoit Girard (:BenWa)]

Ignore what I've said. Let's dump the graphics information at the end. It should represent best what we ran with.

Comment 20

[Avi Halachmi (:avih)]

(In reply to Benoit Girard (:BenWa) from comment #19)
> Ignore what I've said. Let's dump the graphics information at the end. It
> should represent best what we ran with.

See comment 9. I still think it would be beneficial to start the test after the switch.

Comment 21

[Benoit Girard (:BenWa)]

Yes, I agree with comment 9. I guess there shouldn't be much different. Waiting about 10 seconds before running tests is good because we defer some tasks on startup. 10 seconds might not be entirely sufficient but we can catch those later with talos profiling with SPS.

I'm not really set on dumping the data before or after the test. Dumping it before the run will point out if we starting running the test too early (are still with Basic Layers) but dumping it at the end will dump the layers backend that was used for most of the tests. I think either is fine.

Comment 22

[Avi Halachmi (:avih)]

(In reply to Benoit Girard (:BenWa) from comment #21)
> I'm not really set on dumping the data before or after the test. Dumping it
> before the run will point out if we starting running the test too early (are
> still with Basic Layers) but dumping it at the end will dump the layers
> backend that was used for most of the tests. I think either is fine.

Yup, but assuming switching layer manager WILL have an undesirable effect on stability/noise of the results, we should know good enough that we don't expect any surprises anyway.

So while getting the info at the end is less likely to get incorrect info, putting it before the test will force us to know that we're after the switch.

I suggest to have some clear indication which talos can poll (a pref) or be notified of that we're past init and can start the test.

Also, this LM switch happens on windows. What about other platforms? do we also need some wait period? how much?

Comment 23

[Benoit Girard (:BenWa)]

We do it on Mac and I imagine perhaps Linux when we turn it on. We don't do it on mobile where we DON'T support basic.

Comment 24

[Joel Maher ( :jmaher ) (UTC -8)]

I have been working on this unsuccesfully.  Pageloader (the addon that loads pages for us via loadURI) seems to have a problem with a setTimeout(<func>, 10000) call.  I end up getting browser crashes on any value >5000.  So dumping the data prior to the run is not happening.  

Is there a way to listen for an event that we are initialized fully?  maybe a pref?  If I do this in getInfo.html (the profile warmup run), when we launch the browser for the test will it already have the graphics information?  Otherwise, the best solution given what I know now would be dump the graphics information at the end.

Comment 25

[Benoit Girard (:BenWa)]

Sounds like a serious problem. Let's fixed that and maybe block on that.

Comment 26

[Joel Maher ( :jmaher ) (UTC -8)]

I am open for suggestions on how to work around this or debug it more.

Comment 27

[Joel Maher ( :jmaher ) (UTC -8)]

ok, lets revisit this bug, this is overall good information to have.

it seems that we have a 10 second delay when starting firefox due to the memory initializer for pageloader tests:
https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/pageloader/chrome/memory.js?q=path%3Atesting%2Ftalos%2Ftalos%2Fpageloader%2Fchrome%2Fmemory.js&redirect_type=single#40

given that, I believe we could print out the graphics information after memory initialization is completed.

for startup tests, we can do it in the initialization as normal- since these are the same machines, it will be interesting to compare the dumped information on a ts_paint vs a tp5o run.

Comment 28

[Joel Maher ( :jmaher ) (UTC -8)]

we dump out about:support now