1470450 - Add a debug mode to raptor

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement)

Description

[Robert Wood [:rwood]]

If there's some kind of web extension error (i.e. code syntax, or a webext API has broken) raptor will just timeout awaiting the pageload, and there are no error messages in the raptor console/production log.

We need a way to capture js errors in the webext itself (i.e. webext errors that are dumped to the js web console) and dump them to the terminal/production logs.

Comment 1

[Robert Wood [:rwood]]

We definitely need this, without it, to debug the raptor webext you need to:

- hack the control_server code to not shutdown the browser
- hack the raptor webext runner code to not close the browser tab

Then run raptor - during the 30 second startup pause, go to about:debugging, turn on webext debugging, and choose to debug the Raptor webext (window pops up, choose console). That will display console output for the main raptor runner.js

Then when the pageload test starts (a new tab is opened and page is loaded), open the web console for that page (that will display console output from the raptor measure.js content) 

We need a debug mode that captures the console from runner.js and measure.js and dumps to terminal.

Comment 2

[Robert Wood [:rwood]]

Note, when running raptor on android geckoview the webext console.log statements already appear in the adb logcat; as :bc noted on irc we should pull those into the main logging (not sure how).

When running on Desktop not sure how to get the web ext console.log statements or if that's possible.

Comment 3

[Robert Wood [:rwood]]

I don't see a way to do this. Although if we add a debug mode, we can startup Firefox with the browser console open, that will display the console.log messages there from the raptor web extension.

Use this bug to implement a debug mode for raptor that will at least:

- get rid of the 30 second browser init pause
- set pagecycles to 1
- open browser console on launch

Comment 4

[Robert Wood [:rwood]]

One way to get console.log messages out of the raptor webext and into the terminal/logs:

Inside the raptor webext js, instead of console.log, use a wrapper function that:

- accepts the string to log;
- dumps the string to console.log;
- adds the string to a growing list of strings already logged

Then in the webext cleanup (or at a timeout) send the list of console strings to the raptor control server; and have the raptor control server dunp it to terminal. Something like that may work.

Comment 5

[Robert Wood [:rwood]]

Sweet, starting Firefox with '-jsconsole' opens the browser console; and all the console.log messages from both the main raptor runner as well as the measure.js content are printed there. No need to capture the console.log errors for the control server/terminal.

Comment 6

[Robert Wood [:rwood]]

Attachment: Bug 1470450 - Add a debug mode to raptor; r?jmaher

Comment 7

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d23fbdf5f3adaf11ac01c0cf31f51ebcfbf8740f

Comment 8

[Robert Wood [:rwood]]

I also added a section about debug-mode to the Raptor wiki:

https://wiki.mozilla.org/Performance_sheriffing/Raptor#Raptor_Debug_Mode

Comment 9

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a14d6098037134dc27c49bc902261fc3c3d3a385

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8fb8a2697446
Add a debug mode to raptor; r=jmaher

Comment 11

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/8fb8a2697446