1269468 - "Copy as cURL" does not include request body

Status: RESOLVED FIXED

(DevTools :: Netmonitor, defect, P2)

Description

[Kyle Lahnakoski [:ekyle]]

Please have "Copy as cURL" include the body of a POST request, using the -d curl option.

Comment 1

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Yep, this could be useful. Note that you might want to use "Copy POST Data" together with "Copy as cURL" in the meantime.

Honza

Comment 2

[Vangelis Katsikaros]

I have the impression this is no longer relevant. With:
>Version 	55.0a1
>Build ID 	20170413080758
>User Agent 	Mozilla/5.0 (X11; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0
from 352751:819a666afddc central

Output of "Copy As cURL" on a POST request, note the --data:
```
curl 'https://www.adzuna.co.uk/jobs/create_user' -H 'Host: www.adzuna.co.uk' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://www.adzuna.co.uk/' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'X-Requested-With: XMLHttpRequest' -H 'Cookie: __utma=91298250.1120090921.1492237912.1492237912.1492237912.1; __utmb=91298250.3.9.1492237931420; __utmc=91298250; __utmz=91298250.1492237912.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmt=1; adzuna_session_start=1492237916125; adzuna_last_visited=jobs; _tq_id.TV-277227-1.5dfe=73c0930df401b82b.1492237918.0.1492237918..' -H 'Connection: keep-alive' --data 'email=vangelis%example.com&password=lalala'
```

For sanity checking, output of "Copy POST Data"
```
email=[e-mail]
password=lalala
```

Comment 3

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Thanks for testing this, Vangelis!

Yes works also on my machine, closing.

Honza

Comment 4

[Julien Wajsberg [:julienw]]

Hey Honza,

Actually it doesn't work for me on this website, so I reopen this bug.

STR:
1. Go to http://tag.ratp.fr/tag-paris-webapp/?t=2&id=2222
2. Open NetMonitor.
3. Press the refresh button ("double arrows") at the top right of the window, to refresh the data.
=> Notice the POST request to "infosArret"
4. Right click on that request, select "Copy as cURL"
5. Paste the request somewhere.

I get:

curl 'http://tag.ratp.fr/tag-paris-webapp/app/infosArret' -H 'Accept: */*' --compressed -H 'Accept-Language: fr,en-US;q=0.7,en;q=0.3' -H 'Cache-Control: max-age=0' -H 'Connection: keep-alive' -H 'Content-Type: text/x-gwt-rpc; charset=utf-8' -H 'Cookie: __cfduid=d545a3d7fd5f7303bb33ba6ce46745dff1517526992' -H 'DNT: 1' -H 'Host: tag.ratp.fr' -H 'Referer: http://tag.ratp.fr/tag-paris-webapp/?t=2&id=2222' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0' -H 'X-GWT-Module-Base: http://tag.ratp.fr/tag-paris-webapp/app/' -H 'X-GWT-Permutation: ADCEBA6A689A3BFD78EAE715302B1C10' --data ''

Notice the empty --data at the end.

Copying POST data works and I get:

7|0|6|http://tag.ratp.fr/tag-paris-webapp/app/|758D906AB66DD1326044BCDB0230B3D1|biz.ixxi.tagparis.webapp.client.services.InfosArretService|getNextPlaceNextStops|java.lang.String/2004016611|2222|1|2|3|4|1|5|6|

Comment 5

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Yes, I can reproduce the problem using STRs from comment #4

Thanks Julien for testing this!

Honza

Comment 6

[Jan Honza Odvarko [:Honza] (always need-info? me)]

A few comments for anyone interested in fixing this:

1) There is `copyAsCurl` command
https://searchfox.org/mozilla-central/rev/33cc9e0331da8d9ff3750f1e68d72d61201176cb/devtools/client/netmonitor/src/widgets/RequestListContextMenu.js#302

2) The command is built on top of `Curl` helper
https://searchfox.org/mozilla-central/rev/33cc9e0331da8d9ff3750f1e68d72d61201176cb/devtools/client/shared/curl.js#42

3) POST data part is generated here
https://searchfox.org/mozilla-central/rev/33cc9e0331da8d9ff3750f1e68d72d61201176cb/devtools/client/shared/curl.js#81
(see the data are passed into the `generateCommand` function

4) It looks like the POST data needs to be manually fetched from the backend. An example of how to do this is here:
https://searchfox.org/mozilla-central/rev/33cc9e0331da8d9ff3750f1e68d72d61201176cb/devtools/client/netmonitor/src/components/ParamsPanel.js#60

Let me know if you need more help!

Honza

Comment 7

[Tom Glowka]

Hey, I would like to fix this!

Comment 8

[Tom Glowka]

Attachment: Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

Review commit: https://reviewboard.mozilla.org/r/226320/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/226320/

Comment 9

[Tom Glowka]

Attachment: Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections

Review commit: https://reviewboard.mozilla.org/r/226334/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/226334/

Comment 10

[Tom Glowka]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/226320/diff/1-2/

Comment 11

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

https://reviewboard.mozilla.org/r/226320/#review233152

Thanks for working on this, the patch looks good!

Can you please yet update the existing test we have, so this feature doesn't regresss?

This is the test we already have:
browser_net_copy_as_curl.js

Honza

Comment 12

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8957395 [details]
Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections

https://reviewboard.mozilla.org/r/226334/#review233154

Comment 13

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=aac0aca13ccaa335238174b03d09dc7e00c19638&selectedJob=167657963

The existing test fails at the moment:
devtools/client/netmonitor/test/browser_net_curl-utils.js

Honza

Comment 14

[Tom Glowka]

Attachment: Bug 1269468 - onRequestPostData returns requestPostData not whole payload

For the sake of coherence with the rest of FirefoxDataProvider interface, to avoid future bugs like the one this commit refers to.

Review commit: https://reviewboard.mozilla.org/r/227864/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/227864/

Comment 15

[Tom Glowka]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/226320/diff/2-3/

Comment 16

[Code Review Bot [:reviewbot]]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

https://reviewboard.mozilla.org/r/226320/#review233698


Code analysis found 1 defect in this patch:
 - 1 defect found by mozlint

You can run this analysis locally with:
 - `./mach lint path/to/file` (JS/Python)


If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx


::: devtools/client/netmonitor/test/html_copy-as-curl.html:23
(Diff revision 3)
>        /* exported performRequest */
>        "use strict";
>  
> -      function performRequest(url) {
> +      function performRequest(url, payload) {
>          let xhr = new XMLHttpRequest();
> -        xhr.open("GET", url, true);
> +        xhr.open(payload ? "POST": "GET", url, true);

Error: Infix operators must be spaced. [eslint: space-infix-ops]

Comment 17

[Tom Glowka]

Attachment: Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections

Review commit: https://reviewboard.mozilla.org/r/227886/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/227886/

Comment 18

[Tom Glowka]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/226320/diff/3-4/

Comment 19

[Tom Glowka]

Comment on attachment 8959009 [details]
Bug 1269468 - onRequestPostData returns requestPostData not whole payload

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/227864/diff/1-2/

Comment 20

[Tom Glowka]

Hey!

Thanks for having an eye on tests, I underestimated a bit the consequences of that onRequestPostData change and needed to rewrite it (whole review here: https://reviewboard.mozilla.org/r/226318/). So first commit is just fixing obvious errors with test adjustment, second one is the little, related fix (no new changes).

Those two actually do the basic job for this bug, but I also added the third one, which is my suggestion of modifying onRequestPostData. Although I describe it below, nothing is more descriptive than a good diff, so commit up to your review on reviewboard as well.

General idea:
1. Return only requestPostData not payload (to pass requestHeadersFromUploadStream) to be consistent with other onRequest*/onReponse* methods and avoid future misuse
2. Set requestPostData.uploadHeaders during execution of requestData() as it is 100% derived from it other fields of requestPostData, to compensate change of onRequestPostData, where no kind of side-effect passing of payload.requestHeadersFromUploadStream.
3. And then using requestPostData.uploadHeaders set requestHeadersFromUploadStream during action.updateReuest() specifically in processNetworkUpdates(). Worth noting: requestHeadersFromUploadStream until now was reset in processNetworkUpdates and set again during exactly the same action, always.

One more that thing I haven't touched, but need some changes now or later. Two calls of actions.updateRequest():

https://searchfox.org/mozilla-central/source/devtools/client/netmonitor/src/connector/firefox-data-provider.js#134
This one works alright.

https://searchfox.org/mozilla-central/source/devtools/client/netmonitor/src/connector/firefox-data-provider.js#429
This one is after passing through onRequest*/onReponse* so payload actually is never payload but single field of payload that is just ignored as it is not in UPDATE_PROPS constant.
The other fact is that those two calls are really one almost instantly after another. I guess it's because the second one is dealing with "*Available" field, but then maybe they can still be merged into one call?

Comment 21

[Tom Glowka]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/226320/diff/4-5/

Comment 22

[Tom Glowka]

Comment on attachment 8959019 [details]
Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/227886/diff/1-2/

Comment 23

[Tom Glowka]

Comment on attachment 8959009 [details]
Bug 1269468 - onRequestPostData returns requestPostData not whole payload

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/227864/diff/2-3/

Comment 24

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Thank for the update!

I am seeing conflicts when applying the third patch.
Can you please rebase on top of m-c, thanks.

Honza


patching file devtools/client/netmonitor/src/connector/firefox-data-provider.js
Hunk #1 FAILED at 178
Hunk #2 FAILED at 534
2 out of 2 hunks FAILED -- saving rejects to file devtools/client/netmonitor/src/connector/firefox-data-provider.js.rej
patching file devtools/client/netmonitor/src/har/har-builder.js
Hunk #1 FAILED at 262
1 out of 1 hunks FAILED -- saving rejects to file devtools/client/netmonitor/src/har/har-builder.js.rej
patching file devtools/client/netmonitor/src/utils/request-utils.js
Hunk #1 FAILED at 435
Hunk #2 FAILED at 470
2 out of 2 hunks FAILED -- saving rejects to file devtools/client/netmonitor/src/utils/request-utils.js.rej
patching file devtools/client/netmonitor/src/widgets/RequestListContextMenu.js
Hunk #1 FAILED at 283
Hunk #2 FAILED at 301
2 out of 2 hunks FAILED -- saving rejects to file devtools/client/netmonitor/src/widgets/RequestListContextMenu.js.rej
patching file devtools/client/netmonitor/test/browser_net_curl-utils.js
Hunk #1 FAILED at 252
1 out of 1 hunks FAILED -- saving rejects to file devtools/client/netmonitor/test/browser_net_curl-utils.js.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working directory
errors during apply, please fix and qrefresh curl3

Comment 25

[Tom Glowka]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/226320/diff/5-6/

Comment 26

[Tom Glowka]

Comment on attachment 8959019 [details]
Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/227886/diff/2-3/

Comment 27

[Tom Glowka]

Comment on attachment 8959009 [details]
Bug 1269468 - onRequestPostData returns requestPostData not whole payload

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/227864/diff/3-4/

Comment 28

[Tom Glowka]

Rebased on m-c, I hope it will be fine now.

Comment 29

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

https://reviewboard.mozilla.org/r/226320/#review235082

Looks good, just one little inline comment.

R+, you don't have to ask for additional review as soon as the comment is resolved.

Thanks,
Honza

::: devtools/client/netmonitor/test/browser_net_copy_as_curl.js:79
(Diff revision 6)
> +
> +  async function performRequest(payload) {
> -  let wait = waitForNetworkEvents(monitor, 1);
> +    let wait = waitForNetworkEvents(monitor, 1);
> -  await ContentTask.spawn(tab.linkedBrowser, SIMPLE_SJS, async function(url) {
> -    content.wrappedJSObject.performRequest(url);
> +    await ContentTask.spawn(tab.linkedBrowser, {
> +      url: SIMPLE_SJS, payload_: payload
> +    }, function* ({url, payload_}) {

Please use async function not generator

Comment 30

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8959009 [details]
Bug 1269468 - onRequestPostData returns requestPostData not whole payload

https://reviewboard.mozilla.org/r/227864/#review235084

Looks reasonable
R+
Honza

::: devtools/client/netmonitor/src/connector/firefox-data-provider.js:190
(Diff revision 4)
>  
>        requestPostData.postData.text = postData;
> -      payload.requestPostData = Object.assign({}, requestPostData);
> -      payload.requestHeadersFromUploadStream = { headers, headersSize };
> +      payload.requestPostData = {
> +        ...requestPostData,
> +        uploadHeaders: { headers, headersSize }
> +      };

Yep, nice

Comment 31

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8959019 [details]
Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections

https://reviewboard.mozilla.org/r/227886/#review235094

Try push looks ok:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=572be92e3f9bb2b8d616b272b4440e50a6b1caac&selectedJob=169179113

R+

Thanks for working on this!
Honza

Comment 32

[Tom Glowka]

Comment on attachment 8957388 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/226320/diff/6-7/

Comment 33

[Tom Glowka]

Attachment: Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections;

Review commit: https://reviewboard.mozilla.org/r/229350/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/229350/

Comment 34

[Tom Glowka]

Attachment: Bug 1269468 - onRequestPostData returns requestPostData not whole payload;

For the sake of coherence with the rest of FirefoxDataProvider interface, to avoid future bugs like the one this commit refers to.

Review commit: https://reviewboard.mozilla.org/r/229352/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/229352/

Comment 35

[Jan Honza Odvarko [:Honza] (always need-info? me)]

You are missing reviewer in the commit message.

> R+, you don't have to ask for additional review as soon as the comment is resolved.
Perhaps, this confused you, but the reviewer name must be there.
(I can give another R+ if it simplify things)

Honza

Comment 36

[Tom Glowka]

Attachment: Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData;

Review commit: https://reviewboard.mozilla.org/r/229398/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/229398/

Comment 37

[Tom Glowka]

Comment on attachment 8960595 [details]
Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/229350/diff/1-2/

Comment 38

[Tom Glowka]

Comment on attachment 8960596 [details]
Bug 1269468 - onRequestPostData returns requestPostData not whole payload;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/229352/diff/1-2/

Comment 39

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8960595 [details]
Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections;

https://reviewboard.mozilla.org/r/229350/#review235116

Comment 40

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8960596 [details]
Bug 1269468 - onRequestPostData returns requestPostData not whole payload;

https://reviewboard.mozilla.org/r/229352/#review235118

Comment 41

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8960627 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData;

https://reviewboard.mozilla.org/r/229398/#review235120

Comment 42

[Tom Glowka]

Actually the only thing that confused me was that when I amended the first (earliest) commit according to your comment, rebased the other two on it, pushed with just "push review" , I lost history of review on reviewboard. I did so previously and it had worked, anyway I must have done something to cause it...

Thanks for r+!

Comment 43

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e4e40aa97fb3
fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData; r=Honza
https://hg.mozilla.org/integration/autoland/rev/4d0ea899c5c5
fix netmonitor copy POST Data by conditonal use of formDataSections; r=Honza
https://hg.mozilla.org/integration/autoland/rev/3981bf998979
onRequestPostData returns requestPostData not whole payload; r=Honza

Comment 44

[Natalia Csoregi [:nataliaCs]]

Backed out for failing netmonitor/test/browser_net_copy_as_curl.js

backout link: https://hg.mozilla.org/integration/autoland/rev/2afb657bb20ea1165e08e2e77711a1e340aabbfa

push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=3981bf99897959753fa07d0927df083782bbd1d2&group_state=expanded

failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=169382533&repo=autoland&lineNumber=4633

06:46:26     INFO -  377 INFO TEST-START | devtools/client/netmonitor/test/browser_net_copy_as_curl.js
06:46:33     INFO -  TEST-INFO | started process screenshot
06:46:33     INFO -  TEST-INFO | screenshot: exit 0
06:46:33     INFO -  Buffered messages logged at 06:46:26
06:46:33     INFO -  378 INFO Entering test bound
06:46:33     INFO -  379 INFO Initializing a network monitor pane.
06:46:33     INFO -  380 INFO Adding a new tab with URL: http://example.com/browser/devtools/client/netmonitor/test/html_copy-as-curl.html
06:46:33     INFO -  381 INFO Console message: [JavaScript Warning: "Use of nsIFile in content process is deprecated." {file: "resource://gre/modules/FileUtils.jsm" line: 170}]
06:46:33     INFO -  382 INFO Console message: [JavaScript Warning: "Use of nsIFile in content process is deprecated." {file: "resource://gre/modules/FileUtils.jsm" line: 170}]
06:46:33     INFO -  383 INFO Tab added and finished loading
06:46:33     INFO -  384 INFO Net tab added successfully: http://example.com/browser/devtools/client/netmonitor/test/html_copy-as-curl.html
06:46:33     INFO -  385 INFO Target remoted.
06:46:33     INFO -  386 INFO Console message: [JavaScript Warning: "Unknown pseudo-class or pseudo-element ‘selection’.  Ruleset ignored due to bad selector." {file: "chrome://devtools/content/sourceeditor/codemirror/lib/codemirror.css" line: 232}]
06:46:33     INFO -  387 INFO Console message: [JavaScript Warning: "Unknown property ‘-moz-border-radius’.  Declaration dropped." {file: "chrome://devtools/content/sourceeditor/codemirror/lib/codemirror.css" line: 241}]
06:46:33     INFO -  388 INFO Console message: [JavaScript Warning: "Unknown pseudo-class or pseudo-element ‘selection’.  Ruleset ignored due to bad selector." {file: "chrome://devtools/content/sourceeditor/codemirror/lib/codemirror.css" line: 324}]
06:46:33     INFO -  389 INFO Console message: [JavaScript Warning: "Unknown property ‘user-select’.  Declaration dropped." {file: "resource://devtools/client/shared/components/reps/reps.css" line: 290}]
06:46:33     INFO -  390 INFO Network monitor pane shown successfully.
06:46:33     INFO -  391 INFO Disabling cache and reloading page.
06:46:33     INFO -  392 INFO Got marker: dom-interactive
06:46:33     INFO -  Buffered messages logged at 06:46:27
06:46:33     INFO -  393 INFO Got marker: dom-complete
06:46:33     INFO -  394 INFO Got two timeline markers, done waiting
06:46:33     INFO -  395 INFO Request server1.conn0.child1/netEvent26 for http://example.com/browser/devtools/client/netmonitor/test/html_copy-as-curl.html done
06:46:33     INFO -  396 INFO All requests finished
06:46:33     INFO -  397 INFO Cache disabled when the current and all future toolboxes are open.
06:46:33     INFO -  398 INFO TEST-PASS | devtools/client/netmonitor/test/browser_net_copy_as_curl.js | Request to reconfigure the tab was recorded. -
06:46:33     INFO -  399 INFO Clearing requests in the console client.
06:46:33     INFO -  400 INFO Clearing requests in the UI.
06:46:33     INFO -  401 INFO Starting test...
06:46:33     INFO -  402 INFO > Network event progress: NetworkEvent: 0/1, PayloadReady: 1/1, got NetMonitor:PayloadReady for server1.conn0.child1/netEvent28
06:46:33     INFO -  403 INFO > Network event progress: NetworkEvent: 1/1, PayloadReady: 1/1, got NetMonitor:NetworkEvent for server1.conn0.child1/netEvent28
06:46:33     INFO -  404 INFO TEST-PASS | devtools/client/netmonitor/test/browser_net_copy_as_curl.js | Clipboard has the given value: 'curl "http://example.com/browser/devtools/client/netmonitor/test/sjs_simple-test-server.sjs" -H "Accept: */*" --compressed -H "Accept-Language: en-US" -H "Cache-Control: no-cache" -H "Connection: keep-alive" -H "Host: example.com" -H "Pragma: no-cache" -H "Referer: http://example.com/browser/devtools/client/netmonitor/test/html_copy-as-curl.html" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" -H "X-Custom-Header-1: Custom value" -H "X-Custom-Header-2: 8.8.8.8" -H "X-Custom-Header-3: Mon, 3 Mar 2014 11:11:11 GMT"' -
06:46:33     INFO -  405 INFO Clipboard contains a cURL command for the currently selected item's url.
06:46:33     INFO -  406 INFO > Network event progress: NetworkEvent: 0/1, PayloadReady: 1/1, got NetMonitor:PayloadReady for server1.conn0.child1/netEvent29
06:46:33     INFO -  407 INFO > Network event progress: NetworkEvent: 1/1, PayloadReady: 1/1, got NetMonitor:NetworkEvent for server1.conn0.child1/netEvent29
06:46:33     INFO -  Buffered messages logged at 06:46:28
06:46:33     INFO -  408 INFO TEST-PASS | devtools/client/netmonitor/test/browser_net_copy_as_curl.js | Clipboard has the given value: 'curl "http://example.com/browser/devtools/client/netmonitor/test/sjs_simple-test-server.sjs" -H "Accept: */*" --compressed -H "Accept-Language: en-US" -H "Cache-Control: no-cache" -H "Connection: keep-alive" -H "Cookie: bob=true; tom=cool" -H "Host: example.com" -H "Pragma: no-cache" -H "Referer: http://example.com/browser/devtools/client/netmonitor/test/html_copy-as-curl.html" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" -H "X-Custom-Header-1: Custom value" -H "X-Custom-Header-2: 8.8.8.8" -H "X-Custom-Header-3: Mon, 3 Mar 2014 11:11:11 GMT"' -
06:46:33     INFO -  409 INFO Clipboard contains a cURL command for the currently selected item's url.
06:46:33     INFO -  410 INFO > Network event progress: NetworkEvent: 0/1, PayloadReady: 1/1, got NetMonitor:PayloadReady for server1.conn0.child1/netEvent30
06:46:33     INFO -  411 INFO > Network event progress: NetworkEvent: 1/1, PayloadReady: 1/1, got NetMonitor:NetworkEvent for server1.conn0.child1/netEvent30
06:46:33     INFO -  Buffered messages finished
06:46:33    ERROR -  412 INFO TEST-UNEXPECTED-FAIL | devtools/client/netmonitor/test/browser_net_copy_as_curl.js | Timed out while polling clipboard for pasted data, got: curl "http://example.com/browser/devtools/client/netmonitor/test/sjs_simple-test-server.sjs" -H "Accept: */*" --compressed -H "Accept-Language: en-US" -H "Cache-Control: no-cache" -H "Connection: keep-alive" -H "Content-Type: text/plain;charset=UTF-8" -H "Cookie: bob=true; tom=cool" -H "Host: example.com" -H "Pragma: no-cache" -H "Referer: http://example.com/browser/devtools/client/netmonitor/test/html_copy-as-curl.html" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" -H "X-Custom-Header-1: Custom value" -H "X-Custom-Header-2: 8.8.8.8" -H "X-Custom-Header-3: Mon, 3 Mar 2014 11:11:11 GMT" --data "Plaintext value as a payload" -
06:46:33     INFO -  Stack trace:
06:46:33     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:putAndVerify:1015
06:46:33     INFO -  Not taking screenshot here: see the one that was previously logged
06:46:33    ERROR -  413 INFO TEST-UNEXPECTED-FAIL | devtools/client/netmonitor/test/browser_net_copy_as_curl.js | Uncaught exception - failed
06:46:33     INFO -  414 INFO Leaving test bound
06:46:33     INFO -  415 INFO Removing tab.
06:46:33     INFO -  416 INFO Waiting for event: 'TabClose' on [object XULElement].
06:46:33     INFO -  417 INFO Got event: 'TabClose' on [object XULElement].
06:46:33     INFO -  418 INFO Tab removed and finished closing
06:46:33     INFO -  419 INFO finish() was called, cleaning up...
06:46:33     INFO -  GECKO(7176) | MEMORY STAT | vsize 2098976MB | vsizeMaxContiguous 129996799MB | residentFast 279MB | heapAllocated 114MB
06:46:33     INFO -  420 INFO TEST-OK | devtools/client/netmonitor/test/browser_net_copy_as_curl.js | took 7386ms

Comment 45

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Tom, the `browser_net_copy_as_curl.js` is still failing...

Honza

Comment 46

[Tom Glowka]

Oh, that's bad. The interesting thing is it passes when running local, I will track down what makes it unstable in various environments.

Comment 47

[Tom Glowka]

Comment on attachment 8960627 [details]
Bug 1269468 - fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/229398/diff/1-2/

Comment 48

[Tom Glowka]

Comment on attachment 8960595 [details]
Bug 1269468 - fix netmonitor copy POST Data by conditonal use of formDataSections;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/229350/diff/2-3/

Comment 49

[Tom Glowka]

Comment on attachment 8960596 [details]
Bug 1269468 - onRequestPostData returns requestPostData not whole payload;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/229352/diff/2-3/

Comment 50

[Tom Glowka]

So the test was passing on Linux, but not Win. I fixed quoting cmdline param in test, which seems perfectly matching the behavior. Honza, can you please run try?
BTW. Do you think it is relevant to get the access, so that I can run try myself in situations like this?

Comment 51

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Tom Glowka from comment #50)
> So the test was passing on Linux, but not Win. I fixed quoting cmdline param
> in test, which seems perfectly matching the behavior. Honza, can you please
> run try?
Sure, try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=08b76a0852e4ae608dbd342a8c4d904dcd1e670a

> BTW. Do you think it is relevant to get the access, so that I can run try
> myself in situations like this?
Yes! Follow the instructions here:
hg import https://reviewboard-hg.mozilla.org/gecko/rev/e398cf76137bc5009ed1614a9cc97d0c4edfa67f
I'll vouch for you.

Honza

Comment 52

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to Jan Honza Odvarko [:Honza] from comment #51)
> Yes! Follow the instructions here:
The right link:
https://wiki.mozilla.org/ReleaseEngineering/TryServer#Getting_access_to_the_Try_Server

Honza

Comment 53

[Tom Glowka]

I have run another try (https://treeherder.mozilla.org/#/jobs?repo=try&revision=5f886702102cd099ee121ed3c06909c6f32ca484&selectedJob=169569022) after studying logs, when I realized that test for this bug belonged to suite mochitest-clipboard and in fact wasn't run before. This time I found it in logs, so everything looks finally fine.


(In reply to Jan Honza Odvarko [:Honza] from comment #52)
> (In reply to Jan Honza Odvarko [:Honza] from comment #51)
> > Yes! Follow the instructions here:
> The right link:
> https://wiki.mozilla.org/ReleaseEngineering/
> TryServer#Getting_access_to_the_Try_Server
> 
> Honza

Thanks for link and vouching for me!

Comment 55

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2e26d62979e0
fix netmonitor Copy as cURL: adjusting requestData calls to current implementation of onRequestPostData; r=Honza
https://hg.mozilla.org/integration/autoland/rev/7608e0de5f5f
fix netmonitor copy POST Data by conditonal use of formDataSections; r=Honza
https://hg.mozilla.org/integration/autoland/rev/ee94b789bd8e
onRequestPostData returns requestPostData not whole payload; r=Honza

Comment 56

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/2e26d62979e0
https://hg.mozilla.org/mozilla-central/rev/7608e0de5f5f
https://hg.mozilla.org/mozilla-central/rev/ee94b789bd8e

Comment 59

[Laurențiu Nicola]

Confirmed to work (via https://bugzilla.mozilla.org/show_bug.cgi?id=1433109#c0).