1052856 - "Copy as curl" should use --compressed instead of -H "accept-encoding gzip..."

Status: RESOLVED FIXED

(DevTools :: Netmonitor, defect)

Description

[--]

When using "Copy as curl" on a web page that was sent with compressed content-encoding (like a page here from bugzilla), you'll get -H 'Accept-Encoding: gzip, deflate' included among the options.

That header will lead to curl asking for a compressed response but without the option that asks it to automatically decompress that data. This leads to the user getting "rubbish" (gzipped data) instead of the expected plain html.

curl's --compressed option includes the correct request header _and_ makes it automatically decompress downloaded compressed content.

Comment 1

[Abhirav Kariya]

hi! I'M new here , Please assign this bug to me!

Comment 2

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Hi Abhirav!  For new contributors, we don't assign the bug until a first patch appears, to keep it available for others until then.

Please take a look at our guide[1] to getting started with the code base.

For this specific bug, I imagine we'd need to modify this block[2] to special case the use of compression headers as Daniel mentions in comment 0.

Also, we should update the cURL tests[3] to cover this change.

If you have any questions, you can check "need info" below and fill in my email, or come to the #devtools IRC room.

[1]: https://wiki.mozilla.org/DevTools/Hacking
[2]: http://dxr.mozilla.org/mozilla-central/source/browser/devtools/shared/Curl.jsm#116
[3]: http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/test/browser_net_copy_as_curl.js

Comment 3

[raymond [:retornam] (inactive)]

Attachment: bug-1052856.patch

Comment 4

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Comment on attachment 8479406 [details] [diff] [review]
bug-1052856.patch

Review of attachment 8479406 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good overall!

You'll also need to update the test[1] to cover this change.

[1]: http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/test/browser_net_copy_as_curl.js

::: browser/devtools/shared/Curl.jsm
@@ +119,5 @@
>        headers = headers.concat(multipartHeaders);
>      }
>      for (let i = 0; i < headers.length; i++) {
>        let header = headers[i];
> +      if (header.name === "Accept-Encoding" || header.name === "Content-Encoding"){

Content-Encoding is a response header, so only Accept-Encoding is needed here.

Comment 5

[raymond [:retornam] (inactive)]

Attachment: bug-1052856.patch

Comment 6

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Comment on attachment 8480066 [details]
bug-1052856.patch

Review of attachment 8480066 [details]:
-----------------------------------------------------------------

Seems good to me!  Do you have try access to run tests, or should I push it to try for you?

Comment 7

[raymond [:retornam] (inactive)]

(In reply to J. Ryan Stinnett [:jryans] from comment #6)
> Comment on attachment 8480066 [details]
> bug-1052856.patch
> 
> Review of attachment 8480066 [details]:
> -----------------------------------------------------------------
> 
> Seems good to me!  Do you have try access to run tests, or should I push it
> to try for you?

I dont have access to try. Please push for me thanks

Comment 8

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Try: https://tbpl.mozilla.org/?tree=Try&rev=614fdf54db04

Comment 9

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

It looks like the tests are failing on try.  Please take a look at the failures.  Also, make sure the tests are passing on your local machine.

Comment 10

[raymond [:retornam] (inactive)]

Attachment: bug-1052856.patch

Comment 11

[raymond [:retornam] (inactive)]

I fixed the issue.

Comment 12

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Comment on attachment 8480925 [details] [diff] [review]
bug-1052856.patch

Review of attachment 8480925 [details] [diff] [review]:
-----------------------------------------------------------------

Hopefully it works this time!

Try: https://tbpl.mozilla.org/?tree=Try&rev=0be411351126

Comment 13

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Try looks good, thanks for working on this!  See the wiki[1] for more DevTools bugs you may be interested in.

[1]: https://wiki.mozilla.org/DevTools/GetInvolved#Mentored_and_Good_First_Bugs

Comment 14

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/integration/fx-team/rev/8d6dc19c9f94

Comment 15

[Ryan VanderMeulen (PTO, back 6-April)]

https://hg.mozilla.org/mozilla-central/rev/8d6dc19c9f94

Comment 16

[Mattias Besemann]

This bug still seems to exist on the macOS version of Firefox - the -H 'Accept-Encoding: gzip, deflate, br' header is being used instead of the --compressed option. Confirmed not to be an issue in the Windows version, either using POSIX or Windows cURL options. I have not yet tested any Linux builds.