Closed Bug 1725182 Opened 3 years ago Closed 3 years ago

[meta] Ensure tests within devtools/client are https-first compliant

Categories

(Core :: DOM: Security, task, P3)

Product:
Core
Component:
DOM: Security
Type:
task

Tracking

()

Status:
RESOLVED FIXED
Milestone:
95 Branch
Tracking Flags:
Tracking Status
firefox95 --- fixed

People

(Reporter: t.yavor, Assigned: jdescottes)

References

(Blocks 1 open bug)

Details

(Keywords: meta, Whiteboard: [domsecurity-active])

Attachments

(2 files)

Bug 1725182 - [devtools] (tmp) Add configuration to run tests with https-first
48 bytes, text/x-phabricator-request
Details | Review
Bug 1725182 - Ensure remaining tests within devtools/client are https-first compliant
48 bytes, text/x-phabricator-request
Details | Review
No description provided.
Severity: -- → S3
Status: NEW → ASSIGNED
Priority: -- → P3
Whiteboard: [domsecurity-active]

Tomer: I plan to work on making all devtools tests work with https-first. Can I reuse this bug and assign myself to it?

Flags: needinfo?(lyavor)

Thank you.
Yes, for sure.

Flags: needinfo?(lyavor)
Assignee: lyavor → nobody
Status: ASSIGNED → NEW
Assignee: nobody → jdescottes
Status: NEW → ASSIGNED
Keywords: meta
Summary: Ensure tests within devtools/client are https-first compliant → [meta] Ensure tests within devtools/client are https-first compliant
Severity: S3 → --
Depends on: 1734177
Depends on: 1734178

Some resources to work on this topic.

Try configuration to run https-first and non https-first in the same push:

See https://hg.mozilla.org/try/rev/92915cec363c44a78ac7c1c45a326f34ff0a3acd. This changeset adds a M-dt-https group that can be selected alongside regular runs.

Note that you can either hg import https://hg.mozilla.org/try/rev/92915cec363c44a78ac7c1c45a326f34ff0a3acd or hg pull https://hg.mozilla.org/try/ -r 92915cec363c44a78ac7c1c45a326f34ff0a3acd to retrieve the code locally.

Suggested fuzzy query: devtools linux mochitest opt !a11y !asan !shippable

List of known failures:

storage: Bug 1736509

devtools/client/storage/test/browser_storage_cache_navigation.js
devtools/client/storage/test/browser_storage_cookies_delete_all.js
devtools/client/storage/test/browser_storage_cookies_navigation.js
devtools/client/storage/test/browser_storage_delete.js
devtools/client/storage/test/browser_storage_delete_all.js
devtools/client/storage/test/browser_storage_delete_usercontextid.js
devtools/client/storage/test/browser_storage_dfpi.js
devtools/client/storage/test/browser_storage_dom_cache_disabled.js
devtools/client/storage/test/browser_storage_dynamic_updates_localStorage.js

webconsole: Bug 1736678

devtools/client/webconsole/test/browser/browser_console_webextension.js
devtools/client/webconsole/test/browser/browser_jsterm_evaluation_context_selector.js
devtools/client/webconsole/test/browser/browser_jsterm_evaluation_context_selector_inspector.js
devtools/client/webconsole/test/browser/browser_jsterm_evaluation_context_selector_pause_in_debugger.js
devtools/client/webconsole/test/browser/browser_jsterm_evaluation_context_selector_targets_update.js
devtools/client/webconsole/test/browser/browser_webconsole_warning_group_content_blocking.js
devtools/client/webconsole/test/browser/browser_webconsole_warning_group_multiples.js
devtools/client/webconsole/test/browser/browser_webconsole_warning_groups.js
devtools/client/webconsole/test/browser/browser_webconsole_warning_groups_filtering.js
devtools/client/webconsole/test/browser/browser_webconsole_warning_groups_outside_console_group.js
devtools/client/webconsole/test/browser/browser_webconsole_warning_groups_toggle.js
devtools/client/webconsole/test/browser/browser_webconsole_websocket.js

Fixed tests:

aboutdebugging: Bug 1734178

devtools/client/aboutdebugging/test/browser/browser_aboutdebugging_devtoolstoolbox_navigate_back_forward.js

debugger: Bug 1734718

devtools/client/debugger/test/mochitest/browser_dbg-unselected-pause.js

inspector: Bug 1734725

devtools/client/inspector/boxmodel/test/browser_boxmodel_update-after-navigation.js
devtools/client/inspector/markup/test/browser_markup_events_jquery_1.0.js
devtools/client/inspector/markup/test/browser_markup_events_jquery_1.1.js
devtools/client/inspector/markup/test/browser_markup_events_jquery_1.11.1.js
devtools/client/inspector/markup/test/browser_markup_events_jquery_1.2.js
devtools/client/inspector/markup/test/browser_markup_events_jquery_1.3.js
devtools/client/inspector/rules/test/browser_rules_color_scheme_simulation.js
devtools/client/inspector/rules/test/browser_rules_copy_styles.js
devtools/client/inspector/rules/test/browser_rules_edit-value-after-name_04.js
devtools/client/inspector/rules/test/browser_rules_imported_stylesheet_edit.js
devtools/client/inspector/rules/test/browser_rules_original-source-link.js
devtools/client/inspector/rules/test/browser_rules_print_media_simulation.js
devtools/client/inspector/rules/test/browser_rules_style-editor-link.js
devtools/client/inspector/rules/test/browser_rules_urls-clickable.js
devtools/client/inspector/test/browser_inspector_fission_frame.js
devtools/client/inspector/test/browser_inspector_fission_frame_navigation.js
devtools/client/inspector/test/browser_inspector_fission_switch_target.js
devtools/client/inspector/test/browser_inspector_highlighter-eyedropper-frames.js
devtools/client/inspector/test/browser_inspector_highlighter-reload.js
devtools/client/inspector/test/browser_inspector_navigate_to_errors.js
devtools/client/inspector/test/browser_inspector_navigation.js
devtools/client/inspector/test/browser_inspector_reload_xul.js
devtools/client/inspector/test/browser_inspector_search-05.js
devtools/client/inspector/test/browser_inspector_select-last-selected.js

framework: Bug 1734177

devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_evaluation_context.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_fission_contentframe_inspector.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_fission_inspector.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_fission_inspector_webextension.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_print_preview.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_rtl.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_ruleview_stylesheet.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_shouldprocessupdates.js
devtools/client/framework/browser-toolbox/test/browser_browser_toolbox_watchedByDevTools.js
devtools/client/framework/test/browser_source_map-01.js
devtools/client/framework/test/browser_source_map-absolute.js
devtools/client/framework/test/browser_source_map-inline.js
devtools/client/framework/test/browser_source_map-late-script.js
devtools/client/framework/test/browser_source_map-reload.js
devtools/client/framework/test/browser_tab_descriptor_fission.js
devtools/client/framework/test/browser_target_cached-resource.js
devtools/client/framework/test/browser_toolbox_backward_forward_navigation.js
devtools/client/framework/test/browser_toolbox_error_count.js
devtools/client/framework/test/browser_toolbox_fission_navigation.js
devtools/client/framework/test/browser_toolbox_hosts.js
devtools/client/framework/test/browser_toolbox_options_enable_serviceworkers_testing.js
devtools/client/framework/test/browser_toolbox_view_source_01.js
devtools/client/framework/test/browser_toolbox_view_source_02.js
devtools/client/framework/test/browser_toolbox_view_source_03.js
devtools/client/framework/test/browser_toolbox_window_title_changes.js

Misc: Bug 1735143

(For test suites which have very few https related failures)
devtools/client/memory/test/browser/browser_memory_allocationStackDisplay_02.js
devtools/client/responsive/test/browser/browser_container_tab.js
devtools/client/shared/test/browser_link.js
devtools/client/styleeditor/test/browser_styleeditor_loading.js
devtools/shared/commands/resource/tests/browser_resources_several_resources.js
devtools/shared/commands/resource/tests/browser_resources_websocket.js

netmonitor: Bug 1735660

devtools/client/netmonitor/src/har/test/browser_net_har_copy_all_as_har.js
devtools/client/netmonitor/test/browser_net_block-pattern.js
devtools/client/netmonitor/test/browser_net_block.js
devtools/client/netmonitor/test/browser_net_cached-status.js
devtools/client/netmonitor/test/browser_net_cause_redirect.js
devtools/client/netmonitor/test/browser_net_cause_source_map.js
devtools/client/netmonitor/test/browser_net_charts-01.js
devtools/client/netmonitor/test/browser_net_charts-02.js
devtools/client/netmonitor/test/browser_net_charts-03.js
devtools/client/netmonitor/test/browser_net_charts-04.js
devtools/client/netmonitor/test/browser_net_charts-05.js
devtools/client/netmonitor/test/browser_net_charts-06.js
devtools/client/netmonitor/test/browser_net_charts-07.js
devtools/client/netmonitor/test/browser_net_column_slow-request-indicator.js
devtools/client/netmonitor/test/browser_net_columns_showhide.js
devtools/client/netmonitor/test/browser_net_columns_time.js
devtools/client/netmonitor/test/browser_net_content-type.js
devtools/client/netmonitor/test/browser_net_copy_as_curl.js
devtools/client/netmonitor/test/browser_net_copy_as_fetch.js
devtools/client/netmonitor/test/browser_net_copy_headers.js
devtools/client/netmonitor/test/browser_net_copy_svg_image_as_data_uri.js
devtools/client/netmonitor/test/browser_net_cors_requests.js
devtools/client/netmonitor/test/browser_net_curl-utils.js
devtools/client/netmonitor/test/browser_net_cyrillic-01.js

Christoph, just to make sure we are not delaying you, what is the timeline for enabling https-first?

Flags: needinfo?(ckerschb)
Depends on: 1734718
Depends on: 1734725

I do not plan on landing this, but will base my stack on it

Depends on: 1734787

Also wonder what we should do with all the tests marked https_first_disabled https://searchfox.org/mozilla-central/search?q=https_first_disabled&path=devtools

Depends on: 1735143

(In reply to Julian Descottes [:jdescottes] from comment #4)

Christoph, just to make sure we are not delaying you, what is the timeline for enabling https-first?

We are really trying to enable https-first in Nightly more or less ASAP. That said, there are still many fallouts where tests in other components needs to be updated. I really appreciate you taking on the work and updating tests within devtools. You are not slowing things done, you are helping a lot which allows us to move faster.

Flags: needinfo?(ckerschb)
Depends on: 1735658
Depends on: 1735660
Depends on: 1736509
Depends on: 1736678

All bugs have patches attached now. Current try push with all patches + https-first enabled: https://treeherder.mozilla.org/#/jobs?repo=try&revision=095ba047efec373935c8718b1015aee9534aa376

There still seems to be a few failures, but not too many. New try push at https://treeherder.mozilla.org/jobs?repo=try&revision=1712b6be473041ccc29a16de5a3ce1274c44a3cb

Depends on D128931

One inspector test was missed in a previous bug, as well as all the framework browser-toolbox tests.

Except for known intermittent we have a fully green try for devtools mochitests with https-first at https://treeherder.mozilla.org/jobs?repo=try&revision=1712b6be473041ccc29a16de5a3ce1274c44a3cb

There might still be some edge cases, because I only pushed to try with https-first for Linux without Fission, but most of the test suite should run now at least.

Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2f5efb27cd35
Ensure remaining tests within devtools/client are https-first compliant r=bomsy

https://hg.mozilla.org/mozilla-central/rev/2f5efb27cd35

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
status-firefox95: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 95 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: