1623937 - Move devtools frontend files to a dedicated folder

Status: ASSIGNED

(DevTools :: Framework, task, P3)

Description

[Julian Descottes [:jdescottes]]

Currently, the devtools/client folder contains mostly "frontend" files. This means files used to build our panels and other UIs (such as about:debugging).

After Bug 1604539, devtools/client will also contain our fronts. Fronts are only used by the client and moving them in devtools/client avoids misunderstandings about what runs on the client and what runs on the server:

• actors are in devtools/server, used only on the server
• fronts are in devtools/client, used only on the client
• specs are in devtools/shared, used both on the client and on the server

However devtools/client is a special folder because we skip it on Android. This is fine because there is no DevTools client on Android, only a server. But, we have xpcshell tests in devtools/shared and devtools/server that use a DevTools client. And today we are skipping most of those test suites on android https://searchfox.org/mozilla-central/search?q=skip-if+%3D.*android&regexp=true&path=devtools%2Fs

In theory those tests should be able to run on Android. Maybe we can split the files differently in order to allow such tests to run on Android?

The idea would be to split the client files between frontend files (panel/ui code) and pure client files (devtools-client, fronts etc...).
The layout could be something like:

• devtools/client: fronts
• devtools/frontend: panels, css, images
• devtools/server: actors and server code
• devtools/shared: specs, transports, protocol, shared helpers

Comment 1

[Alexandre Poirot [:ochameau]]

(In reply to Julian Descottes [:jdescottes] from comment #0)

The idea would be to split the client files between frontend files (panel/ui code) and pure client files (devtools-client, fronts etc...).
The layout could be something like:

• devtools/client: fronts

Front + ResourcesWatcher+TargetList + DevToolsClient and may be some other bits around transport/sockets?
Oh and you mentioned them, but all client/front tests. This may actually help highlighting tests which would have to be rewritten to use the ResourceWatch rather than existing front methods.

• devtools/frontend: panels, css, images
• devtools/server: actors and server code
• devtools/shared: specs, transports, protocol, shared helpers

While we do that, it might be a good time to scan devtools/shared.
It looks like qrcode is no longer used:
https://searchfox.org/mozilla-central/search?q=qrcode&case=false&regexp=false&path=devtools%2F

Otherwise, this layout looks good to me!

Comment 2

[Julian Descottes [:jdescottes]]

Thanks for the feedback!

Front + ResourcesWatcher+TargetList + DevToolsClient and may be some other bits around transport/sockets?

Yes thanks for mentioning it. I said in the last DevTools meeting that I should add a longer description of the folder hierarchy that could be reused in https://searchfox.org/mozilla-central/source/devtools/docs/files/README.md, so here it goes:

Directories Overview

This page provides a very top level overview of what is on each directory in the DevTools source code:

• devtools/client: Client side code for DevTools, apart from the UI code which is in the dedicated devtools/frontend folder. The reason for this split is that devtools/frontend is not built in Firefox for Android, but some server and shared tests can still run on Android and might use a headless client. Such tests can import headless client files from devtools/client, which is included in the Firefox for Android builds.
  • devtools/client/fronts: Contains the client-side part of the DevTools Actors. Most Devtools Actors are represented on the client side by a Front which will be able to query the server-side Actor thanks to protocol.js. (link to documentation about DevTools Actors)
  • devtools/client/locales: Strings used in the front-end. They are still part of devtools/client because moving localized files means migrating existing strings for localized builds and will be done in a second step.
  • devtools/client/resources: Contains client-side helpers built for Fission support in DevTools, such as the TargetList and the ResourceWatcher.
• devtools/frontend: Only contains the code related to the devtools frontend: toolbox, panels, other UIs (about:debugging, RDM). This folder is excluded from the Firefox for Android builds because we don't support the DevTools UI in Firefox for Android, and its size makes it worth excluding.
  • devtools/frontend/shared/vendor: Vendored libraries used by the DevTools frontend, such as React. All frontend vendored libraries should ideally be grouped in this folder.
  • devtools/frontend/themes: CSS and images used in the DevTools frontend.
• devtools/server: Code for the server side of DevTools. Mostly contains DevTools Actors.
  • devtools/server/actors: Contains the server-side part of the DevTools Actors (so, the actual Actors themselves). Files in this folder might not all be Actors, as it can also contain utils and support classes used by Actors. Look for ActorClassWithSpec to check if a given class is an actor.
  • devtools/server/connectors: Contains bootstrap code that is injected in targets we want to debug. Connectors will typically start a server, a transport and finalize the connection with the caller.
• devtools/shared: Code shared between the client+frontend and the server. It contains shared helpers that might be used both from the client and the server, but also several important framework utilities, detailed below
  • devtools/shared/protocol: Contains all the base classes used by Remote Debugging Protocol (RDP): Pool, Front, Actor, Request, Response, types, etc.
  • devtools/shared/specs: Contains the specification files for the DevTools Actors. Specifications describe how a Front should build a request to an Actor, and how an Actor will respond to this request. (link to documentation about DevTools Actors).
  • devtools/shared/transport: Contains various implementations of the DevTools Transport layer. A transport is necessary to communicate between a DevToolsClient and a DevToolsServer, or between several DevToolsServers. Depending on the boundary separating the two entities (different process, different machine, etc.), different transports can be used to cross the boundary and send packets across.

There are some other folders at the root of the devtools folder, which are less often modified:

• devtools/docs: Contains the documentation for DevTools. Used to generate both https://docs.firefox-dev.tools/ and https://firefox-source-docs.mozilla.org/devtools/index.html
• devtools/platform: Contains platform code to inspect JS (note: if anybody has a better description, it's much welcome)
• devtools/startup: Contains the components which are registered when Firefox is starting. DevToolsStartup.jsm in particular contains all the bootstrap code for DevTools, it's loaded very early, and should be kept as small and efficient as possible.
  • devtools/startup/aboutdevtools: Contains the code for about:devtools, which was an experiment for DevTools onboarding and is pending removal.

Comment 3

[Julian Descottes [:jdescottes]]

While writing this overview, I realized we would have a small issue with the planned move: devtools/client/locales contains localized files and moving such files is a bigger problem than just moving code. Typically I think we would need to write a script to move all the corresponding files in the l10n repositories.

For now I propose to keep devtools/client/locales in the same folder and work on moving them in a second step.

Comment 4

[Julian Descottes [:jdescottes]]

Now actively working on this. I wanted to get this in FF78, but it's going to be too short so aiming for FF79 now.
(ran into technical issues related to pushing to try, which slowed down quite a bit my progress here).

Comment 5

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 1: Move and copy files from client to frontend

Comment 6

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 2: Fix moz.build files

Depends on D77024

Comment 7

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 3: Replace most devtools/client paths in devtools

Depends on D77025

Comment 8

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 4: Update eslintrc

Depends on D77026

Comment 9

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 5: Update devtools paths outside of devtools

Depends on D77031

Comment 10

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 6: update xpcshell TEST_HARNESS_FILES

Depends on D77032

Comment 11

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 7: update devtools client paths in tools/rewriting

Depends on D77033

Comment 12

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 8: Fix linting issues

Depends on D77034

Comment 13

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 9: Ignore some forbidden requires

Depends on D77035

Comment 14

[Julian Descottes [:jdescottes]]

Attachment: Bug 1623937 - Part 10: Update docs/files/README.md

Depends on D77036

Comment 15

[Greg Mierzwinski [:sparky]]

:jdescottes, is this bug still valid? The patches have been inactive for over 2 years.

Comment 16

[Julian Descottes [:jdescottes]]

The bug is still valid although I can abandon the patches if that helps for some reason.

Comment 17

[Greg Mierzwinski [:sparky]]

I'm going over older patches (2 years+) in the Waiting on authors section for the perftest group review and many of the patches and bugs were no longer relevant so I abandoned them. It's up to you if you want to abandon these, and it's fine with me if you leave them open.