If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Get a list of source addons for JetPerf testing

RESOLVED FIXED

Status

Testing
Talos
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: Jeff Hammel, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [jetpack+talos])

(Reporter)

Description

6 years ago
WRT JetPerf testing (bug 717036), we need a list of source addons to
be generated with the SDK.  The first should be a "hello world" app
that doesn't do much of anything, then we can expand from there.
(Reporter)

Updated

6 years ago
Whiteboard: [jetpack+talos]
(Reporter)

Updated

6 years ago
Blocks: 717036
(Reporter)

Updated

6 years ago
Blocks: 729205
Here is a list of potential addons to test:
  https://github.com/ochameau/jetperf-test-addons

These addons just prints some messages like this:

  [PERF][1334758986946] Addon running
  [PERF][1334758987115] Content script loaded
  [PERF][1334758987141] DOM Updated

The big number being the result of (new Date().getTime()). 
You may adapt them to use something else, but that gives an idea on what kind of simple addon we may want to test.

I think we would like to test two thing:
 - Do we hit a perf regression on Jetpack? i.e. is Jetpack becoming slower with one particular revision? Like addons are slower to start, or Widget takes longer to load, or content script are executing slower.
 - Do we hit a perf regression against Firefox? i.e. is Jetpack slowing down Firefox with one particular revision? Like web content takes more time to be loaded/display when there is a page-mod running.

Here, with these examples, I'm only trying to verify first case: "Is Jetpack itself running slower".
It would be interesting to run tests against the second case in order to see impact on Firefox that Jetpack has.
(Reporter)

Comment 2

6 years ago
See also https://bugzilla.mozilla.org/show_bug.cgi?id=729205#c16

As I understand it, using the addons from comment 1 will prevent us from installing an addon multiple times.  Do we care about this?
(Reporter)

Comment 3

6 years ago
We should also figure out where these should live in relationship to bug 729205 . If we're running jetperf in production, AFAIK it is not kosher to touch github (or similar) in production runs.  Should we put these addons in the mozharness repository?  Should we put them somewhere on a Mozilla site and download them?  I'm more inclined to include them directly in mozharness where you have a notion of directory.
(In reply to Jeff Hammel [:jhammel] from comment #2)
> See also https://bugzilla.mozilla.org/show_bug.cgi?id=729205#c16
> 
> As I understand it, using the addons from comment 1 will prevent us from
> installing an addon multiple times.  Do we care about this?

You are right, but I don't think we care that much about such usecase, do we?
But I'm just jumping into your project so I may have different/new ideas compared to what you have in mind.
If you think that what I'm suggesting doesn't go in your direction, please tell me what you have in mind for the long run. So that I can help you driving in this particular direction.

(In reply to Jeff Hammel [:jhammel] from comment #3)
> We should also figure out where these should live in relationship to bug
> 729205 . If we're running jetperf in production, AFAIK it is not kosher to
> touch github (or similar) in production runs.  Should we put these addons in
> the mozharness repository?  Should we put them somewhere on a Mozilla site
> and download them?  I'm more inclined to include them directly in mozharness
> where you have a notion of directory.

I used github as a way to show/share these examples with you.
You can consider them as being in public domain and do whatever you think is relevant with them. Host them on some other repo, modify them ...
It is up to you to decided where/how host such files in "Testing universe".
(Reporter)

Comment 5

6 years ago
:aki, what would you like to do with these directories?

Comment 6

6 years ago
Would these fit in http://hg.mozilla.org/projects/addon-sdk ?
That would allow addon sdk developers to update the tests if wanted/needed.
(Reporter)

Comment 7

6 years ago
(In reply to Aki Sasaki [:aki] from comment #6)
> Would these fit in http://hg.mozilla.org/projects/addon-sdk ?
> That would allow addon sdk developers to update the tests if wanted/needed.

It works for me, but I have no jurisdiction over this repo.  :ochameau, does this work for you and the jetpack team or should we seek another solution?
I'm not sure that's the best place to land them.
Any file added to this repository is going to be included in the SDK downloaded by developers, whereas I don't see why they would need them?
I'll try to bring up this question to the Jetpack team.
(Reporter)

Comment 9

6 years ago
So this decision is blocking any further jetperf. It is clear, moving from text addons to directories of addons that most of the jetperf script will need a rewrite and the form of this rewrite is contingent upon where these testing addons live.  I can't really do much here until we figure out a production home for these (that is, presumedly, developer accessible)

Comment 10

6 years ago
I'm fine with a subdirectory in addon-sdk, or a separate sibling repository if that speeds things up.

Comment 11

6 years ago
Why can't we just put these things somewhere to unblock ourselves and make that location a parameter to the test so that it is easy to change if/when the jetpack devs change their minds about where to put them?  I'm not clear why we're letting the location of some files block us here.
(Reporter)

Comment 12

6 years ago
:aki, :ochameau : what about a repository that lives at http://hg.mozilla.org/projects/addon-sdk-jetperf for these addons? This would have the advantage that:

- it is accessible to devs so that they can make changes or add addons as they desire
- it is a mozilla repository so that the buildsystem can be happy

Comment 13

6 years ago
wfm.
WFM as weel.

As I said in comment 4 you can do whatever you like with these addons.
Host them however/whereever is best for you.
Landing them in addon-sdk repository is another question.
(Reporter)

Updated

6 years ago
Depends on: 749681
(Reporter)

Comment 15

6 years ago
Filed https://bugzilla.mozilla.org/show_bug.cgi?id=749681 to this end
(Reporter)

Updated

5 years ago
Depends on: 755058
(Reporter)

Comment 16

5 years ago
Commited: http://hg.mozilla.org/projects/addon-sdk-jetperf-tests/rev/d7e90684309c
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.