23053 - Need to instrument Mail source code for performance measurements

Status: VERIFIED DUPLICATE of bug 78787

(SeaMonkey :: MailNews: Message Display, defect, P3)

Description

[bsharma]

The mail developers needs to put the timing information in the souce code in a
way that it can be automated to get back the timing information.

Please let me know when some of the instrumentation is ready to use
so that I can start designing the scripts required.

Comment 1

[ekrock's old account (dead)]

Corrected code name to the standard, [PERF]. (Don't use [PERFORMANCE].)

Comment 2

[lchiang]

We wanted to run some subset of the performance test (about 5 tests) on a daily
basis and not have to use QA Partner to automate since we want these tests
across all platforms and put out on mozilla.org. For this, Bindu needs some help
from development. For details on what needs to be intrumented, please contact
Bindu.

The instrumentation that Scott helped us with earlier were for use w/QA Partner.

Comment 3

[Phil Peterson]

Bindu or Lisa, please put the specific requirements in this bug report.
Otherwise, this bug report is too vague for us to do anything with.

Comment 4

[lchiang]

Sol, can you list the items that we wanted to measure in here?  I didn't write
them down from yesterday's mail leads mtg.

Bindu, can you put down what you need done?  What requirements you have?
Perhaps, use the browser area as an example if some of the instrumentation is
done for you already by another team.

Comment 5

[bsharma]

Once the document is finalised I am planning to meet with developers and discuss
the requirements with them.
We can discuss what all can be possible to instrument that can be useful for
automation.
Browser Bug # for the source code instrumentation is
http://bugzilla.mozilla.org/show_bug.cgi?id=23036

Comment 6

[sol]

I'd like to instrument all of the tests that are currently being run manually
and the results of which are posted on the following page:

    http://www.mozilla.org/mailnews/performance-results.html

The highest priority items to test are the following:

   Mail Startup after Navigator
   POP: Load summary file
   IMAP: Load summary file
   IMAP: Display 5 Messages without Attachment
   Open Compose Window

Comment 7

[bsharma]

For automating all the tests I have following requirements from developers:
1. The timing information should be provided either on the console that can be
directed to the log file or directly to a file.
2. Setup a flag as environment variable (because in the final product the
console will not exist) to get the performance data.

NOTE: I am not sure how we want to simulate the user interface actions.

Please let me know if you have any questions.

Comment 8

[bsharma]

Adding Paul and Jan to the cc list.

Comment 9

[lchiang]

The problem here is how to simulate a user's action without having QA Partner to
do so; it's not so much a problem of writing timings to the console, right?  For
example: to display 5 messages in an IMAP account, we'd need a way to be able to
open directly into a predefined folder containing these 5 messages and be able
to select each one.  Is there a way to do this by a command line of some sort?

Comment 10

[leger]

Adding perf to keyword field.

Comment 11

[leger]

Putting on dogfood radar for a PDT+ marking

Comment 12

[leger]

PDT and phil agree to PDT-.  suresh is able to get performance date weekly.  
This would be great to get done, but would not hold up beta.

Comment 13

[selmer (gone)]

Phil found a way to weasel out of owning this bug.  Reassigning.

Comment 14

[lchiang]

I am removing dogfood from the keyword list.

Comment 15

[selmer (gone)]

I wouldn't hold any shipment for this.  We need to fix performance for the
release and this is only one way to get there.  Marking M20, please correct if
I've misinterpreted this bug.  Adding putterman to the CC to get his opinion on
this.

Comment 16

[selmer (gone)]

Scott, is there a reasonable time where we can still address this bug?

Comment 17

[lchiang]

move to future target milestone.  adding helpwanted keyword.  This would be good 
for a contributor to do to help us be able to get timings automatically.

Comment 18

[leger]

Putting on nsbeta3 radar.  QA needs this for PR3 perf testing.

Comment 19

[scottputterman]

I guess we could do this but I don't think this is worthwhile unless you are
marking this beta3 for someone outside of Netscape to fix.  Currently we have
performance measurements coming from mailnews QA on all of the items we are
going to focus on for beta3.  Fixing this means we won't fix bugs that are
actually visible in the product.

Comment 20

[scottputterman]

As has been mentioned in the bug, things that require user action may be hard to
do. But perhaps we can cover the more atomic non-user action type timings. I
will look into this but as I said above this still may not be the best user of
our time.  The main reason we haven't done this in the past is because it was
too hard to cover some of the actions and the amount of work required didn't
seem worthwhile.

Comment 21

[scottputterman]

Mail triage marking [nsbeta3-]

Comment 22

[scottputterman]

reassigning to sspitzer.  I'm not sure how important this still is for QA.

Comment 23

[lchiang]

stephen - do you need this?

Comment 24

[[e-mail] (gone - use [e-mail] instead)]

This has lived as bug 78787 for some time, and yes, Seth and I sat down one
evening and we agreed that when he had 'free' time, he would take a look at
logging some key areas.  It's not so critical now that I'll be doing mac and
linux every other week, but it would be nice to have.

*** This bug has been marked as a duplicate of 78787 ***

Comment 25

[[e-mail] (gone - use [e-mail] instead)]

verified dup.