522373 - Make TopFails site faster

Status: RESOLVED WONTFIX

(Testing :: General, enhancement)

Description

[Serge Gautherie (:sgautherie)]

[Not sure where to file this :-/]

1- The site take a long-long time to load.

2- The page misses a little explanation of what the results are.
{
Most recent test failures

    * 2009-10-13 07:56 Firefox Windows: TestEditMessages: WM_PASTE #1, timeline -
    * 2009-09-15 10:24 Firefox Linux: chrome://mochikit/content/chrome/content/xul/templates/tests/chrome/test_tmpl_bindingsreversed.xul, timeline - bindings - reversed
}

Comment 1

[Serge Gautherie (:sgautherie)]

3- While there, my most wanted feature would probably be to be able to query for a failure: to know when it last happened, maybe when it started and how often it happened too...

Comment 2

[(not currently active) Ted Mielczarek]

You can already do that. If you click on one of the test filenames, it will bring up a page listing all known failures for that test.

Comment 3

[Serge Gautherie (:sgautherie)]

(In reply to comment #2)
> You can already do that. If you click on one of the test filenames, it will
> bring up a page listing all known failures for that test.

Hum, yeah :-)

But let's see:

On one hand:
"2009-10-13 07:56 Firefox Windows: TestEditMessages: WM_PASTE #1, timeline  -"
links to
http://tinderbox.mozilla.org/topfails/test?name=TestEditMessages:%20WM_PASTE%20#1
http://tinderbox.mozilla.org/topfails/timeline?name=TestEditMessages:%20WM_PASTE%20#1
But both return "No Tests matches the given query." :-/

On the other hand:
I tried to search for
http://tinderbox.mozilla.org/topfails/timeline?name=/tests/content/media/test/test_mozLoadFrom.html
for bug 518659,
http://tinderbox.mozilla.org/topfails/timeline?name=/tests/layout/style/test/test_parse_url.html
for bug 520048,
http://tinderbox.mozilla.org/topfails/timeline?name=/browser/browser/components/sessionstore/test/browser/browser_461743.js
for bug 521889,
and none where found.
I also tried
http://tinderbox.mozilla.org/topfails/test?name=/tests/toolkit/content/tests/widgets/test_videocontrols.html
http://tinderbox.mozilla.org/topfails/timeline?name=/tests/toolkit/content/tests/widgets/test_videocontrols.html
and it found some entries, but none where the ones for that bug.

Then, I'm wondering what trees/builds are indexed? (as in "could it be Firefox (old) check only, and not Firefox (new) unittests builds"?)

And it doesn't find failure when using the test name only, like "test_videocontrols.html" ... which would be rather nice.

***

To summarize,
I guess it lacks 1+ indexes for the initial report page,
and could use some (inline) documentation/accesses to features.

NB: I hope this could become such a useful tool to track intermittent failures...

Comment 4

[(not currently active) Ted Mielczarek]

Oh, you're absolutely right. It doesn't check the new -Unittest trees. It could probably just be switched from looking at "Firefox" to "Firefox-Unittest" here:
http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/file/f130c2a6c5c0/unittest-log.py#l154

since there aren't really any unittest results on "Firefox" anymore. Additionally, it only knows to look at boxes with "unittest" in the name, so it'd need to know to look at the "mochitest" and "everythingelse" boxes here:
http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/file/f130c2a6c5c0/unittest-log.py#l221

Comment 5

[Serge Gautherie (:sgautherie)]

(In reply to comment #4)
> Oh, you're absolutely right. It doesn't check the new -Unittest trees.

I filed bug 522743.

Comment 6

[Serge Gautherie (:sgautherie)]

Attachment: (Av1) unittest-log.py: fix indentation and whitespaces [Checkin: Comment 8]

Let's start with these trivial nits.

***

Is there a way I could access the actual database, even read-only?

Comment 7

[(not currently active) Ted Mielczarek]

Here's an older copy of the db you can use if you'd like:
http://people.mozilla.com/~tmielczarek/unittest.db.bz2

Also, I don't care quite enough about this script to review changes, so feel free to push changes to it without review. If you're motivated to fix it, then by all means do so!

Comment 8

[Serge Gautherie (:sgautherie)]

Comment on attachment 406839 [details] [diff] [review]
(Av1) unittest-log.py: fix indentation and whitespaces
[Checkin: Comment 8]


http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/e0b688516ffd

Comment 9

[Serge Gautherie (:sgautherie)]

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/234b3a50ed92
(Bv1) unittest-log.py: Fix SQL code format nits

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/b022b602de10
(Cv1) unittest-log.py: Fix some code nits, 'totalchunks' is now an integer

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/8fbcb70d4411
(Dv1) unittest-log.py: Create 'S_IN_H', 'chunksize' is now an integer
[PS: Not Cv1 !]

Comment 10

[Serge Gautherie (:sgautherie)]

(In reply to comment #7)
> feel free to push changes to it without review.

Next patches do behavior changes including modifying the db schema.
When I push them, how do I get the (code and db) site (manually) updated?

Comment 11

[(not currently active) Ted Mielczarek]

You'll have to file an IT bug and ask them to do so. If you're going to modify the schema, you should probably either provide a schema migration tool or make the code automatically handle the out-of-date schema and update it.

Comment 12

[Serge Gautherie (:sgautherie)]

Noted.

I suggest to (automatically) delete data older than (twice) the time the logs are available: that would be 2(->4) months currently.
How long do you think data should be kept ? (forever??)

Comment 13

[Serge Gautherie (:sgautherie)]

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/a28b82cd4468
(Ev1) .hgignore: ignore the SQLite database file.

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/dfdc2d9ad81d
(Fv1) Move CreateDB() to new dbschema.py from unittest-log.py, Add license to both files.

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/f3854f015f52
(Gv1) unittest-log.py: improve imports documentation and code.

Comment 14

[Serge Gautherie (:sgautherie)]

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/818e258d3444
unittest-log.py: synchronize 'logging.*()' and 'print' commands.

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/1ae9cc960c40
unittest-log.py: Remove unused imports from 'time', Import and use 'ctime()', 'endtime' is now an integer, Remove useless and confusing global 'starttime'.

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/e13dc64e8e2f
unittest-log.py: 'build = timerow[index]' rewrite.

Comment 15

[Serge Gautherie (:sgautherie)]

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/ce36110abaf5
(Kv1) DB schema v2: Fix/Improve code from changeset 8 : 1eb0b82cd788, Add auto-update to it.

Comment 16

[(not currently active) Ted Mielczarek]

You're probably aware, but you'll have to file a Server Ops bug to get the production instance updated with your changes.

Comment 17

[Serge Gautherie (:sgautherie)]

Noted, yes, but I'm not yet done with the cleanups ... then I'll fix the actual bug issue(s) ;->

I've got WIP patches, but I want to do only one (for all) schema update at the end...

Comment 18

[timeless]

not to be a nag, but is this really 'tinderbox', or do you guys want your own component?

Comment 19

[Reed Loden [:reed]]

(In reply to comment #18)
> not to be a nag, but is this really 'tinderbox', or do you guys want your own
> component?

New component would be nice.

Comment 20

[(not currently active) Ted Mielczarek]

It's not, but it doesn't really need its own component either. We could just refile it in Testing:General or something to avoid creating a new component for one bug.

Comment 21

[timeless]

ok, deporting, sorry about the paperwork, i'm sure this is an interesting/important topic, but reed/i pay attention to tinderbox and paperwork, so we like having our paperwork current and clean.

Comment 22

[Serge Gautherie (:sgautherie)]

http://hg.mozilla.org/users/tmielczarek_mozilla.com/unittest-log/rev/7225b08144a5
(Lv1) Improve build status handling. Be a little stricter on log parsing.