826084 - Add ability for developers to "push to eideticker"

Status: RESOLVED WONTFIX

(Testing Graveyard :: Eideticker, defect)

Description

[William Lachance (:wlach)]

It would be useful for developers to be able to get eideticker results by pushing a specific revision of the tree and/or by specifying a revision of the tree to test. This would help with:

* Bisection of which revision resulted in a performance regression
* Determining whether a set of changes result in a performance change

This is similar to bug 747515, where we talked about creating a web interface for developers to test with (and get results). This would probably be more useful though.

There are two components to this bug that come to me right off the bat:

(1) The component that allows submission of a specific tree/revision to be tested (no idea how this would work)
(2) The presentation of the results (some work on this is done, but it's tied into the dashboard code in a strange way... it would be great if we could make things more modular)

If I were to work on this, I would probably start with (2) and get some benefit from it (in the form of it being easier to run/present individual eideticker runs). I'm not sure how (1) would work... there may be other efforts in the a-team and releng that we could plug into here.

Comment 1

[William Lachance (:wlach)]

I'm not sure if I'd agree that this bug necessarily "blocks" those bugs in an absolute sense. There are ways of running eideticker manually that would get us what we would need, albeit they are time consuming and only I know them.

One possible compromise might be to make eideticker's standard console mode be able to get intermediate builds from inbound or central (in addition to the current nightlies). This would still require ssh access to the eideticker machine and some command line steps, but it would (1) be way less time consuming than searching through the build result server for timestamped builds and (2) be way easier to explain to developers working on bugs like those mentioned.

Comment 2

[William Lachance (:wlach)]

(In reply to William Lachance (:wlach) from comment #1)
> I'm not sure if I'd agree that this bug necessarily "blocks" those bugs in
> an absolute sense. There are ways of running eideticker manually that would
> get us what we would need, albeit they are time consuming and only I know
> them.
> 
> One possible compromise might be to make eideticker's standard console mode
> be able to get intermediate builds from inbound or central (in addition to
> the current nightlies). This would still require ssh access to the
> eideticker machine and some command line steps, but it would (1) be way less
> time consuming than searching through the build result server for
> timestamped builds and (2) be way easier to explain to developers working on
> bugs like those mentioned.

Hmm, apparently that was such a good idea that I filed a bug for it:

https://bugzilla.mozilla.org/show_bug.cgi?id=756428

Note that the comments on mozregression may be obsolete. I think I found it wasn't working for eideticker's needs, but I can't remember why.

Comment 3

[William Lachance (:wlach)]

(In reply to William Lachance (:wlach) from comment #1)

> One possible compromise might be to make eideticker's standard console mode
> be able to get intermediate builds from inbound or central (in addition to
> the current nightlies). This would still require ssh access to the
> eideticker machine and some command line steps, but it would (1) be way less
> time consuming than searching through the build result server for
> timestamped builds and (2) be way easier to explain to developers working on
> bugs like those mentioned.

I filed bug 838208 for this.

Comment 4

[BMO Automation]

Eideticker has been discontinued, see bug 1361056