Closed Bug 720901 Opened 12 years ago Closed 11 years ago

Set up continuous integration (buildbot) for Jetperf

Categories

(Add-on SDK Graveyard :: General, defect, P2)

Product:
Add-on SDK Graveyard
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: k0scist, Unassigned)

References

Details

(Whiteboard: [jetpack+talos]) 

When the general form of the JetPerf software (currently at http://k0s.org/mozilla/hg/jetperf ) is approved, we should add this to buildbot. 

Currently, jetpack code for buildbot lives in:
- http://hg.mozilla.org/build/tools/file/tip/buildfarm/utils/run_jetpack.py
- http://hg.mozilla.org/build/buildbotcustom/file/tip/misc.py#l3325
Note: this is (and should be) run on changes to  http://hg.mozilla.org/projects/addon-sdk
it sounds like we just need to add a talos test suite to buildbot-configs and have it run only on the addon-sdk branch.  Is there something I am missing here?
I think so?  AFAIK, there isn't a addon-sdk branch of m-c; we want to run on changes to the addon-sdk repo (with a firefox binary).  See links in comment 1. The addon-sdk (again, AFAIK) isn't in m-c.
Depends on: 725012
Current CI source is here:

https://tbpl.mozilla.org/?usetinderbox=1&tree=Jetpack
So I am trying to reproduce the existing TBPL results locally using hg.m.o/build/tools run_jetpack.py.  However, I get timeout errors:

time python run_jetpack.py -p fedora -t http://hg.mozilla.org/projects/addon-sdk/archive/tip.tar.bz2 -b mozilla-central -f 'ftp://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/%(branch)s-%(platform)s' -e linux-i686.tar.bz2

<snip/>
info: executing: fallback
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: tooltip fallbacks to label
info: pass: widget in last test property cleaned itself up
info: executing: tooltip
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: tooltip gets updated
info: pass: widget in last test property cleaned itself up
info: executing: allow
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: scripts are evaluated by default
info: pass: widget in last test property cleaned itself up
info: executing: allow
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: scripts are evaluated when we want to
info: pass: widget in last test property cleaned itself up
info: executing: allow
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: scripts aren't evaluated when explicitly blocked it
info: pass: widget in last test property cleaned itself up
info: executing: first
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: 2nd window has correct number of child elements after first widget
info: executing: second
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: 2nd window has correct number of child elements after second widget
info: pass: 2nd window has correct number of child elements after first destroy
info: pass: 2nd window has correct number of child elements after second destroy
*** e = [Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: chrome://browser/content/utilityOverlay.js :: getShellService :: line 364"  data: no]
info: pass: widget in last test property cleaned itself up
console: [JavaScript Warning: "reference to undefined property this._value" {file: "chrome://browser/content/urlbarBindings.xml" line: 669}]
console: [JavaScript Warning: "setting a property that has only a getter" {file: "chrome://browser/content/browser.js" line: 3252}]
info: executing: first
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: Got first widget view
info: pass: Got second widget view
info: pass: emit on a destroyed view should throw
info: pass: widget in last test property cleaned itself up
info: executing: first
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: 2nd window has correct number of child elements after widget creation
info: pass: 2nd window has correct number of child elements after widget destroy
info: pass: 1st window has an hidden addon-bar
info: pass: 2nd window has an hidden addon-bar
*** e = [Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: chrome://browser/content/utilityOverlay.js :: getShellService :: line 364"  data: no]
info: pass: widget in last test property cleaned itself up
console: [JavaScript Warning: "setting a property that has only a getter" {file: "chrome://browser/content/browser.js" line: 3252}]
info: executing: text
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: a == b == 200
info: pass: a == b == 200
info: pass: a == b == 200
info: pass: a == b == 300
info: pass: a == b == 300
info: pass: widget in last test property cleaned itself up
info: executing: click-content
info: pass: widget node at index
info: pass: widget element is correct
info: pass: widget width is correct
info: pass: container has correct number of child elements
info: pass: found content
info: pass: content is iframe or image
info: pass: right click wasn't sent to click handler
info: pass: widget in last test property cleaned itself up
info: executing 'test-widget.testPanelWidget1'
info: pass: Menu separator should be present
info: pass: Menu separator should be a <menuseparator>
info: pass: Overflow submenu should be absent
error: TEST FAILED: test-widget.testPanelWidget1 (failure)
error: fail: Item should be present in top-level menu
info: Traceback (most recent call last):
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/timer.js", line 38, in notify
    this.callback.apply(null, this.arguments);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1876, in null
    callback.call(self, evt);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 2016, in null
    onshownCallback.call(this, popup);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 573, in null
    test.checkMenu([item], [], []);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1743, in null
    this.checkPresentItems(presentItems);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1806, in null
    recurse.call(this, this.contextMenuPopup, presentItems, true);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1803, in recurse
    }, this);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1792, in null
    "Item should be present in top-level menu");
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 129, in assert
    this.fail(message);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 67, in fail
    this.console.trace();
info: pass: Item DOM element should be a xul:menuitem
info: pass: Item should have correct title
info: pass: Item should not have image
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
info: pass: Item label should be < next item's label
error: fail: Item label should be < next item's label
info: Traceback (most recent call last):
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/timer.js", line 38, in notify
    this.callback.apply(null, this.arguments);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1876, in null
    callback.call(self, evt);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 2016, in null
    onshownCallback.call(this, popup);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 573, in null
    test.checkMenu([item], [], []);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1746, in null
    this.checkSort(presentItems);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1855, in null
    "Item label should be < next item's label");
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 129, in assert
    this.fail(message);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 67, in fail
    this.console.trace();
error: fail: Item label should be < next item's label
info: Traceback (most recent call last):
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/timer.js", line 38, in notify
    this.callback.apply(null, this.arguments);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1876, in null
    callback.call(self, evt);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 2016, in null
    onshownCallback.call(this, popup);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 573, in null
    test.checkMenu([item], [], []);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1746, in null
    this.checkSort(presentItems);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1855, in null
    "Item label should be < next item's label");
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 129, in assert
    this.fail(message);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 67, in fail
    this.console.trace();
error: fail: The first item in sorted order should have the first element in sorted order (15 != 1)
info: Traceback (most recent call last):
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/timer.js", line 38, in notify
    this.callback.apply(null, this.arguments);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1876, in null
    callback.call(self, evt);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 2016, in null
    onshownCallback.call(this, popup);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 573, in null
    test.checkMenu([item], [], []);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1746, in null
    this.checkSort(presentItems);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1860, in null
    "The first item in sorted order should have the " +
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 160, in assertEqual
    this.fail(message);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 67, in fail
    this.console.trace();
info: pass: about:buildconfig page-mod to be executed
info: executing 'test-widget.testPanelWidget2'
info: pass: widget.panel must be a Panel object
info: executing 'test-widget.testPanelWidget3'
info: pass: onClick called on click for widget with both panel and onClick
error: TEST FAILED: test-widget.testPanelWidget3 (failure)
error: fail: Menu separator should be absent
info: Traceback (most recent call last):
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/timer.js", line 38, in notify
    this.callback.apply(null, this.arguments);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1876, in null
    callback.call(self, evt);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 2016, in null
    onshownCallback.call(this, popup);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 581, in null
    test.checkMenu([], [], [item]);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1741, in null
    this.checkSeparator(presentItems);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1835, in null
    this.test.assert(!sep || sep.hidden, "Menu separator should be absent");
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 129, in assert
    this.fail(message);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 67, in fail
    this.console.trace();
info: pass: Overflow submenu should be absent
error: fail: Item should be removed from top-level menu
info: Traceback (most recent call last):
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/timer.js", line 38, in notify
    this.callback.apply(null, this.arguments);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1876, in null
    callback.call(self, evt);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 2016, in null
    onshownCallback.call(this, popup);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 581, in null
    test.checkMenu([], [], [item]);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1745, in null
    this.checkRemovedItems(removedItems);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1815, in null
    this.test.assert(!elt, "Item should be removed from top-level menu");
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 129, in assert
    this.fail(message);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 67, in fail
    this.console.trace();
error: fail: Item should be removed from overflow submenu
info: Traceback (most recent call last):
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/timer.js", line 38, in notify
    this.callback.apply(null, this.arguments);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1876, in null
    callback.call(self, evt);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 2016, in null
    onshownCallback.call(this, popup);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 581, in null
    test.checkMenu([], [], [item]);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1745, in null
    this.checkRemovedItems(removedItems);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/addon-kit/tests/test-context-menu.js", line 1820, in null
    this.test.assert(!elt, "Item should be removed from overflow submenu");
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 129, in assert
    this.fail(message);
  File "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/unit-test.js", line 67, in fail
    this.console.trace();
error: An exception occurred.
Traceback (most recent call last):
  File "chrome://browser/content/tabbrowser.xml", line 1493, in removeTab
    if (!this._beginRemoveTab(aTab, false, null, true))
  File "chrome://browser/content/tabbrowser.xml", line 1548, in _beginRemoveTab
    let ds = browser.docShell;
TypeError: browser is null
info: executing 'test-widget.testWidgetMessaging'
info: pass: a == b == "foo"
info: executing 'test-widget.testWidgetMove'
info: pass: Got first ready event
info: pass: Got second ready event
info: pass: Got message after node move
info: executing 'test-widget.testWidgetViews'
info: pass: WidgetView created
info: pass: Got message in WidgetView
info: pass: WidgetView destroyed
info: executing 'test-widget.testWidgetViewsCustomEvents'
info: pass: event argument is valid on WidgetView
info: pass: event argument is valid on Widget
info: executing 'test-widget.testWidgetViewsTooltip'
info: pass: view tooltip defaults to base widget label
info: pass: tooltip defaults to base widget label
info: executing 'test-widget.testWidgetViewsUIEvents'
info: pass: Got attach event
info: pass: event first argument is equal to the WidgetView
info: pass: widget.getView return the same WidgetView
info: executing 'test-widget.testWidgetWithPound'
console: [JavaScript Warning: "reference to undefined property obj.tagName" {file: "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/cuddlefish.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/sandbox.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/data/content-proxy.js" line: 531}]
console: [JavaScript Warning: "reference to undefined property obj.tagName" {file: "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/cuddlefish.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/sandbox.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/data/content-proxy.js" line: 531}]
info: pass: found widget node in the front-end
info: pass: content updated to pound?
info: executing 'test-clipboard.testNotInFlavor'
info: pass: assertion successful
info: pass: a == b == null
info: executing 'test-clipboard.testWithFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "html"
info: pass: a == b == "hello there"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: executing 'test-clipboard.testWithNoFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: executing 'test-clipboard.testWithRedundantFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: executing 'test-passwords.test addon associated credentials'
info: pass: `addon:` uri is used for add-on credentials
info: pass: username matches
info: pass: password matches
info: pass: realm matches
info: pass: `formSubmitURL` is `null` for add-on credentials
info: pass: usernameField is empty
info: pass: passwordField is empty
info: pass: credential is removed
info: executing 'test-passwords.test can't store same login twice'
info: pass: credential saved
console: [JavaScript Error: "uncaught exception: This login already exists." {file: "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/cuddlefish.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/passwords/utils.js" line: 90}]
info: pass: re-saving credential failed
info: pass: credential was removed
info: executing 'test-passwords.test exceptions in onComplete are reported'
info: pass: Error thrown is reported
info: executing 'test-passwords.test onComplete is optional'
info: pass: exception is not thrown if `onComplete is missing
info: executing 'test-passwords.test remove fails if no login found'
console: [JavaScript Error: "uncaught exception: No matching logins" {file: "resource:///components/nsLoginManager.js" line: 438}]
console: [JavaScript Error: "uncaught exception: [Exception... "'No matching logins' when calling method: [nsILoginManagerStorage::removeLogin]"  nsresult: "0x8057001e (NS_ERROR_XPC_JS_THREW_STRING)"  location: "JS frame :: resource:///components/nsLoginManager.js :: <TOP_LEVEL> :: line 438"  data: no]" {file: "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/cuddlefish.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/passwords/utils.js" line: 100}]
info: pass: can't remove unstored credentials
info: executing 'test-passwords.test site authentication credentials'
info: pass: url matches
info: pass: username matches
info: pass: password matches
info: pass: realm matches
info: pass: formSubmitURL is null
info: pass: usernameField is empty
info: pass: passwordField is empty
info: pass: credential is removed
info: executing 'test-passwords.test store requires `password` field'
console: [JavaScript Error: "uncaught exception: Can't add a login with a null or empty password." {file: "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/cuddlefish.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/passwords/utils.js" line: 90}]
info: pass: '`password` is required
info: executing 'test-passwords.test store requires `realm` field'
console: [JavaScript Error: "uncaught exception: Can't add a login without a httpRealm or formSubmitURL." {file: "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/cuddlefish.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/passwords/utils.js" line: 90}]
info: pass: '`realm` is required
info: executing 'test-passwords.test store requires `username` field'
console: [JavaScript Error: "uncaught exception: Can't add a login with a null username." {file: "resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/cuddlefish.js -> resource://46cc97a8-509d-4ae1-97d1-cba928ab62a6-at-jetpack/api-utils/lib/passwords/utils.js" line: 90}]
info: pass: '`username` is required
info: executing 'test-passwords.test web page associated credentials'
info: pass: url matches
info: pass: username matches
info: pass: password matches
info: pass: realm is null
info: pass: formSubmitURL matches
info: pass: usernameField is matches
info: pass: passwordField matches
info: pass: credential is removed
info: executing 'test-timers.testTimeout'
info: pass: timers.setTimeout works
info: executing 'test-addon-page.test that add-on page has no chrome'
info: pass: chrome is visible for non addon page
info: pass: chrome is not visible for addon page
info: pass: chrome is visible again
info: executing 'test-addon-page.test that add-on pages is closed on unload'
info: pass: add-on page tabs are closed on unload
info: executing 'test-simple-storage.testEmpty'
info: pass: Store file should not exist
info: executing 'test-simple-storage.testMalformed'
info: pass: Malformed storage should cause root to be empty
info: executing 'test-simple-storage.testQuotaExceededHandle'
info: pass: OverQuota was emitted as expected
info: pass: `this` should be simple storage
info: pass: 2
info: pass: x value should be correct
info: pass: y value should be correct
info: executing 'test-simple-storage.testQuotaExceededNoHandle'
info: pass: Value should have persisted: foo
info: pass: OverQuota emitted as expected
info: pass: Over-quota value should not have been written, old value should have persisted: foo
info: executing 'test-simple-storage.testQuotaUsage'
info: pass: quotaUsage should be correct
info: pass: quotaUsage should be correct
info: pass: quotaUsage should be correct
info: executing 'test-simple-storage.testSetGet'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testSetGetRootArray'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testSetGetRootBool'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testSetGetRootFunction'
info: pass: Setting storage to a function should fail
info: executing 'test-simple-storage.testSetGetRootNull'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testSetGetRootNumber'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testSetGetRootObject'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testSetGetRootString'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testSetGetRootUndefined'
info: pass: Setting storage to undefined should fail
info: executing 'test-simple-storage.testSetNoSetRead'
info: pass: Value read should be value set
info: pass: Store file should exist
info: pass: Value should persist
info: executing 'test-simple-storage.testUninstall'
info: pass: Store file should exist
info: pass: Store file should be removed
info: executing 'test-page-worker.testAllowScript'
info: pass: Script runs when allowed to do so.
info: executing 'test-page-worker.testAllowScriptDefault'
info: pass: Script is allowed to run by default.
info: executing 'test-page-worker.testAutoDestructor'
info: pass: Page correctly unloaded.
info: executing 'test-page-worker.testConstructorAndDestructor'
info: pass: Page 1 and page 2 should be different objects.
info: pass: page1 correctly unloaded.
info: pass: page2 correctly unloaded.
info: executing 'test-page-worker.testContentAndAllowGettersAndSetters'
info: pass: Correct value expected for allowScript - 3
info: pass: Correct content expected
info: pass: Correct value expected for allowScript - 3
info: pass: Correct content expected
info: pass: Correct value expected for allowScript - g
info: pass: Correct content expected
info: pass: Correct value expected for allowScript - g
info: pass: Correct content expected
info: pass: Correct value expected for allowScript - 4
info: pass: Correct content expected
info: executing 'test-page-worker.testLoadContentPage'
info: pass: Correct page title accessed directly
info: pass: <p> can be accessed directly
info: pass: Correct text node expected
info: pass: <div> can be accessed directly
info: pass: Correct text node expected
info: executing 'test-page-worker.testMultipleDestroys'
info: pass: Multiple destroys should not cause an error
info: executing 'test-page-worker.testMultipleOnMessageCallbacks'
info: pass: All callbacks were called, in order.
info: executing 'test-page-worker.testOnMessageCallback'
info: pass: onMessage callback called
info: executing 'test-page-worker.testPageProperties'
info: pass: contentURL property is defined on page.
info: pass: allow property is defined on page.
info: pass: contentScriptFile property is defined on page.
info: pass: contentScript property is defined on page.
info: pass: contentScriptWhen property is defined on page.
info: pass: on property is defined on page.
info: pass: postMessage property is defined on page.
info: pass: removeListener property is defined on page.
info: pass: postMessage doesn't throw exception on page.
info: executing 'test-page-worker.testPingPong'
info: pass: pong
info: executing 'test-page-worker.testSimplePageCreation'
info: pass: Page Worker should start with a blank page by default
info: pass: The 'this' object is the page itself.
info: executing 'test-page-worker.testValidateOptions'
info: pass: Validation correctly denied a non-URL contentURL
info: pass: Validation correctly denied a non-function onMessage.
info: pass: Options validation is working.
info: executing 'test-page-worker.testWrappedDOM'
info: pass: getElementById from content script is the native one
info: pass: scrollTo from content script is the native one
info: executing 'test-tabs.test ready event on new window tab'
*** e = [Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: chrome://browser/content/utilityOverlay.js :: getShellService :: line 364"  data: no]
info: pass: ready event was emitted
console: [JavaScript Warning: "setting a property that has only a getter" {file: "chrome://browser/content/browser.js" line: 3252}]
info: executing 'test-tabs.test window focus changes active tab'
console: [JavaScript Error: "no element found" {file: "data:text/xml,%3Cbindings%20xmlns%3D%22http%3A%2F%2Fwww.mozilla.org%2Fxbl%22%3E%3Cbinding%20id%3D%22id%22%20extends%3D%22chrome%3A%2F%2Fglobal%2Fcontent%2Fbindings%2Fpopup.xml%23arrowpanel%22%3E%3Cresources%3E%3Cstylesheet%20src%3D%22data%3Atext%2Fcss%2C.panel-inner-arrowcontent%252C%2520.panel-arrowcontent%2520%257Bpadding%253A%25200%253B%257D%22%2F%3E%3C%2Fresources%3E%3C%2Fbinding%3E%3C%2Fbindings%3E" line: 1}]
console: [JavaScript Warning: "An XBL file is malformed. Did you forget the XBL namespace on the bindings tag?" {file: "data:text/xml,%3Cbindings%20xmlns%3D%22http%3A%2F%2Fwww.mozilla.org%2Fxbl%22%3E%3Cbinding%20id%3D%22id%22%20extends%3D%22chrome%3A%2F%2Fglobal%2Fcontent%2Fbindings%2Fpopup.xml%23arrowpanel%22%3E%3Cresources%3E%3Cstylesheet%20src%3D%22data%3Atext%2Fcss%2C.panel-inner-arrowcontent%252C%2520.panel-arrowcontent%2520%257Bpadding%253A%25200%253B%257D%22%2F%3E%3C%2Fresources%3E%3C%2Fbinding%3E%3C%2Fbindings%3E" line: 0}]
*** e = [Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: chrome://browser/content/utilityOverlay.js :: getShellService :: line 364"  data: no]
*** e = [Exception... "Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]"  nsresult: "0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE)"  location: "JS frame :: chrome://browser/content/utilityOverlay.js :: getShellService :: line 364"  data: no]
info: pass: activate was called on windows focus change.
Traceback (most recent call last):
  File "bin/cfx", line 33, in <module>
    cuddlefish.run()
  File "/home/jhammel/mozilla/src/jetperf/src/tools/buildfarm/utils/addonsdk-poller/addon-sdk-6dd268ad1991/python-lib/cuddlefish/__init__.py", line 517, in run
    test_all(env_root, defaults=options.__dict__)
  File "/home/jhammel/mozilla/src/jetperf/src/tools/buildfarm/utils/addonsdk-poller/addon-sdk-6dd268ad1991/python-lib/cuddlefish/__init__.py", line 345, in test_all
    test_all_packages(env_root, defaults)
  File "/home/jhammel/mozilla/src/jetperf/src/tools/buildfarm/utils/addonsdk-poller/addon-sdk-6dd268ad1991/python-lib/cuddlefish/__init__.py", line 408, in test_all_packages
    env_root=env_root)
  File "/home/jhammel/mozilla/src/jetperf/src/tools/buildfarm/utils/addonsdk-poller/addon-sdk-6dd268ad1991/python-lib/cuddlefish/__init__.py", line 783, in run
    mobile_app_name=options.mobile_app_name)
  File "/home/jhammel/mozilla/src/jetperf/src/tools/buildfarm/utils/addonsdk-poller/addon-sdk-6dd268ad1991/python-lib/cuddlefish/runner.py", line 663, in run_app
    OUTPUT_TIMEOUT)
Exception: Test output exceeded timeout (60s).
TinderboxPrint:Jetpack<br />7/0


real	24m57.626s
user	0m18.285s
sys	0m3.736s

Is it possible I made a mistake running?  I will endeavor to run overnight.
There are a few things to decide here:

* Do we want the new tests to run out of run_jetpack.py ?  (Talking to :lukas, I think yes) 

* do we want to run jetperf for m-c changes or just http://hg.mozilla.org/projects/addon-sdk changes? (see http://hg.mozilla.org/build/tools/file/ba0475a066d8/buildfarm/utils/run_jetpack.py#l93 and be careful; IMHO we want to only run on jetpack changes, especially since m-c jetpack is permared; no point wasting even more build time)

* are we content conflating the jetperf results with the m-c m-a m-b on https://tbpl.mozilla.org/?usetinderbox=1&tree=Jetpack ?  If not this will involve more clever planning
It is also worth noting that m-c on https://tbpl.mozilla.org/?usetinderbox=1&tree=Jetpack is perma-orange.  We should probably green this up before landing new tests or people won't pay attention to them
(In reply to Jeff Hammel [:jhammel] from comment #7)
> It is also worth noting that m-c on
> https://tbpl.mozilla.org/?usetinderbox=1&tree=Jetpack is perma-orange.  We
> should probably green this up before landing new tests or people won't pay
> attention to them

From what I read in bug 725954 and the pull request and IRC, the tests should be greening up as soon as that commit gets mirrored to the mercurial mirror. Sadly, bug 727626 is not letting that happen.
(In reply to Jeff Hammel [:jhammel] from comment #6)
> There are a few things to decide here:
> 
> * Do we want the new tests to run out of run_jetpack.py ?  (Talking to
> :lukas, I think yes) 
> 
yes, we can run them out of there but i think jetpack_perf.py should be its own script that is called in run_jetpack.py via a flag passed in when this builder calls the script that way we can turn perf on/off per branch if needed and it will make it easier to separate out later when we move to config-driven builds in the brilliant log-uploading future
(In reply to Lukas Blakk [:lsblakk] from comment #9)
> (In reply to Jeff Hammel [:jhammel] from comment #6)
> > There are a few things to decide here:
> > 
> > * Do we want the new tests to run out of run_jetpack.py ?  (Talking to
> > :lukas, I think yes) 
> > 
> yes, we can run them out of there but i think jetpack_perf.py should be its
> own script that is called in run_jetpack.py via a flag passed in when this
> builder calls the script that way we can turn perf on/off per branch if
> needed and it will make it easier to separate out later when we move to
> config-driven builds in the brilliant log-uploading future

The jetperf script lives here:
http://k0s.org/mozilla/hg/jetperf/file/tip/jetperf/jetperf.py

I am not sure how CI-ready it is; it relies on mozharness and could be mirrored to hg.m.o/build/mozharness scripts directory. It also relies on Talos.
wrt comment 5, I was able to get an overnight run to complete:

2 of 2 tests passed.
Total time: 4.793178 seconds
Program terminated successfully.
Some package tests were unsuccessful.
Some tests were unsuccessful.
TinderboxPrint:Jetpack<br />5850/713


real	31m47.493s
user	3m17.760s
sys	0m10.521s

I'm not sure why this takes so long
Currently JetPerf is its own package that depends (in the install_requires sense) on mozharness and talos: http://k0s.org/mozilla/hg/jetperf/file/0361eea4b40c 

The active script is http://k0s.org/mozilla/hg/jetperf/file/0361eea4b40c/jetperf/jetperf.py

I'm not sure what the actionable way of getting this on to the appropriate slaves.  The way that it is currently packaged assumes that either setup.py will be run or that mozharness and talos is installed.

I am not sure if mozharness is on these machines or is actionably runnable from http://hg.mozilla.org/build/tools/file/ba0475a066d8/buildfarm/utils/run_jetpack.py .  That will have to be the case.  If it is not presently true a bug should be filed about that.

I am also not sure if we want to assume that talos is installed on the slaves.  If we don't want to do this we should come up with a strategy to do so.  I'm not in charge of the build slaves so this is really up to releng.  :lukas, :aki, can you comment here?
(In reply to Jeff Hammel [:jhammel] from comment #12)
> Currently JetPerf is its own package that depends (in the install_requires
> sense) on mozharness and talos:
> http://k0s.org/mozilla/hg/jetperf/file/0361eea4b40c 
> 
> The active script is
> http://k0s.org/mozilla/hg/jetperf/file/0361eea4b40c/jetperf/jetperf.py
> 
> I'm not sure what the actionable way of getting this on to the appropriate
> slaves.  The way that it is currently packaged assumes that either setup.py
> will be run or that mozharness and talos is installed.

Mozharness is cloned, then scripts are run from scripts/.
So, step 1: modify the script to be reviewable and landable as a mozharness script that requires no virtualenv to run.

The idea is that the mozharness script is the original step; it will install talos and jetperf and whatever else needs to be run as an action.
Depends on: 729205
Thanks for the quick response, Aki.  I've noted the work as bug 729205
Depends on: 753526
Depends on: 753530
Depends on: 759396
I'm currently getting the below from the talos jetperf run:

13:20:21     INFO -  NOISE: [JavaScript Warning: "WARN addons.xpi: Exception running bootstrap method startup on widget-addon@jetpack: [Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsILocalFile.initWithPath]"  nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)"  location: "JS frame :: resource://widget-addon-at-jetpack/api-utils/lib/loader.js -> resource://widget-addon-at-jetpack/api-utils/lib/system.js :: openFile :: line 28"  data: no]" {file: "resource://widget-addon-at-jetpack/api-utils/lib/loader.js -> resource://widget-addon-at-jetpack/api-utils/lib/system.js" line: 28}]
13:20:21     INFO -  NOISE: [JavaScript Warning: "WARN addons.xpi: Exception running bootstrap method startup on empty-addon@jetpack: [Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsILocalFile.initWithPath]"  nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)"  location: "JS frame :: resource://empty-addon-at-jetpack/api-utils/lib/loader.js -> resource://empty-addon-at-jetpack/api-utils/lib/system.js :: openFile :: line 28"  data: no]" {file: "resource://empty-addon-at-jetpack/api-utils/lib/loader.js -> resource://empty-addon-at-jetpack/api-utils/lib/system.js" line: 28}]

Not sure if this is a problem or expected or what, but needs to be figured out
Depends on: 767117
So I'm unsure what the status is here.  It appears that all of the issues have been worked out at this point in time.  What needs to be done?  Do I need to do it?
I'm also unsure of the staging process.  Should I endeavor to setup buildbotcustom to ensure that things work locally?
The closer this is to running, the easier it will be to find someone with time to push it over the finish line.

Have you verified that this runs on all platforms with the checked in config files?
Is all that needs doing scheduling jetperf runs in buildbot, or do we have to debug runtime issues as well?
I'm mostly asking "how to test on all platforms" in a production-oriented way at this point (where all platforms == mac, linux, win [whatever versions] at this point; I don't have much interest in android until the rest has proven stable). The checked in production config, http://hg.mozilla.org/build/mozharness/file/326fdd7c546f/configs/jetperf/prod_config.py, is tailored for build machines so i'm not entirely sure how to check it.

The jetperf.py script does work on linux without issues.  I usually run via 

python scripts/jetperf.py --config-file configs/jetperf/test_config --binary-path `which firefox`
Does ateam still have a staging environment?
If not, buildduty can give you access to the missing platforms.
ABICT, our staging environment is currently down (if it still works at all) and it is very out of date.  Whether we want to resurrect it or not is an open discussion.
(In reply to Aki Sasaki [:aki] from comment #20)
> Does ateam still have a staging environment?
> If not, buildduty can give you access to the missing platforms.

The Ateam environment was never put into puppet (we tried, years ago, and failed, the sordid history is only worth repeating over copious amounts of beer), so even if it is running, it is so far out of date that testing on it is worthless. Let's test this in releng staging, please.
Ok.
If Jeff's willing, buildduty can give him access to test machines.

Still, any testing on windows and mac, even on non-puppeted machines, will give us a better idea of how things will work than none.
Depends on: 787479
Depends on: 786885
Depends on: 795236
Per :jgriffin, this is no longer in our plans.
If that's not accurate, please let me know.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.