1454466 - Move raptor in-tree

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement)

Description

[Robert Wood [:rwood]]

We've been developing Raptor in Github [1] but we need to move it in-tree now, in order to proceed to the next development phase (integration with mach, mitmproxy, try/production/mozharness) etc.

[1] https://github.com/rwood-moz/raptor

Comment 1

[Robert Wood [:rwood]]

Attachment: Bug 1454466 - Move raptor in-tree;

Review commit: https://reviewboard.mozilla.org/r/237304/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/237304/

Comment 2

[Code Review Bot [:reviewbot]]

Comment on attachment 8968609 [details]
Bug 1454466 - Move raptor in-tree;

https://reviewboard.mozilla.org/r/237304/#review243106


Code analysis found 192 defects in this patch (only the first 30 are reported here):
 - 192 defects found by mozlint

You can run this analysis locally with:
 - `./mach lint path/to/file` (JS/Python)


If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx


::: testing/raptor/raptor/control_server.py:1
(Diff revision 1)
> +# This Source Code Form is subject to the terms of the Mozilla Public

Error: Missing from __future__ import absolute_import [py2: require absolute_import]

::: testing/raptor/raptor/gen_test_config.py:1
(Diff revision 1)
> +# This Source Code Form is subject to the terms of the Mozilla Public

Error: Missing from __future__ import absolute_import [py2: require absolute_import]

::: testing/raptor/raptor/manifest.py:1
(Diff revision 1)
> +# This Source Code Form is subject to the terms of the Mozilla Public

Error: Missing from __future__ import absolute_import [py2: require absolute_import]

::: testing/raptor/raptor/playback/__init__.py:1
(Diff revision 1)
> +from mozlog import get_proxy_logger

Error: Missing from __future__ import absolute_import [py2: require absolute_import]

::: testing/raptor/raptor/playback/base.py:1
(Diff revision 1)
> +# abstract class for all playback tools

Error: Missing from __future__ import absolute_import [py2: require absolute_import]

::: testing/raptor/setup.py:1
(Diff revision 1)
> +import os

Error: Missing from __future__ import absolute_import [py2: require absolute_import]

::: testing/raptor/unittests/conftest.py:1
(Diff revision 1)
> +import json

Error: Missing from __future__ import absolute_import [py2: require absolute_import]

::: testing/raptor/webext/raptor/auto_gen_test_config.js:4
(Diff revision 1)
> +
> +    // this file is auto-generated by raptor, do not edit directly
> +    function getTestConfig() {
> +      return {'browser': 'firefox', 'test_settings_url': 'http://localhost:8000/raptor-firefox-tp7.json'};

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/auto_gen_test_config.js:4
(Diff revision 1)
> +
> +    // this file is auto-generated by raptor, do not edit directly
> +    function getTestConfig() {
> +      return {'browser': 'firefox', 'test_settings_url': 'http://localhost:8000/raptor-firefox-tp7.json'};

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/auto_gen_test_config.js:4
(Diff revision 1)
> +
> +    // this file is auto-generated by raptor, do not edit directly
> +    function getTestConfig() {
> +      return {'browser': 'firefox', 'test_settings_url': 'http://localhost:8000/raptor-firefox-tp7.json'};

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/auto_gen_test_config.js:4
(Diff revision 1)
> +
> +    // this file is auto-generated by raptor, do not edit directly
> +    function getTestConfig() {
> +      return {'browser': 'firefox', 'test_settings_url': 'http://localhost:8000/raptor-firefox-tp7.json'};

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/auto_gen_test_config.js:6
(Diff revision 1)
> +
> +    // this file is auto-generated by raptor, do not edit directly
> +    function getTestConfig() {
> +      return {'browser': 'firefox', 'test_settings_url': 'http://localhost:8000/raptor-firefox-tp7.json'};
> +    }
> +    

Error: Trailing spaces not allowed. [eslint: no-trailing-spaces]

::: testing/raptor/webext/raptor/auto_gen_test_config.js:6
(Diff revision 1)
> +
> +    // this file is auto-generated by raptor, do not edit directly
> +    function getTestConfig() {
> +      return {'browser': 'firefox', 'test_settings_url': 'http://localhost:8000/raptor-firefox-tp7.json'};
> +    }
> +    

Error: Newline required at end of file but not found. [eslint: eol-last]

::: testing/raptor/webext/raptor/benchmark-relay.js:8
(Diff revision 1)
> + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
> +
> +// receives result from benchmark and relays onto our background runner
> +
> +function receiveMessage(event) {
> +  console.log('received message!');

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/benchmark-relay.js:10
(Diff revision 1)
> +// receives result from benchmark and relays onto our background runner
> +
> +function receiveMessage(event) {
> +  console.log('received message!');
> +  console.log(event.origin);
> +  if (event.origin == 'http://localhost:8081') {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/benchmark-relay.js:18
(Diff revision 1)
> +}
> +
> +function sendResult(_type, _value) {
> +  // send result back to background runner script
> +  console.log("sending result back to runner: " + _type + " " + _value);
> +  chrome.runtime.sendMessage({"type": _type, "value": _value}, function(response) {

Error: 'chrome' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/benchmark-relay.js:23
(Diff revision 1)
> +  chrome.runtime.sendMessage({"type": _type, "value": _value}, function(response) {
> +    console.log(response.text);
> +  });
> +}
> +
> +window.addEventListener("message", receiveMessage, false);

Error: Addeventlistener's third parameter can be omitted when it's false. [eslint: mozilla/no-useless-parameters]

::: testing/raptor/webext/raptor/measure.js:31
(Diff revision 1)
> +
> +function contentHandler() {
> +  // retrieve test settings from local ext storage
> +  if (typeof(browser) !== "undefined") {
> +    // firefox, returns promise
> +    browser.storage.local.get("settings").then(function(item) {

Error: 'browser' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:32
(Diff revision 1)
> +function contentHandler() {
> +  // retrieve test settings from local ext storage
> +  if (typeof(browser) !== "undefined") {
> +    // firefox, returns promise
> +    browser.storage.local.get("settings").then(function(item) {
> +      setup(item['settings']);

Error: ["settings"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:32
(Diff revision 1)
> +function contentHandler() {
> +  // retrieve test settings from local ext storage
> +  if (typeof(browser) !== "undefined") {
> +    // firefox, returns promise
> +    browser.storage.local.get("settings").then(function(item) {
> +      setup(item['settings']);

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:36
(Diff revision 1)
> +    browser.storage.local.get("settings").then(function(item) {
> +      setup(item['settings']);
> +    });
> +  } else {
> +    // chrome, no promise so use callback
> +    chrome.storage.local.get("settings", function(item) {

Error: 'chrome' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:37
(Diff revision 1)
> +      setup(item['settings']);
> +    });
> +  } else {
> +    // chrome, no promise so use callback
> +    chrome.storage.local.get("settings", function(item) {
> +      setup(item['settings']);

Error: ["settings"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:37
(Diff revision 1)
> +      setup(item['settings']);
> +    });
> +  } else {
> +    // chrome, no promise so use callback
> +    chrome.storage.local.get("settings", function(item) {
> +      setup(item['settings']);

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:43
(Diff revision 1)
> +    });
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:43
(Diff revision 1)
> +    });
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];

Error: ["measure"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:43
(Diff revision 1)
> +    });
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];

Error: ["fnbpaint"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:43
(Diff revision 1)
> +    });
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:44
(Diff revision 1)
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];

Error: ["measure"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:44
(Diff revision 1)
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:44
(Diff revision 1)
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:44
(Diff revision 1)
> +  }
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];

Error: ["fcp"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:45
(Diff revision 1)
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:45
(Diff revision 1)
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {

Error: ["measure"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:45
(Diff revision 1)
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:45
(Diff revision 1)
> +}
> +
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {

Error: ["hero"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:47
(Diff revision 1)
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {
> +    getHero = true;
> +    heroesToCapture = settings['measure']['hero'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:47
(Diff revision 1)
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {
> +    getHero = true;
> +    heroesToCapture = settings['measure']['hero'];

Error: ["measure"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:47
(Diff revision 1)
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {
> +    getHero = true;
> +    heroesToCapture = settings['measure']['hero'];

Error: ["hero"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/measure.js:47
(Diff revision 1)
> +function setup(settings) {
> +  getFNBPaint = settings['measure']['fnbpaint'];
> +  getFCP = settings['measure']['fcp'];
> +  if (settings['measure']['hero'].length !== 0) {
> +    getHero = true;
> +    heroesToCapture = settings['measure']['hero'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:50
(Diff revision 1)
> +  if (settings['measure']['hero'].length !== 0) {
> +    getHero = true;
> +    heroesToCapture = settings['measure']['hero'];
> +  }
> +  if (getHero) {
> +    console.log('hero elements to measure: ' + heroesToCapture);

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:54
(Diff revision 1)
> +  if (getHero) {
> +    console.log('hero elements to measure: ' + heroesToCapture);
> +    measureHero();
> +  }
> +  if (getFNBPaint) {
> +    console.log('will be measuring fnbpaint');

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:58
(Diff revision 1)
> +  if (getFNBPaint) {
> +    console.log('will be measuring fnbpaint');
> +    measureFNBPaint();
> +  }
> +  if (getFCP) {
> +    console.log('will be measuring first-contentful-paint');

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:72
(Diff revision 1)
> +  console.log("found " + heroElementsFound.length + " hero elements in the page");
> +
> +  if (heroElementsFound) {
> +    function callbackHero(entries, observer) {
> +      entries.forEach(entry => {
> +        heroFound = entry.target.getAttribute('elementtiming');

Error: 'herofound' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:72
(Diff revision 1)
> +  console.log("found " + heroElementsFound.length + " hero elements in the page");
> +
> +  if (heroElementsFound) {
> +    function callbackHero(entries, observer) {
> +      entries.forEach(entry => {
> +        heroFound = entry.target.getAttribute('elementtiming');

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:74
(Diff revision 1)
> +  if (heroElementsFound) {
> +    function callbackHero(entries, observer) {
> +      entries.forEach(entry => {
> +        heroFound = entry.target.getAttribute('elementtiming');
> +        // mark the time now as when hero element received
> +        perfData.mark(heroFound);

Error: 'herofound' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:75
(Diff revision 1)
> +    function callbackHero(entries, observer) {
> +      entries.forEach(entry => {
> +        heroFound = entry.target.getAttribute('elementtiming');
> +        // mark the time now as when hero element received
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);

Error: 'herofound' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:77
(Diff revision 1)
> +        heroFound = entry.target.getAttribute('elementtiming');
> +        // mark the time now as when hero element received
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,

Error: Read-only global 'name' should not be modified. [eslint: no-native-reassign]

::: testing/raptor/webext/raptor/measure.js:77
(Diff revision 1)
> +        heroFound = entry.target.getAttribute('elementtiming');
> +        // mark the time now as when hero element received
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,

Error: Infix operators must be spaced. [eslint: space-infix-ops]

::: testing/raptor/webext/raptor/measure.js:78
(Diff revision 1)
> +        // mark the time now as when hero element received
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,

Error: 'startmark' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:78
(Diff revision 1)
> +        // mark the time now as when hero element received
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,

Error: Infix operators must be spaced. [eslint: space-infix-ops]

::: testing/raptor/webext/raptor/measure.js:79
(Diff revision 1)
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,
> +                         endMark=heroFound);

Error: 'endmark' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:79
(Diff revision 1)
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,
> +                         endMark=heroFound);

Error: Infix operators must be spaced. [eslint: space-infix-ops]

::: testing/raptor/webext/raptor/measure.js:79
(Diff revision 1)
> +        perfData.mark(heroFound);
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,
> +                         endMark=heroFound);

Error: 'herofound' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:80
(Diff revision 1)
> +        console.log("found hero:" + heroFound);
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,
> +                         endMark=heroFound);
> +        perfResult = perfData.getEntriesByName(resultType);

Error: 'perfresult' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:81
(Diff revision 1)
> +        // calculcate result: performance.timing.fetchStart - time when we got hero element
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,
> +                         endMark=heroFound);
> +        perfResult = perfData.getEntriesByName(resultType);
> +        var _result = perfResult[0].duration;

Error: 'perfresult' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:82
(Diff revision 1)
> +        perfData.measure(name=resultType,
> +                         startMark=startMeasure,
> +                         endMark=heroFound);
> +        perfResult = perfData.getEntriesByName(resultType);
> +        var _result = perfResult[0].duration;
> +        var resultType = "hero:" + heroFound;

Error: 'herofound' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:95
(Diff revision 1)
> +    var options = {root: null, rootMargin: "0px", threshold: [1]};
> +    try {
> +      obs = new window.IntersectionObserver(callbackHero, options);
> +      heroElementsFound.forEach(function(el) {
> +        // if hero element is one we want to measure, add it to the observer
> +        if (heroesToCapture.indexOf(el.getAttribute('elementtiming')) > -1)

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:104
(Diff revision 1)
> +      console.log(err);
> +    }
> +  } else {
> +      console.log("couldn't find hero element");
> +  }
> +  return;

Error: Unnecessary return statement. [eslint: no-useless-return]

::: testing/raptor/webext/raptor/measure.js:117
(Diff revision 1)
> +    return;
> +  }
> +  if (x > 0) {
> +    console.log("got fnbpaint");
> +    gRetryCounter = 0;
> +    startTime = perfData.timing.fetchStart;

Error: 'starttime' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:118
(Diff revision 1)
> +  }
> +  if (x > 0) {
> +    console.log("got fnbpaint");
> +    gRetryCounter = 0;
> +    startTime = perfData.timing.fetchStart;
> +    sendResult("fnbpaint", x - startTime);

Error: 'starttime' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/measure.js:132
(Diff revision 1)
> +  }
> +}
> +
> +function measureFirstContentfulPaint() {
> +  // see https://developer.mozilla.org/en-US/docs/Web/API/PerformancePaintTiming
> +  var resultType = 'fcp';

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:135
(Diff revision 1)
> +function measureFirstContentfulPaint() {
> +  // see https://developer.mozilla.org/en-US/docs/Web/API/PerformancePaintTiming
> +  var resultType = 'fcp';
> +  var result = 0;
> +
> +  let performanceEntries = perfData.getEntriesByType('paint');

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/measure.js:161
(Diff revision 1)
> +}
> +
> +function sendResult(_type, _value) {
> +  // send result back to background runner script
> +  console.log("sending result back to runner: " + _type + " " + _value);
> +  chrome.runtime.sendMessage({"type": _type, "value": _value}, function(response) {

Error: 'chrome' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/runner.js:26
(Diff revision 1)
> +var pageCycles = 0;
> +var pageCycle = 0;
> +var pageCycleDelay = 1000;
> +var testURL;
> +var testTabID = 0;
> +var results = {'page': '', 'measurements': {}};

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:26
(Diff revision 1)
> +var pageCycles = 0;
> +var pageCycle = 0;
> +var pageCycleDelay = 1000;
> +var testURL;
> +var testTabID = 0;
> +var results = {'page': '', 'measurements': {}};

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:26
(Diff revision 1)
> +var pageCycles = 0;
> +var pageCycle = 0;
> +var pageCycleDelay = 1000;
> +var testURL;
> +var testTabID = 0;
> +var results = {'page': '', 'measurements': {}};

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:45
(Diff revision 1)
> +  return new Promise(resolve => {
> +
> +    fetch(settingsURL).then(function(response) {
> +      response.text().then(function(text) {
> +        console.log(text);
> +        settings = JSON.parse(text)['raptor-options'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:48
(Diff revision 1)
> +      response.text().then(function(text) {
> +        console.log(text);
> +        settings = JSON.parse(text)['raptor-options'];
> +
> +        // parse the test settings
> +        testType = settings['type'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:48
(Diff revision 1)
> +      response.text().then(function(text) {
> +        console.log(text);
> +        settings = JSON.parse(text)['raptor-options'];
> +
> +        // parse the test settings
> +        testType = settings['type'];

Error: ["type"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:49
(Diff revision 1)
> +        console.log(text);
> +        settings = JSON.parse(text)['raptor-options'];
> +
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:49
(Diff revision 1)
> +        console.log(text);
> +        settings = JSON.parse(text)['raptor-options'];
> +
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];

Error: ["page_cycles"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:50
(Diff revision 1)
> +        settings = JSON.parse(text)['raptor-options'];
> +
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:50
(Diff revision 1)
> +        settings = JSON.parse(text)['raptor-options'];
> +
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];

Error: ["test_url"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:51
(Diff revision 1)
> +
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:51
(Diff revision 1)
> +
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;

Error: ["page"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:52
(Diff revision 1)
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;
> +        results['type'] = testType;

Error: ["type"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:52
(Diff revision 1)
> +        // parse the test settings
> +        testType = settings['type'];
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;
> +        results['type'] = testType;

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:54
(Diff revision 1)
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;
> +        results['type'] = testType;
> +
> +        if (settings['page_timeout'] !== undefined) {

Error: ["page_timeout"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:54
(Diff revision 1)
> +        pageCycles = settings['page_cycles'];
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;
> +        results['type'] = testType;
> +
> +        if (settings['page_timeout'] !== undefined) {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:55
(Diff revision 1)
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;
> +        results['type'] = testType;
> +
> +        if (settings['page_timeout'] !== undefined) {
> +          pageTimeout = settings['page_timeout'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:55
(Diff revision 1)
> +        testURL = settings['test_url'];
> +        results['page'] = testURL;
> +        results['type'] = testType;
> +
> +        if (settings['page_timeout'] !== undefined) {
> +          pageTimeout = settings['page_timeout'];

Error: ["page_timeout"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:59
(Diff revision 1)
> +        if (settings['page_timeout'] !== undefined) {
> +          pageTimeout = settings['page_timeout'];
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:60
(Diff revision 1)
> +          pageTimeout = settings['page_timeout'];
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:60
(Diff revision 1)
> +          pageTimeout = settings['page_timeout'];
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];

Error: ["measure"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:60
(Diff revision 1)
> +          pageTimeout = settings['page_timeout'];
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];

Error: ["fnbpaint"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:60
(Diff revision 1)
> +          pageTimeout = settings['page_timeout'];
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:61
(Diff revision 1)
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:61
(Diff revision 1)
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];

Error: ["measure"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:61
(Diff revision 1)
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];

Error: ["fcp"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:61
(Diff revision 1)
> +        }
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:62
(Diff revision 1)
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];
> +          if (settings['measure']['hero'].length !== 0) {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:62
(Diff revision 1)
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];
> +          if (settings['measure']['hero'].length !== 0) {

Error: ["measure"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:62
(Diff revision 1)
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];
> +          if (settings['measure']['hero'].length !== 0) {

Error: ["hero"] is better written in dot notation. [eslint: dot-notation]

::: testing/raptor/webext/raptor/runner.js:62
(Diff revision 1)
> +        console.log("using page timeout (ms): " + pageTimeout);
> +
> +        if (testType == 'pageload') {
> +          getFNBPaint = settings['measure']['fnbpaint'];
> +          getFCP = settings['measure']['fcp'];
> +          if (settings['measure']['hero'].length !== 0) {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:68
(Diff revision 1)
> +            getHero = true;
> +          }
> +        }
> +
> +        // write options to storage that our content script needs to know
> +        if (browserName === 'firefox') {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:71
(Diff revision 1)
> +
> +        // write options to storage that our content script needs to know
> +        if (browserName === 'firefox') {
> +          ext.storage.local.clear().then(function() {
> +            ext.storage.local.set({settings}).then(function() {
> +              console.log('wrote settings to ext local storage');

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:78
(Diff revision 1)
> +            });
> +          });
> +        } else {
> +          ext.storage.local.clear(function() {
> +            ext.storage.local.set({settings}, function() {
> +              console.log('wrote settings to ext local storage');

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:90
(Diff revision 1)
> +  });
> +}
> +
> +function getBrowserInfo() {
> +  return new Promise(resolve => {
> +    if (browserName === 'firefox') {

Error: Strings must use doublequote. [eslint: quotes]

::: testing/raptor/webext/raptor/runner.js:91
(Diff revision 1)
> +}
> +
> +function getBrowserInfo() {
> +  return new Promise(resolve => {
> +    if (browserName === 'firefox') {
> +      ext = browser;

Error: 'browser' is not defined. [eslint: no-undef]

::: testing/raptor/webext/raptor/runner.js:92
(Diff revision 1)
> +
> +function getBrowserInfo() {
> +  return new Promise(resolve => {
> +    if (browserName === 'firefox') {
> +      ext = browser;
> +      var gettingInfo = browser.runtime.getBrowserInfo();

Error: 'browser' is not defined. [eslint: no-undef]

Comment 3

[Andrew Halberstadt [:ahal]]

Comment on attachment 8968609 [details]
Bug 1454466 - Move raptor in-tree;

https://reviewboard.mozilla.org/r/237304/#review243108

Thanks, this looks good! Nothing major, but a few minor issues so would like to do a re-review before landing.

::: testing/raptor/requirements.txt:1
(Diff revision 1)
> +mozrunner
> +mozprofile

You'll want to pin these versions as only the latest ones have the changes raptor will need. I recommend using the `~=` operator so future minor versions will also be accepted:

    mozrunner ~= 7.0
    mozprofile ~= 6.1
    
More info:
https://www.python.org/dev/peps/pep-0440/#compatible-release

I'd also recommend doing this for manifestparser because might as well :)

::: testing/raptor/setup.py:18
(Diff revision 1)
> +      version=version,
> +      description=description,
> +      url='https://github.com/rwood-moz/raptor',
> +      author='Mozilla',
> +      author_email='[e-mail]',
> +      license='MPL',

nit: MPL 2.0

::: testing/raptor/unittests/conftest.py:45
(Diff revision 1)
> +    try:
> +        from moztest.selftest import fixtures
> +    except ImportError:
> +        pytest.xfail(reason="A newer version of moztest is required")

You can remove this try/except now that we're in-tree.

::: testing/raptor/unittests/python.ini:1
(Diff revision 1)
> +[DEFAULT]

I'm probably being too pedantic here, but I think we should just call the directory `test` as we have both integration tests and unit tests in there.

::: testing/raptor/unittests/test_cmdline.py:20
(Diff revision 1)
> +    verify_options(parser, args)  # assert no exception
> +
> +if __name__ == '__main__':

This would be a flake8 error (not enough blank lines). While we're at it we might as well enable at least the flake8 linter on this directory by adding it to:

    tools/lint/flake8.yml
    
You can also add the `py2`, `py3` and `codespell` linters if you like (doing it the same way).

Test with `mach lint --outgoing`

::: testing/raptor/webext/raptor/auto_gen_test_config.js:6
(Diff revision 1)
> +
> +    // this file is auto-generated by raptor, do not edit directly
> +    function getTestConfig() {
> +      return {'browser': 'firefox', 'test_settings_url': 'http://localhost:8000/raptor-firefox-tp7.json'};
> +    }
> +    

nit: whitespace

Comment 4

[Robert Wood [:rwood]]

Comment on attachment 8968609 [details]
Bug 1454466 - Move raptor in-tree;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/237304/diff/1-2/

Comment 5

[Robert Wood [:rwood]]

Comment on attachment 8968609 [details]
Bug 1454466 - Move raptor in-tree;

Sorry didn't mean to reset the review flag yet

Comment 6

[Robert Wood [:rwood]]

Comment on attachment 8968609 [details]
Bug 1454466 - Move raptor in-tree;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/237304/diff/2-3/

Comment 7

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=38bb37c859266a90a0d29f1e41cc517f4fcb0d34

Comment 8

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c444db35d8c5ace98b5bd715ca7215432d1ea2e8

Comment 9

[Andrew Halberstadt [:ahal]]

Comment on attachment 8968609 [details]
Bug 1454466 - Move raptor in-tree;

https://reviewboard.mozilla.org/r/237304/#review243148

Thanks, looks great! Fyi if you wanted to get the python tests running in taskcluster you can add a task to this file:
https://searchfox.org/mozilla-central/source/taskcluster/ci/source-test/python.yml

Feel free to do this in a follow-up though.

::: testing/raptor/raptor/playback/base.py:1
(Diff revisions 1 - 3)
>  # abstract class for all playback tools

nit: missing license block

Comment 10

[Robert Wood [:rwood]]

Comment on attachment 8968609 [details]
Bug 1454466 - Move raptor in-tree;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/237304/diff/3-4/

Comment 11

[Robert Wood [:rwood]]

(In reply to Andrew Halberstadt [:ahal] from comment #9)

> Thanks, looks great! Fyi if you wanted to get the python tests running in
> taskcluster you can add a task to this file:
> https://searchfox.org/mozilla-central/source/taskcluster/ci/source-test/
> python.yml
> 
> Feel free to do this in a follow-up though.
> 
> ::: testing/raptor/raptor/playback/base.py:1
> (Diff revisions 1 - 3)
> >  # abstract class for all playback tools
> 
> nit: missing license block

Added both of those changes, thanks!

Comment 12

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4e17a0ee5d22242ec487807349c98fdeab3842d3

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c75acc48f536
Move raptor in-tree; r=ahal

Comment 14

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/c75acc48f536