Closed Bug 1443768 Opened 7 years ago Closed 7 years ago

./mach test-info is failing due to missing platform information

Categories

(Testing :: General, defect)

Product:
Testing
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jmaher, Unassigned)

Details

I was looking to gather some additional information about a test: ./mach test-info browser/components/customizableui/test/browser_901207_searchbar_in_panel.js and it failed on me due to missing record['build']['platform']: KeyError: u'platform' File "c:\Users\elvis\mozilla-inbound\testing/mach_commands.py", line 619, in test_info self.report_test_results() File "c:\Users\elvis\mozilla-inbound\testing/mach_commands.py", line 815, in report_test_results data.sort(key=self.get_platform) File "c:\Users\elvis\mozilla-inbound\testing/mach_commands.py", line 762, in get_platform platform = record['build']['platform'] I hacked the code a bit to print out record and continue on and here is what I see: JMAHER: missing platform: {u'count': 1949, u'failures': 4.0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'debug'}} JMAHER: missing platform: {u'count': 1661, u'failures': 1.0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'opt'}} JMAHER: missing platform: {u'count': 1086, u'failures': 0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'pgo'}} JMAHER: missing platform: {u'count': 214, u'failures': 0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'asan'}} JMAHER: missing platform: {u'count': 41, u'failures': 0, u'build': {u'type': u'opt'}} JMAHER: missing platform: {u'count': 16, u'failures': 0, u'run': {u'type': u'e10s'}, u'build': {u'type': u'opt'}} JMAHER: missing platform: {u'count': 1949, u'failures': 4.0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'debug'}} 4 failures in 1949 runs JMAHER: missing platform: {u'count': 1661, u'failures': 1.0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'opt'}} 1 failures in 1661 runs JMAHER: missing platform: {u'count': 1086, u'failures': 0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'pgo'}} 0 failures in 1086 runs JMAHER: missing platform: {u'count': 214, u'failures': 0, u'run': {u'type': [u'e10s', u'chunked']}, u'build': {u'type': u'asan'}} 0 failures in 214 runs JMAHER: missing platform: {u'count': 41, u'failures': 0, u'build': {u'type': u'opt'}} 0 failures in 41 runs JMAHER: missing platform: {u'count': 16, u'failures': 0, u'run': {u'type': u'e10s'}, u'build': {u'type': u'opt'}} 0 failures in 16 runs in fact, no platforms exist at all. Possibly this is fallout from activedata etl changes?
I believe this is the query: https://searchfox.org/mozilla-central/source/testing/mach_commands.py#786 and the raw text: { "from":"unittest", "format":"list", "limit":100, "where":{"and":[ {"eq":{"result.test":"browser/components/customizableui/test/browser_901207_searchbar_in_panel.js"}}, {"in":{"build.branch":["mozilla-central","mozilla-inbound","autoland"]}}, {"gt":{"run.timestamp":{"date":"2018-02-28"}}}, {"lt":{"run.timestamp":{"date":"2018-03-07"}}} ]}, "groupby":["build.platform","build.type","run.type"], "select":[ {"aggregate":"count"}, { "aggregate":"sum", "default":0, "name":"failures", "value":{"case":[{"then":1,"when":{"eq":{"result.ok":"F"}}}]} } ] } :ekyle, did the etl change?
Flags: needinfo?(klahnakoski)
I will look into this today. I will keep then eedinfo flag on until I have solved the problem.
I have found the problem, and it was a mistake I introduced to the ETL. I confirmed the problem; and we can see a distinct dip in the build.platform count: date build.platform count 21-Feb-18 225,285 22-Feb-18 214,302 23-Feb-18 4,314 24-Feb-18 293 26-Feb-18 3,193 27-Feb-18 2,995 https://activedata.allizom.org/tools/query.html#query_id=ctRQYTQI I fixed it: https://github.com/klahnakoski/ActiveData-ETL/commit/5063ba1ee314a8958188a7412d91d08db81c5d4f I added a check to ensure I get alerts on this particular problem in the future: https://github.com/klahnakoski/ActiveData-ETL/commit/30f543c60ba318a80b6c1c0f84ab5f6c91dc5c7c I am running a backfill on these days to correct the historical record: https://github.com/klahnakoski/ActiveData-ETL/commit/392daa742fe8c4c73f61fa2293c9816679b53efa This backfill will take a full 24 hours to be realized, with most recent data processed first.
This is done.
Flags: needinfo?(klahnakoski)
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.