If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Aurora rc4 jobs permafailing with testSessionHistory | The Toolbar title is mochi.test:8888/tests/robocop/robocop_blank_02.html - got mochi.test:8888/tests/robocop/robocop_blank_01.html, expected mochi.test:8888/tests/robocop/robocop_blank_02.html

RESOLVED FIXED

Status

()

Firefox for Android
Testing
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: KWierso, Assigned: Margaret)

Tracking

({intermittent-failure})

Trunk
intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox47 affected)

Details

(Reporter)

Description

2 years ago
https://treeherder.mozilla.org/logviewer.html#?job_id=1994490&repo=mozilla-aurora


Last green job was on Friday. Everything since then, and every new retrigger prior to that is failing.
It looks like some aurora rc4 runs were green, but now fail on re-trigger (on the same push).

:margaret -- Switchboard related?
Flags: needinfo?(margaret.leibovic)
(Assignee)

Comment 2

2 years ago
(In reply to Geoff Brown [:gbrown] from comment #1)
> It looks like some aurora rc4 runs were green, but now fail on re-trigger
> (on the same push).
> 
> :margaret -- Switchboard related?

The experiment shouldn't be enabled on Aurora... but it looks like it is, sigh.

So, we should uplift the patch for bug 1247366 to fix this. And I'm going to look into why this experiment is live on Aurora.

Miles, does the "org.mozilla.fennec" match also happen to match "org.mozilla.fennec_aurora"? If so, we should change that so it doesn't.

https://github.com/leibovic/switchboard-experiments/blob/master/experiments.json#L4
Depends on: 1247366
Flags: needinfo?(margaret.leibovic) → needinfo?(mcrabill)
(Assignee)

Comment 3

2 years ago
FYI, this failure will go away after the merge happens, but I still want to figure out what's going wrong with the Switchboard config. We need to be able to control how experiments are rolled out to different release channels.
(Assignee)

Comment 4

2 years ago
I filed an issue: https://github.com/mozilla-services/switchboard-server/issues/4
(In reply to :Margaret Leibovic from comment #2)
> (In reply to Geoff Brown [:gbrown] from comment #1)
> > It looks like some aurora rc4 runs were green, but now fail on re-trigger
> > (on the same push).
> > 
> > :margaret -- Switchboard related?
> 
> The experiment shouldn't be enabled on Aurora... but it looks like it is,
> sigh.
> 
> So, we should uplift the patch for bug 1247366 to fix this. And I'm going to
> look into why this experiment is live on Aurora.
> 
> Miles, does the "org.mozilla.fennec" match also happen to match
> "org.mozilla.fennec_aurora"? If so, we should change that so it doesn't.
> 
> https://github.com/leibovic/switchboard-experiments/blob/master/experiments.
> json#L4

Hi, yes - it does also match. That is because as it is substring matches are accepted... I could require exact matches, but this way you have more power. The way to deal with this is to use start and end delimiters in your regular expressions for matching.

Here is an example based on the issue you're describing:

{
  "fennec_aurora": {
    "match": {
      "appId": "^org.mozilla.fennec_aurora$"
    },
    "buckets": {
      "min": "0",
      "max": "100"
    }
  },
  "fennec": {
    "match": {
      "appId": "^org.mozilla.fennec$"
    },
    "buckets": {
      "min": "0",
      "max": "100"
    }
  }
}

I've updated the switchboard-server tests to account for exactly this problem. There's still a lot of work to do on testing switchboard-server, but at least this is in there now!
Flags: needinfo?(mcrabill)
(Assignee)

Comment 6

2 years ago
This should be resolved with this change I just made:
https://github.com/mozilla-services/switchboard-experiments/commit/acecc0854fe109bc8291e9fed66fe3e5e257b505
(Assignee)

Comment 7

2 years ago
Let me know if this goes away (it should!) and we can close this bug.
Flags: needinfo?(wkocher)
(Assignee)

Updated

2 years ago
Assignee: nobody → margaret.leibovic

Comment 8

2 years ago
65 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-aurora: 65

Platform breakdown:
* android-4-3-armv7-api15: 65

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1250289&startday=2016-02-22&endday=2016-02-28&tree=all
(Reporter)

Comment 9

2 years ago
Yeah, this seems to have gone away.
Flags: needinfo?(wkocher)
(Assignee)

Updated

2 years ago
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.