1314456 - some windows update tests are ignoring config file versions and failing

Status: RESOLVED WONTFIX

(Testing :: Firefox UI Tests, defect)

Description

[Jordan Lund (:jlund)]

all jobs can be found here: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=d7f98b92fdec&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-localtest(

e.g. of bug in question:

you can see it uses 48.0.2 in this job: https://treeherder.mozilla.org/logviewer.html#?job_id=322430&repo=mozilla-release#L821

even though it should be 47.0.1 defined here: http://mm-ci-production.qa.scl3.mozilla.com:8080/job/ondemand_update/47106/


side note, I'm seeing other windows jobs fail with error lines like: `INFO - TEST-UNEXPECTED-ERROR | test_direct_update.py TestDirectUpdate.test_update | AttributeError: 'dict' object has no attribute 'type' `

example of that here: https://treeherder.mozilla.org/logviewer.html#?job_id=322779&repo=mozilla-release#L866


may be of interest: these failures are against a 'special' mozilla-release branch that is based off 47.0. Current m-r based tests (50.0) may not be affected.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Jordan Lund (:jlund) from comment #0)
> all jobs can be found here:
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> release&revision=d7f98b92fdec&group_state=expanded&filter-tier=3&filter-
> searchStr=Fxup-release-localtest(
> 
> e.g. of bug in question:
> 
> you can see it uses 48.0.2 in this job:
> https://treeherder.mozilla.org/logviewer.html#?job_id=322430&repo=mozilla-
> release#L821
> 
> even though it should be 47.0.1 defined here:
> http://mm-ci-production.qa.scl3.mozilla.com:8080/job/ondemand_update/47106/

I actually do not know what the problem is in this case. We successfully downloaded and installed version 47.0.1. So I can only assume that on this machine an older Firefox process with version 48.0.2 was still running and Marionette connected to it. This can happen in some rare cases including if jobs are getting aborted. If you have such a case again in the future please mark the machine as offline in Jenkins with an appropriate message.

I re-run this job on mm-win-xp-32-2 and it was using Firefox 47.0.1 this time.

> side note, I'm seeing other windows jobs fail with error lines like: `INFO -
> TEST-UNEXPECTED-ERROR | test_direct_update.py TestDirectUpdate.test_update |
> AttributeError: 'dict' object has no attribute 'type' `
> 
> example of that here:
> https://treeherder.mozilla.org/logviewer.html#?job_id=322779&repo=mozilla-
> release#L866

So while running the above test I have seen that 47.0.2 is a watershed! Some suspicion I mentioned already yesterday in the channel. So is 47.0.2 really expected to update to 50? That's what I have seen. And as such I think we see something related to bug 1247596 which I fixed a while ago. Sadly all failures have been starred against this old fixed bug and no new one got filed. So I will do so shortly.

> may be of interest: these failures are against a 'special' mozilla-release
> branch that is based off 47.0. Current m-r based tests (50.0) may not be
> affected.

May there be a problem in the update rules given that we are trying to update users eg. on Windows XP as next step to 50? Or is this the effect for having release-localtest open for 50.0rc testing?

Comment 2

[Rail Aliiev [:rail]]

Yes, the 47.0.2 release is special, it's not final for users on 47.0.1. When we tested the update path looked as 47.0.1 -> 47.0.2 -> 49.0.2. Later the same they we posted the 50.0 build, but I don't think it matters in this case.

Bug 1314550 should make it clearer, wen we have another update and the 48.0.2 issue I bet is just a leftover of one of the previous tests (as you described). The latter probably requires some pre run clean up.

Comment 3

[Jordan Lund (:jlund)]

sounds like we need to have better integration between watersheds and update tests. clearing needinfo unless I can provide more info behind what rail said.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We are not going to fix anything here, also because it didn't seem to happen all the last year.