1026730 - [1.4 => 2.0] Wi-Fi networks are erased after an OTA update

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::First Time Experience, defect)

Description

[Johan Lorenzo [:jlorenzo]]

Steps to reproduce
1. Flash your device to the last 1.4 nightly build.
1bis. (Workaround) Change the channel to aurora with that script https://github.com/nhirata/B2G-flash-tool/blob/new_channels/change_channel.sh
2. Connect to a public network (like Mozilla Guest)
3. Update via OTA
4. Once update is complete, go to Settings -> Wi-Fi -> Manage networks

Actual results
"No known networks" is displayed. Wi-Fi is on but not connected

Additional notes
Update channel have recently changed. The script in step 1bis has been modified to handle that change.

Comment 1

[Johan Lorenzo [:jlorenzo]]

QA Wanted to see if this issue is reproducible from 1.3 to 2.0.

Comment 2

[Johan Lorenzo [:jlorenzo]]

This will be a standard migration path. Marked as 2.0? blocker.

Comment 3

[Dave Hylands [:dhylands]]

I think that this might be a dup of bug 1025034?

Comment 4

[Jason Smith [:jsmith]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #1)
> QA Wanted to see if this issue is reproducible from 1.3 to 2.0.

I'd also check 1.3 --> 1.4.

Comment 5

[Dave Hylands [:dhylands]]

OTA from any working version to 2.0 should be fine (since OTA can't touch the wifi drivers).

Doing a full flash of the final version (2.0) should reproduce the problem.

FOTA may modify the wifi stuff, so I don't know if a FOTA update will result in working or non-working Wifi.

Comment 6

[Chuck Lee [:chucklee, Triggered by NI, Review, and Feedback requests]]

I think the most possible cause might be there's a command in FOTA commands to erase or reset /data/misc/wifi/.

Comment 7

[Henry Chang [:hchang]]

(In reply to Chuck Lee [:chucklee] from comment #6)
> I think the most possible cause might be there's a command in FOTA commands
> to erase or reset /data/misc/wifi/.

More specifically speaking, /data/misc/wifi/wpa_supplicant.conf may be removed after OTA.

Comment 8

[bhavana bajaj [:bajaj]]

waiting for QA input here

Comment 9

[Andrew Overholt [:overholt]]

User data (known wifi networks) shouldn't be erased during an update.  If that is reproducible as a regular user with an OTA update that would be presented to a regular user, this should block.

Comment 10

[Johan Lorenzo [:jlorenzo]]

We are not able to test 1.3 => X as we don't have Flame builds on that version (except the base image). And Buri 1.3 => 1.4 OTA is not working right now. 

But as Andrew said, we should block on this.

Comment 12

[howie [:howie]]

Hi Dave, could you help to check the FOTA behavior as Comment 6 and Comment 7 mentioned? If that's the case, the command might need to be removed to fix this problem, thank you!

Comment 13

[Dave Hylands [:dhylands]]

OTA can only touch files in /system/b2g
FOTA can wipe anything.

I've never seen a FOTA wipe /data/misc/wpa_supplicant.conf, and as far as I'm aware the issue here isn't that the file got wiped, but rather than wifi didn't come up properly due to some missing files.

Bug 1025034 is probably the cause here.

Comment 14

[Vincent Chang[:vchang][changyihsin]]

Hi Gerry, can you help to reproduce this?

Comment 15

[Gerry Chang [:gchang]]

Yes, I can recreate the problem. Device is flame.
1. Flash to 1.4 first
Gaia      9ec45071c9c6bf921061580363257907e0465e16
Gecko     https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/7280e2146451
BuildID   20140703160201
Version   30.0

2. Connect to "Mozilla Mobile" network
3. Change OTA url to 2.0 (http://update.boot2gecko.org/flame/2.0.0/nightly/update_20140703160208.xml)
4. Perform OTA
5. After complete, go to Settings -> Wi-Fi -> Manage networks.

Gaia      6b6b7d7fe829ebea85b01aa4ed44cf5ada366bbe
Gecko     https://hg.mozilla.org/releases/mozilla-aurora/rev/9457a67841b3
BuildID   20140703160208
Version   32.0a2

Actual results:
"No known networks" is displayed.

Comment 16

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: ota_1.4_to_2.0.log

I can reproduce the same result for v1.4 -> v2.0, and the attached file is adb logcat.
I also notice that after the device is updated, it will start into FTU "Skip/Start tour" page.
So some data could be erased.

Comment 17

[Vincent Chang[:vchang][changyihsin]]

Based on comment 13, it seems like doing the OTA is something similar with doing the "flash gecko" command because it only updates files in /system/b2g folder. I tried to update flame to 1.4 using tool in https://github.com/Mozilla-TWQA/B2G-flash-tool. The command is "./auto_flash_from_pvt.sh -w". Then, I do "flash gecko" to update b2g to latest mc build and check file in /data/misc/wifi/wpa_supplicant.conf. I found it remain the same after b2g process restart. The SSID information is still there without modification. Looks like there is some trick when doing the OTA update to make the different result.

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So as reported on the newly duped bug 1033983 I see the exact same behavior when I initially flashed version 2.0 onto my new Flame device. All networks which were available before with the pre-installed version 1.3 are gone.

(In reply to Dave Hylands [:dhylands] from comment #13)
> Bug 1025034 is probably the cause here.

If that is the case, on which branches has this bug been fixed yet? I assume only 2.1 for now, or?

Comment 20

[howie [:howie]]

Hi Dave, is running FTU again after OTA the correct behavior? Comment 16, also ni UX to confirm. And do you know what additional actions OTA does besides flash gecko? We might find some clues there. Thank you!

Comment 21

[Dave Hylands [:dhylands]]

(In reply to howie [:howie] from comment #20)
> Hi Dave, is running FTU again after OTA the correct behavior? Comment 16,
> also ni UX to confirm. And do you know what additional actions OTA does
> besides flash gecko? We might find some clues there. Thank you!

I wouldn't have thought so.

OTA only updates files in /system/b2g
FOTA can pretty much do anything (wipe partitions, update the kernel, whatever)

I do believe that there are some hooks someplace for allow some type of update script to run, which we use for updating the databases when we change versions. So maybe it's possible that one of those scripts might do something to cause FTU to run again?

Comment 22

[Jacqueline Savory [:jsavory] UX]

Hi Howie, I believe that the expected behaviour is to run the tutorial flow (Skip/start tour) with only the tutorials relevant to the update. I don't think that the update should run the FTU setup.

Comment 23

[Ken Chang[:kenkai]]

Hi Viral,
  I wonder if you can help this feature. According to comment 7 and comment 17, I am not sure if you or vendors need to take care this FOTA problem.

Comment 24

[viral [:viralwang]]

Hi Francis,

Need your help to discuss this issue with vendor.
Thanks!!

Comment 25

[Francis Lee [:frlee]]

hi All, 

this issue requires further investigation: first, issue is reproduced with Moz' own OTA mechanism. vendor doesn't really have visibility about our OTA.

i have talked to Kai-Zhen and he will check our OTA update process step by step to find out the root cause. in the mean time, Gerry will try to reproduce this issue with Buri. so that we have a better idea if this is a generic issue or not.

Comment 26

[Gerry Chang [:gchang]]

Hi Francis,
1. Buri has problem upgrading to 2.0 due to invalid signature.
The reason is that all builds on PVT are based on old base image but now we only have newer one. We are trying to get old base image from partner.
2. From PVT, only buri and flame support 2.0. There are no any 2.0 builds for other device.

Comment 27

[Kai-Zhen Li [:seinlin][:kli]]

I doubt this could be the root cause.
http://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/nsUpdateService.js#3384

I also think the data could be reset by the 'updater' which is included in the update.mar download from the server.

Comment 29

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Kai-Zhen Li from comment #27)
> I doubt this could be the root cause.
> http://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/
> nsUpdateService.js#3384
> 

I verified that when it is not loading default preferences in OTA update, wifi setting will also be erased. So the above doubt can be neglected.

Comment 30

[Kai-Zhen Li [:seinlin][:kli]]

I found that wifi config was erased during the first launch of b2g after OTA update is done successfully. And it is because WifiWorker receive 'WifiManager:forget' message.

Comment 31

[Evelyn Hung]

Hi EJ, do we erase configuration while performing OTA? Thanks.

Comment 32

[Kai-Zhen Li [:seinlin][:kli]]

I found that it is erased by ftu. After device is upgraded from 1.4 to 2.0, ftu will be launched to display tour page. Wifi config will be erased when ftu is launched.

Comment 33

[Francis Lee [:frlee]]

Kai-Zhen, great finding, thanks a lot for your effort.
let's see if EJ can find out potential fix for that.

Comment 34

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: ota_ftu.patch

This can be fixed by adding a version check before forget network. If this is reasonable I can send a PR.

Comment 35

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

No, we won't do anything in Settings/wifi for OTA. Based on comment 32, it seems that the bug is caused by FTU.

Thanks :)

Comment 36

[Kai-Zhen Li [:seinlin][:kli]]

Comment on attachment 8454304 [details] [diff] [review]
ota_ftu.patch

Hi Francisco, I think wifi config should not be forgot during ftu after OTA update. Do you think if this patch is reasonable?

Comment 37

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Kai-Zhen Li from comment #36)
> Hi Francisco, I think wifi config should not be forgot during ftu after OTA
> update. Do you think if this patch is reasonable?

I don't see how this would fix the problem with a full flash. When I do that, no wifi network is found. How is this bug related? I ask because my bug 1033983 got marked as duplicated of that one.

Comment 38

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

Note that Bug 1031369 may fix this situation. I reordered the FTU initialization logic so taht if we're in an upgrade situation, none of the FTU modules other than tutorial and navigation are initialized. Therefore we'd never initialize the wifi module and shouldn't run forget. That said, I'm not sure how I'd test this situation at the moment.

Comment 39

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Kyle Machulis [:kmachulis] [:qdot] from comment #38)
> Note that Bug 1031369 may fix this situation. I reordered the FTU
> initialization logic so taht if we're in an upgrade situation, none of the
> FTU modules other than tutorial and navigation are initialized. Therefore
> we'd never initialize the wifi module and shouldn't run forget. That said,
> I'm not sure how I'd test this situation at the moment.

Yes, Bug 1031369 can fix this issue.

Comment 40

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

Ok, I'll go ahead and take this then, will resolve once Bug 1031369 lands.

Comment 41

[Ross Kuhlman ([e-mail])]

I encountered this issue while updating from 2.0 to current Master.All WiFi networks were disconnected after the OTA process

v2.0 Environmental Variables:
Device: Flame v2.0
BuildID: 20140711000201
Gaia: 18c44a1bc31b374ba00a069904465a8d07971a60
Gecko: f880dae4fdbe
Version: 32.0a2
Firmware Version: v122

Comment 42

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

Yup, it's gonna happen on any version where we have tutorial-on-upgrade currently. Bug 1031369 will be landing to 2.0 and master after review, should fix it on both.

Comment 43

[Peter Bylenga [:PBylenga]]

This fails the ota smoketest case for 2.1.  For 2.0 Smoketest we did an upgrade from 1.3 to 2.0 using base image + 1.3 gaia as a start point and we did not reproduce this issue.

Comment 44

[Jason Smith [:jsmith]]

Make sure you finish off the triage workflow here after you get needinfo'd - the bug still have a triage? on it.

Comment 45

[Kyle Machulis [:qdot] [:kmachulis] (INACTIVE)]

Fixed as Bug 1031369 landed

Comment 46

[Andrew Stole [:astole]]

Verified fix on the latest 2.0

2.0 Environmental Variables:
Device: Flame 2.0
BuildID: 20140716040207
Gaia: d29773d2a011825fd77d1c0915a96eb0911417b6
Gecko: 691ffea49efb
Version: 33.0a1
Firmware Version: v122

Comment 47

[Andrew Stole [:astole]]

Correction to comment 46: Verified on the latest 2.1 after updating from 2.0

Environmental Variables are for the latest 2.1