Fennec 38 caused huge decrease in submitted MLS stumbling data

VERIFIED FIXED in Firefox 38

Status

defect
--
critical
VERIFIED FIXED
4 years ago
2 years ago

People

(Reporter: hschlichting, Assigned: garvan)

Tracking

Firefox 38
Firefox 41
All
Android
Bug Flags:
in-testsuite ?

Firefox Tracking Flags

(firefox38+ verified, firefox38.0.5+ fixed, firefox39+ verified, firefox40+ verified, firefox41+ verified, relnote-firefox 38+, fennec38+)

Details

Attachments

(1 attachment)

(Reporter)

Description

4 years ago
Since the release of Fennec 38 on the play store, we are seeing a huge decrease in submitted stumbling data. Instead of 3-6 requests per second, we are down to 1-2 requests / second.

The service logs confirm that we do get data from both Fennec 37.0.2 and 38.0, so it's not generally broken.

We did not see a similar decrease in traffic around March 31 (the last major Fennec release).

Currently we are stumped, as to what and why this is happening.
(Reporter)

Updated

4 years ago
OS: Unspecified → Android
Hardware: Unspecified → All
Version: unspecified → Firefox 38
(Assignee)

Comment 1

4 years ago
My patch in bug 1155237 flipped a boolean check incorrectly for 'is wifi available', resulting in no uploading.
Attachment #8605279 - Flags: review?(rnewman)
(Assignee)

Updated

4 years ago
Severity: normal → critical
Attachment #8605279 - Flags: review?(rnewman) → review+
tracking-fennec: --- → ?
Component: General → Geolocation
Product: Firefox for Android → Android Background Services
Assignee: nobody → gkeeley
Status: NEW → ASSIGNED
(Assignee)

Comment 2

4 years ago
Comment on attachment 8605279 [details] [diff] [review]
bug1164468.diff

Approval Request Comment
[Feature/regressing bug #]: The patch for bug 1155237 for Fennec stumbler caused this. 
[User impact if declined]: None for the user. 
However, for Mozilla Location Service this is very bad. Fennec release is 2/3 of the stumbling data we receive.
[Describe test coverage new/current, TreeHerder]: None
[Risks and why]: Zero risk, a boolean got flipped in a prev patch, this patch corrects that.
[String/UUID change made/needed]: None
Attachment #8605279 - Flags: approval-mozilla-release?
Attachment #8605279 - Flags: approval-mozilla-beta?
Attachment #8605279 - Flags: approval-mozilla-aurora?
[Tracking Requested - why for this release]:
(Assignee)

Comment 4

4 years ago
(In reply to Kevin Brosnan [:kbrosnan] from comment #3)
> [Tracking Requested - why for this release]:

For Mozilla Location Service this is very bad. Fennec release is 2/3 of the stumbling data we receive.
(Assignee)

Comment 5

4 years ago
Note: 38.0.5 is in 3 weeks (Jun 2nd)
Is there an automated test?
Flags: in-testsuite?
(Assignee)

Comment 7

4 years ago
(In reply to Aaron Train [:aaronmt] from comment #6)
> Is there an automated test?

Not on Fennec. The Fennec test coverage only tests that stumbler gets turned on/off according to the Fennec UI. Full test coverage is on Mozilla Stubmler github.
Comment on attachment 8605279 [details] [diff] [review]
bug1164468.diff

One character typo fix. (Second typo requiring that we ship a fix in the 38 cycle.) Approved across the board.

Note that this patch needs to land on the 38.0 relbranch as well as m-r.
Attachment #8605279 - Flags: approval-mozilla-release?
Attachment #8605279 - Flags: approval-mozilla-release+
Attachment #8605279 - Flags: approval-mozilla-beta?
Attachment #8605279 - Flags: approval-mozilla-beta+
Attachment #8605279 - Flags: approval-mozilla-aurora?
Attachment #8605279 - Flags: approval-mozilla-aurora+
Release Note Request (optional, but appreciated)
[Why is this notable]: Stumbling is mostly broken in 38.
[Suggested wording]: Fixed: Mozilla Location Service (MLS) stumbler may not submit all data
[Links (documentation, blog post, etc)]:

Garvan - Can you please confirm that the wording of this note is ok or help me revise?
Flags: needinfo?(gkeeley)
(Assignee)

Comment 12

4 years ago
(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #10)
> Release Note Request (optional, but appreciated)
> [Why is this notable]: Stumbling is mostly broken in 38.
> [Suggested wording]: Fixed: Mozilla Location Service (MLS) stumbler may not
> submit all data
> [Links (documentation, blog post, etc)]:
> 
> Garvan - Can you please confirm that the wording of this note is ok or help
> me revise?

This is correct. I'll give you more explanation. There is a check to see if there is a network connection, then upload collected stumbling data (typically 1x per day it does this). If not, the collected stumbling data will not get uploaded, and after a few days, is discarded. That boolean check got broken, making it appear there is never a network connection, and stumbling data gets discarded. So no harm to the user, but very harmful for MLS.
tracking-fennec: ? → 38+
https://hg.mozilla.org/mozilla-central/rev/5c12b34d73d3
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 41
Is there a way for QE to reproduce to verify the fix?
Flags: needinfo?(gkeeley)
(Assignee)

Comment 15

4 years ago
Yes, to test:
- After Fennec is installed do:
`adb shell setprop log.tag.PassiveStumbler DEBUG`
 (documented here https://wiki.mozilla.org/QA/Fennec/MozStumbler)
This turns on logging.
- In Fennec, enable stumbling pref
- Install Mozilla Stumbler, turn on, verify that the observation counter increases (1 observation is fine). A long as you are within 2-3 meters of an office window you should get a GPS fix.
- In Fennec, turn stumbling off pref off and on again. (This force-triggers the upload process that has the bug.)
- In 30 seconds, if you do not see the following in logcat (exact phrase, you can grep for this):
"not on WiFi, not sending"
Then it is working.
- BONUS: If the phone is rooted you can check /data/data/org.mozilla.fennec/reports and check if the directory contains timestamped report files that are prior to the time the stumbling pref was toggled (indicating they were not uploaded as expected).
Flags: needinfo?(gkeeley)
Ioana - Does your team still have time to verify this bug today?
Flags: needinfo?(ioana.chiorean)

Comment 17

4 years ago
Catalin is working on it already - he will add a final status to it soon.
Flags: needinfo?(ioana.chiorean) → needinfo?(csuciu)

Comment 18

4 years ago
Checked that on Firefox 38.0 the following line was present in logcat:
D/Stumbler_AsyncUploader(27310): not on WiFi, not sending


Verified as fixed on Firefox 38.0.1 on Nexus 5 (Android 5.1) since the "not on WiFi, not sending" is not present in the logs:

Logs:
D/Stumbler_ScanManager(11347): Scanning started...
D/Stumbler_ScanManager(11347): New passive location
V/Stumbler_WifiScanner(11347): Activate Periodic Scan
D/WifiService(  745): acquireWifiLockLocked: WifiLock{MozStumbler type=2 binder=android.os.BinderProxy@3a37123}
V/Stumbler_WifiScanner(11347): WiFi Scanning Timer fired
V/Stumbler_WifiScanner(11347): WiFi Scanning Timer fired
D/Stumbler_Reporter(11347): Received bundle
V/Stumbler_WifiScanner(11347): WiFi Scanning Timer fired
V/Stumbler_WifiScanner(11347): WiFi Scanning Timer fired
V/Stumbler_WifiScanner(11347): Deactivate periodic scan
D/WifiService(  745): releaseWifiLockLocked: WifiLock{MozStumbler type=2 binder=android.os.BinderProxy@3a37123}
D/Stumbler_GPSScanner(11347): Discard fused/network location.
Flags: needinfo?(csuciu)
(Assignee)

Comment 19

4 years ago
Thanks, that output matches correct behaviour.
Verified as fixed on Firefox 39 Beta 1
Ioana, Catalin: Based on comment 21 and given that the same fix landed on moz-aurora, moz-beta, could you please mark the status as "verified" on FF40 and FF41? Post that, could you also change the bug's status to "verified"? Thanks!
Flags: needinfo?(chiorean.ioana)
Flags: needinfo?(catalin.suciu)
Status: RESOLVED → VERIFIED
Flags: needinfo?(catalin.suciu)
Flags: needinfo?(chiorean.ioana)
You need to log in before you can comment on or make changes to this bug.