1560260 - Empty pocket cards shown without retry

Status: VERIFIED FIXED

(Firefox :: New Tab Page, defect, P1)

Description

[Ed Lee :Mardak]

Attachment: empty cards

Seems like bug 1542863 was to fix when things aren't loading, so I'm not sure if stories failed to refresh but I definitely had stories previously showing on new tabs. I came back to my computer and opened up a new tab with empty cards and no sub sections (so top sites then empty pocket 3 cards then highlights). The second new tab was fine, so the previously preloaded new tab had no pocket content for some reason.

Not sure if there was some network failure / computer offline at some point and the feed tried to refresh ?

Comment 1

[Jessilyn Davis]

Scott- any insight here?

Comment 2

[Scott [:thecount] Downe]

I wasn't able to reproduce with no network. My guess is the issue is a slow network. Our current timeout is 45 seconds which is a lot.

I wasn't able to determine if mardak had it open for 45 seconds or not, but in theory it could have just be loading really slow on a hotel wifi, or another weird case (close browser with a slow network before page finishes fetching, the reopening again under no network, or some other combination), it could just be taking a long time to show the error.

The timeout is mostly a performance thing (we don't want a bunch of fetches sticking around not resolving) and the 45 seconds is the time recommended for how long you should wait on a fetch before freeing up the request. I would guess from a UX point of view 45 seconds is far too long. We can tweak at some point if we want, but for now I don't have a strong reason to suspect a bug quite yet (but certainly possible)

Comment 3

[Jessilyn Davis]

Let's reopen if it happens again and we can get more info.

Comment 4

[Ed Lee :Mardak]

I happened to see this with the new layout so it showed 7 rows of empty cards. I was able to extract the DiscoveryStream redux state and it showed that articles were loaded but spocs sub-object had an endpoint value but everything else was initial state, e.g., loaded = false:

https://github.com/mozilla/activity-stream/blob/29da72b53d637493ca9d659c4cce3e60a9574fd3/common/Reducers.jsm#L597-L600

I believe DISCOVERY_STREAM_SPOCS_ENDPOINT is getting dispatched causing existing spocs data to get reset to INITIAL_STATE.

I let a profile run while logging all actions, and I noticed that there would be some DISCOVERY_STREAM_SPOCS_ENDPOINT after a SYSTEM_TICK. My hunch is that on the 5-minute interval, something has expired resulting in refreshAll({ updateOpenTabs: false }) calling loadLayout to dispatch SPOCS_ENDPOINT (updating the main process redux store to spocs INITIAL_STATE). In the meantime component feeds and spocs are being loaded and haven't updated state and a new tab is opened getting the intermediate state.

Checking the timestamps of the actions:

15:29:56.910 DISCOVERY_STREAM_SPOCS_ENDPOINT
15:29:57.348 DISCOVERY_STREAM_SPOCS_UPDATE

I'm on a pretty fast connection right now, so there was only a window of less than half a second where opening a new tab would have resulted in the empty cards. But any slow network delay between SPOCS_ENDPOINT and SPOCS_UPDATE should result in blank cards.

Arguably SPOCS_ENDPOINT probably shouldn't dispatch if it didn't actually change. Or the reducer notices that it's the same value, don't clear out the other values in the object. ??

Comment 5

[Ed Lee :Mardak]

[Tracking Requested - why for this release]: Turning on discovery stream by default in 69 and this bug will "randomly" show up for users 2 times per hour for a small window of time (worse for users with slow internet)

Comment 6

[Ed Lee :Mardak]

Attachment: recording immediately after a spocs request starts (slowed down to GPRS speed)

Comment 7

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/5246

Comment 8

[Scott [:thecount] Downe]

Attachment: uplift 1560260 - Empty pocket cards shown without retry

Comment 9

[Scott [:thecount] Downe]

Comment on attachment 9084859 [details] [review]
Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/5246

Beta/Release Uplift Approval Request

• User impact if declined: Bad user experience of empty content that won't load
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: In general this is tricky to test because it's time based.
  more importantly, testing for regressions is probably the more important thing here.

To test:

1. Open browser toolbox so you can see network requests.
2. Set throttling to GPRS
3. Wait for the spoc request to come through, should be a post to "https://spocs.getpocket.com/spocs", it should take roughly a half hour, but you can lilely change your system time to have it expire, and it should update in 5 mins instead.
4. As soon as you see the request in netwrok, open a few new tabs.

expected, they should have pocket stories.
Not expected: bunch of empty cards.

• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It's only 1 addition in line changes, and a bit of linting to accommodate the new line.

Once it's verified on nightly this should be low risk.

• String changes made/needed:

Comment 10

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/7d5723904550

Comment 11

[Scott [:thecount] Downe]

Deleted, I was confused.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9084863 [details]
uplift 1560260 - Empty pocket cards shown without retry

Fix for empty cards which won't load content sometimes. Approved for 69.0b14.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/4dd7c91ad1f9

Comment 14

[Daniel Bodea [:danibodea]]

I have attempted reproducing the steps in the uplift request, but I am not sure how/when the sprocket POST event appears. Are the steps complete? When should the POST event appear in Network?

Comment 15

[Daniel Bodea [:danibodea]]

I have succeeded reproducing this issue on Nightly v69.0a1 (2019-06-19) (64-bit) with these steps:

1. Open the browser with the preference to NOT update automatically.
2. Press F12 (Devtools), go to DevTools Settings and check "Enable browser chrome and add-on debugging toolboxes" and "Enable remote debugging" checkboxes.
3. Go to about:config and set pref "browser.search.region" to "US" (to enable the recommended by pocket cards);
4. Restart browser.
5. Press CTRL+ALT+SHIFT+i to open the Browser Toolbox.
6. Set Throttling to GPRS on the Browser Toolbox.
7. Set time an hour ahead.
8. Wait for the spoc post request to come through.
9. After that (step 8), open several new tabs and observe that in the "Recomended by pocket" section are a bunch of empty cards.

I have verified this fix in Nightly v70.0a1 from 2019-08-14 and Beta v69.0b14. Empty cards haven't been displayed anymore.

Comment 16

[Scott [:thecount] Downe]

:danibodea sounds like you got it sorted out, is that correct?

Yeah sorry about that this one was a tricky one.

Comment 17

[Daniel Bodea [:danibodea]]

Yup, received some unexpected help from and managed to reproduce and verify it. Thanks to Cipri Muresan.