962225 - [basket] Find a way to deal with the malformed data coming from FxOS

Status: RESOLVED FIXED

(www.mozilla.org :: Newsletters, defect)

Description

[Paul [:pmac] McLanahan]

Bug 949170 is being fixed to deal with the malformed basket requests, but until that makes it onto devices (could be a while) it'd be very nice to not miss out on all of these subscribers. Hopefully we can read the raw URL-encoded data coming from the FxOS requests and parse it manually to make it work in the interim.

Comment 1

[[github robot]]

Commits pushed to master at https://github.com/mozilla/basket

https://github.com/mozilla/basket/commit/12f65f580e9a23855ceb0a808d2a1bf50be279ee
Fix bug 962225: Support malformed POSTs from FxOS

A patch is in for FxOS, but this should accept the
requests until the device code is fixed. We should
revert this once bug 949170 is fixed and on devices.

https://github.com/mozilla/basket/commit/3ef587fb9b1ecf83f45b7d8a0c68a02237f76e57
Merge pull request #92 from pmclanahan/support-bad-post-fxos-962225

Fix bug 962225: Support malformed POSTs from FxOS

Comment 2

[[github robot]]

Commits pushed to master at https://github.com/mozilla/basket

https://github.com/mozilla/basket/commit/afc14f017292fbc7af33adab68ad6672b08e8e23
Bug 962225: Fix issue with unescaped characters in emails.

The string from FxOS is _NOT_ urlencoded, therefore common
email symbols like "+" are not escaped. This will cause them
to be converted to " " by QueryDict. Fix this by doing a much
more simple parse of the string.

https://github.com/mozilla/basket/commit/357f3fd0759190477d84d71c806474a2a6b8b935
Merge pull request #93 from pmclanahan/support-bad-post-fxos-962225

Bug 962225: Fix issue with unescaped characters in emails.

Comment 3

[[github robot]]

Commit pushed to master at https://github.com/mozilla/basket

https://github.com/mozilla/basket/commit/00e7eff7ee193674d64d49ad7bd04fa06fbb5060
Bug 962225: Improve test to check for string manipulation.

Comment 4

[Paul [:pmac] McLanahan]

Jess, this is now in production. Please keep an eye on signups to the FIREFOX_OS_USER list to see if it's really working. If so please mark this verified.

/me crosses fingers

Comment 5

[Jessilyn Davis]

Woah! Great work, pmac! I just checked and there are 10,323 with this flag now!

On one hand, this could mean that since implementing this code 10k people have signed up = crazy.

On the other, I'm not sure if that's the case and we should see how these folks could have gotten into our system with this flag.

Question: Does the FTU keep retrying if it wasn't able to contact basket successfully?

If so, does it keep retrying forever? Or just until it's successful?

Comment 6

[Paul [:pmac] McLanahan]

It does indeed appear to keep trying forever! This is crazy fantastic :) The code (my reading of it) adds an event listener to the "online" event. So whenever the phone goes from offline to online the event fires, and this tries the subscription again. If it succeeds it removes the email address from storage and removes the event listener. So it appears that this system works quite well :)

Comment 7

[Jessilyn Davis]

\o/

This IS Crazy fantastic! Now we just need to get country + language as part of this bad boy in Bug 949170 and all will be looovely. :D