Closed Bug 998477 Opened 12 years ago Closed 11 years ago

Password reset emails never sent

Categories

(addons.mozilla.org Graveyard :: Public Pages, defect)

Product:
addons.mozilla.org Graveyard
Component:
Public Pages
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: kmag, Assigned: magopian)

References

Details

We've received reports from several users that they never receive password reset emails. I tried myself and also never received an email. I've already manually reset passwords for three users in the past week. I'd rather not go on this way.
Assigning to magopian, but also marking qawanted and CCing Jason as I don't think there has been a code change around this in quite some time. Jason - any reason on the ops side this would be happening? Can you try sending mail off the django CLI and seeing if it shows up? Krupa - can you reproduce this?
Assignee: nobody → mathieu
Keywords: qawanted
OK, it turns out I did eventually receive my password reset email. But the three other users I had to reset passwords for in the past week suggest that there's still something wrong.
See Also: → 990500
Looks like mail is working from django cli: >>> from django.core.mail import send_mail >>> print settings.EMAIL_HOST 10.32.128.21 >>> send_mail('Test Mail', 'Test Message.', 'mail@mozilla.com', ... ['jason@jasonthomas.in'], fail_silently=False) 1 Apr 19 12:28:19 log1 postfix/smtp[13097]: 439B0240160: to=<jason@jasonthomas.in>, relay=smtp.socketlabs.com[142.0.179.10]:25, delay=0.63, delays=0/0/0.47/0.16, dsn=2.0.0, status=sent (250 2.0.0 Message received and queued as 70000002fbb258.) Received mail instantly. I did this from one of the prod web nodes.
:kmag is this still relevant? Did you have other requests from users with the same issue? I can't reproduce locally either, and it's all done in django's own views, so unless it's something relevant to the upgrade to Django1.6 or not working at the system level, I can't see what could be going wrong.
I tried resetting my password and immediately received the email. I tried with my gmail and mozilla accounts. I cannot reproduce this issue.
Keywords: qawanted
It does seem to work whenever I try it, but I keep getting users complaining that it hasn't worked, starting about 2 weeks ago, the most recent being today, for https://addons.mozilla.org/user/marketing5/ I do see a log entry saying the email was sent. I see from the code that that log entry gets added before the email is sent, but I also don't see a log entry saying it failed, so I'm at a loss for the moment.
Hi I'm https://addons.mozilla.org/en-US/firefox/user/another_sam/ and I found the same problem on yesterday. Tried again today and is not working either. Please contact me if I can test anything for you. Thanks. (i want to push an update for WR Translator but it is not urgent; no manual reset pretended :) )
Jason - do you have any ideas? I don't know what else we can do here. Are there logs for sent mail we could look at for someone's address?
I see a log entry for this password reset email, too: Apr 28 11:53:19 ...mozilla.com: [<anon>][...] z.users:INFO Password reset email sent for user (207140: another_sam) :.../olympia/apps/users/forms.py:86 another_sam, can you go to Gmail and search for 'in:all from:nobody@mozilla.org` to make sure that the email was not sorted into a non-obvious place? Thanks.
I am not able to reset my password for two of my accounts on addons.mozilla.org (jason@lithiumfox.com and jthomas@mozilla.com). I am not seeing any issues on our postfix servers, they seem to be working fine. I am also unable to reset my password on addons-dev.allizom.org which uses totally different postfix servers than prod. Can we add more logging around this? Apr 29 14:55:45 dev1.addons.phx1.mozilla.com: [<anon>][71.167.42.204] z.users:INFO Password reset email sent for user (6108771: jason) :/data/addons-dev/www/addons-dev.allizom.org/deploy-olympia-dev-20140429020716-d7d787a516/olympia/apps/users/forms.py:86
I traced the olympia process on -dev while performing a password reset and I don't see any connections being made to the postfix server.
Out of my own curiosity I did a quick code walk starting from the password reset form to see what it checks before sending the email. https://github.com/django/django/blob/stable/1.6.x/django/contrib/auth/forms.py#L216 One is checking if the password is "usable". I ran the following on -dev against one of the password strings mentioned in comment 10 and the result is False: >>> from django.contrib.auth.models import User >>> user = User.objects.get(email='jason@lithiumfox.com') >>> user.has_usable_password() False When that happens Django completely skips the whole password reset. It looks like it has something to do with password hashers and detecting that it's a password we support: https://docs.djangoproject.com/en/dev/topics/auth/passwords/
(In reply to Kris Maglione [:kmag] from comment #9) > I see a log entry for this password reset email, too: > > Apr 28 11:53:19 ...mozilla.com: [<anon>][...] z.users:INFO Password reset > email sent for user (207140: another_sam) :.../olympia/apps/users/forms.py:86 > > another_sam, can you go to Gmail and search for 'in:all > from:nobody@mozilla.org` to make sure that the email was not sorted into a > non-obvious place? > > Thanks. @kmag Is not there :( I stay available for more tests or so. Thanks.
(In reply to Jason Thomas [:jason] from comment #11) > I traced the olympia process on -dev while performing a password reset and I > don't see any connections being made to the postfix server. I'm pretty sure emails on -dev are just logged to the DB, and never actually sent to the mail server. (In reply to Rob Hudson [:robhudson] from comment #12) > https://github.com/django/django/blob/stable/1.6.x/django/contrib/auth/forms. > py#L216 > > One is checking if the password is "usable". I ran the following on -dev > against one of the password strings mentioned in comment 10 and the result > is False: OK. That's interesting. The problem is this, then: Passwords for accounts imported from getpersonas have `sha512+base64` set as their algorithm, and the only hasher that we define in `settings.PASSWORD_HASHERS` only accepts `sha512`. The two easiest solutions are probably to either subclass that hasher and allow it to process `sha512+base64` hashes, or to run a migration to fix the old hashes (which may or may not be possible). We should probably just also add a check for `has_usable_password`, and either notify users when it's going to fail, or reset the hash (assuming it doesn't start with '!', meaning disabled) to a randomly generated password.
Ah, actually, there are at least two cases where this happens: 1) As above, sha512+base64 algorithm 2) `password` column is empty in the `auth_user` table, but not in the `users` table.
(In reply to Kris Maglione [:kmag] from comment #14) > (In reply to Jason Thomas [:jason] from comment #11) > > I traced the olympia process on -dev while performing a password reset and I > > don't see any connections being made to the postfix server. > > I'm pretty sure emails on -dev are just logged to the DB, and never actually > sent to the mail server. I created a new account just now on addons-dev.allizom.org 'testjason@lithiumfox.com'. I received a password reset email.
I have the same problem. I'm not getting any emails. My profile: https://addons.mozilla.org/en-US/firefox/user/CIAvash/
I'm -1 on reseting the hash to a randomly generated password: that means one could simply reset someone's password without even using the generated password reset link. It wouldn't give him access to the account, but it would still be a huge inconvenience for the target. What is the sha512+base64 algorithm doing? If it's base64 encoding the sha512, then I guess we could run a migration to unencode all those or create a custom hasher. If it's the other way round, we're out of luck. That being said, how can users with a sha512+base64 algorithm login to the site if there's no corresponding password hasher? There's also the issue with the auth_user and users passwords not being properly synchronized: I believe this is an issue, and should be fixed when we switch to using the custom user model (soon! https://github.com/mozilla/olympia/pull/63). If you believe there's a lot of accounts that have this specific issue, and if this issue is urgent, we'll have to run a migration to copy the UserProfile.password to the auth.User.password (and make sure it's synchronized everywhere in the code). I'd rather not go down this path if we can just wait for the custom user model. In the meantime, here's a PR that simply warns the user his account can't have a password reset: https://github.com/mozilla/olympia/pull/67
(In reply to Kris Maglione [:kmag] from comment #15) > 2) `password` column is empty in the `auth_user` table, but not in the > `users` table. Ah, and this is somewhat problemmatic in that we don't seem to ever set the password column in the `auth_user` table under most circumstances... (In reply to Mathieu Agopian [:magopian] from comment #18) > I'm -1 on reseting the hash to a randomly generated password: that means one > could simply reset someone's password without even using the generated > password reset link. It wouldn't give him access to the account, but it > would still be a huge inconvenience for the target. I meant only when the hash was considered invalid, but, yeah, it does still have some potential for abuse. > What is the sha512+base64 algorithm doing? If it's base64 encoding the > sha512, then I guess we could run a migration to unencode all those or > create a custom hasher. If it's the other way round, we're out of luck. It's just b64 encoding the salt. Which means we can only conceivably unencode it if the decoded form doesn't interfere with the hash format (i.e., it has no $ in it). I don't have the getpersonas code in front of me, so I don't know how the salts were generated. > That being said, how can users with a sha512+base64 algorithm login to the > site if there's no corresponding password hasher? I'm not sure what we actually use the hashers for, other than for that has_usable_password check. We do password checking manually in 1UserProfile.check_password` > There's also the issue with the auth_user and users passwords not being > properly synchronized: I believe this is an issue, and should be fixed when > we switch to using the custom user model (soon! > https://github.com/mozilla/olympia/pull/63). If you believe there's a lot of > accounts that have this specific issue, and if this issue is urgent, we'll > have to run a migration to copy the UserProfile.password to the > auth.User.password (and make sure it's synchronized everywhere in the code). > I'd rather not go down this path if we can just wait for the custom user > model. Given the number of users this seems to be affectiong, I'd rather this be done ASAP. I don't really care about syncing the password columns, though. Let's just check `has_usable_password` and give the `auth_user` column a dummy value if it fails.
I stumbled across this and thought it might be useful to mention: https://github.com/mozilla/zamboni/blob/master/apps/users/models.py#L71-L74
I reworked/completed the pull-request with a rather radical approach: until we have the custom user model (https://github.com/mozilla/olympia/pull/63), we monkey-patch the "has_usable_password" test (that now only checks the password isn't empty), and should thus not prevent emails from being sent. PR is still https://github.com/mozilla/olympia/pull/67
Hi Magopian - we've been seeing a lot of problems resetting passwords lately. If you could get this in for our push this week it would be good. Let me know if you need anything.
Flags: needinfo?(mathieu)
Fixed in https://github.com/mozilla/olympia/commit/ff96997206956615db18c677acf010fc284813da (ready for the push ;)
Status: NEW → RESOLVED
Closed: 11 years ago
Flags: needinfo?(mathieu)
Resolution: --- → FIXED
I am a simpleton user who fell victim to this bug multiple times. If someone would kindly tell me when to test it, I will try again to re-set my password.
We still experience this issue when attempting to reset our password. Any idea when the fix is going to be deployed?
Flags: needinfo?(mathieu)
When is this going to be pushed to prod?
I was able to reset my password 4 days ago. So the code must have been pushed, right?
Given that the schemas in the prod DB have changed as expected, I'd say yes.
I'll ask the user to try again, there might be a different bug at play.
Success, thank you very much!
Working for me as well. Thanks! (y)
Now it does not work again. Now I do receive the email. I click on the link and I go to https://addons.mozilla.org/en-US/firefox/users/pwreset/somehashhere , then I fill with new password, but after pressing "Update" it says: "Oops! Not allowed. You tried to do something that you weren't allowed to. Try going back to the previous page, refreshing and then trying again." I've tried going back to the previous page, refreshing and then trying again. It does not work either.
Can you send me the URL, including the hash, in an email? The only reasons I would expect a "Not allowed" page in that circumstance would be an expired password reset hash or CSRF token. But an expired hash should lead to the page not loading at all, and CSRF should not be an issue unless you're doing something strange with cookies.
OK, that's strange. The form for the password reset page is generated without a CSRF token unless you visit the login page first. Can you try clicking the 'Log in' link in the header and then visiting the password reset page again?
Now it worked. The previous time I was resetting from a different Firefox profile - different set of cookies... so that was the strange thing with cookies! Sorry I did not realize before. Thank you for your support.
(In reply to Martin Havelka from comment #25) > We still experience this issue when attempting to reset our password. Any > idea when the fix is going to be deployed? From the best of my knowledge, this fix has already been deployed to production.
(In reply to Mathieu Agopian [:magopian] from comment #36) > (In reply to Martin Havelka from comment #25) > > We still experience this issue when attempting to reset our password. Any > > idea when the fix is going to be deployed? > > From the best of my knowledge, this fix has already been deployed to > production. Thank you, we got the password updated manually meanwhile. I have checked the reset, and it seems to work.
Flags: needinfo?(mathieu)
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.