887753 - Server not found after reconnecting to etherpad

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[Honza Bambas (:mayhemer)]

http://hg.mozilla.org/releases/mozilla-aurora/rev/40c9d5acd1b7

https://etherpad.mozilla.org/U7EheATozG

I always just sleep my machine, never shutdown.  After wake up (after some 12 hours, or even less!) when I go to the tab with etherpad, there is a popup telling me to Reconnect.  I press the Reconnect button and get regular error page "Server not found".

Wireshark tells me we do a request for 'etherpad.mozilla.org' and get answer:

12	13:51:09.702044	fe80::219d:2101:836a:1eec	fe80::1	DNS	100	Standard query 0x220b  A etherpad.mozilla.org

16	13:51:10.242770	fe80::1	fe80::219d:2101:836a:1eec	DNS	218	Standard query response 0x220b  A 63.245.217.90

18	13:51:10.700628	192.168.5.12	192.168.5.1	DNS	80	
Standard query 0x2594  A etherpad.mozilla.org

19	13:51:10.711489	192.168.5.1	192.168.5.12	DNS	198	Standard query response 0x2594  A 63.245.217.90

I don't have a NSPR log, but I can create one if necessary.

100% reproducable for me.

Comment 1

[Patrick McManus [:mcmanus]]

honza, jst managed to get a nspr log for something very similar sounding during the workweek. Its got a bunch of cookies/etc in it so I won't attach it here but jst will probably let me share it if you need it.

however, we might not need to because I've at least got a guess at what is going on from the log. I can't repro it - but I have seen it once or twice.

If you can repro it can you try out a try build with the potential fix?

(try is closed right now, I'll update the bug later with a cset)

Comment 2

[Patrick McManus [:mcmanus]]

  https://tbpl.mozilla.org/?tree=Try&rev=06a8d89754dd

Comment 4

[Patrick McManus [:mcmanus]]

tagging NI as I need someone from {jst, dougt, mayhemer} to evaluate try build and see if it fixes this issue.

Comment 5

[Honza Bambas (:mayhemer)]

So far inconclusive.  I always hit this when I sit to my machine new day and wake it and go to the tab with etherpad.  I left your build running during the night and tried exactly those steps.  It worked.  But then I tried to do the same in my regular profile with Aurora and it worked too.  So it's hard to say if it's fixed or not.

I'll do more attempts.

Comment 6

[Honza Bambas (:mayhemer)]

Patrick, I can no longer reproduce on my regular profile with Aurora :(  Bitch...

So I cannot confirm the fix.

Comment 7

[Johnny Stenback (:jst)]

I've ran a stock nightly build side by side with Patrick's try build today and I've seen the error show up 3 times so far in the stock nightly build, and zero times in the try build!

Comment 8

[Doug Turner (:dougt)]

post patches - inquiring minds want to know!

Comment 9

[Patrick McManus [:mcmanus]]

Attachment: server not found after reconnecting to etherpad

Comment 10

[Patrick McManus [:mcmanus]]

Attachment: server not found after reconnecting to etherpad

Comment 11

[Patrick McManus [:mcmanus]]

Attachment: server not found after reconnecting to etherpad

Comment 12

[Patrick McManus [:mcmanus]]

Attachment: server not found after reconnecting to etherpad

Comment 13

[Patrick McManus [:mcmanus]]

Attachment: 887753

jst's nspr log suggests a very quick dns failure. I suspect that's the inappropriate use of a negative cache entry. Its at least consistent.

negative entries are important to prefetch - because they can see a LOT of invalid names quickly and really jam up the q's with redundant references to them. (we consilidate overlapping requests, but it is still a problem without the negative cache)

So this patch just says to only use negative cache entries for < HIGH (i.e. speeculative) priority requests.. as a matter of hygiene it also dooms negative entries if they are used for a connection attempt.

Comment 14

[Patrick McManus [:mcmanus]]

heh.. bzexport gave me an error each time I ran the command but it attached the diff anyhow each time (which I didn't notice). Sorry for the bugspam.

Comment 15

[Steve Workman [:sworkman] (INACTIVE)]

Comment on attachment 778764 [details] [diff] [review]
887753

Review of attachment 778764 [details] [diff] [review]:
-----------------------------------------------------------------

Had a good chat with Pat over email about this. Patch looks good to me r=me.

Comment 16

[Patrick McManus [:mcmanus]]

remote:   https://hg.mozilla.org/integration/mozilla-inbound/rev/540de81dc176

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/540de81dc176