1700508 - Perma Assertion failure: !(aSource == kCharsetFromChannel && !aChannelHadCharset) (If charset is from channel, channel must have had charset.), at /builds/worker/checkouts/gecko/parser/html/nsHtml5StreamParser.h:245

Status: RESOLVED FIXED

(Thunderbird :: Upstream Synchronization, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: geoff [at] darktrojan.net
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=334221901&repo=comm-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cx4cFWQ7QfurPfV5WYlfxQ/runs/0/artifacts/public/logs/live_backing.log

---

Comment 1

[Intermittent Failures Robot]

16 failures in 778 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• comm-central: 16

Platform and build breakdown:

• linux64: 4
  • debug: 4
• windows10-64: 4
  • debug: 4
• windows7-32: 4
  • debug: 4
• macosx1014-64: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1700508&startday=2021-03-23&endday=2021-03-23&tree=all

Comment 2

[Masatoshi Kimura [:emk]]

I think this line should be changed to

  parser->SetDocumentCharset(encoding, charsetSource, charsetSource == kCharsetFromChannel);

.

Comment 3

[Henri Sivonen (:hsivonen)]

(In reply to Masatoshi Kimura [:emk] from comment #2)

I think this line should be changed to

  parser->SetDocumentCharset(encoding, charsetSource, charsetSource == kCharsetFromChannel);

.

nsSAXXMLReader is never supposed to combine with nsHtml5StreamParser, so I suspect the reason is elsewhere.

Comment 4

[Masatoshi Kimura [:emk]]

Sure.

Comment 5

[Masatoshi Kimura [:emk]]

Here is another hypothesis.

nsMessenger::SetDisplayCharset calls SetHintCharacterSetSource(kCharsetFromChannel):
https://searchfox.org/comm-central/rev/938bb1a03265a9e9c82ed0abcb7ec42c66fdbccf/mailnews/base/src/nsMessenger.cpp#279

So charsetSource may change to kCharsetFromChannel without updating channelHadCharset here:
https://searchfox.org/mozilla-central/rev/6c9ff2820d3aae683ec87c53c79e5108d54f3f76/dom/html/nsHTMLDocument.cpp#483

Comment 6

[Intermittent Failures Robot]

16 failures in 164 pushes (0.098 failures/push) were associated with this bug yesterday.

Repository breakdown:

• comm-central: 16

Platform and build breakdown:

• linux64: 4
  • debug: 4
• windows10-64: 4
  • debug: 4
• windows7-32: 4
  • debug: 4
• macosx1014-64: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1700508&startday=2021-03-27&endday=2021-03-27&tree=all

Comment 7

[Intermittent Failures Robot]

42 failures in 3967 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #34 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 42

Platform and build breakdown:

• linux64: 10
  • debug: 10
• windows10-64: 10
  • debug: 10
• windows7-32: 8
  • debug: 8
• macosx1014-64: 13
  • debug: 12
  • opt: 1
• macosx1014-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1700508&startday=2021-03-22&endday=2021-03-28&tree=all

Comment 8

[Magnus Melin [:mkmelin]]

Should the channelHadCharset assignment just be moved down to after the tries have concluded?

Comment 9

[Geoff Lankow (:darktrojan)]

I've been looking at this. We should look at fixing our channel implementations to return a charset. Hard-wiring it to UTF-8 almost works, as pointed out here libmime always converts to UTF-8. However if we just output the raw data, for example to view source, it might not be UTF-8. Could we set the channel charset only if libmime gets involved? I think that would prevent this assertion failure – nsHtml5StreamParser's not going to be involved in any other situation that I can think of.

Comment 10

[Geoff Lankow (:darktrojan)]

Here's a Try run showing what happens if you hard-wire UTF-8 in the channels. Only the view source test is complaining.

https://treeherder.mozilla.org/jobs?repo=try-comm-central&group_state=expanded&revision=685fc2ef6c30cf85246764a796832d3ce5f742ad

Comment 11

[Masatoshi Kimura [:emk]]

(In reply to Magnus Melin [:mkmelin] from comment #8)

Should the channelHadCharset assignment just be moved down to after the tries have concluded?

No, if TryUserForcedCharset changes the charsetSource to something stronger than kCharsetFromChannel, charsetSource is not kCharsetFromChannel while channelHadCharset is true. channelHadCharset has been introduced to handle this exact situation.

Comment 12

[Henri Sivonen (:hsivonen)]

Since the source is an internal component that always sets the encoding to UTF-8, I suggest changing the source value to kCharsetFromBuiltIn.

Comment 13

[Henri Sivonen (:hsivonen)]

However if we just output the raw data, for example to view source, it might not be UTF-8.

Except kCharsetFromBuiltIn doesn't make sense for that case.

Comment 14

[Geoff Lankow (:darktrojan)]

I don't think that case should be coming through here, so that's alright.

Comment 15

[Geoff Lankow (:darktrojan)]

I just realised my Try run in comment 10 was opt when it should've been debug. I wondered why it went so well. :-/

Comment 16

[Geoff Lankow (:darktrojan)]

Attachment: Bug 1700508 - Fix assertion failure in nsHtml5StreamParser.h. r?mkmelin

Thanks to hsivonen for the suggestion.

Comment 17

[Intermittent Failures Robot]

17 failures in 647 pushes (0.026 failures/push) were associated with this bug yesterday.

Repository breakdown:

• comm-central: 17

Platform and build breakdown:

• linux64: 4
  • debug: 4
• windows10-64: 4
  • debug: 4
• macosx1014-64: 5
  • debug: 5
• windows7-32: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1700508&startday=2021-03-29&endday=2021-03-29&tree=all

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/0aa11a1c79fa
Fix assertion failure in nsHtml5StreamParser.h. r=mkmelin

Comment 19

[Intermittent Failures Robot]

17 failures in 3125 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 17

Platform and build breakdown:

• linux64: 4
  • debug: 4
• windows10-64: 4
  • debug: 4
• macosx1014-64: 5
  • debug: 5
• windows7-32: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1700508&startday=2021-03-29&endday=2021-04-04&tree=all