568317 - [k] AssertionError: Node has no children

Status: VERIFIED FIXED

(support.mozilla.org :: Search, defect)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

https://support-stage-new.mozilla.com/en-US/search?q=freezing&page=2:

Traceback (most recent call last):

  File "/data/virtualenvs/kitsune/src/django/django/core/handlers/base.py", line 101, in get_response
    response = callback(request, *callback_args, **callback_kwargs)

  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/search/views.py", line 341, in search
    excerpt = fc.excerpt(forum_thread.data, cleaned['q'])

  File "/data/www/support-stage-new.mozilla.com/kitsune/apps/search/clients.py", line 149, in excerpt
    return self.bleach.clean(excerpt)

  File "/data/virtualenvs/kitsune/src/bleach/bleach/__init__.py", line 71, in clean
    return force_unicode(_serialize(parser.parseFragment(string)))

  File "/data/virtualenvs/kitsune/src/bleach/bleach/__init__.py", line 151, in _serialize
    return serializer.render(stream)

  File "/data/virtualenvs/kitsune/lib64/python2.6/site-packages/html5lib/serializer/htmlserializer.py", line 224, in render
    return u"".join(list(self.serialize(treewalker)))

  File "/data/virtualenvs/kitsune/lib64/python2.6/site-packages/html5lib/serializer/htmlserializer.py", line 103, in serialize
    for token in treewalker:

  File "/data/virtualenvs/kitsune/lib64/python2.6/site-packages/html5lib/treewalkers/_base.py", line 143, in __iter__
    firstChild = self.getFirstChild(currentNode)

  File "/data/virtualenvs/kitsune/lib64/python2.6/site-packages/html5lib/treewalkers/simpletree.py", line 52, in getFirstChild
    assert node.hasContent(), "Node has no children"

AssertionError: Node has no children

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

https://support-stage-new.mozilla.com/en-US/search?q=192.168.1.1 might be an easier way to reproduce this?

Comment 2

[James Socol [:jsocol, :james]]

Jeff tells us empty strings can cause this, but only with certain extra configs.

Comment 3

[Jeff Balogh (:jbalogh)]

>>> import bleach
>>> bleach.Bleach().bleach(u'')
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
  File "/Users/jeff/dev/zamboni/vendor/src/bleach/bleach/__init__.py", line 60, in bleach
    return self.linkify(self.clean(string))
  File "/Users/jeff/dev/zamboni/vendor/src/bleach/bleach/__init__.py", line 71, in clean
    return force_unicode(_serialize(parser.parseFragment(string)))
  File "/Users/jeff/dev/zamboni/vendor/src/bleach/bleach/__init__.py", line 151, in _serialize
    return serializer.render(stream)
  File "/Users/jeff/dev/zamboni/vendor/packages/html5lib/src/html5lib/serializer/htmlserializer.py", line 224, in render
    return u"".join(list(self.serialize(treewalker)))
  File "/Users/jeff/dev/zamboni/vendor/packages/html5lib/src/html5lib/serializer/htmlserializer.py", line 103, in serialize
    for token in treewalker:
  File "/Users/jeff/dev/zamboni/vendor/packages/html5lib/src/html5lib/treewalkers/_base.py", line 143, in __iter__
    firstChild = self.getFirstChild(currentNode)
  File "/Users/jeff/dev/zamboni/vendor/packages/html5lib/src/html5lib/treewalkers/simpletree.py", line 52, in getFirstChild
    assert node.hasContent(), "Node has no children"
AssertionError: Node has no children

Comment 4

[Jeff Balogh (:jbalogh)]

Quick fix is to bail early on the empty string.

Comment 5

[Paul Craciunoiu [:paulc]]

(In reply to comment #4)
> Quick fix is to bail early on the empty string.
That won't work because of stripping comments (fails the 2nd test here [1])

Seems like prefixing it with whitespace does the trick, though:
[1] http://github.com/pcraciunoiu/bleach/commit/acd3ec022f82fad58a32252f0971114ac105559a

Comment 6

[Paul Craciunoiu [:paulc]]

A similar things needs to happen to linkify - except for some reason prepending a space there causes this test to fail, like so:

======================================================================
FAIL: bleach.tests.test_links.test_mangle_link
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/paulc/.virtualenvs/kitsune/lib/python2.6/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/Users/paulc/.virtualenvs/kitsune/src/bleach/bleach/tests/test_links.py", line 25, in test_mangle_link
    c.linkify('http://example.com'))
  File "/Users/paulc/.virtualenvs/kitsune/lib/python2.6/site-packages/nose/tools.py", line 31, in eq_
    assert a == b, msg or "%r != %r" % (a, b)
AssertionError: '<a href="http://bouncer/?u=http%3A%2F%2Fexample.com" rel="nofollow">http://example.com</a>' != u'<a href="http://bouncer/?u=http%3A%2F%2Fbouncer%2F%3Fu%3Dhttp%253A%252F%252Fexample.com" rel="nofollow">http://example.com</a>'


James: maybe it has to do with the linkify regex?

Comment 7

[Paul Craciunoiu [:paulc]]

(In reply to comment #6)
> James: maybe it has to do with the linkify regex?
Nevermind, I was Doing It Wrong(tm)

Comment 8

[Paul Craciunoiu [:paulc]]

Thanks to Jeff and James for all the feedback.

http://github.com/jsocol/bleach/commit/84ade19ff1dc5af36ab9a32e5c4b1f0eb7580295

Comment 9

[James Socol [:jsocol, :james]]

(In reply to comment #8)
> Thanks to Jeff and James for all the feedback.
> 
> http://github.com/jsocol/bleach/commit/84ade19ff1dc5af36ab9a32e5c4b1f0eb7580295

There were changes to py-wikimarkup, too, right? You should link those here.

Comment 10

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED; tested on both https://support-stage-new.mozilla.com/en-US/search?q=192.168.1.1 and https://support-stage-new.mozilla.com/en-US/search?q=freezing&page=2

Comment 11

[Paul Craciunoiu [:paulc]]

(In reply to comment #9)
> There were changes to py-wikimarkup, too, right? You should link those here.
I linked them in bug 566101.