634390 - [traceback] 500 error with invalid parameters to search

Status: VERIFIED FIXED

(developer.mozilla.org Graveyard :: Demo Studio / Dev Derby, defect)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

https://developer-stage9.mozilla.org/en-US/demos/search/?q=%BF%27%22%28&sort=likes triggers an exception, though I don't have a call stack/traceback.

Comment 1

[Luke Crouch [:groovecoder]]

I do! I have 225 of them in my mailbox and they're piling up!

Traceback (most recent call last):

  File "/data/www/django/developer.mozilla.org/mdn/vendor/packages/Django/django/core/handlers/base.py", line 100, in get_response
    response = callback(request, *callback_args, **callback_kwargs)

  File "/data/www/django/developer.mozilla.org/mdn/apps/demos/views.py", line 106, in search
    template_name='demos/listing_search.html')

  File "/data/www/django/developer.mozilla.org/mdn/vendor/packages/Django/django/views/generic/list_detail.py", line 101, in object_list
    return HttpResponse(t.render(c), mimetype=mimetype)

  File "/data/www/django/developer.mozilla.org/mdn/lib/utils.py", line 104, in render
    return self.template.render(context_dict)

  File "/data/www/django/developer.mozilla.org/mdn/vendor/packages/jinja2/jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)

  File "/data/www/django/developer.mozilla.org/mdn/apps/demos/templates/demos/listing_search.html", line 5, in top-level template code
    {% set query = ' ' %}

  File "/data/www/django/developer.mozilla.org/mdn/apps/demos/templates/demos/base.html", line 1, in top-level template code
    {% extends "base_compact.html" %}

  File "/data/www/django/developer.mozilla.org/mdn/templates/base_compact.html", line 1, in top-level template code
    {% extends "base.html" %}

  File "/data/www/django/developer.mozilla.org/mdn/templates/base.html", line 33, in top-level template code
    {% block extrahead %}{% endblock %}

  File "/data/www/django/developer.mozilla.org/mdn/apps/demos/templates/demos/listing_search.html", line 15, in block "extrahead"
    href="{{ url('demos_feed_search', format='atom', query_string=query) }}" />

  File "/data/www/django/developer.mozilla.org/mdn/apps/devmo/helpers.py", line 57, in url
    return reverse(viewname, args=args, kwargs=kwargs)

  File "/data/www/django/developer.mozilla.org/mdn/apps/devmo/urlresolvers.py", line 27, in reverse
    url = django_reverse(viewname, urlconf, args, kwargs, prefix)

  File "/data/www/django/developer.mozilla.org/mdn/vendor/packages/Django/django/core/urlresolvers.py", line 350, in reverse
    *args, **kwargs)))

  File "/data/www/django/developer.mozilla.org/mdn/vendor/packages/Django/django/core/urlresolvers.py", line 296, in reverse
    "arguments '%s' not found." % (lookup_view_s, args, kwargs))

NoReverseMatch: Reverse for 'demos_feed_search' with arguments '()' and keyword arguments '{'query_string': u"-Search%20the%20Demo%20Studio'OR/**/1=1/**/AND/**/ISNULL(ASCII(SUBSTRING(CAST((SELECT/**/@@version)AS/**/varchar(8000)),1,1)),0)>0", 'format': 'atom'}' not found.

Comment 2

[Les Orchard [:lorchard]]

I'm hoping that this fixes the search exceptions:
https://github.com/fwenzel/mdn/commit/8456b75b0b50e43046ed6aed97d277e4f65b0eeb

But, I might need to see a bunch of them to tell what's going on if this doesn't do the trick. That specific exception in comment 1 (ie. "Reverse for 'demos_feed_search'...not found") isn't a MySQL-related exception like the one I reproduced, and is instead coming from somewhere in Django internals. That would be annoying.

Comment 3

[Les Orchard [:lorchard]]

https://github.com/fwenzel/mdn/commit/c9e708a18791408a9feedf1501d66f4fb28eb1e4

This should address the specific error from comment 1, where Django URL mappings don't like URL segments with slashes in them

Comment 4

[Stephen Donner [:stephend] Not actively reading bugmail]

To be clear, Les, you just need the traceback from comment 0, then?

http://pastebin.mozilla.org/1070329 has it, if so.

Comment 5

[Les Orchard [:lorchard]]

No, I need all tracebacks. 

One bug was exposed by comment 0, and a second bug was exposed by the traceback Luke posted in comment 1. Without seeing a good sample of tracebacks, I can't know if those were the only two bugs exposed in testing.

Comment 6

[Luke Crouch [:groovecoder]]

http://pastebin.mozilla.org/1070329 is representative of 300+ tracebacks I've seen. from 

[Django] Error (EXTERNAL IP): /en-US/demos/search/
and
[Django] Error (EXTERNAL IP): /en-US/demos/feeds/atom/search/
and
[Django] Error (EXTERNAL IP): /zh-TW/demos/search/

Comment 7

[Les Orchard [:lorchard]]

Hmm, the DB error is fixed on my dev machine after applying the migration, but still seems to be breaking on stage9.

Maybe the migration I checked in to ensure all tables were UTF8 didn't apply? Operating blind there, since I can't see what happens on staging.

Comment 8

[Luke Crouch [:groovecoder]]

Looks like migrations failed or never ran?

e.g.,

mysql> show create table demos_submission;
...
) ENGINE=MyISAM AUTO_INCREMENT=29 DEFAULT CHARSET=latin1 | 

Jeremy can we try to run the migrations manually as in the update_staging.sh script?

mdn$ python26 vendor/src/schematic/schematic migrations/

Comment 9

[Les Orchard [:lorchard]]

Fixing one exception reveals another, which I've also fixed:
https://github.com/fwenzel/mdn/commit/16b05096c267690615ba5b592f233a674036e122

Comment 10

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED; see screenshot.

Comment 11

[Stephen Donner [:stephend] Not actively reading bugmail]

Actually, instead of a screenshot, this is more helpful:

https://developer-stage9.mozilla.org/en-US/demos/search/?q=+/
Attacking urls (GET)...
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=http%3A%2F%2Fwww.google.com%2F
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=%2Fetc%2Fpasswd
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=%2Fetc%2Fpasswd%00
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=c%3A%5C%5Cboot.ini
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=c%3A%5C%5Cboot.ini%00
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2Fetc%2Fpasswd
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2Fetc%2Fpasswd%00
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2Fboot.ini
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2Fboot.ini%00
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=a%3Benv
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=a%29%3Benv
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=%2Fe%00
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=%BF%27%22%28
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=<script>var+pf_68747470733a2f2f646576656c6f7065722d7374616765392e6d6f7a696c6c612e6f72672f656e2d55532f64656d6f732f7365617263682f_71=new+Boolean();</script>
+ https://developer-stage9.mozilla.org/en-US/demos/search/?q=http%3A%2F%2Fwww.google.com%0D%0APowerfuzzer%3A+v1+BETA
Looking for permanent XSS