1324026 - DXR crash when trying to access an old file rev living in a folder which moved

Status: RESOLVED FIXED

(Webtools Graveyard :: DXR, defect)

Description

[Ludovic Hirlimann [:Usul]]

https://dxr.mozilla.org/mozilla-central/rev/8103c612b79c2587ea4ca1b0a9f9f82db4b185b8/testing/marionette/harness/marionette/runner/base.py#933
500 internla sever error.

Nothing in nagios

Comment 1

[Jake Watkins [:dividehex]]

This is the traceback being logged from apache.  

[Fri Dec 16 17:11:30 2016] [error] Exception on /mozilla-central/rev/8103c612b79c2587ea4ca1b0a9f9f82db4b185b8/testing/marionette/harness/marionette/runner/base.py [GET]
[Fri Dec 16 17:11:30 2016] [error] Traceback (most recent call last):
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
[Fri Dec 16 17:11:30 2016] [error]     response = self.full_dispatch_request()
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
[Fri Dec 16 17:11:30 2016] [error]     rv = self.handle_user_exception(e)
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
[Fri Dec 16 17:11:30 2016] [error]     reraise(exc_type, exc_value, tb)
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
[Fri Dec 16 17:11:30 2016] [error]     rv = self.dispatch_request()
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
[Fri Dec 16 17:11:30 2016] [error]     return self.view_functions[rule.endpoint](**req.view_args)
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/dxr/app.py", line 593, in rev
[Fri Dec 16 17:11:30 2016] [error]     contents = file_contents_at_rev(abs_path, revision)
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/dxr/vcs.py", line 414, in file_contents_at_rev
[Fri Dec 16 17:11:30 2016] [error]     return cls.get_contents(abspath, revision, stderr=devnull)
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/dxr/vcs.py", line 191, in get_contents
[Fri Dec 16 17:11:30 2016] [error]     return cls.invoke_vcs(['cat', '-r', revision, tail], head, stderr=stderr)
[Fri Dec 16 17:11:30 2016] [error]   File "/data/prod/dxr-prod/virtualenv/lib/python2.7/site-packages/dxr/vcs.py", line 57, in invoke_vcs
[Fri Dec 16 17:11:30 2016] [error]     return subprocess.check_output([cls.command] + args, cwd=cwd, **kwargs)
[Fri Dec 16 17:11:30 2016] [error]   File "/usr/lib64/python2.7/subprocess.py", line 566, in check_output
[Fri Dec 16 17:11:30 2016] [error]     process = Popen(stdout=PIPE, *popenargs, **kwargs)
[Fri Dec 16 17:11:30 2016] [error]   File "/usr/lib64/python2.7/subprocess.py", line 710, in __init__
[Fri Dec 16 17:11:30 2016] [error]     errread, errwrite)
[Fri Dec 16 17:11:30 2016] [error]   File "/usr/lib64/python2.7/subprocess.py", line 1335, in _execute_child
[Fri Dec 16 17:11:30 2016] [error]     raise child_exception
[Fri Dec 16 17:11:30 2016] [error] OSError: [Errno 2] No such file or directory: '/data/prod/src/mozilla-central/testing/marionette/harness/marionette/runner'

Comment 2

[Jake Watkins [:dividehex]]

:erik, fubar is on PTO this week and I'm not very familiar with the dxr undercarriage.  Would you have any insight to what is going on here?  It looks like paths where changed on m-c between rev 8103c612b79c and @ therefore the path causing the 'No such file or directory' is no longer valid to what is on disk.

I'll keep poking around at this unless you can point me certain direction here.
Thanks

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So I reported this problem earlier today on the #moc channel. What I can add is that this file has been moved to another location in a newer commit. It looks like that this situation can happen for references to specific files in an old changeset, whereby the file is location somewhere else nowadays.

Specifically here it means it is at testing/marionette/harness/marionette_harness/runner/base.py now.

Comment 4

[Erik Rose [:erik][:erikrose]]

So /data/prod/src/mozilla-central/testing/marionette/harness/marionette/runner is indeed no longer around? (I can't get onto the VPN right now, or I'd check myself.) That would make sense, then: the indexed tree is, of course, a little behind what's on disk. I suspect the problem is that DXR's VCS module, which is kind of creaky, is trying to cd to the old, nonexistent dir before running the `hg cat` command. I'll see if I can repro it by doing that.

Comment 5

[Erik Rose [:erik][:erikrose]]

Repro'd.

Comment 6

[Erik Rose [:erik][:erikrose]]

We can fix this by cd-ing to the root of the source tree, not to the folder deep within. I don't know why we were traversing so far in to begin with. Creaky!

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/dxr

https://github.com/mozilla/dxr/commit/f3cc71a98975c99c34483fdc827489245b9387b9
When pulling a file out of a VCS revision, don't assume its containing folder still exists. Fixes bug 1324026.

It may well have moved in the checkout's revision. So stop cd-ing to its containing folder. Instead, go into the deepest existing folder on its containing path, and run the VCS commands against the remainder of the relative path from there. Going into the deepest existing folder is a way to get the correct VCS invoked, even if there are nested repositories in the tree.

Refactor to create GenerativeTestCase, which should let us write these VCS test cases without tarring up an .hg or .git dir, which is hard to modify or examine and tempts us to couple too many tests together.