Closed Bug 1280948 Opened 8 years ago Closed 8 years ago

index add-ons

Categories

(Webtools Graveyard :: DXR, defect)

Product:
Webtools Graveyard
Component:
DXR
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: fubar, Assigned: fubar)

References

Details

We need to index the addons tree, same as we've done on MXR. 

:jason, is there anything you need to do on your side (eg ip addr restrictions on the ssh key), or can I just copy the bits from mxr-processor1:~/addons_update and change to suit ?
Flags: needinfo?(jthomas)
I assume we'll need some kind of auth before showing the addons results as was the case with MXR. Not all the add-ons have a license that allows republishing, and some add-ons we scan are not public at all.
never mind -- I now see bug 1279107 in the dependency list.
(In reply to Kendall Libby [:fubar] from comment #0)
> :jason, is there anything you need to do on your side (eg ip addr
> restrictions on the ssh key), or can I just copy the bits from
> mxr-processor1:~/addons_update and change to suit ?

there isn't; currently syncing over to the dxr data store.
(In reply to Kendall Libby [:fubar] from comment #0)
> :jason, is there anything you need to do on your side (eg ip addr
> restrictions on the ssh key), or can I just copy the bits from
> mxr-processor1:~/addons_update and change to suit ?

I am currently restricting by IP Address and only allow the rsync command. What IP Address should I allow on our end?
(In reply to Jason Thomas [:jason] from comment #4) 
> I am currently restricting by IP Address and only allow the rsync command.
> What IP Address should I allow on our end?

10.22.75.78, but I think we must have set that up back in December, since the rsync command just finished. :-)
Flags: needinfo?(jthomas)
Blocks: 1281443
Assignee: nobody → klibby
So here's a fun twist. There are ~44,000 directories in /addons, some number of which don't have a recognized VCS (which makes some sense, there are actually dumps from the addons db, and who knows what the creator is actually using).

Erik, Peter: how do we work around this?

Starting tree 'addons'.
/builds/dxr-build-env/dxr/dxr/vcs.py:186: UserWarning: Your git remote is not supported yet. Please use a GitHub remote if you would like version control naviagtion links to show.
  warn("Your git remote is not supported yet. Please use a "
fatal: Not a git repository (or any parent up to mount point /builds/dxr-build-env/src)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Traceback (most recent call last):
  File "venv/bin/dxr", line 9, in <module>
    load_entry_point('dxr==2.0', 'console_scripts', 'dxr')()
  File "/builds/dxr-build-env/dxr/dxr/cli/__init__.py", line 29, in main
    return dxr.invoke(ctx)
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/builds/dxr-build-env/dxr/dxr/cli/index.py", line 26, in index
    index_and_deploy_tree(tree, verbose=verbose)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 61, in index_and_deploy_tree
    index_name = index_tree(tree, es, verbose=verbose)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 208, in index_tree
    vcs_cache = VcsCache(tree)
  File "/builds/dxr-build-env/dxr/dxr/vcs.py", line 359, in __init__
    self.repos = tree_to_repos(tree)
  File "/builds/dxr-build-env/dxr/dxr/vcs.py", line 314, in tree_to_repos
    attempt = vcs.claim_vcs_source(cwd, dirs, tree)
  File "/builds/dxr-build-env/dxr/dxr/vcs.py", line 195, in claim_vcs_source
    return cls(path)
  File "/builds/dxr-build-env/dxr/dxr/vcs.py", line 167, in __init__
    self.invoke_vcs(['ls-files'], self.root).splitlines())
  File "/builds/dxr-build-env/dxr/dxr/vcs.py", line 56, in invoke_vcs
    return subprocess.check_output([cls.command] + args, cwd=cwd, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['git', 'ls-files']' returned non-zero exit status 128
Flags: needinfo?(peter.elmers)
Flags: needinfo?(erik)
Currently DXR decides that the directory visited has a git repo based on the existence of a .git folder, but we should also check some git command can execute without failing before claiming the folder as tracked.

Same for hg, I imagine.
Flags: needinfo?(peter.elmers)
Looking into this, I'm expecting we'll have to clean up the upstream URL handling; Mercurial will fail if a "default" path does not exist, and Git will fail if the "origin" remote does not exist.
What's a .git folder doing at the root level anyway? It seems like getting rid of that would work around the problem for the moment.
Flags: needinfo?(erik)
Tom saw this problem coming, https://github.com/mozilla/dxr/pull/508, so I'll review that now.
(In reply to Erik Rose [:erik][:erikrose] from comment #9)
> What's a .git folder doing at the root level anyway? It seems like getting
> rid of that would work around the problem for the moment.

there isn't one. nor a .hg dir. they're all in the individual addon subfolder, if at all.
(In reply to Peter Elmers [:new_one] from comment #10)
> Tom saw this problem coming, https://github.com/mozilla/dxr/pull/508, so
> I'll review that now.

that definitely helped. but now we've got a new and exciting error!


(top few lines are "normal" in that there are about a bajillion of them)

664166/data/Example/gmail.js Error Line 1: Unexpected token (
664130/bootstrap.js Error Line 542: Unexpected token (
6639/chrome/easydragtogo/content/easydragtogoConfig.js Error Line 12: Unexpected identifier

<--- Last few GCs --->

 3860967 ms: Mark-sweep 1386.9 (1435.0) -> 1386.8 (1435.0) MB, 1482.0 / 0 ms (+ 0.6 ms in 1 steps since start of marking, biggest step 0.6 ms) [allocation failure] [GC in old space requested].
 3862221 ms: Mark-sweep 1386.8 (1435.0) -> 1386.8 (1435.0) MB, 1254.0 / 0 ms [last resort gc].
 3863474 ms: Mark-sweep 1386.8 (1435.0) -> 1386.8 (1435.0) MB, 1252.9 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3aab57fc9e59 <JS Object>
    1: consumeSemicolon(aka consumeSemicolon) [/builds/dxr-build-env/dxr/dxr/plugins/js/analyze_js/node_modules/esprima/esprima.js:2602] [pc=0x1bfd4528d394] (this=0x3aab57f04189 <undefined>)
    2: parseStatement(aka parseStatement) [/builds/dxr-build-env/dxr/dxr/plugins/js/analyze_js/node_modules/esprima/esprima.js:~4737] [pc=0x1bfd45367ca3] (this=0x3aab57f04189 <undefined>)
    3: parseStatem...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
Running post_build [####################################] 100%
Indexing folders   [-----------------------------------#]
Traceback (most recent call last):
  File "venv/bin/dxr", line 9, in <module>
    load_entry_point('dxr==2.0', 'console_scripts', 'dxr')()
  File "/builds/dxr-build-env/dxr/dxr/cli/__init__.py", line 29, in main
    return dxr.invoke(ctx)
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/builds/dxr-build-env/dxr/dxr/cli/index.py", line 26, in index
    index_and_deploy_tree(tree, verbose=verbose)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 61, in index_and_deploy_tree
    index_name = index_tree(tree, es, verbose=verbose)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 266, in index_tree
    index_files(tree, tree_indexers, index, pool, es)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 611, in index_files
    index_folders(tree, index, es)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 599, in index_folders
    'is_folder': True})
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/pyelasticsearch/client.py", line 93, in decorate
    return func(*args, query_params=query_params, **kwargs)
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/pyelasticsearch/client.py", line 366, in index
    query_params)
  File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/pyelasticsearch/client.py", line 278, in send_request
    raise InvalidJsonResponseError(exc.args[0])
pyelasticsearch.exceptions.InvalidJsonResponseError: {'path': ['553992/locale/ru-RU/\x8d\xae\xa2\xa0\xef \xaf\xa0\xaf\xaa\xa0'], 'folder': '553992/locale/ru-RU', 'name': '\x8d\xae\xa2\xa0\xef \xaf\xa0\xaf\xaa\xa0', 'is_folder': True}
I think the invalid JSON error is due to that non-UTF-8-encoded path. We should skip paths that can't be UTF-8 decoded or, to get fancier, throw chardet at them.
https://github.com/mozilla/dxr/commit/bcecadc60e5b9525356833777be642d26fa0d079 will just skip indexing any folders/files under non-utf8-decodeable paths.
A test run on Friday evening got to 41% through indexing files before this:

A worker failed while indexing /builds/dxr-build-env/src/addons/619174/chrome/DropTimePlus/locale/jp-JP/コピー - clock.dtd:
Traceback (most recent call last):
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 578, in index_chunk
    index_file(tree, tree_indexers, path, es, index)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 471, in index_file
    append_update(needles, file_to_index.needles())
  File "/builds/dxr-build-env/dxr/dxr/utils.py", line 122, in append_update
    for k, v in pairs:
  File "/builds/dxr-build-env/dxr/dxr/plugins/core.py", line 456, in needles
    if self.is_link():
  File "/builds/dxr-build-env/dxr/dxr/indexers.py", line 321, in is_link
    return islink(self.absolute_path())
  File "/builds/dxr-build-env/venv/lib/python2.7/posixpath.py", line 142, in islink
    st = os.lstat(path)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-75: ordinal not in range(128)
Some looking around (http://stackoverflow.com/questions/2076708/python-os-stat-and-unicode-file-names) suggests that os.stat tries to decode the path to system codec (LANG, I think). Can we confirm the machine's set to utf-8 by default?
Flags: needinfo?(klibby)
(In reply to Peter Elmers [:new_one] from comment #16)
> Some looking around
> (http://stackoverflow.com/questions/2076708/python-os-stat-and-unicode-file-
> names) suggests that os.stat tries to decode the path to system codec (LANG,
> I think). Can we confirm the machine's set to utf-8 by default?

nope.

jenkins@1a4dd6e4ae63:/builds/dxr-build-env$ locale -a
C
C.UTF-8
POSIX
jenkins@1a4dd6e4ae63:/builds/dxr-build-env$ locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
...

will try another run with LANG=C.UTF-8
Flags: needinfo?(klibby)
Same error this go around, but at a different point in the code:

A worker failed while indexing /builds/dxr-build-env/src/addons/619174/chrome/DropTimePlus/locale/jp-JP/コピー - clock.dtd:
Traceback (most recent call last):
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 578, in index_chunk
    index_file(tree, tree_indexers, path, es, index)
  File "/builds/dxr-build-env/dxr/dxr/build.py", line 468, in index_file
    file_to_index = tree_indexer.file_to_index(rel_path, contents)
  File "/builds/dxr-build-env/dxr/dxr/plugins/clang/indexers.py", line 255, in file_to_index
    self._csv_map[sha1(path).hexdigest()],
UnicodeEncodeError: 'ascii' codec can't encode characters in position 40-42: ordinal not in range(128)
https://github.com/mozilla/dxr/commit/34a8f729d27cbbd011583145790cfba77d9f8c9b should fix this issue.
However it seems to me that we don't need some of the plugins for addon indexing, in particular clang, extmatch, rust, xpidl, and maybe python.
Blocks: 1270680
(In reply to Kendall Libby [:fubar] from comment #6)
> So here's a fun twist. There are ~44,000 directories in /addons, some number
> of which don't have a recognized VCS (which makes some sense, there are
> actually dumps from the addons db, and who knows what the creator is
> actually using).

The add-ons that have VCS directories have them because of packaging errors.
We warn developers that hidden directories shouldn't be included in their
packages, but we don't enforce it. The VCS directories that do exist should be
ignored.
(In reply to Kendall Libby [:fubar] from comment #17)
> (In reply to Peter Elmers [:new_one] from comment #16)
> > Some looking around
> > (http://stackoverflow.com/questions/2076708/python-os-stat-and-unicode-file-
> > names) suggests that os.stat tries to decode the path to system codec (LANG,
> > I think). Can we confirm the machine's set to utf-8 by default?
> 
> will try another run with LANG=C.UTF-8

Don't expect all add-ons to have file and directory names that are valid
UTF-8. Many of them use arbitrary encodings, and some of them have names that
aren't valid in any encoding.
The addons tree could use an enabled_plugins = pygmentize urllink js
we've finally made it past the 41% mark, from above. it's a slow, slow job, but I'm hopeful that it'll finish.
Depending on how fresh we want to keep this index, this could be a good motivator for us to get a limited form of incremental indexing going. Addons isn't all in one VCS (or any VCS at all), is it? Would whatever update method we're doing give us believeable mod dates?
Hi Kendall, do we have an ETA now? Thanks.
(In reply to Jonathan Hao [:jhao] from comment #25)
> Hi Kendall, do we have an ETA now? Thanks.

Test index finally finished over the weekend, so you can find it at https://dxr.mozilla.org/addons/source/ (requires LDAP/Okta auth). \o/

Full index run too 21 hours, so we're not going to be able to index daily, but I'll look at having it run ever few days. Using pypy may help, too.
ofc, I say that and the go searching for something and get an ISE. *sigh* will see what's up...
(In reply to Kendall Libby [:fubar] from comment #27)
> ofc, I say that and the go searching for something and get an ISE. *sigh*
> will see what's up...

E_NOTENUFCOFFEE. dxr.config had not been updated on the web heads. however, there's a weird problem with okta that I'm going to have to get some help from webops. you get in eventually, though.

I'm setting up the indexing job to run three times a week: sunday, tuesday, and thursday, at 1000UTC. The indexing run takes about 21hrs atm, so the updated index should be available the following day.
All set.

:erikrose, there's an odd bit where if you go to /addons/source/ and scroll down, it stops around /372876/ even though the directories carry on up to /699705/. thoughts?

I don't imagine it's hugely important, since browsing the directory tree at that level means going through ~44000 entries, but it's unexpected. Searching for things in those directories DOES work, thankfully.
Flags: needinfo?(erik)
It's the hard-coded limit "10,000". ES makes us specify one. I've increased it to a million in https://github.com/mozilla/dxr/pull/593. That ought to be enough for anyone. ;-)
Flags: needinfo?(erik)
two issues:

1) something on the cloudops side was causing errors so that not everything was coming over to us. :jason's sorted that out.

2) the cron schedule isn't correct, so addons isn't indexing on the correct schedule. with some of the memory/swap issues, and subsuequent reduction of # of workers, that might not be the worst thing ever.

however, there's a significant backlog of builds atm. I'm working with IT to get more memory on the VMs as well as a 4th...
(In reply to Kendall Libby [:fubar] from comment #31)
> 
> 2) the cron schedule isn't correct, so addons isn't indexing on the correct
> schedule. with some of the memory/swap issues, and subsuequent reduction of
> # of workers, that might not be the worst thing ever.

... because it was configured as a normal repo which uses pollscm to trigger. converted to tree-type which uses a timed schedule (a la cron).

> however, there's a significant backlog of builds atm. I'm working with IT to
> get more memory on the VMs as well as a 4th...

increasing memory definitely helped and we've chewed through most of the backlog already. \o/
Ran into disk space issues on the docker image end of last week. Fixed that and now we're back to unicode issues:


Running post_build [####################################] 100%
Indexing folders   [-----------------------#------------]
Traceback (most recent call last):
  File "venv/bin/dxr", line 9, in <module>
    load_entry_point('dxr==2.0', 'console_scripts', 'dxr')()
  File "/home/jenkins/dxr/dxr/cli/__init__.py", line 29, in main
    return dxr.invoke(ctx)
  File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/home/jenkins/dxr/dxr/cli/index.py", line 26, in index
    index_and_deploy_tree(tree, verbose=verbose)
  File "/home/jenkins/dxr/dxr/build.py", line 60, in index_and_deploy_tree
    index_name = index_tree(tree, es, verbose=verbose)
  File "/home/jenkins/dxr/dxr/build.py", line 265, in index_tree
    index_files(tree, tree_indexers, index, pool, es)
  File "/home/jenkins/dxr/dxr/build.py", line 617, in index_files
    index_folders(tree, index, es)
  File "/home/jenkins/dxr/dxr/build.py", line 604, in index_folders
    needles.update(dict(folder_to_index(name, tree, folder).needles()))
  File "/home/jenkins/dxr/dxr/plugins/descriptor/__init__.py", line 35, in needles
    for entry in sorted(listdir(self.path)):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa5 in position 5: ordinal not in range(128)

LANG is set to C.UTF-8 in the above
Depends on: 1290945
Are any of the LC_ vars set (LANG functioning as only a default in the absence of those)? For at least my test data, setting LC_ALL=C.UTF-8 makes it work. Meanwhile, I'm seeing if I can squash it properly, but it's not looking great, since the crash is actually within the Python stdlib--the part written in C, not even Python.
OTOH, since Linux treats filenames as random bags of bytes, I could see it still failing if the bag wasn't valid UTF-8. (My test case was.)
Alright, now that bug 1290945 is fixed, let's see what add-ons does after the next deploy.
Currently running a test build, storing the logs and temp files outside of the docker image. The js plugin is currently eating up 44gb of space! And it hasn't changed in quite a while. It feels like maybe we could be doing something a lot more efficiently there.

Storage folks have kindly bumped the NFS mount up by 50g for the weekend so that it (hopefully) doesn't fill that, too, but we need to come up with a better long term solution.
Doing a gzip on the temp files would be a considerable gain since the lines emitted are very repetitive.
Why don't we just turn off the js plugin for the addons tree until we've optimized it a bit? I think they'll still be able to do everything they could in MXR and more.
current issue blocking addons indexing is a very large json file:

$ du -ksh addons/710739/bg/data/enamdict.json
88M	enamdict.json
$ file addons/710739/bg/data/enamdict.json
enamdict.json: ASCII English text, with very long lines, with no line terminators

I've added that file to ignore_patterns for the addons tree; is there a better option?
I deduce you figured out that's causing the timeouts, as in https://jenkins-dxr.mozilla.org/job/addons-tree/15/console? If so, ignoring it is probably the best bet for now. We could also bump up es_indexing_timeout, but we have it at 120s already, and I don't know how far we'd have to go. I suppose it wouldn't hurt to increase it if you like.

I'm not really sure what to do if ES takes 5 minutes to ingest a line like this; a line is a very small unit of indexing already, and it would be a lot of hacking to subdivide. Maybe newer versions of ES are better, but I have no reason to believe that. Long-term, we could add a configurable max line length after which to truncate or give up on the entire file, but that's going to make searches inaccurate. :-(
It seems reasonable to skip indexing data files that are larger than, say, 16MB. I can only hope we won't actually run into JS files that big, but those should probably be indexed regardless of size.
Depends on: 1299169
(In reply to Kris Maglione [:kmag] from comment #20)
> The add-ons that have VCS directories have them because of packaging errors.
> We warn developers that hidden directories shouldn't be included in their
> packages, but we don't enforce it. The VCS directories that do exist should
> be ignored.


Erik, what's the best way to ignore VCS files, or otherwise work around this:

 Starting tree 'addons'.
Traceback (most recent call last):
  File "/home/jenkins/venv/bin/dxr", line 9, in <module>
    load_entry_point('dxr==2.0', 'console_scripts', 'dxr')()
  File "/home/jenkins/dxr/dxr/cli/__init__.py", line 29, in main
    return dxr.invoke(ctx)
  File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/home/jenkins/dxr/dxr/cli/index.py", line 26, in index
    index_and_deploy_tree(tree, verbose=verbose)
  File "/home/jenkins/dxr/dxr/build.py", line 61, in index_and_deploy_tree
    index_name = index_tree(tree, es, verbose=verbose)
  File "/home/jenkins/dxr/dxr/build.py", line 207, in index_tree
    vcs_cache = VcsCache(tree)
  File "/home/jenkins/dxr/dxr/vcs.py", line 432, in __init__
    self.repos = tree_to_repos(tree)
  File "/home/jenkins/dxr/dxr/vcs.py", line 387, in tree_to_repos
    attempt = vcs.claim_vcs_source(cwd, dirs, tree)
  File "/home/jenkins/dxr/dxr/vcs.py", line 266, in claim_vcs_source
    return cls(path)
  File "/home/jenkins/dxr/dxr/vcs.py", line 201, in __init__
    self.revision = self.invoke_vcs(['rev-parse', 'HEAD'], self.root).strip()
  File "/home/jenkins/dxr/dxr/vcs.py", line 57, in invoke_vcs
    return subprocess.check_output([cls.command] + args, cwd=cwd, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128
Flags: needinfo?(erik)
It would help to know git's actual output. Do you have it handy?
Flags: needinfo?(erik)
Misses this in the three preceding log lines:

10:35:21 fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
10:35:21 Use '--' to separate paths from revisions, like this:
10:35:21 'git <command> [<revision>...] -- [<file>...]'
It sounds like there's a malformed (or even completely empty) git repo in the tree. Le sigh.... Trying to add some more error handling....
Commit pushed to master at https://github.com/mozilla/dxr

https://github.com/mozilla/dxr/commit/220fd2564683bda434930d4079a0b169f3d3ad53
Don't crash when encountering an empty git repo. Refs bug 1280948.
Okay, fubar, try now.
The most recent build, https://jenkins-dxr.mozilla.org/job/addons-tree/32/console, looks like it failed due to files disappearing from the FS. That's weird.

12:00:43 Indexing files
12:00:43 Indexing files
11:46:49 A worker failed while indexing /home/jenkins/src/addons/705477/assets/flags/4x3/.af.svg.vqbYpz:
11:46:49 Traceback (most recent call last):
11:46:49   File "/home/jenkins/dxr/dxr/build.py", line 577, in index_chunk
11:46:49     index_file(tree, tree_indexers, path, es, index)
11:46:49   File "/home/jenkins/dxr/dxr/build.py", line 442, in index_file
11:46:49     contents = unicode_contents(path, tree.source_encoding)
11:46:49   File "/home/jenkins/dxr/dxr/build.py", line 368, in unicode_contents
11:46:49     with open(path, 'rb') as source_file:
11:46:49 IOError: [Errno 2] No such file or directory: '/home/jenkins/src/addons/705477/assets/flags/4x3/.af.svg.vqbYpz'
11:46:49 
11:46:49 Traceback (most recent call last):
11:46:49   File "/home/jenkins/venv/bin/dxr", line 9, in <module>
11:46:49     load_entry_point('dxr==2.0', 'console_scripts', 'dxr')()
11:46:49   File "/home/jenkins/dxr/dxr/cli/__init__.py", line 29, in main
11:46:49     return dxr.invoke(ctx)
11:46:49   File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 991, in invoke
11:46:49     return _process_result(sub_ctx.command.invoke(sub_ctx))
11:46:49   File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 837, in invoke
11:46:49     return ctx.invoke(self.callback, **ctx.params)
11:46:49   File "/home/jenkins/venv/local/lib/python2.7/site-packages/click/core.py", line 464, in invoke
11:46:49     return callback(*args, **kwargs)
11:46:49   File "/home/jenkins/dxr/dxr/cli/index.py", line 26, in index
11:46:49     index_and_deploy_tree(tree, verbose=verbose)
11:46:49   File "/home/jenkins/dxr/dxr/build.py", line 61, in index_and_deploy_tree
11:46:49     index_name = index_tree(tree, es, verbose=verbose)
11:46:49   File "/home/jenkins/dxr/dxr/build.py", line 265, in index_tree
11:46:49     index_files(tree, tree_indexers, index, pool, es)
11:46:49   File "/home/jenkins/dxr/dxr/build.py", line 640, in index_files
11:46:49     raise type, value  # exits with non-zero
11:46:49 IOError: [Errno 2] No such file or directory: '/home/jenkins/src/addons/705477/assets/flags/4x3/.af.svg.vqbYpz'
11:47:02 Build step 'Execute shell' marked build as failure
11:47:02 Sending e-mails to: fubar@mozilla.com erose@mozilla.com
11:47:02 Finished: FAILURE

Any idea what could cause that, fubar?
Flags: needinfo?(klibby)
Almost certainly due to the indexing job running over the rsync job. If it happens again, we can adjust when things run, but we had a successful index over the weekend after you disabled the pygments plugin. \o/
Flags: needinfo?(klibby)
Okay, we got through one! Next: JS (which will require optimizing the plugin first so as not to fill the disk) and getting syntax coloring happening at render time (which I'm surprised it's not already). But I think I'll close this and open new tickets about those.
Bug 1307146 is about the JS plugin.
Bug 1307160 is about the render-time coloring.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: Webtools → Webtools Graveyard
You need to log in before you can comment on or make changes to this bug.