Open Bug 685957 Opened 8 years ago Updated 7 months ago
We shouldn't litter our tree with .pyc files
When you import a file in python, by default a .pyc file will be created in the same directory. These require manual wiping sometimes and otherwise litter the tree. Using e.g. python -B should fix this problem, though other solutions are also worth considering
It sucks that python doesn't have a way to specify where .pyc files should go instead of next to the source file, but I really don't think that we want to disable .pyc files by default, since that could significantly affect build times. If it really bugs you, can you just specify PYTHONDONTWRITEBYTECODE=x in your environment?
It less bothers me than leads to either genuine or imagined errors and the recommendation of wiping .pyc files before rebuilding. Maybe `make -f client.mk` or similar should do find . -iname '*.pyc' -delete instead?
I've had build failures that required deleting a bad .pyc file to get going again.
I must admit I've never experienced that form of error. Do we know what caused it? Python is normally pretty good at checking the timestamps of .pyc files and rebuilding them on any mismatch.
Unfortunately, it happened to me a few days ago. I asked on IRC and Ms2ger suggested deleting the .pyc file it was complaining about, and it worked. Is there a way to find old posts from a certain user on pastebin.m.o?
I was given the same advice. finding and deleteing the bad .pyc files on make takes almost no time on my system. IMHO, its better doing this than chancing on yet another way things can break
Maybe the thing to do is to make `make clobber` delete the .pyc files and maybe check-sync-dirs too
The former is feasible, the latter is not. We run check-sync-dirs for every build, so we don't want to clobber the .pyc files.
(In reply to Benjamin Smedberg [:bsmedberg] from comment #4) > I must admit I've never experienced that form of error. Do we know what > caused it? Python is normally pretty good at checking the timestamps of .pyc > files and rebuilding them on any mismatch. In the last case I saw, it happened because a source file changed from a plain .py file to a preprocessed .py.in file, but the old .pyc file was still in the srcdir, and so (as far as I can tell) it was used rather than the .py file which was now in the objdir. Maybe that could be avoided if objdir directories were always before srcdir directories in the python path?
Erik Rose and I were talking Python packaging and I was lamenting over the state of pyc files and lack of control. He informed me of a clever solution employed by Debian: symlinks. Essentially you create a directory tree that mirrors the source tree. e.g. topsrcdir mozbuild backend make.py topobjdir mozbuild backend Then you create symlinks back to the source .py files: topobjdir mozbuild backend make.py -> topsrcdir/mozbuild/backend/make.py You then point sys.path at topobjdir. Python doesn't follow the symlinks and it creates the .pyc files in the same directory as the symlink: topobjdir mozbuild backend make.py -> topsrcdir/mozbuild/backend/make.py make.pyc This could work for us on platforms where symlinks are supported. Windows might be a little funky. A downside is we'd need to walk mapped directories to reconcile the destination file list, probably on every build. But we do have code in mozpack that should make this relatively pain free. If we can pull it off, I think I like this more than the custom importer solution because the custom importer will be extremely hacky, especially on 2.7.
You need to log in before you can comment on or make changes to this bug.