464107 - [silme] filter unneeded L10nPackages/L10nObjects/EntityLists/Objects/Entities while reading them

Status: NEW

(Localization Infrastructure and Tools :: Silme, defect)

Description

[Adrian Kalla [:adriank]]

Attachment: filter.py handler

One of the missing and at the same time really nice to have features in Silme is being able to decide what objects you don't want to have *while* reading.
It makes it possible to filter images, binary files and other unneeded files before reading them. Because of that it saves a lot of time.

Beside of that filter.py handling is needed for compare-locales feature compatibility.


There is already a really nice method to filter folder, files or entities with a filter.py file created by Pike: http://hg.mozilla.org/mozilla-central/file/27c3c042ffb7/browser/locales/filter.py

I already ported this to Silme some time ago. 
The first step is to have a filter.py handler, the second is to enable it in lib/silme/io/file.py :

Comment 1

[Adrian Kalla [:adriank]]

Attachment: patch for file.py enabling use of the filter.py handler

Comment 2

[Zibi Braniecki [:zbraniecki][:gandalf]]

I find it a really specific task for now, as we want to solve it ultimately with ability to select what to do with which files (so not only filtering them out, but also selective loading objects or just marking them in package).

So my opinion is that you should overload the default functions with yours in the script.

You can do this like this:

overload_file.py:
import silme.io.file

def myGetL10nObject():
  pass

silme.io.file.FileClient.getL10nObject = myGetL10nObject

Comment 3

[Zibi Braniecki [:zbraniecki][:gandalf]]

and since it doesn't break the API it's not a blocker

Comment 4

[Zibi Braniecki [:zbraniecki][:gandalf]]

Adrian: Once you have it working in 0.5.1, could we close this bug? :)

This feature eventually did land for 0.5 and with a patch for 0.5.1 should solve the issue.

Comment 5

[Adrian Kalla [:adriank]]

Gandalf: I can confirm, that we can now filter everything BUT entities.
So to close this bug we would need a way to filter entities too.

To do that we could use the _should_ignore method e.g. here: http://hg.mozilla.org/users/zbraniecki_mozilla.com/silme/file/4aa68586ff68/lib/silme/format/dtd/parser.py#l29
(BTW. It would be a good idea to have an own method for just that for-loop, so that it could be easily overriden by scripts e.g. for logging duplicates.)

I'm not sure how to do that for L10nObjects...

Comment 6

[Francesco Lodolo [:flod]]

Moving Silme bugs to its component.

[Mass change filter: silme-move]