1894632 - non-UTF-8 files are shown as 0 bytes

Status: RESOLVED FIXED

(Webtools :: Searchfox, defect)

Description

[Tooru Fujisawa [:arai]]

The filesize in the directory listing is calculated in the following code, which expects the file content being UTF-8.
If the file content is not UTF-8, or simply binary, it takes Err(_) => (None, 0), path and the filesize is shown as 0 bytes.

https://github.com/mozsearch/mozsearch/blob/e17ac42e1cc11fda488eb1ebe62064b75488c56e/tools/src/file_format/repo_data_ingestion.rs#L576

            let (description, file_size) = match fs::read_to_string(&raw_file_path) {
                Ok(contents) => {
...

                            (Some(description), metadata.len())
                        }
                        None => (None, metadata.len()),
                    }
                }
                Err(_) => (None, 0),
            };

example: TwemojiMozilla.ttf in https://searchfox.org/mozilla-central/source/browser/fonts says it's 0 bytes.

The file size calculation actually doesn't depend on the file content.
Unless there's any issue with accessing the binary file metadata, the calculation for the description and the file size can be done separately, so that binary file's size becomes correct value.

Comment 1

[Tooru Fujisawa [:arai]]

Attachment: GitHub Pull Request