Open Bug 1674520 Opened 4 years ago Updated 5 months ago

External dictionaries cannot be used

Categories

(Core :: Spelling checker, defect, P3)

78 Branch
x86_64
Linux
defect

Tracking

()

UNCONFIRMED

People

(Reporter: crxssi, Unassigned)

References

(Regression)

Details

(Keywords: regression, regressionwindow-wanted)

Attachments

(2 files)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0

Steps to reproduce:

I am running vanilla Firefox 78.3.0 ESR (from Mozilla, not from a distro) under Linux and neither $DICPATH nor the spellchecker.dictionary_path preference is working (also not working in Firefox 68ESR). I have a customized medical dictionary that I have used for many years under older versions of Firefox that I now cannot use. I assume it is supposed to be as simple as:

$ ls /opt/dictionary
$ file *
en-US.aff: ASCII text
en-US.dic: UTF-8 Unicode text

$ export DICPATH="/opt/dictionary"
$ firefox

Yet it is NOT using the external dictionary I defined (it shows words in that external dictionary as misspelled, still). I tried also adding export DICTIONARY="en-US" before launching and that didn't help. I am hoping someone might have some idea what is going on. Is this option now broken?

OS: Unspecified → Linux
Hardware: Unspecified → x86_64
See Also: → 1587101
Component: Untriaged → Spelling checker
Product: Firefox → Core

I just confirmed it also does not work in Firefox 82 (using either or both environment variables or the about:config setting). Also on two different platforms- CentOS 8 and Mageia 7.

I doubt there is anything wrong with my dictionary files, but I am attaching them to this bug.
One of several test words I am using is "metoprolol" which is not in the stock dictionary, but is in mine.

medical dictionary files for testing

The severity field is not set for this bug.
:smaug, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(bugs)

Based on some code inspection it looks like
https://bugzilla.mozilla.org/show_bug.cgi?id=1410214 broke some of this - or perhaps the actual breakage happened later when that
code started to be used.
Addons seem to have higher priority.

It isn't quite clear to me why such order even when using just the built-in dictionary.

(for now one could probably create dictionary addon and install that)

Severity: -- → S3
Flags: needinfo?(bugs) → needinfo?(kmaglione+bmo)
Priority: -- → P3
Regressed by: 1410214
Has Regression Range: --- → yes

(In reply to Olli Pettay [:smaug] from comment #4)

Based on some code inspection it looks like
https://bugzilla.mozilla.org/show_bug.cgi?id=1410214 broke some of this - or perhaps the actual breakage happened later when that
code started to be used.

That seems likely. I can't say, because before this, we were simply replacing the centrally-included non-addon dictionary files with our own. We later found out that the ability to do this was intentionally removed from Firefox, so then we started searching for another way. Using DICPATH was the offered solution, which would be perfect, if it worked. So far, I know it is not working from 68 through 82.

(for now one could probably create dictionary addon and install that)

That might be a work-around for a single, expert user. However:

  1. Our system is a multi-user, thin-client environment. That would mean instead of having a single copy of a large dictionary, we would have many gigabytes tied up in hundreds of redundant duplications of the same large dictionary.

  2. Any time we have to modify the dictionary, we will have to manually update hundreds of users, instead of just one file.

  3. It is not obvious how to create such an addon. This will be especially off-putting for newer or non-expert users.

  4. The add-on method is incompatible with Linux distros, which will want to use the same dictionary settings for all the installed programs and allow the user to switch them, centrally.

To us, this is a pretty bad regression :( I am surprised I am [apparently] the first to report it, since it has been broken a long time. Of course, we are not the typical type of installation.

QA Whiteboard: [qa-regression-triage]

Hi crxssi,

Could you try to run Mozregression and confirm that the regressor bug is indeed Bug 1410214?

Flags: needinfo?(crxssi)

Hello,

Attempted to find a regression window for the issue but all checked builds (ran a window from 2017-01-01 until the present day) appeared to be affected.

I’m not sure I proceeded correctly with the installation of the dictionary from Comment 2 though. On Ubuntu, I’ve found out that dictionaries are stored in /usr/share/hunspell (as can be seen from the value of spellchecker.dictionary_path from newer versions of Firefox) and replaced the default ones with the ones from Comment 2. Testing with the word “metoprolol” revealed that the dictionary was indeed not used, as the word was red underlined.

(In reply to Alex Cornestean from comment #7)

Hello, Attempted to find a regression window for the issue but all checked builds (ran a window from 2017-01-01 until the present day) appeared to be affected. I’m not sure I proceeded correctly with the installation of the dictionary from Comment 2 though. On Ubuntu, I’ve found out that dictionaries are stored in /usr/share/hunspell (as can be seen from the value of spellchecker.dictionary_path from newer versions of Firefox) and replaced the default ones with the ones from Comment 2. Testing with the word “metoprolol” revealed that the dictionary was indeed not used, as the word was red underlined.


Thanks. Testing for this is actually pretty simple. In older versions of firefox, there was no spellchecker.dictionary_path so I am not sure how that can be tested with older versions. So I am focusing on $DICPATH, which should work on all versions (presumably).

I have the dictionary files (attached to this bug report) just in my $HOME. I just downloaded Firefox 57.04 and tested and "metoprolol" was underlined (not in the dictionary). Then exited firefox and issued "export DICPATH=$HOME" and relaunched firefox from that shell, and checked "metoprolol" was then properly recognized in firefox spellcheck. I haven't tried a full regression test yet, but this proves $DICPATH was working in 57.04. I don't know how the dates exactly line up with releases, but I think 57 was in November 2017.

I am going to try the regression tool next...

Flags: needinfo?(crxssi)

The mozregression GUI tool is surprisingly easy to use. I chose firefox, 64 bit, shippable, from 2017-11-01 to 2019-07-08. However, after testing 9 versions, it came back with "unable to find enough data to bisect." I am not sure what to do now, so I took a screenshot of it. The list was as follows:
2018-09-04 bad
2018-04-04 good
2018-06-19 bad
2018-05-12 good
2018-05-31 bad
2018-05-22 good
2018-05-27 bad
2018-05-25 good
2018-05-26 good

I repeated it again, just in case, but widened to from 2017-11-01 to 2020-11-17 and got the same error message after 10 tests. There were lots of "skipped" versions with warnings for some reason or another. This "critical" in the log might be the most important:

2020-11-17T20:05:02.298000: CRITICAL : Last build db78be8dbc1c is missing, but mozregression can't find a build after - so it is excluded, but it could contain the regression!

I will attach the entire regression log from the second attempt. I hope this was helpful.

Attached file regression_log.txt

Regression log, cut and pasted into a text file.

As per comment 9 and comment 10 is it ok to remove "regressionwindow-wanted" from keywords and check "has regression range" field?
Best,
Clara

Flags: needinfo?(acornestean)

Looking over the log I’m not seeing a final result i.e. a regressor being found. My opinion is that for the moment the flag should remain.

Thank you !

Flags: needinfo?(acornestean)

(In reply to Alex Cornestean from comment #12)

Looking over the log I’m not seeing a final result i.e. a regressor being found. My opinion is that for the moment the flag should remain.

Granted, I had issues with the regression tool for some reason. But it seems to have at least narrowed it to a single day:
2018-05-26 good
2018-05-27 bad

Is that a correct interpretation? Is there something else I can do or try? Or do we have to wait for someone else?

Based on the window you provided (2018-05-26 to 2018-05-27) and some research I’ve done, I might have found a possible cause for the issue.
From https://hg.mozilla.org/mozilla-unified/pushloghtml?startdate=May+26+2018&enddate=May+27+2018 , it is possible that this enhancement – https://bugzilla.mozilla.org/show_bug.cgi?id=1457321 , especially part 3 of it (https://hg.mozilla.org/mozreview/gecko/rev/07d42e3f8ab6b558115c9fc3032ed49091ceb9c2#index_header) might have broken something.

Alex, you are probably onto something. Originally, they were trying to speed up Firefox by having the dictionary built-in and NOT scanning "standard" locations for directories. But there is no reason to scan for or include dictionaries now, UNLESS $DICPATH or spellchecker.dictionary_path is used/set. Testing if either is set/used takes no time at all (so it doesn't hurt performance). I looked at the code changes, but can't make much sense of what could have broken the overrides working (I am not competent in that regard, unfortunately).

Not sure if it is appropriate, but I included the two reviewers of that change on this regression report. I apologize in advance if this was improper.

Clear a needinfo that is pending on an inactive user.

Inactive users most likely will not respond; if the missing information is essential and cannot be collected another way, the bug maybe should be closed as INCOMPLETE.

For more information, please visit BugBot documentation.

Flags: needinfo?(kmaglione+bmo)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: