Closed Bug 1241494 Opened 4 years ago Closed 4 years ago

Updating the en-US dictionary to SCOWL 2016.01.19 FAILED

Categories

(Core :: Spelling checker, defect)

defect
Not set

Tracking

()

RESOLVED DUPLICATE of bug 1240916

People

(Reporter: ehsan, Unassigned)

References

Details

Attachments

(2 files, 1 obsolete file)

I noticed that these somehow got added to 5-mozilla-added in the
SCOWL 2016.01.19 update in bug 1240916.
I noticed that these somehow got added to 5-mozilla-added in the
SCOWL 2016.01.19 update in bug 1240916.
Attachment #8710456 - Flags: review?(mozilla)
Comment on attachment 8710456 [details] [diff] [review]
Remove the possessive forms of give, get, him and her from the en-US dictionary

This is a dictionary update, should be safe to backport to Aurora to include it in ESR45.
Attachment #8710456 - Flags: approval-mozilla-aurora?
There is something fishy going on here.

First we need to know that
her's
him's
give's
get's
were always part of SCOWL. That's why I complained about this upstream:
https://github.com/kevina/wordlist/issues/141

Kevin removed those possessive forms:
https://github.com/kevina/wordlist/commit/6b25d6138913ec820065f1a2a33a59c785aff493

The update from SCOWL should work this way:
If a word came from SCOWL and is removed, it should also be removed from the Mozilla dictionary.
It should not move to the Mozilla added words. That's what the complicated list handling is all about in make-new-dict.

So removing them from the dictionary is just covering up some other error.

Otherwise the patch is fine but I'd like to understand what's gone on here.

Let me see what happened with your refresh in bug 1240916. Do you still have the files which are not version controlled? 1-base.txt, 2-mozilla-rem, etc.?
Flags: needinfo?(ehsan)
I'm not sure what went wrong, Kevin?  I'll attach everything in dictionary-sources after running the scripts in a sec.
Flags: needinfo?(ehsan)
Flags: needinfo?(kevin.bugzilla)
Attached file files.zip
Well, if this is the same set of files you had when you created the refresh in bug 1240916, then something has gone terribly wrong. This explains why her's him's give's and get's weren't removed.

1-base.txt is empty!! This should contain the SCOWL provided words from the previous SCOWL edition:
Look at make-new-dict: expand $ORIG/en_US-custom.aff > 1-base.txt
(I can imagine that step went wrong since something changed in the affix file).

Consequently *all* words in the Mozilla dictionary are considered Mozilla added since 2-mozilla.txt and 2-mozilla-add are identical.

If all words are considered Mozilla-added, then nothing will be removed.

In other words, your merge in bug 1240916 went wrong.

These temporary files are there for a reason, one should do a minimal sanity check on them. Don't trust the machine ;-)
Flags: needinfo?(kevin.bugzilla)
Comment on attachment 8710456 [details] [diff] [review]
Remove the possessive forms of give, get, him and her from the en-US dictionary

While the patch fixes the problem introduced in bug 1240916, I'd much prefer to do the merge properly instead of picking up the broken pieces later.
Attachment #8710456 - Flags: review?(mozilla)
Attachment #8710456 - Flags: approval-mozilla-aurora?
I suspect this is caused by https://github.com/kevina/wordlist/issues/144.

I actually wanted Kevin to look at this.  Restoring the needinfo.
Flags: needinfo?(kevin.bugzilla)
Sorry about the NI, but we're in the wrong bug here. The upstream update failure should be discussed in bug 1240916.
I can reproduce the problem. And am looking into it.  I will fix the problem and create a patch to my scripts and then let Ehsan try again to make sure there are no problems.

Can we change the subject to something better like "Updating the en-US dictionary to SCOWL 2016.01.19 FAILED".
Sure, we can change the subject if you wish. Somehow I don't think we need two bugs for one upstream update. Either this or bug 1240916 could be closed.
Flags: needinfo?(kevin.bugzilla)
Summary: Remove the possessive forms of give, get, him and her from the en-US dictionary → Updating the en-US dictionary to SCOWL 2016.01.19 FAILED
(In reply to Kevin Atkinson from comment #10)
> I can reproduce the problem. And am looking into it.  I will fix the problem
> and create a patch to my scripts and then let Ehsan try again to make sure
> there are no problems.

Thanks!

> Can we change the subject to something better like "Updating the en-US
> dictionary to SCOWL 2016.01.19 FAILED".

Let's just close this bug.  (BTW I just granted you the canconfirm and editbugs privileges on bugzilla so that you can make changes to the subject and whatnot yourself in the future.  :-)
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1240916
Sorry about that Ehsan.  The attached patch fixes my scripts so the upgrade should work smoothly.  Please first apply this patch, then do the upgrade.  Do an sanity check to make sure 1-base.txt is not empty and that 5-mozilla-added does not contain "get's" or any non-ascii words.
Attachment #8710456 - Attachment is obsolete: true
Flags: needinfo?(ehsan)
Sorry our messages crossed.  Will post my patch to the other bug.
Flags: needinfo?(ehsan)
You need to log in before you can comment on or make changes to this bug.