Closed
Bug 1842782
Opened 2 years ago
Closed 1 years ago
Add comparisons with teacher models
Categories
(Firefox :: Translations, task)
Firefox
Translations
Tracking
()
RESOLVED
MOVED
People
(Reporter: marco, Unassigned)
Details
In our evaluation code, we currently compare our student models against models from other vendors.
We should add our teacher models to the comparison, so we can know how much quality we are losing when distilling.
This will help figure out if bug 1842781 is worth it or not.
|
||
Comment 1•2 years ago
|
||
The teacher models are too large to submit to the evaluation repo. In fact, we should track all metrics for all models across the training pipeline. Then we'll be able to see how the quality degrades after different steps. We can check it manually now. Integration with the proper experiment tracking toolset will help here.
|
Reporter | |
Comment 2•1 years ago
|
||
Status: NEW → RESOLVED
Closed: 1 years ago
Resolution: --- → MOVED
You need to log in
before you can comment on or make changes to this bug.
Description
•