Closed Bug 1259423 Opened 4 years ago Closed 4 years ago

freshclam fails to update the DB

Categories

(Release Engineering :: Release Automation: Other, defect)

defect
Not set

Tracking

(firefox48 fixed)

RESOLVED FIXED
Tracking Status
firefox48 --- fixed

People

(Reporter: rail, Assigned: kmoir)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

Assignee: nobody → kmoir
I pulled down the image locally and was able to run the command to update the db.  Perhaps a intermittent network issue? Or we can't connect to this db address from the container when it runs in the build.
Yeah, it's intermittent. Probably related to the load on the clamav mirrors.

Some ideas:

1) we can regenerate the images regularly
 * pros: no need to run freshclam
 * cons: need to use ":latest" in the tasks or use some service to tell the last good sha256; need to schedule this

2) don't fail if freshclam fails. Probably the easiest way to go. Failures are not frequent, so it won't probably hurt.
We can probably refresh the images as a separate task and publish the images as artifacts, but I'm not sure how to use them
Yeah, I looked at how to run a commmand in docker (freshclam) and not fail starting the image if it failed but didn't find a way to do that.
another idea:

1) create a task to generate an image and use dockerSave feature (see http://docs.taskcluster.net/workers/docker-worker/)

2) reference the image as in https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/tests/eslint-gecko.yml#11-14. Not sure if the dependencies are tracked automatically
Blocks: 1204281
https://gist.github.com/rail/fae3cf4b1eccee3d8ecd may be something that would address the "don't fail if freshclam fails". I haven't tested it though.
Attached patch bug1259423.patchSplinter Review
I tested it and it works
Attachment #8735490 - Flags: review?(rail)
Comment on attachment 8735490 [details] [diff] [review]
bug1259423.patch

I think we also need to update all beetmover calls in releasetasks and add "--refresh-antivirus". Would be great to have add a test to catch beetmover tasks without "--refresh-antivirus" passed
Attachment #8735490 - Flags: review?(rail) → review+
We don't need to update the beetmover calls in releasetasks.  When I tested it, I just used an existing taskid and the same call to mh (with a bogus bucket/credentials).  Beetmover is just called like this via releasetasks:

python scripts/release/beet_mover.py --template configs/beetmover/source.yml.tmpl --platform linux64 --version {{ version }} --locale en-US --taskid {{ stableSlugId(buildername) }} --build-num build{{ buildNumber }} --bucket {{ beetmover_candidates_bucket }}

The specific mh actions aren't referenced in the call.
Attachment #8735490 - Flags: checked-in+
We need a new image for rail/funsize-update-generator too but perhaps this should be part of bug 1259425
(In reply to Pulsebot from comment #10)
> https://hg.mozilla.org/integration/mozilla-inbound/rev/2f84b90500fe

Can you uplift this patch also to aurora, beta, and esr45?

(In reply to Kim Moir [:kmoir] from comment #11)
> We need a new image for rail/funsize-update-generator too but perhaps this
> should be part of bug 1259425

We can tackle that separately. Updated the image:

Digest: sha256:d1c88af7f13d124e08a37d10203e9e60655a231301123e5f2127fab5468ef5dd
https://hg.mozilla.org/mozilla-central/rev/2f84b90500fe
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
keeping open to address the remaining issues.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
other changes were needed to be added for beta and esr45 since funsize files were missing there. 

I landed the relevant changes on m-a and m-b but m-esr45 is closed now so will try again later
merged this pull request and landed changes on mozilla-esr45 too now that it is open
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → FIXED
Blocks: 1306635
Blocks: 1411358
No longer blocks: 1411358
You need to log in before you can comment on or make changes to this bug.