Closed Bug 1380893 Opened 2 years ago Closed 2 years ago

Stand up regular builds with sccache disabled

Categories

(Firefox Build System :: General, enhancement)

enhancement
Not set

Tracking

(firefox57 wontfix, firefox61 fixed)

RESOLVED FIXED
mozilla61
Tracking Status
firefox57 --- wontfix
firefox61 --- fixed

People

(Reporter: glandium, Assigned: gps)

References

Details

Attachments

(1 file, 1 obsolete file)

This is the opposite end of bug 1373460. Having spent some time looking at cache miss issues, rust build times and having seen first hand the large difference of build times between a build that has cache and one that doesn't, it now seems to me imperative that we set up builds that *never* use any form of caching, and track the build times for those, as they will more closely match the experience of clobber builds for local developers. That's something that's entirely lost in the noise of normal builds that, most of the time, get large cache hits (even when there are cache misses).
How about standing up tasks that basically just do `mach build` and nothing else? No symbol dumping. No test archive generation. Maybe not even using mozharness. Just configure+make.

(Although, mozharness does have the perfherder integration today, so we may need mozharness in some limited capacity unless we want to move that code from desktop_build.py to the build system.)
Fair point, the build times we measure are including those things, and they're just noise compared to how local builds perform.
Assignee: nobody → gps
Status: NEW → ASSIGNED
Comment on attachment 8944964 [details]
Bug 1380893 - Run build configuration that performs a clobber without caches;

https://reviewboard.mozilla.org/r/215104/#review220722

::: taskcluster/ci/build/macosx.yml:34
(Diff revision 1)
>          - linux64-libdmg
>          - linux64-llvm-dsymutil
>          - linux64-rust-macos
>          - linux64-sccache
>  
>  macosx64/opt:

macosx64-clobber/opt
Attachment #8944964 - Flags: review+
Comment on attachment 8944964 [details]
Bug 1380893 - Run build configuration that performs a clobber without caches;

https://reviewboard.mozilla.org/r/215104/#review220770

::: build/mozconfig.common:25
(Diff revision 2)
> +    # mozconfig.automation enables additional build steps. We don't care
> +    # about those in a no caches build. So we tie them to the config. This
> +    # isn't logically correct. But it works for our needs.
> -. "$topsrcdir/build/mozconfig.automation"
> +    . "$topsrcdir/build/mozconfig.automation"

It seems to me skipping mozconfig.automation should be gated on something else. That seems like something desirable for many other jobs that currently include it. In fact, I'd go as far as saying the contents of that mozconfig should be set from taskcluster configs instead.

If you don't want to go that far, I'd just set the various environment for those jobs only, they will override the contents of mozconfig.automation.

::: build/mozconfig.common:29
(Diff revision 2)
> +    # mozconfig.automation enables additional build steps. We don't care
> +    # about those in a no caches build. So we tie them to the config. This
> +    # isn't logically correct. But it works for our needs.
> -. "$topsrcdir/build/mozconfig.automation"
> +    . "$topsrcdir/build/mozconfig.automation"
> -. "$topsrcdir/build/mozconfig.rust"
> -. "$topsrcdir/build/mozconfig.cache"
> +    . "$topsrcdir/build/mozconfig.cache"

I'd rather see bug 1417421 fixed than hiding this before some environment variable.

::: taskcluster/taskgraph/transforms/job/common.py:20
(Diff revision 2)
>  def docker_worker_add_workspace_cache(config, job, taskdesc, extra=None):
>      """Add the workspace cache.
>  
>      ``extra`` is an optional kwarg passed in that supports extending the cache
>      key name to avoid undesired conflicts with other caches."""
> +    if job['worker'].get('disable-default-caches'):

should this instead be done in e.g. build_docker_worker_payload, avoiding any caches that would be set elsewhere in the future?
Attachment #8944964 - Flags: review?(mh+mozilla)
Product: Core → Firefox Build System
Attachment #8944964 - Attachment is obsolete: true
Comment on attachment 8969727 [details]
Bug 1380893 - Add CI for plain builds;

https://reviewboard.mozilla.org/r/238532/#review246694

::: testing/mozharness/mozharness/mozilla/building/buildbase.py:1787
(Diff revision 1)
>  
> +    def ensure_upload_path(self):
> +        env = self.query_mach_build_env()
>  
> +        # Some Taskcluster workers don't like it if an artifacts directory
> +        # is defined by no artifacts are uploaded. Guard against this by always

by no -> but no
Attachment #8969727 - Flags: review+
Attachment #8969727 - Flags: review?(core-build-config-reviews)
Attachment #8969727 - Flags: review?(core-build-config-reviews)
https://hg.mozilla.org/mozilla-central/rev/3accdb3ed507
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla61
Duplicate of this bug: 1207666
You need to log in before you can comment on or make changes to this bug.