pkg_config is not found during comm-central-l10n repacks for linux32 and linux64

RESOLVED FIXED

Status

Release Engineering
General
RESOLVED FIXED
2 years ago
14 days ago

People

(Reporter: Takanori MATSUURA, Assigned: aleth)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 5 obsolete attachments)

(Reporter)

Description

2 years ago
According to the latest comm-central-l10n build log on linux64, configure process is failed by checking for pkg_config.

checking for pkg_config... not found
DEBUG: pkg_config: Trying /builds/slave/tb-c-cen-l64-l10n-ntly-0000000/build/comm-central/mozilla/gtk3/usr/local/bin/pkg-config
ERROR: Cannot find pkg_config

comm-central-l10n build log on linux64:
https://archive.mozilla.org/pub/thunderbird/nightly/2016/09/2016-09-28-03-02-22-comm-central-l10n/comm-central-linux64-l10n-nightly-ja-bm74-build1-build1014.txt.gz
(Reporter)

Updated

2 years ago
Blocks: 1267523
(Assignee)

Comment 1

2 years ago
Looks like TOOLTOOL_DIR isn't set correctly.
(Assignee)

Comment 2

2 years ago
A similar problem was solved for TB beta builds in bug 1286440:
https://hg.mozilla.org/build/buildbot-configs/rev/e2f6fb3d8204ec5162295c56a1bdaf872260d7a0
https://hg.mozilla.org/build/buildbotcustom/rev/c400d63b386dcfe91030e5706fe51fc0e5992142

I'm not sure the same approach (adding the TOOLTOOL_DIR to the env during signing) will work for c-c and c-a, as the SigningScriptFactory only seems to be used for release builds?
Flags: needinfo?(jlund)
(Assignee)

Updated

2 years ago
Summary: pkg_config is not found during comm-central-l10n build for linux64 → pkg_config is not found during comm-central-l10n repacks for linux64
(Assignee)

Updated

2 years ago
Blocks: 1306194
(Assignee)

Comment 3

2 years ago
(In reply to aleth [:aleth] from comment #2)
> I'm not sure the same approach (adding the TOOLTOOL_DIR to the env during
> signing) will work for c-c and c-a, as the SigningScriptFactory only seems
> to be used for release builds?

Hmm, actually that's not correct, it's also used from misc.py in various places.

Maybe something like this would work after all?
(Assignee)

Comment 4

2 years ago
Created attachment 8799186 [details] [diff] [review]
Set extra_signing_env for non-release builds, buildbotcustom part
Attachment #8799186 - Flags: feedback?(jlund)
(Assignee)

Updated

2 years ago
Assignee: nobody → aleth
Status: NEW → ASSIGNED
(Assignee)

Comment 5

2 years ago
Created attachment 8799188 [details] [diff] [review]
WIP: Set extra_signing_env for TB non-release builds, buildbot-config part
Attachment #8799188 - Flags: feedback?(jlund)
(Reporter)

Updated

2 years ago
Summary: pkg_config is not found during comm-central-l10n repacks for linux64 → pkg_config is not found during comm-central-l10n repacks for linuclinux64
(Reporter)

Updated

2 years ago
Summary: pkg_config is not found during comm-central-l10n repacks for linuclinux64 → pkg_config is not found during comm-central-l10n repacks for linux32 and linux64

Comment 6

2 years ago
(In reply to aleth [:aleth] from comment #5)
> Created attachment 8799188 [details] [diff] [review]
> WIP: Set extra_signing_env for TB non-release builds, buildbot-config part

FYI - I am away until Oct 24th. Please feel free to leave any requests open to me or redirect them to coop@mozilla.com if you need a response sooner.
(Assignee)

Updated

2 years ago
Attachment #8799186 - Flags: feedback?(jlund) → feedback?(nthomas)
(Assignee)

Updated

2 years ago
Attachment #8799188 - Flags: feedback?(jlund) → feedback?(nthomas)
Comment on attachment 8799186 [details] [diff] [review]
Set extra_signing_env for non-release builds, buildbotcustom part

I tried this out, and it doesn't change the master configuration. For jobs like 'Thunderbird comm-central linux l10n nightly' you need to look at NightlyRepackFactory, and give the treatment like bug 1221502. Or adjust the env passed in at line 1376 of misc.py. FWIW, Firefox isn't using NightlyRepackFactory any more.
Flags: needinfo?(jlund)
Attachment #8799186 - Flags: feedback?(nthomas) → feedback-
Attachment #8799188 - Flags: feedback?(nthomas) → feedback-
(Assignee)

Comment 8

2 years ago
Created attachment 8799937 [details] [diff] [review]
Set TOOLTOOL_DIR for l10n repacks

With luck, this might actually be enough to do the trick? I'm not sure if comm-aurora uses the NightlyRepackFactory though.
Attachment #8799937 - Flags: review?(nthomas)
(Assignee)

Updated

2 years ago
Attachment #8799186 - Attachment is obsolete: true
(Assignee)

Updated

2 years ago
Attachment #8799188 - Attachment is obsolete: true
(Assignee)

Comment 9

2 years ago
(In reply to aleth [:aleth] from comment #8)
> I'm not sure if comm-aurora uses the NightlyRepackFactory though.

Looks like it does - at least the builds appear in pub/thunderbird/nightly/.
(Assignee)

Comment 10

2 years ago
Created attachment 8799947 [details] [diff] [review]
Set TOOLTOOL_DIR for l10n repacks
Attachment #8799947 - Flags: review?(nthomas)
(Assignee)

Updated

2 years ago
Attachment #8799937 - Attachment is obsolete: true
Attachment #8799937 - Flags: review?(nthomas)
Comment on attachment 8799947 [details] [diff] [review]
Set TOOLTOOL_DIR for l10n repacks

>         env.update({'EN_US_BINARY_URL': enUSBinaryURL})
>+        env.update({'TOOLTOOL_DIR': WithProperties('%s/%s' % (basedir, branchName))})

This causes
  NameError: global name 'basedir' is not defined
so I you'd want this instead
  env.update({'TOOLTOOL_DIR': WithProperties('%(basedir)s/' + branchName)})
That causes quiet a lot of changes in the dump of the master - a lot of steps are getting the extra TOOLTOOL_DIR arg in the env, including the call to configure you're wanting to fix and make installers-<locale>. It looks harmless, but if it turns out to cause bustage then modifying BaseRepackFactory.configure() at http://hg.mozilla.org/build/buildbotcustom/file/default/process/factory.py#l2849 may be the way to go.

I see tooltool running in /builds/slave/tb-c-cen-lx-l10n-ntly-00000000/build/comm-central in [1], but this doesn't match NightlyRepackFactory.doRepack() at line 3461, which has
  printconfig_env.update({'TOOLTOOL_DIR': WithProperties('%(basedir)s/build')})

Could you dig a little further and work out which is right ?


[1] https://archive.mozilla.org/pub/thunderbird/nightly/2016/10/2016-10-10-03-02-12-comm-central-l10n/comm-central-linux-l10n-nightly-af-bm74-build1-build1226.txt.gz
Attachment #8799947 - Flags: review?(nthomas) → review-
(Assignee)

Comment 12

2 years ago
(In reply to Nick Thomas [:nthomas] from comment #11)
> That causes quiet a lot of changes in the dump of the master - a lot of
> steps are getting the extra TOOLTOOL_DIR arg in the env, including the call
> to configure you're wanting to fix and make installers-<locale>. It looks
> harmless, but if it turns out to cause bustage then modifying
> BaseRepackFactory.configure() at
> http://hg.mozilla.org/build/buildbotcustom/file/default/process/factory.
> py#l2849 may be the way to go.

If it turns out we have to move it to BaseRepackFactory.configure, it will probably need an if(Thunderbird).

> I see tooltool running in
> /builds/slave/tb-c-cen-lx-l10n-ntly-00000000/build/comm-central in [1], but
> this doesn't match NightlyRepackFactory.doRepack() at line 3461, which has
>   printconfig_env.update({'TOOLTOOL_DIR':
> WithProperties('%(basedir)s/build')})

From the log, tooltool unpacks gtk3 into /builds/slave/tb-c-cen-lx-l10n-ntly-00000000/build/comm-central/, 
so it looks like that line is wrong, and now TOOLTOOL_DIR should already be in the env anyway.

> Could you dig a little further and work out which is right ?

On beta we have {'TOOLTOOL_DIR': '%(basedir)s/comm-beta'} [1], which is what I followed here. 
For central/aurora it looks like we need the extra build/ in the path, good catch!

I'm not sure why this differs between nightly/release/repack, but let's just get it working for now.

[1] https://ftp.mozilla.org/pub/thunderbird/candidates/50.0b1-candidates/build2/logs/release-comm-beta-linux64_repack_10-bm91-build1-build6.txt.gz
(Assignee)

Comment 13

2 years ago
Created attachment 8800194 [details] [diff] [review]
Set TOOLTOOL_DIR for l10n repacks
Attachment #8800194 - Flags: review?(nthomas)
(Assignee)

Updated

2 years ago
Attachment #8799947 - Attachment is obsolete: true
(Assignee)

Comment 14

2 years ago
> > this doesn't match NightlyRepackFactory.doRepack() at line 3461, which has
> >   printconfig_env.update({'TOOLTOOL_DIR':
> > WithProperties('%(basedir)s/build')})
> 
> From the log, tooltool unpacks gtk3 into
> /builds/slave/tb-c-cen-lx-l10n-ntly-00000000/build/comm-central/, 
> so it looks like that line is wrong, and now TOOLTOOL_DIR should already be
> in the env anyway.

Hmm, doRepack() uses self.env, so that won't be affected by the changes in this patch.
(Assignee)

Comment 15

2 years ago
Created attachment 8800197 [details] [diff] [review]
Set TOOLTOOL_DIR for l10n repacks
Attachment #8800197 - Flags: review?(nthomas)
(Assignee)

Updated

2 years ago
Attachment #8800194 - Attachment is obsolete: true
Attachment #8800194 - Flags: review?(nthomas)
(In reply to aleth [:aleth] from comment #12)
> If it turns out we have to move it to BaseRepackFactory.configure, it will
> probably need an if(Thunderbird).

I don't think so. The only class using BaseRepackFactory is NightlyRepackFactory, and and on releng-infra only Thunderbird uses that. Seamonkey probably does on their own infra too. So there's no difference modifying either of the two choices from that point of view.
 
> > Could you dig a little further and work out which is right ?
> 
> On beta we have {'TOOLTOOL_DIR': '%(basedir)s/comm-beta'} [1], which is what
> I followed here. 

That's a release build, which uses SigningScriptFactory instead of NightlyBuildFactory (which is why the first round of patches was a miscue).
(Assignee)

Comment 17

2 years ago
(In reply to Nick Thomas [:nthomas] from comment #16)
> I don't think so. The only class using BaseRepackFactory is
> NightlyRepackFactory, and and on releng-infra only Thunderbird uses that.
> Seamonkey probably does on their own infra too. So there's no difference
> modifying either of the two choices from that point of view.

Thanks, I wasn't sure what still used buildbot for this. (Seamonkey iirc have their own fork of buildbotcustom.)
 
> > > Could you dig a little further and work out which is right ?
> > 
> > On beta we have {'TOOLTOOL_DIR': '%(basedir)s/comm-beta'} [1], which is what
> > I followed here. 
> 
> That's a release build, which uses SigningScriptFactory instead of
> NightlyBuildFactory (which is why the first round of patches was a miscue).

To be clear, by "here" I meant the previous patch.
Comment on attachment 8800197 [details] [diff] [review]
Set TOOLTOOL_DIR for l10n repacks

Let's try it.

http://hg.mozilla.org/build/buildbotcustom/rev/01f02115479b  (default)
http://hg.mozilla.org/build/buildbotcustom/rev/c3eb75097fe3  (production-0.8)
Attachment #8800197 - Flags: review?(nthomas)
Attachment #8800197 - Flags: review+
Attachment #8800197 - Flags: checked-in+
(Assignee)

Comment 19

2 years ago
Looks like it worked! Thanks for the quick review & deployment.
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.