408811 - warning in Bootstrap::Stage (Use of uninitialized value in hash element)

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Nick Thomas [:nthomas] (UTC+12)]

I saw this during Tb1.5.0.14 staging, but it's also present in Fx2.0.0.11 and Fx3.0b1 staging logs (but at different line numbers). eg:

log: WARNING: Unknown file type in tree: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/win32_signing_rc1.log
Use of uninitialized value in hash element at Bootstrap/Step/Stage.pm line 644.
log: Deleting unwanted locale deliverable: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/win32_signing_rc1.log
log: WARNING: Unknown file type in tree: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/linux_info.txt
Use of uninitialized value in hash element at Bootstrap/Step/Stage.pm line 644.
log: Deleting unwanted locale deliverable: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/linux_info.txt
...
log: WARNING: Unknown file type in tree: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/macosx_info.txt
Use of uninitialized value in hash element at Bootstrap/Step/Stage.pm line 644.
log: Deleting unwanted locale deliverable: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/macosx_info.txt
...
log: WARNING: Unknown file type in tree: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/win32_info.txt
Use of uninitialized value in hash element at Bootstrap/Step/Stage.pm line 644.
log: Deleting unwanted locale deliverable: /data/cltbld/firefox-2.0.0.11/batch1/prestage-trimmed/win32_info.txt

It's this code (in Stage.pm, IsValidLocaleDeliverable()), barfing on files that don't fit our preconceptions of the filename:

    .... # a bunch of other tests on files that ship, similar to this elsif
    } elsif ($dirent =~ /\.mar/) {
        # e.g. firefox-2.0.0.2.tr.win32.[partial,complete].mar
        $locale = $parts[$partsCount - 4];
        $platform = $DELIVERABLE_TO_PLATFORM{$parts[$partsCount - 3]};
    } else {
        $this->Log(msg => "WARNING: Unknown file type in tree: $dirent");
    }

--> foreach my $allowedPlatform (@{$this->{'localeManifest'}->{$locale}}) {
        return 1 if ($allowedPlatform eq $platform);
    }

The --> indicates the line the error occurs on (644 in rev1.23 for RELEASE_AUTOMATION_M6_1, 655 in rev 1.26 - the current tip). The problem is that $locale and $platform are not defined when we hit the else clause with the BuildID files and the win32 signing log.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Add BuildID and signing log to list of excluded files

This is one approach to fix it, but we could also "return 0" in the else block in comment #0 if we want a blanket exclusion for new files. What do you prefer Rob ?

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [checked in] Add BuildID and signing log to list of excluded files - but don't hardcode rc1

Oops, hardcoded rc1 in there, instead of rc\d+

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to comment #1)
> Created an attachment (id=293655) [details]
> Add BuildID and signing log to list of excluded files
> 
> This is one approach to fix it, but we could also "return 0" in the else block
> in comment #0 if we want a blanket exclusion for new files. What do you prefer
> Rob ?

I think we should at least make it so encountering new files isn't a syntax error. Since we don't know what to do, we should probably make it fatal.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 293656 [details] [diff] [review]
[checked in] Add BuildID and signing log to list of excluded files - but don't hardcode rc1

Checking in Bootstrap/Step/Stage.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/Stage.pm,v  <--  Stage.pm
new revision: 1.27; previous revision: 1.26
done

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Comment #3 is still to be done. Feel free to grab it if you have time.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [checked in] Crash and burn on files that are unexpected

This just converts the existing warning into something fatal. Stage::Execute exits after populating prestage-trimmed.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 299209 [details] [diff] [review]
[checked in] Crash and burn on files that are unexpected

Checking in Bootstrap/Step/Stage.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/Stage.pm,v  <--  Stage.pm
new revision: 1.34; previous revision: 1.33
done