Resolve pipeline ingestion mock build issues

RESOLVED INCOMPLETE

Status

Cloud Services
Metrics: Pipeline
P3
normal
RESOLVED INCOMPLETE
6 months ago
a month ago

People

(Reporter: whd, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [SvcOps])

(Reporter)

Description

6 months ago
Depending on whether we decide to move our ingestion infrastructure to docker (bug #1357263), we need to resolve a couple of issues with the current pipeline packaging and building steps.

It's been a while since I looked into this, but there are two mains issues I recall.

First, we reply on people.mo for production builds. Currently :trink publishes the lsb, lsb extensions, hindsight and hindsight-admin packages there every sprint. These packages are picked up in the pipeline build and packaging script and then used to build openresty/nginx-moz-ingest. Per operations standards, we should build these packages from source on our build hosts, but (IIRC) issues related to building hindsight-admin on Redhat-based systems prevent us from doing so. We should determine what the issues are and resolve them.

Second, we currently don't have a way of building parquet-cpp, and as a result we are using an old version from when we were able to build it. I believe this is because at some point an external build dependency that was relied upon by an old version of parquet-cpp became inaccessible, and :trink's cmake patches for RPM generation don't apply cleanly to parquet-cpp master anymore. We should update the :cmake patches to work with newer parquet-cpp versions, and get them merged upstream if possible.

Updated

3 months ago
Blocks: 1384208
Closing abandoned bugs in this product per https://bugzilla.mozilla.org/show_bug.cgi?id=1337972
Status: NEW → RESOLVED
Last Resolved: a month ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.