Develop model for reasonable splits of countries
Categories
(Growth :: Modeling, task)
Tracking
(Not tracked)
People
(Reporter: frank, Assigned: vsabino)
Details
Our marketing campaigns have many hypotheses we'd like to test - e.g. multiple creatives, different CPAs, and possible other targeting criteria in the future. We cannot run campaigns in parallel in a region - they eat each other and the results don't indicate which campaign performed better.
Instead, we're going to develop a method to split countries into multiple regions, such that performance of each region is relatively similar (within some covariates). We can run a single hypothesis in each region, and compare them directly to see which performed better.
In the immediate future, we will use this to test multiple cost-per-action (CPA) campaigns, alongside a CPI campaign.
This ticket is for developing a model we can use to split countries into reasonable subregions. For the immediate campaign we should see if we can split into 5 regions; however it would be optimal to allow changes to that based on action counts (if e.g. we run a campaign on a cost-per-action that has fewer events).
|
Reporter | |
Comment 1•3 years ago
|
||
Note: Initially we're going to target splits for our top countries in both NA and EU - US, CA, FR, and DE.
|
Assignee | |
Comment 2•3 years ago
|
||
The split was done based on random assignments of the subregions of the selected countries.
We used data from the last three months of the most recent campaign to find the best assignment over a 1000x simulation, looking for groups that were distributed evenly according to the criteria:
- total number of new profiles
- total number of activated profiles
- correlation (daily) of new profiles
Code and analysis can be found on this Colab notebook and the resulting split for North America and Europe in this spreadsheet.
Description
•