1541001 - Error checking filter for <recipe_name> when using the Stage server set-up

Status: RESOLVED INVALID

(Firefox :: Normandy Client, defect)

Description

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

Attachment: image.png

[Affected versions]:

• Firefox Release 66.0.2, Build ID 20190326175229
• Firefox Beta 67.0b6, Build ID 20190328152334
• Firefox Nightly 68.0a1, Build ID 20190401215651

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• On a new profile, set the following prefs to the following values:
  • Set the "security.content.signature.root_hash" pref to "DB:74:CE:58:E4:F9:D0:9E:E0:42:36:BE:6C:C5:C4:F6:6A:E7:74:7D:C0:21:42:7A:03:BC:2F:57:0C:8B:9B:90";
  • Set the "extensions.shield-recipe-client.api_url" pref to "https://stage.normandy.nonprod.cloudops.mozgcp.net/api/v1";
  • Set the "app.normandy.dev_mode" pref to true;
  • Set the "app.normandy.logging.level" to 0;
  • Set the "cipri" pref to true;

[Steps to reproduce]:

1. Open the Browser with the profile from prerequisites.
2. Open the Browser Console and observe the contents.

[Expected results]:

• 4 recipes are executed.

[Actual results]:

• No recipes are executed and an error is thrown for each one.

[Notes]:

• The issue reproducible with all recipe types (heartbeat, pref experiment, opt-out experiment, rollout)
• Attached a screenshot of the issue.

Comment 1

[Michael Cooper [:mythmon]]

A shorter STR:

1. Open the Browser Console
2. Run the following code:

const {RecipeRunner} = ChromeUtils.import("resource://normandy/lib/RecipeRunner.jsm");
await RecipeRunner.checkFilter({
  id: 1,
  name: "test recipe",
  filter_expression: 'cipri|preferenceExists',
})

With the smaller test case, the problem is easier to see. The given filter expression is cipri|preferenceExists when it should be "cipri"|prefrenceExists. Since cipri is not a value present in the context, this is similar to a filter expression undefined|preferenceExists. We could give a better error message here, but the system is working correctly. The filter is syntactically correct, but invalid since there is no value named cipri in the context.

Nice error messages for filter expressions is not something we have worried about historically, and I don't think we're going to start now. The fact that this errored out and sent Telemetry is enough.

Comment 2

[Michael Cooper [:mythmon]]

Additionally, the original STR were not complete, since they did not include any details about the recipe in question. Recipes contain vital information that affects the behavior of Normandy. In this case the bug was entirely in the Recipe. In the future, please include the relevant details of the recipes created that caused the problem.

Comment 3

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

In the past, I've used the same set-up for recipe creation and did not hit this issue. I will keep this in mind to add the recipes I use for the following bugs I log, sorry for the omission.

Shouldn't the Delivery Console have caught this issue with the additional filter I used? Should I file a bug for this?

Comment 4

[Ciprian Muresan [:cmuresan], Formerly of Ecosystem QA - please needinfo @mcoman for things regarding the team]

Looking back through the recipes I've created in the past it seems that what I remembered was wrong and I've always used " or ' when using the "preferenceExisits" filter. Still, the question about the Delivery Console catching this still stands.

Comment 5

[Michael Cooper [:mythmon]]

It would be reasonable for Delivery Console to show a warning in this case, though it shouldn't block recipe edits. It would be a decent sized piece of work though, because it would likely require a custom parser, executor, or heavy modification to JEXL to accomplish. That kind of work has been something we've been avoiding thus far.