1007280 - Allow Mozharness to have r/w options (live_config) with caching capabilities to help individual steps easily

Status: RESOLVED WONTFIX

(Release Engineering :: Applications: MozharnessCore, defect)

Description

[Jordan Lund (:jlund)]

This will help give Mozharness's actions the ability to be more stand alone.

Actions should be standalone. But some actions do have to depend on each other.

For example, if you have two actions: 'download-application', and 'test-application', and you can only know the filename of the application during the actual download action, the test action depends on the download action.

But what if you have already run both actions in a previous script run and now you only want to 'test-application'? If you haven't clobbered this should be doable.

What we need here is a way to store config items on the fly (self.live_config) and then also be able to cache items for future runs.

Comment 1

[Jordan Lund (:jlund)]

Attachment: 140507_mozharn_live_config-mh.patch WIP DRAFT

this needs nosetests before proceeding

Comment 2

[Jordan Lund (:jlund)]

I think when checking if we can use the cache:

I should switch to using self.__class__.__name__ over sys.argv[0] for verifying that the cache config was made from the same script as the current live_config one.

so:
<snippet>
+    def dump_live_config_as_cache(self):
+        # allow for custom dumps
+        self.pre_dump_live_config()
+        self.info('Dumping live_config as a cache')
+        if self.live_config:
+            # script_used_against will be whatever baseclass name we called
+            # this script from
+            self.live_config["script_used_against"] = self.__class__.__name__
+            self.dump_config(self.live_config.cache_path, self.live_config)
+        else:
+            self.info("live_config was not used. Nothing to dump.")
+
+    def is_cache_config_usable(self, cache_config):
+        is_usable = True
+        # current_script_name will be whatever baseclass name we called this
+        # script from
+        current_script_name = self.__class__.__name__
+        previous_script_name = cache_config.get("script_used_against")
+        if current_script_name is not previous_script_name:
+            self.info("Ignoring cache as it was created against a different "
+                      "or unknown script than this one.")
+            is_usable = False

Comment 3

[Aki Sasaki (not active)]

Hm. Mozharness is in maintenance/when-do-we-deprecate mode. We may want to determine if this is worth doing.