Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0
    • Component/s: Pipelines
    • Labels:
      None
    • Release Notes:
      Changed the format of Hydrator configuration. All pipeline stages are now in a 'stages' array instead of broken up into 'source', 'transforms', and 'sinks' arrays.
    • Rank:
      1|hzz87j:

      Description

      The etl apps use configs where the plugin types are at the top level. We need to make this more generic because we are starting to add more and more plugin types.

      To support additional types, we should move the plugin type down into the plugin section of each stage. The config then becomes a list of stages, a list of connections, resources, and any other options needed.

      In addition, we need to start putting the config version in the config, to allow us to handle incompatibilities more fluidly.

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: