Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10576

Importing a valid pipeline to Hydrator fails

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5.0
    • Fix Version/s: 3.5.0
    • Component/s: Pipelines
    • Labels:
      None
    • Release Notes:
      Removes the dependency of using labels from plugins in pipelines being imported in UI. Any pipeline configuration publishable from CLI or REST api should be publishable from UI too.
    • Rank:
      1|hzzfzb:

      Description

      I am testing the documentation example pipeline JSON by importing our examples into Hydrator.

      The attached example fails, with this error:

      07/13/16 8:48:29 pm: undefinedNode with the same name already exists.
      

      The reason why it is failing is because neither of the stages has a "label"; instead, they just have "names", though they are unique names.

      If I add a single "label":

      "label": "Stream",
      

      in the first stages' plugin section, at line 16, the pipeline loads.

      But the error message is indicating that it actually attempting to use that "undefined" labels as some sort of key. I can't imagine that would work too well when it came to connections...

      The pipeline validates, even though the second one is missing a label, which is marked in red in the UI.

      These things should happen:

      1. We need a written specification for the format for these pipelines.
      2. It would be good if the schema wasn't a string but instead a JSON object.
      3. The UI should not have anything stored outside the __ui__ object.
      4, The UI should key off the name of the stage, not the label. Or maybe if there is no label, it uses the name?

      This first pipeline (streamETLApp_config.json) can be loaded with a curl call:

      curl -w"\n" -X PUT localhost:10000/v3/namespaces/default/apps/streamETLApp \
              -H "Content-Type: application/json" -d @streamETLApp_config.json
      

      indicating that it is valid as JSON and a pipeline.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ajai Ajai Narayan
                Reporter:
                John John Jackson
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: